Yarn 日志无法访问: Aggregation may not be complete, Check back later or try the nodemanager at ....

雨点打透心脏的1/2处 2023-07-13 08:21 14阅读 0赞

用的 ambari 日志看不到 yarn运行结果日志. 折腾我了一个半小时...炸毛!!!!!!

报错截图:

Logs not available for container_e08_1583895535087_0001_01_000001. Aggregation may 
    not be complete, Check back later or try the nodemanager at henghe-047:45454

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70][]

解决方案:

1.修改配置, 在HDFS > Configurations > Custom core-site > Add Property

>     hadoop.http.staticuser.user=yarn

不改会报错: User \[dr.who\] is not authorized to view the logs for application

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 1][]

2.删除配置:

YARN > Configurations > Custom yarn-site

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 2][]

3.修改配置 YARN > Configurations >Resource Manager > 勾选 Enable Log Aggregation

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 3][]

4.修改配置 YARN > Configurations >Node Manager

修改 yarn.nodemanager.remote-app-log-dir  值为 /tmp/logs

\[目录必须这个,别问我为啥,我也不知道,知道的可以告诉我一下,试了很多遍就是不起作用,我也很无奈....\]

> 有兄弟@qq\_37549847 跟我说要 \`将yarn.nodemanager.remote-app-log-dir配置的目录 执行下 chmod +t 设置了黏着位 \` ,这个我还没试.  先记录一下.

5. 重启 hdfs, 重启 yarn ,重启 mr . \[别偷懒, 只重启 yarn 不起作用...\]

我之前的 yarn 配置更改前跟更改后的对比. 截图. 有知道为啥这么操作的请大神告知....

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 4][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70]: /images/20230529/60cbf626e403479986ae34e1b32e418f.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 1]: /images/20230529/119e85b696ca4a9f9a14b0f319396bf1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 2]: /images/20230529/5815eb4506b64824ac01b71526571deb.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 3]: /images/20230529/574469da04934b1bb8bb3075dc3f32be.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nbG9uZ180NDQ0_size_16_color_FFFFFF_t_70 4]: /images/20230529/37271031e0044c659b84d7691a6a3fbd.png