现象：在提交spark任务时指定executor-memory 2g时报错


Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, Required executor memory (2048+384 MB) is above the max threshold (2048 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocation-mb' and/or 'yarn.nodemanager.resource.memory-mb'.
java.lang.IllegalArgumentException: Required executor memory (2048+384 MB) is above the max threshold (2048 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocation-mb' and/or 'yarn.nodemanager.resource.memory-mb'.
executor memory的上限时2g，参数配置中executor memory 2g但是集群自身的jvm也需要内存导致资源超出上限；



先改了

yarn.scheduler.maximum-allocation-mb
为4g但是仍然报同样的错误；

又改参数

yarn.nodemanager.resource.memory-mb

顺利跑完 ；
---------------------
作者：向阳飞行
来源：CSDN
原文：https://blog.csdn.net/bigdataprimary/article/details/83176572
版权声明：本文为博主原创文章，转载请附上博文链接！

奈斯