Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Block rdd_123_456 could not be removed as it was not found on disk or in memory.
The execution plan of a DataFrame is too complicated.
Not enough memory to …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Stack, heap and off-heap are all memory that stored in a computer's RAM.
Stack is used for static memory allocation. Variables allocated on the stack are stored directly to the …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Exception in thread "main" java.lang.IllegalArgumentException: System memory 466092032 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
High disk and memory spill when doing shuffle.
Insufficient executor memory (you can monitor this spill metrics from Spark UI).
Increase executor memory.
--executor-memory=4G
For jobs that …