Ben Chuanlong Du's Blog

It is never too late to learn.

Tips on Valgrind

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Memory in JVM

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Stack, heap and off-heap are all memory that stored in a computer's RAM.

Stack

Stack is used for static memory allocation. Variables allocated on the stack are stored directly to the …

Spark Issue: IllegalArgumentException: System Memory Must Be At Least

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptom

Exception in thread "main" java.lang.IllegalArgumentException: System memory 466092032 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration …