Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Issue
Total size of serialized results is bigger than spark.driver.maxResultSize
Solutions
-
Eliminate unnecessary
broadcast
orcollect
. -
If one of the tables for joining contains too large number of partitions …