Ben Chuanlong Du's Blog

It is never too late to learn.

Spark Issue: Duplicated Partitions

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

There seems to be an issue in Spark that it might fail to overwrite files even if mode of spark.write is set to be "overwrite".

Comments