Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Symptoms
java.lang.IllegalArgumentException: Wrong FS: hdfs://..., expected: viewfs://...
Possible Causes
The Spark cluster has migrated to Router-based Federation (RBF) namenodes,
and viewfs:// (instead of hdfs://) is required to access HDFS paths.
Possible Solutions
-
Use
viewfs://instead ofhdfs://. -
Use relative HDFS paths (without
viewfs://orhdfs://prefixes).