Ben Chuanlong Du's Blog

It is never too late to learn.

Spark Issue: IllegalArgumentException: Wrong FS

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptoms

java.lang.IllegalArgumentException: Wrong FS: hdfs://..., expected: viewfs://...

Possible Causes

The Spark cluster has migrated to Router-based Federation (RBF) namenodes, and viewfs:// (instead of hdfs://) is required to access HDFS …

IO in Rust

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!