Ben Chuanlong Du's Blog

It is never too late to learn.

Spark Issue: IllegalArgumentException: Wrong FS

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptoms

java.lang.IllegalArgumentException: Wrong FS: hdfs://..., expected: viewfs://...

Possible Causes

The Spark cluster has migrated to Router-based Federation (RBF) namenodes, and viewfs:// (instead of hdfs://) is required to access HDFS …

Spark Issue: ViewFs: Cannot Initialize: Empty Mount Table in Config

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptoms

java.io.IOException: ViewFs: Cannot initialize: Empty Mount table in config for viewfs://cluster-name-ns02/

Possible Causes

As the error message says, viewfs://cluster-name-ns02 is not configured.

  1. It is possible that …