Ben Chuanlong Du's Blog

It is never too late to learn.

Command Line Tools for Mac File System

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Note: When work both in Linux and macOS, a painful burden to use command-line is to remember subtle differences between Linux commands and macOS commands. A a matter of fact, GNU …

Tips on NFS

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. Make sure nfs service is not blocked. Check iptables and "/etc/hosts.allow".

  2. Make sure nfs service is running on the server.

    sudo service nfs-kernel-server start

  3. You can choose the version …

Check Whether a File Exists in Spark

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

org.apache.hadoop.fs.FileSystem

val conf = sc.hadoopConfiguration
val fs = org.apache.hadoop.fs.FileSystem.get(conf)
val exists = fs.exists(new org.apache.hadoop.fs.Path("/path/on/hdfs …