Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Official Zeppelin Docker Image

1. Pull the official Zeppelin Docker image.

   docker pull apache/zeppelin
   

2. Launch the image in a container.

   docker run -d -p 8080:8080 \
       -v $PWD/logs:/logs \
       -v …

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

1. Spark MLlib RDD-based API supports stratified sampling but the DataFrame-based API hasn't implemented it yet as of Spark 2.4.3.

sample keys (not rows) with equal probability

References

https://spark …

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

org.apache.hadoop.fs.FileSystem

val conf = sc.hadoopConfiguration
val fs = org.apache.hadoop.fs.FileSystem.get(conf)
val exists = fs.exists(new org.apache.hadoop.fs.Path("/path/on/hdfs …