String¶

Regular Expression Comparison¶

https://stackoverflow.com/questions/229551/how-to-check-if-a-string-contains-a-substring-in-bash

Bash Programming¶

1. case ... esac is different from switch(){} in C/C++. You do not have to return/exit manually after each branch.

not condition

if [ ! -d "$1" ]; then
fi

Shell Programing Trick¶

1. Generate 00, 01, ..., to 25.

    for i in {00..25}; do echo $i; done
   

   or

    seq -f %02g 0 25

Shell Scripting¶

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Comments¶

1. You must assign a value to each enum member.

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Comments¶

There is no (direct) way of select all columns except a few from a table using SQL. However, this is easily doable with DataFrame APIs (pandas, Spark/PySpark, etc.).

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

The HDFS Way¶

You can use the hdfs dfs -du /path/to/table command or hdfs dfs -count -q -v -h /path/to/table to get the size of an HDFS path (or table). However, this only works if the cluster supports HDFS. If a Spark cluster exposes only JDBC/ODBC APIs, this method does not work.