Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Since a Spark DataFrame is immutable, you cannot update or delete records from a physical table (e.g., a Hive table) directly using Spark DataFrame/SQL API. However …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Hive is case-insensitive, both keywords and functions
You can use both double and single quotes for strings
use = rather than == for equality comparison
but it seems that == also works
use % rather …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
15/12/29 17:22:27 ERROR yarn.ApplicationMaster: User class …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
You can use the hdfs dfs -du /path/to/table command
or hdfs dfs -count -q -v -h /path/to/table
to get the size of an HDFS path (or table).
However,
this only works if the cluster supports HDFS.
If a Spark cluster exposes only JDBC/ODBC APIs,
this method does not work.