Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Caused by: org.apache.spark.SparkException: Could not execute broadcast in 600 secs. You can increase the timeout for broadcasts via spark.sql.broadcastTimeout or disable broadcast join by setting …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Most new programming languages (such as Rust, Go, Kotlin, Scala, etc.) support functional programming style and have clear distinguishment on mutable vs immutable variables. So, is functional programming superior to imperative …
Bucketed column is only supported in Hive table at this time.
A Hive table can have both partition and bucket columns.
Suppose t1 and t2 are 2 bucketed tables and with the number of buckets b1 and b2 respecitvely.
For bucket optimization to kick in when joining them:
- The 2 tables must be bucketed on the same keys/columns.
- Must joining on the bucket keys/columns.
- `b1` is a multiple of `b2` or `b2` is a multiple of `b1`.
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
The issue can be fix by stripping null characters using the following command.
tr -d '\000' < filein > fileout
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Enum is the preferred way to constrcut a sum type of several types (which does not implemente the same trait).
The Rust crate
either
provides an enum Either (with variants Left …
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
This post collects some typical pure Python errors in PySpark applications.
object has no attribute
Fix the attribute name.
No such file or directory