Ben Chuanlong Du's Blog

It is never too late to learn.

Spark Issue Libc Not Found

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptom

/lib64/libc.so.6: version `GLIBC_2.18' not found (required by ...)

Cause

The required version of GLIBC by the binary executor is not found on Spark nodes.

Solution

Recompile your …

Read and Write CSV Files in Rust

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips and Traps

  1. By defaut, csv::Reader requires headers.

  2. When the csv crate is used together with the serde crate for deserialization, CSV files to be parsed have to be strictly well formatted. For example, the headers in CSV files have to match the defintion in the serde struct. Otherwise, the code will panic with an error of "missing fields".

IO in Rust

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

General Tips on Programming

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. Do NOT chase the latest versions of libraries/software/tools. Wait for some time for them to be tested thoroughly before adopting them.

  2. Follow a good Semantic Versioning if you release …