Ben Chuanlong Du's Blog

It is never too late to learn.

SQL Database Client-server Protocols

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. Apache Arrow Flight is the future protocol for querying Databases! It use columnar data and leverages Apache Arrow to avoid unnecessary copy of data, which makes it able to query large …

SQL Equivalent

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

SQL translation is a great tool that transalte any SQL statement(s) to a different dialetc using the JOOQ Parser.

SQL Variant Code
List
databases [1 …

Tips on JSON

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Shortcomes of JSON

It is suggested that you avoid using the JSON format! TOML and YAML are better text-based alternatives. If readability is not a concern, a binary serialization format is …

Improve the Performance of Spark

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Plan Your Work

  1. Have a clear idea about what you want to do is very important, especially when you are working on an explorative project. It often saves you time to …

Data Quality

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  • Upper and lower bounds tests and Inter Quartile Range Checks(IQR) and standard deviations

  • Aggregate level checks (after manipulating data, there should still be the ability to explain how the data …