Ben Chuanlong Du's Blog

It is never too late to learn.

Logging in PySpark

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. Excessive logging is better than no logging! This is generally true in distributed big data applications.

  2. Use loguru if it is available. If you have to use the logging module, be …

Markdown vs RestructureText vs MyST for Documentation

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Comparison

  1. Compared to Markdown, RestructuredText is more fully-featured, much more standardized and uniform, and has built-in support for extensions. However, ReStructuredText is also criticized for its complex and confusing syntax. The …

Markedly Structured Text

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

MySt is an extension of CommonMarkdown with some goodies introduced from RestructuredText. The jupyter-book project supports MySt syntax.

References

https://github.com/executablebooks/MyST-Parser

https://myst-parser.readthedocs.io/en/latest/

Dunder/Magic Methods in Python

__call__ (Object Invocation)

We can make any object callable like a regular function by adding the call dunder method. Let’s consider the below toy(not very meaningful) example just to showcase the call method.