Logging in PySpark
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
-
Excessive logging is better than no logging! This is generally true in distributed big data applications.
-
Use
loguru
if it is available. If you have to use thelogging
module, be …
Markdown vs RestructureText vs MyST for Documentation
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Comparison
- Compared to Markdown, RestructuredText is more fully-featured, much more standardized and uniform, and has built-in support for extensions. However, ReStructuredText is also criticized for its complex and confusing syntax. The …
Markedly Structured Text
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
MySt is an extension of CommonMarkdown with some goodies introduced from RestructuredText. The jupyter-book project supports MySt syntax.
References
https://github.com/executablebooks/MyST-Parser
https://myst-parser.readthedocs.io/en/latest/
Dunder/Magic Methods in Python
__call__
(Object Invocation)¶
We can make any object callable like a regular function by adding the call dunder method. Let’s consider the below toy(not very meaningful) example just to showcase the call method.
Online Practices for SQL
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!