Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

1. omegaconf can parse command-line options too. However, unlike argparse it does not enforce any constraint on command-line options.

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

After installing a Python package, you can use the command pip3 show pkg to check detailed information about it. What if you want to get information about a Python package without installing? The RESTAPI https://pypi.org/pypi/pkg/json returns detailed information of the Python package pkg.

Note: The recursive version of dependencies implemented bellow does not take into consideration of versions, so it might not return the correct dependencies of a Python packages.

References¶

https://github.com/databricks/koalas

https://databricks.com/blog/2020/08/11/interoperability-between-koalas-and-apache-spark.html

https://notebooks.gesis.org/binder/jupyter/user/databricks-koalas-mxw72n1l/notebooks/docs/source/getting_started/10min.ipynb

https://spark.apache.org/docs/latest/api/python/pyspark.sql.html

https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame

https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.Column

https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#module-pyspark.sql.functions