Ben Chuanlong Du's Blog

It is never too late to learn.

Builtin Objects Python

Python has built-in functions and object that users can use directly (no need to import). However, if you import another module which hide a built-in function or object, you cannot use it anymore. For example, sum is a built-in function in Python which can be used directly. However, if you use PySpark import SQL functions (from pyspark.sql.functions import *

Python Virtual Environment

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. venv is a standard Python library and is the recommended way for managing virtual environments in Python3.

  2. When developing a Python project, it is recommended that you use Poetry to manage …

Hands on the Python module dask

Installation

  1. You have to install the complete version of Dask (using the command pip3 install dask[complete]) if you need support of extended memory (for handling big data) and schedulers (for performance). The default installation version (pip3 install dask) of Dask does not include those features out-of-box.