Comment¶
pathlib.Path.glob is preferred to the
globmodule!!The list returned is not sorted.
GPU Computing in Python
Hands on the Python module numba
Installation¶
numba can be installed using the following command.
:::bash
pip3 install numba
If you need CUDA support, you have to install CUDA drivers.
:::bash
sudo apt-get install cuda-10-1
Instead of going through the hassle of configuring numba for GPU, a better way is to run numba in a Nvidia Docker container. The Docker image nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 has CUDA runtime installed, so it is as easy as installing numba on top it and you are ready to go. For more detailed instructions, please refer to
Write Unit Tests Using unittest in Python
Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
python -m unittest test_module1 test_module2 python -m unittest test_module.TestClass python -m unittest test_module.TestClass.test_method python -m unittest tests/test_something.py
References
Speed up Python Using Native Code
Tips and Traps¶
- It is never a good idea to do pre-mature optimization. Always profile (please refer to Profile Performance of Python Applications and Python Profiler for JupyterLab Notebooks) your code before you optimize it. If you do decide to optimize you Python code, there are multiple approaches. First, think about whether you can improve your algorithm. Please refer to Python Performance Tips
Broadcast Arrays in Numpy
Tips and Traps¶
The broadcast concept in numpy is essentially a way to "virtually" duplicate data in a numpy array so that it is "virtually" reshaped to be compatible with another numpy array for a certain operation. Do not confused yourself about it with the broadcast concept in Spark which sends a full copy of a (small) DataFrame to each work node for
BroadCastJoin