Ben Chuanlong Du's Blog

It is never too late to learn.

Path Pattern Matching in Python

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips and Traps

  1. Pathspec is preferred over zgitignore as the latter is not actively maintained.

pathspec

pathspec is a utility library for gitignore style pattern matching of file paths.

zgitignore

zgitignore checks if a file is ignored by a .zgitignore

Tips on sshuttle

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. sudo permission is required to run sshuttle.

  2. It's valid to run sshuttle more than once simultaneously on a single client machine, connecting to a different server every time, so you can …

Tips on Jinja

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips on Feature Engineering for Machine Learning

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

https://towardsdatascience.com/automatic-feature-engineering-using-deep-learning-and-bayesian-inference-application-to-computer-7b2bb8dc7351

Feature selection Feature extraction Adding features through domain expertise

FeatureTools a Python library for feature engineering Deep neural network can extract features too

whether feature engineering is …

Get the Dependencies of a Python Package Without Installing It

After installing a Python package, you can use the command pip3 show pkg to check detailed information about it. What if you want to get information about a Python package without installing? The RESTAPI https://pypi.org/pypi/pkg/json returns detailed information of the Python package pkg.

Note: The recursive version of dependencies implemented bellow does not take into consideration of versions, so it might not return the correct dependencies of a Python packages.

Access Control in Spark SQL

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Grant Permission to Users

GRANT
    priv_type [, priv_type ] ...
    ON database_table_or_view_name
    TO principal_specification [, principal_specification] ...
    [WITH GRANT OPTION];

Examples:

GRANT SELECT ON table1 TO USER user1;
GRANT SELECT ON DATABASE db1 TO USER user1 …