Ben Chuanlong Du's Blog

It is never too late to learn.

Tips on Eclipse Che

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

You can launch an Eclipse Che server using the command below.

docker run -it -v /var/run/docker.sock:/var/run/docker.sock -v /workdir/che:/data eclipse/che start

The …

Share Memory Among Processes in Python

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

https://www.youtube.com/watch?v=uWbSc84he2Q

https://docs.python.org/3.8/library/multiprocessing.html#multiprocessing.Value

Use XGBoost With Spark

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

The split-by-leaf mode (grow_policy="lossguide") is not supported in distributed training, which makes XGBoost4J on Spark much slower than LightGBM on Spark.

XGBoost with Spark

https://towardsdatascience.com/build-xgboost-lightgbm-models-on-large-datasets-what-are-the-possible-solutions-bf882da2c27d

https://xgboost …

Tips on Amazon AWS

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips and Traps

  1. By default, AWS shows your resources (VMs, etc.) in one location (data center) only. This can be tricky if you have VMs in mutiple locations (data centers) as …