Ben Chuanlong Du's Blog

It is never too late to learn.

Optimization Method in Machine Learning

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

L-BFGS converges faster and with better solutions on small datasets. However, ADAM is very robust for relatively large datasets. It usually converges quickly and gives pretty good performance. SGD with momentum …

Optimization Algorithms

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

If the objective function has derivative, then an optimization problem is equivalent to finding the root of the derivative of the object function.

Finding Root

  • Linear Search Only good for 1-dimensional …

Tips on XGBoost

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

  1. It is suggested that you use the sklearn wrapper classes XGBClassifier and XGBRegressor so that you can fully leverage other tools of the sklearn package.

  2. There are 2 types of boosters …

String vs Bytecode in Python

Comments

  1. Be careful that some functions return bytecode instead of string in Python 3.

  2. bytecode has similar APIs to str. Most string functions can be applied to bytecode as well.

String to Bytecode

Show Inferred Type in IntelliJ IDEA

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

You can use the shortcut ctrl + shift + p to show inferred type of a variable when the cursor is on the variable in IntelliJ IDEA. You can also configure IntelliJ to …