Ben Chuanlong Du's Blog

It is never too late to learn.

Data Engineering Tools

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

https://github.com/linkedin/datahub

https://engineering.linkedin.com/blog/2019/data-hub DataHub: A generalized metadata search & discovery tool

GPU Related Issues and Solutions

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips

  1. Training a model requires significantly more CPU/GPU memories than running inference using the model.

  2. torch.cuda.empty_cache() doesn't help if memory is not enough

  3. It is suggested that you …

Benchmark of Popular Deep Learning Models

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Memory

  1. ResNet152 inferencing requies about (a little bit less than) 2G GPU memory and training can be done with 8G GPU memory (need to set a small batch size). Deeper ResNet …

Dual Boot Linux and Windows

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Mount the Windows File System

When you dual boot your machine with Linux (e.g., Ubuntu) and Windows, the Windows disk/partition might not be mounted automatically. In that case, you …