Ben Chuanlong Du's Blog

It is never too late to learn.

Retrieval-Augmented Generation for LLM

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

What is RAG

What is retrieval-augmented generation? https://research.ibm.com/blog/retrieval-augmented-generation-RAG

What is retrieval-augmented generation? @ YouTube https://www.youtube.com/watch?v=T-D1OfcDW1M

Fine-Tune LLM

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

LoRA: Low-Rank Adaptation of Large Language Models

https://arxiv.org/abs/2106.09685

Tips on ONNX

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

ONNX Runtime

ONNX Runtime is a cross-platform inference and training machine-learning accelerator. ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as …

JIT and Domain Specific Compilers

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

numba

Hummingbird

Hummingbird compiles trained ML models into tensor computation for faster inference.

jax

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

XLA

deep learning …

Rust for Machine Learning

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

candle

Candle is a minimalist ML framework for Rust with a focus on easiness of use and on performance (including GPU support).

dfdx

dfdx is an ergonomics & safety focused deep learning …

Hardware for AI

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

TVM for deep learning is kind of like LLVM for programming languages.

Nvidia TensorRT

Embedded/Edge AI

Jetson Nano

Google Coral

Intel Neural Compute Stick 2

https://heartbeat.fritz.ai/edge-tpu-google-coral-usb-accelerator-cf0d79c7ec56 …