Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

ONNX Runtime

ONNX Runtime is a cross-platform inference and training machine-learning accelerator. ONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms.

tract

tract is a Neural Network inference toolkit. It can read ONNX or NNEF, optimize them and run them.