Projects

LLM Inference Engine

C++ / Python

GPT-2 inference in C++ with KV caching, SIMD kernels, and INT8/INT4 quantisation.