A C++23 module-based DNN library for GPU-first LLM inference — explicit forward passes, no hidden execution engine, work at the metal. Validated token-for-token on Gemma 4 Unified, Llama 3.x and GPT-2 with compile-time FP8/FP4 weight quantization.
training api deep-learning neural-network gpu cuda transformers inference mnist tensors inference-server gpt-2 cpp23 cpp20-modules llm local-ai llama3-2 fp8-quantization fp4-quantization gemma4-12b
-
Updated
Jun 24, 2026 - C++