A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

Create characters in Unity with LLMs!

llama.go is like llama.cpp in pure Golang!

A cosy home for your LLMs.

Build and run AI agents using Docker Compose. A collection of ready-to-use examples for orchestrating open-source LLMs, tools, and agent runtimes.

React Native binding of llama.cpp

Self-evaluating interview for AI coders

Run AI ✨ assistant locally! with simple API for Node.js 🚀

LLama.cpp rust bindings

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.

A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

Complete offline AI ecosystem for Android: Chat (GGUF/LLMs), Images (Stable Diffusion 1.5), Voice (TTS/STT), and Knowledge (RAG Data-Packs), zero subscriptions, no data harvesting. Open-source privacy-first AI on your terms.

The Swiss Army Knife of Offline AI. Chat, Speak, and Generate Images - Privacy First, Zero Internet. Download an LLM and use it on your mobile device. No data ever leaves your phone. Supports text-to-text, vision, text-to-image

Local ML voice chat using high-end models.

Run LLMs locally. A clojure wrapper for llama.cpp.