Github开源的本地 LLM 推理项目

AI探索2个月前发布 8KMM
654 0 0
1transformersTransformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.123,99624,6311,058434145Apache License 2.00 days, 8 hrs, 18 mins
2ChatGPT-Next-WebA cross-platform ChatGPT/Gemini UI (Web / PWA / Linux / Win / MacOS). 一键拥有你自己的跨平台 ChatGPT/Gemini 应用。66,99254,41221817558MIT License0 days, 9 hrs, 12 mins
3gpt4allgpt4all: run open-source LLMs anywhere63,4256,9573909612MIT License5 days, 16 hrs, 0 mins
4gpt4freeThe official gpt4free repository, various collection of powerful language models56,51112,78189188109GNU General Public License v3.00 days, 12 hrs, 31 mins
5llama.cppLLM inference in C/C++54,8417,7536104781,651MIT License0 days, 8 hrs, 6 mins
6gpt_academic为GPT/GLM等LLM大语言模型提供实用化交互接口,特别优化论文阅读/润色/写作体验,模块化设计,支持自定义快捷按钮&函数插件,支持Python和C++等项目剖析&自译解功能,PDF/LaTex论文翻译&总结功能,支持并行问询多种LLM模型,支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss等。54,1486,8392357728GNU General Public License v3.00 days, 8 hrs, 40 mins
7ollamaGet up and running with Llama 2, Mistral, Gemma, and other large language models.53,8753,75176717353MIT License0 days, 13 hrs, 24 mins
8privateGPTInteract with your documents using the power of GPT, 100% privately, no data leaks51,2636,813194707Apache License 2.01 days, 5 hrs, 29 mins
9text-generation-webuiA Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.35,4514,72122729938GNU Affero General Public License v3.00 days, 23 hrs, 42 mins
10lobe-chatLobe Chat – an open-source, modern-design LLMs/AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Perplexity / Bedrock / Azure / Mistral / Ollama ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT chat application.26,8796,06630089510MIT License0 days, 8 hrs, 46 mins
11chatbot-uiAI chat for every model.25,8927,08389410MIT License1 days, 9 hrs, 50 mins
12localGPTChat with your documents on your local device using GPT models. No data leaves your device and 100% private.19,0152,102452420Apache License 2.010 days, 9 hrs, 5 mins
13LocalAI🤖 The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.18,8031,3812537943MIT License0 days, 8 hrs, 11 mins
14chatboxChatbox is a desktop client for ChatGPT, Claude and other LLMs, available on Windows, Mac, Linux18,1731,8702372855GNU General Public License v3.08 days, 14 hrs, 22 mins
15vllmA high-throughput and memory-efficient inference and serving engine for LLMs17,4452,25278726723Apache License 2.00 days, 8 hrs, 22 mins
16mlc-llmEnable everyone to develop, optimize and deploy AI models natively on everyone’s devices.16,5621,2631961001Apache License 2.00 days, 12 hrs, 16 mins
17janJan is an open source alternative to ChatGPT that runs 100% offline on your computer16,4379011684319GNU Affero General Public License v3.00 days, 8 hrs, 10 mins
18ChuanhuChatGPTGUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.14,6222,2241034621GNU General Public License v3.00 days, 14 hrs, 19 mins
19llamafileDistribute and run LLMs with a single file.12,800611552911Other1 days, 1 hrs, 48 mins
20open-webuiUser-friendly WebUI for LLMs (Formerly Ollama WebUI)12,4701,2361098916MIT License0 days, 10 hrs, 11 mins
21anything-llmA multi-user ChatGPT for any LLMs and vector database. Unlimited documents, messages, and storage in one privacy-focused app. Now available as a desktop application with a built-in LLM!10,6681,13087340MIT License0 days, 17 hrs, 52 mins
22h2ogptPrivate chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo:,2691,14123867129Apache License 2.00 days, 9 hrs, 33 mins
23LibreChatEnhanced ChatGPT Clone: Features OpenAI, Assistants API, Azure, Groq, GPT-4 Vision, Mistral, Bing, Anthropic, OpenRouter, Google Gemini, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. More features in development9,5141,7267610437MIT License0 days, 8 hrs, 38 mins
24chathubAll-in-one chatbot client9,451948277120GNU General Public License v3.011 days, 9 hrs, 8 mins
25FlexGenRunning large language models on a single GPU for throughput-oriented scenarios.8,97151654180Apache License 2.012 days, 15 hrs, 30 mins
26web-llmBringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.8,95353780281Apache License 2.00 days, 17 hrs, 37 mins
27OpenLLMRun any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint, locally and in the cloud.8,6255399125110Apache License 2.02 days, 1 hrs, 51 mins
28text-generation-inferenceLarge Language Model Text Generation Inference7,6448321397438Apache License 2.00 days, 9 hrs, 29 mins
29serverThe Triton Inference Server provides an optimized cloud and edge inferencing solution.7,2441,36442411165BSD 3-Clause “New” or “Revised” License0 days, 16 hrs, 13 mins
30TensorRT-LLMTensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.6,299627550104Apache License 2.01 days, 3 hrs, 46 mins
31llama-cpp-pythonPython bindings for llama.cpp6,210730327123120MIT License0 days, 10 hrs, 47 mins
32chat-uiOpen source codebase powering the HuggingChat app5,767772180607Apache License 2.01 days, 0 hrs, 8 mins
33SillyTavernLLM Frontend for Power Users.5,6331,7582989975GNU Affero General Public License v3.00 days, 8 hrs, 32 mins
34big-agiGenerative AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. It features AI personas, AGI functions, multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.3,9559001263415MIT License0 days, 10 hrs, 39 mins
35lollms-webuiLord of Large Language Models Web User Interface3,6934701313619Apache License 2.00 days, 10 hrs, 55 mins
36koboldcppA simple one-file way to run various GGML and GGUF models with KoboldAI’s UI3,62126416747573GNU Affero General Public License v3.01 days, 7 hrs, 14 mins
37llmAccess large language models from the command-line2,7921301521924Apache License 2.00 days, 14 hrs, 3 mins
38exllamav2A fast inference library for running LLMs locally on modern consumer-class GPUs2,785207903117MIT License3 days, 0 hrs, 7 mins
39inferenceReplace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you’re empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.2,3171832184154Apache License 2.00 days, 8 hrs, 45 mins
40lmdeployLMDeploy is a toolkit for compressing, deploying, and serving LLMs.2,163193974425Apache License 2.00 days, 8 hrs, 38 mins
41LLamaSharpA cross-platform library to run 🦙LLaMA/LLaVA model (and others) on your local device efficiently.1,803242893915MIT License2 days, 3 hrs, 5 mins
42nitroAn inference server on top of llama.cpp. OpenAI-compatible API, queue, & scaling. Embed a prod-ready, local inference engine in your apps. Powers Jan1,52373302269GNU Affero General Public License v3.00 days, 8 hrs, 59 mins
43chatbot-ollamaChatbot Ollama is an open source chat UI for Ollama.9941601861Other47 days, 18 hrs, 30 mins
44LLMFarmllama and other large language models on iOS and MacOS offline using GGML library.8084411124MIT License1 days, 1 hrs, 8 mins
45maidMaid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.5746471225MIT License0 days, 8 hrs, 4 mins
46oterma text-based terminal client for Ollama511275614MIT License8 days, 9 hrs, 25 mins
47amicaAmica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.4737041164MIT License1 days, 1 hrs, 41 mins
48FreeChatllama.cpp based AI chat app for macOS347251840MIT License15 days, 23 hrs, 4 mins
49exuiWeb UI for ExLlamaV2325272070MIT License4 days, 11 hrs, 2 mins
50avaAll-in-one desktop app for running LLMs locally.29412820Other6 days, 20 hrs, 28 mins
51tenereTUI interface for LLMs written in Rust21471512GNU General Public License v3.021 days, 22 hrs, 56 mins
52emeltalLocal ML voice chat using high-end models.1056010


© 版权声明