Sign in Publish
contextetech/

llama.cpp server for a GGUF model

v2
Contexte Tech

254 publication(s) · 322 uses

Source: github.com/ggml-org/llama.cpp ↗ Configuration English License: MIT updated 2 days ago
# llama.cpp server for a GGUF model
code: 8hyjqo | type: inference | language: en | licence: MIT | version: v2
description: Run any GGUF model with the llama.cpp server: OpenAI-compatible API on port 8080, works without a GPU too.
page: https://contextetech.com/en/inference/llama-cpp-server-gguf-8hyjqo
mcp: connect https://contextetech.com/mcp then call load_resource with code "8hyjqo" (or ask "load 8hyjqo")
json: https://contextetech.com/api/resources/contextetech--llama-cpp-server-gguf
python: pip install contexte && contexte get 8hyjqo
source: https://github.com/ggml-org/llama.cpp
citation (required by the licence): contextetech. (2026). llama.cpp server for a GGUF model (Version 2) [Inference config]. Contexte Tech. https://contextetech.com/en/inference/llama-cpp-server-gguf-8hyjqo. Data source: https://github.com/ggml-org/llama.cpp

Run any GGUF model with the llama.cpp server: OpenAI-compatible API on port 8080, works without a GPU too.

Modelmodel.gguf
Tested hardwareCPU only, or a GPU (use -ngl to offload layers); 8 GB of memory for a 7B model in Q4_K_M

Launch command

llama-server -m model.gguf -c 8192 --host 127.0.0.1 --port 8080

Community

No comments yet. Share your feedback, it will help the next person.