Qwen2.5 7B with vLLM on a 24 GB GPU
# Qwen2.5 7B with vLLM on a 24 GB GPU code: l4t26x | type: inference | language: en | licence: MIT | version: v2 description: Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command. page: https://contextetech.com/en/inference/qwen2-5-7b-vllm-24gb-gpu-l4t26x mcp: connect https://contextetech.com/mcp then call load_resource with code "l4t26x" (or ask "load l4t26x") json: https://contextetech.com/api/resources/contextetech--qwen2-5-7b-vllm-24gb-gpu python: pip install contexte && contexte get l4t26x source: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct citation (required by the licence): contextetech. (2026). Qwen2.5 7B with vLLM on a 24 GB GPU (Version 2) [Inference config]. Contexte Tech. https://contextetech.com/en/inference/qwen2-5-7b-vllm-24gb-gpu-l4t26x. Data source: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct
Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command.
| Model | Qwen/Qwen2.5-7B-Instruct |
|---|---|
| Tested hardware | NVIDIA GPU with 24 GB (for example RTX 4090 or L4) |
Launch command
vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --host 127.0.0.1 --port 8000
- Identifier
- contextetech--qwen2-5-7b-vllm-24gb-gpu
- File
- qwen2-5-7b-vllm-24gb-gpu.inference.json
- Format
- JSON (application/json)
- Size
- 524 bytes
- Encoding
- UTF-8
- Estimated tokens
- ≈ 146
- Commercial use
- Allowed
- Personal data
- None
- Version
- 1.0
- Published on
- October 2, 2026 at 9:54 PM
- Updated on
- October 16, 2026 at 11:19 PM
- Storage
- in database
Statistics
Likes : 0 · Comments : 0
Click a counter to show or hide it, hover the chart for details. One view per visitor per day, bots excluded; counting started on 30 September 2026.
Estimated cost
- Input tokens per call
- ≈ 176
| Model | Per call | Per month |
|---|---|---|
| GPT-6 Luna · OpenAIbest price | €0.00002 | €0.02 |
| DeepSeek Flash · DeepSeek | €0.00005 | €0.05 |
| Mistral Large · Mistral AI | €0.00008 | €0.08 |
| DeepSeek V4 Pro · DeepSeek | €0.00021 | €0.21 |
| GPT-6 Sol · OpenAI | €0.00031 | €0.31 |
| Claude Sonnet 5.5 · Anthropic | €0.00031 | €0.31 |
| Claude Opus 5.5 · Anthropic | €0.00062 | €0.62 |
| Self-hosted open-source model (Ollama, vLLM) | €0 in API fees (server cost only) | |
Input cost only, excluding the model's answer. Providers' public standard prices (Mistral AI, DeepSeek, OpenAI, Anthropic) converted from USD to euros at the ECB rate of October 13, 2026. Estimate: 1 token ≈ 3.6 characters.
Common to all 3 regions
Sources under a compatible licence.
Sources: Qwen/Qwen2.5-7B-Instruct model, Apache 2.0 license (Hugging Face model card)
Commercial use allowed · credit the author · changes allowed.
Text or data only: no access to files, network or commands.
Hosted in France (Scaleway, Paris). No transfer outside the EU.
European Union · 5/5
No personal data.
No training data: no summary required.
United States · 5/5
No personal data.
No training data: no documentation to publish.
China · 5/5
No personal data.
No training data; AI-generated content published in China must be labelled (2025).
Indicative summary as of 30/09/2026, not a legal certification. Method and sources →
Content quality
Performance not measured: no measured speed and no passed test
The detailed criteria are for members.
Every criterion, the points removed and a tip to fix each one.
Content analysis
The detailed criteria are for members.
Every criterion, the points removed and a tip to fix each one.
Computed from the content of the resource, by rules, without AI. The score is in the Quality tab.
MCP contexte
https://contextetech.com/mcp
load l4t26x
Add the URL as a connector in your AI apps' settings, then send the request above.
API
import json, urllib.request url = "https://contextetech.com/api/resources/contextetech--qwen2-5-7b-vllm-24gb-gpu" fiche = json.load(urllib.request.urlopen(url)) print(fiche["title"] or fiche["name"])
qwen2-5-7b-vllm-24gb-gpu.inference.json
{
"format": "contextetech/inference/v1",
"id": "contextetech/qwen2-5-7b-vllm-24gb-gpu",
"description": "Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command.",
"license": "MIT",
"language": "en",
"runtime": "vllm",
"model": "Qwen/Qwen2.5-7B-Instruct",
"config": "",
"command": "vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --host 127.0.0.1 --port 8000",
"quant": "bf16",
"contextLength": 8192,
"ramGb": 24
}Launch command
vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --host 127.0.0.1 --port 8000
Community
No comments yet. Share your feedback, it will help the next person.