Qwen2.5 7B avec vLLM sur un GPU de 24 Go

Servir Qwen2.5 7B Instruct avec vLLM sur une carte graphique de 24 Go : API compatible OpenAI, contexte 8k, une seule commande.

Launch command

vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --port 8000

Estimated cost

Input tokens per call : ≈ 167

Input cost only, excluding the model's answer. Providers' public standard prices (Mistral AI, DeepSeek, OpenAI, Anthropic) converted from USD to euros at the ECB rate of September 29, 2026. Estimate: 1 token ≈ 3.6 characters.

Identifier

contextetech--qwen2-5-7b-vllm-gpu-24go

Type

Inference configs

File

qwen2-5-7b-vllm-gpu-24go.inference.json

Format

JSON (application/json)

Size

581 bytes

Encoding

UTF-8

Estimated tokens

≈ 161

Language

French

License

MIT

Commercial use

Allowed

Personal data

None

Version

1.0

Published on

October 1, 2026 at 6:55 PM

Updated on

October 1, 2026 at 6:55 PM

Storage

in database

Uses

0

Likes

0

qwen2-5-7b-vllm-gpu-24go.inference.json

{
  "format": "contextetech/inference/v1",
  "id": "contextetech/qwen2-5-7b-vllm-gpu-24go",
  "description": "Servir Qwen2.5 7B Instruct avec vLLM sur une carte graphique de 24 Go : API compatible OpenAI, contexte 8k, une seule commande.",
  "tags": [
    "vllm",
    "qwen",
    "gpu"
  ],
  "license": "MIT",
  "language": "fr",
  "runtime": "vllm",
  "model": "Qwen/Qwen2.5-7B-Instruct",
  "config": "",
  "command": "vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --port 8000",
  "quant": "bf16",
  "contextLength": 8192,
  "ramGb": 24
}

Launch command

vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --port 8000