Qwen2.5 7B with vLLM on a 24 GB GPU

Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command.

Launch command

vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --port 8000

Estimated cost

Input tokens per call : ≈ 171

Input cost only, excluding the model's answer. Providers' public standard prices (Mistral AI, DeepSeek, OpenAI, Anthropic) converted from USD to euros at the ECB rate of September 29, 2026. Estimate: 1 token ≈ 3.6 characters.

Identifier

contextetech--qwen2-5-7b-vllm-24gb-gpu

Type

Inference configs

File

qwen2-5-7b-vllm-24gb-gpu.inference.json

Format

JSON (application/json)

Size

558 bytes

Encoding

UTF-8

Estimated tokens

≈ 155

Language

English

License

MIT

Commercial use

Allowed

Personal data

None

Version

1.0

Published on

October 2, 2026 at 9:54 PM

Updated on

October 2, 2026 at 9:54 PM

Storage

in database

Uses

0

Likes

0

qwen2-5-7b-vllm-24gb-gpu.inference.json

{
  "format": "contextetech/inference/v1",
  "id": "contextetech/qwen2-5-7b-vllm-24gb-gpu",
  "description": "Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command.",
  "tags": [
    "vllm",
    "qwen",
    "gpu"
  ],
  "license": "MIT",
  "language": "en",
  "runtime": "vllm",
  "model": "Qwen/Qwen2.5-7B-Instruct",
  "config": "",
  "command": "vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --port 8000",
  "quant": "bf16",
  "contextLength": 8192,
  "ramGb": 24
}

Launch command

vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --port 8000