Sign in Publish
contextetech/

Qwen2.5 7B with vLLM on a 24 GB GPU

v2
Contexte Tech

254 publication(s) · 322 uses

Source: huggingface.co/Qwen/Qwen2.5-7B-Instruct ↗ Configuration English License: MIT updated 2 days ago
# Qwen2.5 7B with vLLM on a 24 GB GPU
code: l4t26x | type: inference | language: en | licence: MIT | version: v2
description: Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command.
page: https://contextetech.com/en/inference/qwen2-5-7b-vllm-24gb-gpu-l4t26x
mcp: connect https://contextetech.com/mcp then call load_resource with code "l4t26x" (or ask "load l4t26x")
json: https://contextetech.com/api/resources/contextetech--qwen2-5-7b-vllm-24gb-gpu
python: pip install contexte && contexte get l4t26x
source: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct
citation (required by the licence): contextetech. (2026). Qwen2.5 7B with vLLM on a 24 GB GPU (Version 2) [Inference config]. Contexte Tech. https://contextetech.com/en/inference/qwen2-5-7b-vllm-24gb-gpu-l4t26x. Data source: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct

Serve Qwen2.5 7B Instruct with vLLM on a 24 GB GPU: OpenAI-compatible API, 8k context, a single command.

ModelQwen/Qwen2.5-7B-Instruct
Tested hardwareNVIDIA GPU with 24 GB (for example RTX 4090 or L4)

Launch command

vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 8192 --gpu-memory-utilization 0.90 --host 127.0.0.1 --port 8000

Community

No comments yet. Share your feedback, it will help the next person.