Qwen/Qwen3.5-0.8B
Qwen3.5 tiny dense multimodal model (0.8B) — ultra-low-VRAM / edge serving with 262K context
Tiny Qwen3.5 dense for edge / draft-model use
Default configuration: 8× H200 · Tensor Parallel · BF16
export VLLM_USE_RUST_FRONTEND=1 vllm serve Qwen/Qwen3.5-0.8B \ --trust-remote-code \ --tensor-parallel-size 1 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3
The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3.5-0.8B.md.
Overview
Qwen3.5-0.8B is the smallest member of the Qwen3.5 family — same hybrid gated delta networks architecture and 262K context, at a size suited to edge devices or as a draft model for speculative decoding with larger Qwen3.5 checkpoints.
Prerequisites
- vLLM version: >= 0.17.0
- Hardware: any modern GPU (>=4 GB VRAM) or Intel Arc Pro B60/B70
Client Usage
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
# The server's model name is the checkpoint it runs: the selected variant's.
model = client.models.list().data[0].id
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Hi!"}],
max_tokens=64,
)
print(resp.choices[0].message.content)