vLLM/Recipes
Search
Qwen

Qwen/Qwen3.5-2B

Qwen3.5 mini dense multimodal model (2B) — edge / low-VRAM serving with 262K context

Edge-scale Qwen3.5 dense — fits on 8 GB GPUs or 1x Intel Arc Pro B60/B70

dense2B262,144 ctxvLLM 0.17.0+Multimodal
Default configuration: 8× H200 · Tensor Parallel · BF16
export VLLM_USE_RUST_FRONTEND=1
vllm serve Qwen/Qwen3.5-2B \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3.5-2B.md.

Guide

Overview

Qwen3.5-2B is a miniature dense Qwen3.5 model — the full gated delta networks architecture, vision encoder, and 262K context, in a form small enough for 8 GB consumer GPUs or edge inference.

Prerequisites

  • vLLM version: >= 0.17.0
  • Hardware: single 8 GB GPU or Intel Arc Pro B60/B70

Client Usage

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
# The server's model name is the checkpoint it runs: the selected variant's.
model = client.models.list().data[0].id
resp = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Hi!"}],
    max_tokens=64,
)
print(resp.choices[0].message.content)

References