{
  "hf_id": "Qwen/Qwen3.5-27B",
  "meta": {
    "title": "Qwen3.5-27B",
    "slug": "qwen3.5-27b",
    "provider": "Qwen",
    "description": "Qwen3.5 dense multimodal model (27B) with gated delta networks hybrid attention, MTP, and 262K context",
    "date_added": "2026-04-22",
    "date_updated": "2026-10-27",
    "tasks": [
      "multimodal",
      "text"
    ],
    "performance_headline": "Qwen3.5 flagship dense — single-GPU FP8 or 2x GPU or 2-4x Intel Arc Pro B60/B70 BF16",
    "related_recipes": [
      "Qwen/Qwen3.5-397B-A17B",
      "Qwen/Qwen3.5-35B-A3B",
      "Qwen/Qwen3.5-9B"
    ],
    "hardware": {
      "arc_pro_b60": "verified",
      "arc_pro_b70": "verified",
      "apple_32gb": "supported",
      "apple_64gb": "supported"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "hub": "huggingface",
    "checkpoint_url": "https://huggingface.co/Qwen/Qwen3.5-27B",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {
      "VLLM_USE_RUST_FRONTEND": "1"
    },
    "docker_image": "vllm/vllm-openai:latest",
    "command": "vllm serve Qwen/Qwen3.5-27B \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --reasoning-parser qwen3 \\\n  --mm-encoder-tp-mode data",
    "argv": [
      "vllm",
      "serve",
      "Qwen/Qwen3.5-27B",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "qwen3_coder",
      "--reasoning-parser",
      "qwen3",
      "--mm-encoder-tp-mode",
      "data"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  -e VLLM_USE_RUST_FRONTEND=1 \\\n  vllm/vllm-openai:latest Qwen/Qwen3.5-27B \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser qwen3_coder \\\n  --reasoning-parser qwen3 \\\n  --mm-encoder-tp-mode data",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "-e",
      "VLLM_USE_RUST_FRONTEND=1",
      "vllm/vllm-openai:latest",
      "Qwen/Qwen3.5-27B",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "qwen3_coder",
      "--reasoning-parser",
      "qwen3",
      "--mm-encoder-tp-mode",
      "data"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "multi_node_tp": "/Qwen/Qwen3.5-27B/strategies/multi_node_tp.json",
      "kv_store_centralized_mooncake": "/Qwen/Qwen3.5-27B/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/Qwen/Qwen3.5-27B/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/Qwen/Qwen3.5-27B/hw/h200.json",
      "h100": "/Qwen/Qwen3.5-27B/hw/h100.json",
      "b200": "/Qwen/Qwen3.5-27B/hw/b200.json",
      "gb200": "/Qwen/Qwen3.5-27B/hw/gb200.json",
      "b300": "/Qwen/Qwen3.5-27B/hw/b300.json",
      "gb300": "/Qwen/Qwen3.5-27B/hw/gb300.json",
      "mi300x": "/Qwen/Qwen3.5-27B/hw/mi300x.json",
      "mi325x": "/Qwen/Qwen3.5-27B/hw/mi325x.json",
      "mi350x": "/Qwen/Qwen3.5-27B/hw/mi350x.json",
      "mi355x": "/Qwen/Qwen3.5-27B/hw/mi355x.json",
      "arc_pro_b60": "/Qwen/Qwen3.5-27B/hw/arc_pro_b60.json",
      "arc_pro_b70": "/Qwen/Qwen3.5-27B/hw/arc_pro_b70.json"
    }
  },
  "model": {
    "model_id": "Qwen/Qwen3.5-27B",
    "min_vllm_version": "0.17.0",
    "default_frontend": "rust",
    "architecture": "dense",
    "parameter_count": "27B",
    "active_parameters": "27B",
    "context_length": 262144,
    "base_args": [
      "--trust-remote-code"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "Enable automatic tool choice with Qwen3 Coder parser",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "qwen3_coder"
      ],
      "hardware_overrides": {
        "metal": {
          "args": [
            "--enable-auto-tool-choice",
            "--tool-call-parser",
            "qwen3_coder"
          ]
        }
      }
    },
    "reasoning": {
      "description": "Enable chain-of-thought reasoning with Qwen3 parser",
      "args": [
        "--reasoning-parser",
        "qwen3"
      ],
      "hardware_overrides": {
        "metal": {
          "args": [
            "--reasoning-parser",
            "qwen3"
          ]
        }
      }
    },
    "spec_decoding": {
      "description": "Multi-token prediction speculative decoding for lower latency",
      "args": [
        "--speculative-config",
        "{\"method\":\"mtp\",\"num_speculative_tokens\":1}"
      ]
    },
    "text_only": {
      "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.",
      "args": [
        "--language-model-only"
      ]
    },
    "encoder_parallel": {
      "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.",
      "args": [
        "--mm-encoder-tp-mode",
        "data"
      ]
    },
    "long_context": {
      "description": "YaRN RoPE scaling past the native 262,144-token window to ~1,010,000 tokens. Static YaRN holds the scaling factor constant regardless of input length, so leave this off unless you actually serve long prompts; for ~524K, halve the factor to 2.0.",
      "args": [
        "--hf-overrides",
        "{\"text_config\": {\"rope_parameters\": {\"mrope_interleaved\": true, \"mrope_section\": [11, 11, 10], \"rope_type\": \"yarn\", \"rope_theta\": 10000000, \"partial_rotary_factor\": 0.25, \"factor\": 4.0, \"original_max_position_embeddings\": 262144}}}",
        "--max-model-len",
        "1010000"
      ],
      "env": {
        "VLLM_ALLOW_LONG_MAX_MODEL_LEN": "1"
      }
    }
  },
  "opt_in_features": [
    "spec_decoding",
    "text_only",
    "long_context"
  ],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 65,
      "description": "Full precision BF16 — fits on 1x H200, 2x H100, 4x Intel Arc Pro B60, 2x Intel Arc Pro B70"
    },
    "fp8": {
      "model_id": "Qwen/Qwen3.5-27B-FP8",
      "precision": "fp8",
      "vram_minimum_gb": 33,
      "description": "Qwen official FP8 checkpoint — single 40 GB GPU",
      "json": "/Qwen/Qwen3.5-27B-FP8.json"
    },
    "gptq_int4": {
      "model_id": "Qwen/Qwen3.5-27B-GPTQ-Int4",
      "precision": "int4",
      "vram_minimum_gb": 17,
      "description": "GPTQ Int4 checkpoint — fits on a single 24 GB GPU",
      "json": "/Qwen/Qwen3.5-27B-GPTQ-Int4.json"
    },
    "mlx_int4": {
      "label": "MLX 4-bit",
      "model_id": "mlx-community/Qwen3.5-27B-4bit",
      "description": "Qwen3.5-27B, a larger model, with room for a conversation of about 24,000\nwords.",
      "precision": "int4",
      "vram_minimum_gb": 20,
      "supported_hardware": [
        "apple_32gb",
        "apple_64gb"
      ],
      "min_vllm_version": "0.31.0",
      "install": {
        "pip": {
          "note": "Requires Apple Silicon and macOS 15 or later. Install vLLM Metal.",
          "command": "curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash\nsource ~/.venv-vllm-metal/bin/activate\n"
        },
        "docker": false
      },
      "supported_frontends": [
        "python"
      ],
      "supported_features": [
        "tool_calling",
        "reasoning"
      ],
      "base_args": [
        "--host",
        "127.0.0.1",
        "--max-model-len",
        "32768",
        "--max-num-batched-tokens",
        "1024"
      ],
      "json": "/mlx-community/Qwen3.5-27B-4bit.json"
    }
  },
  "hardware_overrides": {
    "xpu": {
      "extra_args": [
        "--enforce-eager"
      ]
    }
  },
  "notes": [
    {
      "title": "Intel Arc Pro validation",
      "when": {
        "hardware": [
          "arc_pro_b60",
          "arc_pro_b70"
        ]
      },
      "body": "Validated on 4× Intel Arc Pro B60 or 2× Intel Arc Pro B70 (B60 24 GB, B70 32 GB per card) with the official\nvLLM XPU image `vllm/vllm-openai-xpu:latest` with `--max-model-len 8192`.\n"
    },
    {
      "title": "Memory settings",
      "when": {
        "hardware": [
          "apple_32gb",
          "apple_64gb"
        ],
        "variants": [
          "mlx_int4"
        ]
      },
      "body": "- The first run downloads 16 GB.\n- It is slow to read long text. For faster replies, use Qwen3.5-9B.\n- `--max-num-batched-tokens 1024` reads long text in smaller steps, so the\n  model needs less memory. Without it this model does not fit on 32 GB.\n"
    }
  ],
  "troubleshooting": [
    {
      "title": "assert num_cache_lines >= batch",
      "when": {
        "hardware": [
          "nvidia",
          "amd"
        ]
      },
      "body": "The CUDA graph capture size is larger than the Mamba cache. Reduce\n`--max-cudagraph-capture-size` (default 512); see\n[vllm#34571](https://github.com/vllm-project/vllm/pull/34571).\n"
    },
    {
      "title": "Prefix caching is experimental",
      "body": "Prefix caching for the Mamba cache runs in the experimental \"align\" mode.\n"
    }
  ],
  "guide": "## Overview\n\n[Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) is the flagship dense\nmodel of the Qwen3.5 family. It uses the same gated delta networks hybrid\nattention as its MoE siblings, supports vision+text input, and natively serves\n262K context. MTP (multi-token prediction) is supported out of the box for\nlow-latency decoding.\n\n## Prerequisites\n\n- **vLLM version:** >= 0.17.0\n- **Hardware (BF16):** 1x H200, 2x H100, 4x Intel Arc Pro B60, 2x Intel Arc Pro B70\n- **Hardware (FP8):** single 40 GB GPU (H100/H200/L40S)\n- **Hardware (Int4):** single 24 GB GPU\n\n## Client Usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\n# The server's model name is the checkpoint it runs: the selected variant's.\nmodel = client.models.list().data[0].id\nresp = client.chat.completions.create(\n    model=model,\n    messages=[{\"role\": \"user\", \"content\": \"Write a haiku about gated delta networks.\"}],\n    max_tokens=256,\n)\nprint(resp.choices[0].message.content)\n```\n\nQwen3.5 thinks before it answers. To turn thinking off for one request, pass\n`extra_body={\"chat_template_kwargs\": {\"enable_thinking\": False}}`; for every\nrequest, start the server with\n`--default-chat-template-kwargs '{\"enable_thinking\": false}'`.\n\n## References\n\n- [Model card](https://huggingface.co/Qwen/Qwen3.5-27B)\n- [FP8 checkpoint](https://huggingface.co/Qwen/Qwen3.5-27B-FP8)\n- [GPTQ-Int4 checkpoint](https://huggingface.co/Qwen/Qwen3.5-27B-GPTQ-Int4)\n- [Qwen3.5-397B-A17B recipe](../Qwen3.5-397B-A17B)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "1bf414e97acf7563297726531f702cdbea6b2921",
    "path": "recipes/models/Qwen/Qwen3.5-27B.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/Qwen/Qwen3.5-27B.yaml",
    "overlays": [
      {
        "source": "vllm-metal",
        "repo": "vllm-project/vllm-metal",
        "ref": "main",
        "sha": "3451a0149531e4d2889488ed1a80b5a74f6bb2c5",
        "path": "recipes/overlays/Qwen/Qwen3.5-27B.yaml",
        "edit_url": "https://github.com/vllm-project/vllm-metal/edit/main/recipes/overlays/Qwen/Qwen3.5-27B.yaml"
      }
    ]
  }
}