Qwen3.5-35B-A3B-Base: 35 Billion Parameters, 3 Billion Active

Qwen has published Qwen3.5-35B-A3B-Base to Hugging Face. The model contains 35 billion total parameters but uses a mixture-of-experts (MoE) architecture that routes each token through roughly 3 billion active parameters per forward pass [1]. This is a base (pre-trained, non-instruct) checkpoint.

The practical consequence of the 3B active footprint: inference compute costs land closer to a dense 3B model than a dense 35B model, while the total parameter budget provides a much larger knowledge capacity. For teams evaluating local deployment on workstation-class GPUs, the VRAM and throughput profile should be substantially more accessible than a comparably-sized dense model.

Architecture Notes

MoE models split their feed-forward layers into multiple expert sub-networks and use a gating mechanism to select a small subset per token. The “35B total / 3B active” ratio implies a sparsity factor of roughly 10:1 — each token sees less than 10% of the model’s weights.

For comparison, other recent MoE releases have used different sparsity ratios:

Qwen’s ratio sits between Mixtral’s relatively dense routing and DeepSeek’s extreme sparsity, but the absolute active count — 3B — is notably low for a model in this total-parameter class.

Deployment Implications

The 3B active parameter count is the headline for infrastructure planning. Inference latency and memory bandwidth requirements scale primarily with active parameters, not total parameters. VRAM requirements still scale with total parameters (all weights must be loaded), but quantization to 4-bit would bring the ~35B weights to roughly 17-18 GB — within range of a single consumer GPU with 24 GB VRAM.

This makes Qwen3.5-35B-A3B a candidate for:

• Local workstation deployment without cloud API costs
• Edge inference where bandwidth or latency constraints rule out API calls
• Fine-tuning experiments where the base checkpoint serves as a starting point

The base model designation means no instruction tuning or RLHF alignment is included. Teams would need to apply their own fine-tuning or wait for a community or official instruct variant.

References

1. Qwen3.5-35B-A3B-Base — Hugging Face Model Card

---