Nvidia Nemotron 3.5 Bolt Fits 74B Model Into 24GB
Nvidia launched Nemotron 3.5 Bolt, a 74B MoE model compressed via 2-bit quantization to run agentic coding and browser workflows on 24GB hardware.
Open-weight model applies 2-bit quantization to execute local command-line and browser agent workflows without cloud dependencies.
Nvidia introduced Nemotron 3.5 Bolt 74B-A9B on Thursday (October 15, 2026), targeting local software agents that run directly on developer hardware. The release addresses a recurring barrier in local AI development: running capable automation workflows on a personal machine without routing commands through external cloud APIs. Announced through the NVIDIA Blog and detailed by NVIDIA Research, the model compresses 74 billion raw parameters to operate within the 24GB memory ceiling common to flagship GPUs and mobile workstations.
How Nemotron 3.5 Bolt Compresses 74B Into 24GB
Bolt relies on a hybrid Mamba-Transformer architecture using a Mixture-of-Experts (MoE) routing layout. Instead of engaging the entire neural network on every generated token, the model directs computation to specialized expert subnetworks. During inference, Bolt activates 9 billion parameters per token from its total pool of 74 billion. The foundation comes from Nvidia’s 120-billion-parameter Nemotron 3.5 Super VL, slimmed down through an internal compression framework called Star Elastic.
Fitting those parameters into 24GB of video memory required native 2-bit quantization during fine-tuning, an approach known as Quantization-Aware Training (QAT). By accounting for lower numerical precision while updating weights rather than truncating them after the fact, the engineering team packed the entire weight payload into 22GB to 24GB. Nvidia also announced the Nemotron 3.5 Bolt E4B, a 4GB variant scheduled for October 23, 2026 to serve robotics and lightweight conversational setups.
Command-Line Speed and Agent Benchmarks
Rather than functioning as a general-purpose chat assistant, Nemotron 3.5 Bolt is tuned specifically for agentic execution: issuing terminal commands, managing directory structures, and manipulating browser windows. On Terminal-Bench 2.1, an evaluation focused on real command-line problem solving, Bolt registered 51.0 points. That score placed it ahead of larger open models running at standard 16-bit precision (BF16), including Qwen3.6-35B-A3B (44.4) and Gemma4 26B-A4B (37.2). Technical reporting from The New Stack also noted first place in its size class on PinchBench and second on BrowseComp.
In generation throughput, official documentation reports 80 to 84 tokens per second on RTX Spark laptops. On dedicated desktop cards such as the GeForce RTX 5090, inference climbed past 200 tokens per second.
Open Ecosystem and RTX Spark Hardware
Nvidia uploaded the weights to Hugging Face under the permissive OpenMDW 1.1 license, which authorizes commercial deployment and user modifications. The open-source runtime ecosystem added immediate support for GGUF formats across llama.cpp, Ollama, vLLM, and LM Studio, alongside compatibility with agent frameworks like Hermes and OpenClaw.
The software drop coincided with hardware disclosures for the NVIDIA RTX Spark line. The platform combines a 6,144-core Blackwell graphics processor with a 20-core Grace CPU through an NVLink-C2C interconnect, supporting up to 128GB of unified memory. OEM partners including ASUS, Dell, HP, Lenovo, and Microsoft Surface will begin global laptop distribution on October 16, 2026. Official pricing and availability dates for Brazil have not been disclosed.
Long-Context Limits and Real-World Memory
Operating within an exact 24GB memory envelope imposes strict physical boundaries on autonomous tasks. The claim circulating in community discussions that Bolt can process a full 256,000-token context window entirely within 24GB of VRAM without spilling into system RAM is an unconfirmed RUMOR.
The limitation stems from memory allocation math: with model weights taking up 22GB to 24GB on their own, virtually no space remains for the Key-Value (KV) cache. In long agent sessions, this working cache stores command history and tool output, consuming substantial memory within minutes. Furthermore, whether aggressive 2-bit quantization avoids reasoning degradation outside curated benchmarks remains an open question that requires independent evaluation in production workloads.