H Company Speeds VLM Serving with NVIDIA Dynamo

French startup combines peer-to-peer RDMA transfers and vLLM streaming to make elastic scaling viable for computer-use agents on AWS.

Launching a new vision-language model replica on Kubernetes clusters used to take nearly half an hour on H Company’s infrastructure. To make on-demand VLM serving practical for autonomous computer-use agents, the French startup overhauled its inference pipeline using vLLM and ModelExpress, an open-source component of NVIDIA Dynamo. The engineering team published their findings on October 8, 2026, in the post Booting Fast and Slow, which was subsequently featured by NVIDIA AI on X.

Bottlenecks in VLM Serving

Agents that interact directly with computer interfaces work in bursts, alternating quiet periods with sudden streams of screenshots and mouse actions. Leaving dozens of GPU instances running without active requests burns cash quickly. Turning them off, however, created a painful delay when demand spiked back up.

Under H Company’s initial AWS setup, starting a pod running the 397-billion-parameter FP8 variant of Qwen3.5 took up to 27 minutes. That latency came from container pulls, downloading hundreds of gigabytes of weights from Amazon S3 buckets, Python module imports, and runtime CUDA kernel compilation. In practice, the team had to over-provision GPU instances just to spare users from long waits.

Direct GPU-to-GPU Transfers

The largest performance jump came during warm starts, when a new replica launches on a node where existing instances already hold model weights in video memory. Instead of pulling from S3 again or routing weights through host system RAM, the startup deployed NVIDIA Dynamo ModelExpress, an Apache 2.0-licensed tool.

The system copies weights directly between GPUs using RDMA across the AWS Elastic Fabric Adapter (EFA) network and NVIDIA’s NIXL library. Reaching transfer speeds of 341 Gbps, weight loading for the 397B FP8 model dropped from over 12 minutes to 26 seconds. Accounting for the full startup sequence, replica activation ran 12 times faster.

Distributed Streaming and Open Source Fixes

For clean cold starts where a machine boots from scratch and must fetch weights from Amazon S3, the team used distributed streaming via the RunAI model streamer. The eight GPUs on each node pull separate chunks of the weights simultaneously, synchronizing data across NVLink using the NCCL library. That approach brought initial download times from roughly 11 minutes down to 3.4 minutes.

H Company also targeted JIT compilation overhead. By sharing cached FP8 routines (DeepGEMM) and pre-recorded CUDA graphs across replicas, initialization warmup ran four to five times faster.

The production work led to upstream improvements. Engineers Emrick Sinitambirivoutin, Victor Goubet, and Avshalom Manevich contributed OpenTelemetry startup tracing to the main vLLM repository and fixed a ModelExpress issue that triggered silent fallbacks to slower TCP connections.

What This Means for Autonomous Agents

H Company’s setup underscores that making autonomous operating-system agents viable is as much a networking problem as a modeling challenge. Without spin-up times measured in seconds, cloud elasticity falls apart, leaving teams with runaway bills for idle GPU clusters.

The startup has not shared its exact dollar savings on AWS, and memory process snapshotting remains under active development. Even so, combining peer-to-peer RDMA with distributed weight streaming provides a practical blueprint for running heavy visual models without keeping expensive clusters hot around the clock.