Microsoft Research Unveils Agent Lightning for AI Training

Open-source framework plugs reinforcement learning into existing agent codebases through an API proxy layer.

Microsoft Research released code and technical documentation for Agent Lightning v1.0, an open-source framework designed to train AI agents with reinforcement learning without rewriting their tools or execution logic. Developed by the lab’s team in Shanghai, the project is published on GitHub under an MIT license.

An API proxy for Agent Lightning workflows

Training multi-step software agents typically requires developers to port their systems into custom simulation environments. When teams build on LangChain, AutoGen, or OpenAI’s Agents SDK, they usually rely on frozen weights. Connecting those agents to reinforcement learning previously meant restructuring the application around specialized training frameworks.

The team behind Agent Lightning, including researchers Xufang Luo and Luna K. Qiu, bypassed that bottleneck by inserting a lightweight API gateway between the agent and the inference backend. The proxy mimics the OpenAI API format. The agent runs in its standard environment without code changes, while the gateway captures generation steps, tool calls, and token probabilities to route them directly to GPU training clusters.

LightningRL algorithm and core architecture

Long execution chains often cause policy gradient methods to struggle with unstable updates. To handle multi-turn trajectories, the system introduces LightningRL. The algorithm splits long agent executions into individual model decisions and uses hierarchical credit assignment to evaluate each step, making it compatible with single-turn algorithms like PPO and GRPO.

For the v1.0 release, the researchers streamlined the core implementation to roughly 3,500 lines of Python. The framework packages integration with verl and vLLM runtimes, along with native support for Kubernetes clusters so teams can deploy training runs on standard infrastructure without proprietary sandboxes.

Benchmark numbers across coding and search tasks

In technical evaluations published by Microsoft Research, applying Agent Lightning RL fine-tuning to the open-weight Qwen3.5-9B model raised its success rate on SWE-bench Verified from 41.8% to 56.4% across 6,000 training instances.

The team also reported performance gains across other structured workflows. These included text-to-SQL generation using LangChain on the Spider benchmark, multi-hop question answering with OpenAI’s Agents SDK on MuSiQue, and tool-based math calculations using AutoGen.

Current limits and scope

While the GitHub repository has gathered more than 18,500 stars, Microsoft has not scheduled any integration with commercial services like Copilot Studio or Azure AI Agent Service. The reported benchmarks also reflect internal lab tests on mid-sized open models, leaving open questions about the compute expenses and stability of applying the system to proprietary frontier models.