AWS and UC Berkeley release Infinity agent runtime

AWS and UC Berkeley have introduced Infinity, a research project under the Hydro initiative designed to handle extreme concurrency in AI agents. By treating agents like stackless coroutines, the runtime saves conversation history to a store and suspends the process during tool calls or wait periods. This approach allows for running up to 75,000 agents on a Raspberry Pi 5 and enables efficient deployment on serverless platforms.

How Infinity manages agent concurrency

• Uses a stackless coroutine model where agents are executed in bounded slices.
• Instead of holding memory during tool calls, the runtime persists the conversation history to a pluggable store and discards in-memory state.
• Tool results and user inputs arrive via a shared input queue, triggering a new execution slice.
• Designed to be extremely lightweight; can fit 75,000 concurrent agents in 8 GB of RAM on a Raspberry Pi 5.
• Enables serverless deployment because the function can terminate while the agent is suspended waiting for a tool.

Why it matters

Infinity provides a way to scale AI agentic workflows without the massive memory overhead typically required by blocking processes, making complex agent systems viable on edge hardware and serverless infrastructure.