LLVM's lit test runner gets major performance overhaul in GSoC 2026

LLVM's integrated tester, lit, received dedicated performance work for the first time through a GSoC 2026 project. The contributor replaced multiprocessing.Pool with asyncio and ProcessPoolExecutor, moved frequently-used shell utilities like cat and diff in-process to eliminate Python interpreter subprocess overhead, and optimized hot paths including the ShLexer string builder. A single ShLexer optimization reduced CodeGen/X86 suite time by 4.4%. The work was constrained by LLVM's Python 3.8 minimum, forcing fallback to __slots__ instead of @dataclass(slots=True).

What changed in lit's execution model

• Execution engine: Replaced multiprocessing.Pool with asyncio + ProcessPoolExecutor to reduce IPC and queueing bottlenecks on high core-count systems
• Shell utilities: Moved cat and diff in-process; eliminated per-invocation Python subprocess spawning for builtin commands
• ShLexer optimization: Converted str += token building to list joins, dropping from O(N²) to linear; cut CodeGen/X86 suite from 99.31s to 94.96s
• Python modernization: Replaced zip(range(len(x)), x) with enumerate(x), updated super() calls, added __slots__ to ShellCommandResult and related hot objects
• Python 3.8 constraint: @dataclass(slots=True) blocked; used plain __slots__ with migration comments tied to GitHub issue #200531
• Profiling-driven: Scalene profiling used to validate changes; functools.lru_cache and os.scandir() deferred until profiling justified them

Why it matters

LLVM's monorepo test suites run on every commit across thousands of developer machines and CI nodes; reducing lit overhead directly translates to faster feedback for compiler and toolchain developers. The asyncio migration may also serve as a template for other Python-based testing infrastructure facing similar scaling limits.