Meta fixes kernfs scalability bottlenecks for Linux 7.4 after production incidents

Meta engineer Shakeel Butt traced production incidents where system daemons hung for minutes to scalability bottlenecks in kernfs, the framework underlying sysfs and cgroupfs. The resulting 18 patches for Linux 7.4 reduce lock contention dramatically—one benchmark showed kernfs_rwsem read contentions dropping from 48.6 million to 1.3 million—and improve reader path-walk throughput by 35-53% on an 8-CPU VM under churn.

What the 18 patches change

• Lock handling rework: Multiple patches improve how kernfs acquires and releases locks, reducing contention between readers and writers.
• Benchmark results on 8-CPU VM: kernfs_rwsem read contentions fell from 48,593,360 to 1,280,280; top call site revalidations dropped from 1,744,517 to 429,846 and then became absent.
• Throughput gain: Reader path-walk throughput improved 35-53% under workload with concurrent cgroup churn, netdev renames, and device add/remove operations.
• Bug fixes included: Several patches address pre-existing kernfs bugs discovered during the scalability investigation.
• Real-world trigger: Meta identified the problem when cgroup-related interfaces caused system daemons to stall for minutes across their server fleet.

Why it matters

Large-scale deployments and cloud providers running heavy cgroup or sysfs workloads stand to benefit most, though the improvements apply broadly to any multi-core system with active container or device management.