Alexander Ioffe: Why is my Whale Query 1000x Slower in Prod? Databases hate skew, each hates it differently
Ten enterprise customers own 80% of the orders.
Executive Summary
Ten enterprise customers own 80% of the orders. On uniform test data the query runs in ~1ms; on that real skew it runs ~1,000x slower. The fix doesn't transfer between engines: Postgres is rescued by one free session setting, SQL Server can't be touched by it and needs a completely different access-path fix. Measured on Neon and Azure SQL Database using ExoBench.
Database Engine & Storage Subsystem Architecture
From an OLTP/OLAP systems architecture, storage tiering, and data durability perspective: - **Zero-Vendor Lock-in Formats:** Leveraging open specifications (e.g. Parquet, Iceberg) ensures multi-engine interoperability without ingestion lock-in. - **Vector & In-Memory Throughput:** Modern memory layouts and SIMD-accelerated execution pipelines minimize latency in real-time retrieval. - **Distributed Resiliency:** Decoupled storage and compute layers enable elastic horizontal scaling with strict ACID guarantees.
Impact on the Open Ecosystem
Empowers data engineers with open, self-hosted alternatives to monopolistic cloud database providers.