b11632: cuda, hip: GDN chunked kernel (#29353)
cuda, hip: GDN chunked kernel Ported-from: tetherto#257 Source-commit: 53f0027 Co-authored-by: Aman Gupta [email protected] fix older builds review fixes cuda: enable GDN MMA on tested Ampere and Ada GPUs Enable SM86 and SM89 after RTX 3090 and RTX 4090 correctness and prefill benchmarks.
Executive Summary
cuda, hip: GDN chunked kernel Ported-from: tetherto#257 Source-commit: 53f0027 Co-authored-by: Aman Gupta [email protected] fix older builds review fixes cuda: enable GDN MMA on tested Ampere and Ada GPUs Enable SM86 and SM89 after RTX 3090 and RTX 4090 correctness and prefill benchmarks. Keep other untested architectures on the existing fallback. Assisted-by: Codex cuda : use refactored swizzle API in GDN MMA ggml : GDN only guarantees the final state when n_tokens > K cuda : add short GDN MMA phase comments Co-authored-by: gianni-cor [email protected] Co-authored-by: Aman Gupta [email protected]
Artificial Intelligence Architecture & Model Evaluation
From an artificial intelligence architecture, model weights governance, and inference efficiency perspective: - **Weights Accessibility & Sovereignty:** Evaluates whether weights are open for private self-hosting or locked behind centralized cloud APIs. - **Quantization & Edge Performance:** Kernel optimizations (4-bit/8-bit GGUF, AWQ, EXL2) allow high tokens-per-second on consumer GPUs and Apple Silicon. - **Reasoning & Architectural Scaling:** Scrutinizes mixture-of-experts (MoE), attention mechanisms, and fine-tuning datasets against open community benchmarks.
Impact on the Open Ecosystem
Protects developers and enterprises from proprietary black-box entrapment, fostering auditable, sovereign AI infrastructure.