Jef Spaleta: I’ve started my LLM journey
I’m going to be providing dispatches in blog form that detail my learning journey concerning LLMs.
Executive Summary
I’m going to be providing dispatches in blog form that detail my learning journey concerning LLMs. I’m going to be experimenting with fully open source inference that I can run locally on hardware I control. Local Language Models; the other LLM. What utility can I get without reaching for the frontier “as a service” models? I’ve made a commitment to myself to learn, and the way I learn best is experiential.
Artificial Intelligence Architecture & Model Evaluation
From an artificial intelligence architecture, model weights governance, and inference efficiency perspective: - **Weights Accessibility & Sovereignty:** Evaluates whether weights are open for private self-hosting or locked behind centralized cloud APIs. - **Quantization & Edge Performance:** Kernel optimizations (4-bit/8-bit GGUF, AWQ, EXL2) allow high tokens-per-second on consumer GPUs and Apple Silicon. - **Reasoning & Architectural Scaling:** Scrutinizes mixture-of-experts (MoE), attention mechanisms, and fine-tuning datasets against open community benchmarks.
Impact on the Open Ecosystem
Protects developers and enterprises from proprietary black-box entrapment, fostering auditable, sovereign AI infrastructure.