#local-ai

Posts on X2Q tagged local-ai. Updated as new posts are published.

• Jul 13, 2026

  First look at the NVIDIA DGX Spark — what the box actually reports (2026)

  The NVIDIA DGX Spark reports itself as a 20-core Arm machine (10× Cortex-X925 + 10× Cortex-A725), 121 GiB of unified LPDDR5x memory, a single GB10 Blackwell GPU on CUDA 13, and a 4 TB Samsung NVMe — all in a gold box the size of a thick paperback. This first-look post reads the whole machine back from the command line (lscpu, free -h, nvidia-smi, lspci, lsblk) and shows it doing the one thing no consumer GPU can: holding a 67 GB llama4:scout and a 65 GB gpt-oss:120b model in memory at the same time.

• Jun 24, 2026

  Five alternatives to pyannote: speaker diarization for phone calls (2026)

  pyannote-audio 3.1 works, but requires a Hugging Face token and takes 5 seconds per call. Benchmark of five alternatives on 8 real 2-party phone calls: WeSpeaker + spectral clustering (0.2 s), multi-scale AHC (0.6 s), PLDA + spectral (0.3 s), Silero VAD + spectral (2.1 s) and cross-call gallery nearest-neighbour (0.2 s). The Silero VAD approach won at 76.1% agreement and is 2.4× faster. Surprise: all five collapsed on mono calls — pyannote stays in production.

• Jun 22, 2026

  Apple M1 Pro vs RTX 4070 Ti for local AI — GPU matmul and LLM benchmarks (2026)

  RTX 4070 Ti hits 75 TFLOPS at fp16 via tensor cores; M1 Pro MPS reaches 4.2 TFLOPS — an 18× compute gap. But on real LLM inference (minicpm-v4.6, qwen3.5, gemma4:12b via ollama), the RTX is only 3–4× faster. LLM inference at batch=1 is memory-bandwidth-bound, not compute-bound, and the M1's unified-memory architecture closes most of the gap. Apple's AMX extensions also give its CPU 2.7× better matmul throughput than an Intel i5 with the same thread count.

• Jun 22, 2026

  Ollama complete guide — run LLMs locally (2026)

  Ollama lets you run large language models locally with a single command. This guide covers installation on Mac and Linux, how to pick the right model (with a size table), a CLI cheat sheet, the REST API with curl and Python, customisation with Modelfile and system prompts, and performance tricks that halve VRAM usage.

• Jun 10, 2026

  Transcribing 33,000 Danish voice logs on home GPUs — the local pipeline (2026)

  Business phone calls had been recorded for 13 months — ~33,000 Danish mp3s, ~570 hours of 32 kbps phone audio. The job: transcribe everything, name the speakers, summarize per call/day/week, browsable on a site, and do it locally with no LLM API. This is the build: a benchmark of Danish ASR models, a dual-model + Claude-fusion transcription pipeline, 'phone-first' speaker identification from metadata, self-healing infrastructure across two GPUs, and Claude Code subagents as the (API-less) summary layer. Plus what it teaches about applying the same pipeline to a customer-service function.

• Jun 10, 2026

  Whisper backend shootout: faster-whisper vs HF Transformers vs whisper.cpp (2026)

  Three ways to run Whisper — faster-whisper (CTranslate2), Hugging Face Transformers (batched, SDPA), and whisper.cpp (ggml) — benchmarked on real Danish telephone audio on an RTX 4070 Ti. faster-whisper + large-v3-turbo is the fastest overall (~56× real-time); whisper.cpp CUDA is close at a fraction of the VRAM and even matches GPU speed on CPU for short files. Full speed (RTF), VRAM, and consensus-WER quality tables, plus a recommendation per use case.

• Jun 4, 2026

  Local speech-to-text with whisper.cpp — a self-hosted Google Speech API alternative (2026)

  Years ago I ran transcription through the Google Speech API. In 2026 you don't need a cloud API for that — whisper.cpp runs OpenAI's Whisper models locally on a laptop, CPU or GPU, with no keys and nothing leaving the machine. Here's the full path from clone to transcript.