// ITS FOSS — LINUX & OPEN SOURCE
Local AI Weekly #4: The Fine Print of Running AI Locally
"Local AI" keeps meaning different things these days. Sometimes the model runs on your computer. Sometimes only the app does, while the model and your chat still run elsewhere.
There are lots of such "local AI" tools that are not really local. Before you pick a local AI tool, ask three things: where does inference happen, what account or network service is still required, and what does the license let you do? The best local AI tool is the one that doesn't need any of that.
Team It's FOSS has moved from Discord to Buzz for internal chat. Buzz, from Jack Dorsey of Twitter fame, is a decentralized communication tool for humans and agents. Agents can run on remote servers or a local harness over ACP.
It has quirks. Clipboard screenshots won't paste into chat, and desktop notifications only fire for direct messages. Manageable for now.
Kubutu developer Rick Timmis is working on Klara, a local desktop AI assistant for KDE Plasma. The idea is to let you control the desktop via voice input. It is a work in progress for now.
OpenMuse is an MIT-licensed personal-agent app with a browser worker, durable tasks, and an optional Docker-based Linux computer. You can host it yourself, but setup still needs a CopilotKit Intelligence key, and open-ended tasks default to cloud providers. You can point it at an OpenAI-compatible endpoint, but there's no documented native Ollama path. Self-hosted software, not an assured offline agent.
The Linux Foundation now offers AI certifications. The Model Context Protocol Associate one could interest you if you're building MCP integrations, or want an AI credential on your resume. I plan to take it just for the sake of learning new skills.
OpenDecider is a more modest use for local models: answer bounded questions, like which queue a ticket should go to, instead of writing long replies. Its nano model is about 400M parameters; the 4B small model is a Qwen-based adapter. The author says both were distilled from larger teachers, and reports 2.0 GiB for nano and 8.9 GiB for small in the tested setups.
Basically, you don't always need a giant model to route routine requests. A small student model can be the triage step ahead of a slower agent.
NVIDIA's September PAIR announcement also promises easier local-model setup in Hermes and OpenClaw. The one-click Hermes path launched on Windows, with Linux "coming soon." Don't confuse that future path with the Linux PAIR beta available now.