When LLMs Aren’t the Answer — MLOps Community

Looking forward to making this AI stuff pay when I get my cut of this [https://techcrunch.com/2026/01/26/youtubers-sue-snap-for-alleged-copyright-infringement-in-training-its-ai-models/]. ## Inference Inflation LLMs are a luxury tax on simple logic - if a CPU-bound model handles it for pennies, you're burning margin for hype. What wins your production vote: Spacy or GPT? [SPACY](https://gatewaze.mlops.community/offer/surveys/?sid=yesno&question=LLMs+are+a+luxury+tax+on+simple+logic+-+if+a+CPU-bound+model+handles+it+for+pennies%2C+you%27re+burning+margin+for+hype.+What+wins+your+production+vote%3A+Spacy+or+GPT%3F&y=SPACY&n=GPT&oneclick=true&accept=true) [GPT](https://gatewaze.mlops.community/offer/surveys/?sid=yesno&question=LLMs+are+a+luxury+tax+on+simple+logic+-+if+a+CPU-bound+model+handles+it+for+pennies%2C+you%27re+burning+margin+for+hype.+What+wins+your+production+vote%3A+Spacy+or+GPT%3F&y=SPACY&n=GPT&oneclick=true&accept=true) ## Remembering to forget A close one, as 57% said they default to forgetting with long-term agent memory. ## Inference Is Where AI Loses Control Most AI failures in production don’t come from bad models. They come from uncontrolled inference. Costs spike unexpectedly. Latency drifts. Model behavior varies across workflows. Guardrails exist in theory, not in execution. CLōD helps you take back control over the AI inference layer. Developers can define how each request behaves, optimizing for speed, cost, safety, or compliance all in one single API, without hardcoding logic across fragmented systems. With CLōD, you can: * Set cost and latency targets per request * Route intelligently across multiple models with fallback built in * Enforce data residency and sovereignty when required * Add observability, policies, and audit trails without slowing teams down AI doesn’t scale on model power alone. It scales when inference becomes predictable. 👉 Take Back Control of AI with CLōD Today [👉 Take Back Control of AI with CLōD Today](https://clod.io/ai-control-developers) [https://clod.io/ai-control-developers](https://clod.io/ai-control-developers) ## Curated finds to help you stay ahead ## Moltbot: Under the Shell of a Local-First Agent I love a good pun, which is why it’s disappointing that lobster-inspired agentic assistant Clawdbot had to change its name to Moltbot. Still, that doesn’t stop it challenging how personal AI assistants run, splitting execution from interaction. The assistant keeps running locally (next to repos, credentials, and filesystem access), while you control it from anywhere. A single Gateway instance stays bound to localhost [http://localhost] and manages the assistant, channel connections (WhatsApp, Telegram, Slack, Discord, Google Chat, Signal, iMessage), and tool access. You reach it over Tailscale Serve (tailnet-only) or Funnel (public HTTPS), or via SSH tunnels. The macOS/iOS/Android apps connect as clients over WebSocket, letting you pair device nodes for camera, screen recording, or system commands. The execution model doesn't change when you're away. Bash runs where the Gateway lives. Device-specific actions (camera snap, screen capture, local notifications) route to paired nodes via node.invoke. Groups get isolated Docker sandboxes when you set agents.defaults.sandbox.mode: "non-main", so the assistant can handle group chats without exposing host access. Sessions persist independently of connection state. You can reset a conversation from your phone, pick it up in WebChat, then resume later in Slack without losing context. The Gateway tracks presence, typing indicators, and usage, and runs scheduled jobs (cron, Gmail Pub/Sub triggers, webhooks) whether you're connected or not. You extend the assistant with skills. Bundled skills ship with Moltbot. Managed skills live on ClawdHub (the public registry). Workspace skills are local to a project. The assistant can search ClawdHub, pull in new capabilities as needed, and combine multiple skills for a single task. VoltAgent's skill collection (565+ community skills across 30+ categories) demonstrates this extensibility at scale. If you run the Gateway on Linux and pair a macOS node, the assistant can execute remote commands while preserving macOS TCC permissions for screen recording or camera access. The same pattern works for iOS/Android nodes. Execution locality becomes a deployment choice, not an architecture constraint. The Browser tool launches a dedicated Chrome/Chromium instance with CDP control. The Canvas tool pushes a live visual workspace to paired devices (macOS/iOS/Android). Voice Wake and Talk Mode run on device but coordinate through the Gateway. Multi-agent orchestration uses supervisor runtimes and a sessions tool suite (sessions_list, sessions_history, sessions_send) so agents coordinate without jumping between chat surfaces. Moltbot treats messaging channels as first-class surfaces. Each channel has pairing policies (DM access requires approval by default), group activation modes (mention-only or always-on), and delivery controls (reply tags, chunking, typing indicators). You configure allowlists, set per-channel media limits, and control which groups the assistant joins. This fits longer-running work. The assistant doesn't need you present to finish tasks, respond to triggers, or manage ongoing conversations. Oversight happens when you check in, not continuously. Useful links * Moltbot GitHub [https://github.com/moltbot/moltbot] - Core gateway, channels, and tools architecture * Moltbot Documentation [https://docs.molt.bot/] - Setup wizard, configuration reference, security model * Awesome Moltbot Skills [https://github.com/VoltAgent/awesome-moltbot-skills] - Curated collection of 565+ community skills * VoltAgent Framework [https://github.com/VoltAgent/voltagent] - TypeScript framework for building custom agents and workflows * ClawdHub [https://clawdhub.com/] - Public registry for discovering and installing agent skills * Getting Started Guide [https://docs.molt.bot/start/getting-started] - Onboarding wizard walkthrough * Gateway Operational Runbook [https://docs.molt.bot/gateway] - Running the Gateway and remote access patterns ## A Playground for AI Engineers LLMs are great until you have 10,000 support tickets in Portuguese and latency starts costing real money. This conversation breaks down how one team keeps “boring” classical NLP in production for speed and cost, while using LLM agents where the interface matters and the risk is higher. * Classic NLP still wins at scale Cheap, fast ticket classification and entity extraction using logistic regression and spaCy, running as Kubernetes microservices, with continuous retraining to handle drift. * LLM agents for user-facing work RAG-backed AI tutors constrained to course content, plus guardrails for emotional or out-of-scope prompts and fallbacks when vendors fail. * Metrics that hurt if you ignore them Thumbs up/down per answer, weekly return usage, and retention lift versus non-agent courses as the proxy for real value. Classical ML handles the infrastructure, LLMs handle the conversations, and metrics decide which problems deserve which tools. [https://podcasts.apple.com/us/podcast/a-playground-for-ai-ml-engineers/id1505372978?i=1000746370918](https://podcasts.apple.com/us/podcast/a-playground-for-ai-ml-engineers/id1505372978?i=1000746370918) [https://home.mlops.community/home/videos/a-playground-for-ai-engineers](https://home.mlops.community/home/videos/a-playground-for-ai-engineers) [https://open.spotify.com/episode/55PNrsip8JNhpkQWmMhEje?si=9a2f670eec304a86](https://open.spotify.com/episode/55PNrsip8JNhpkQWmMhEje?si=9a2f670eec304a86) ## Cracking the Black Box: Real-Time Neuron Monitoring & Causality Traces Regulation feels abstract until you realize the EU AI Act is backed by thousands of pages of concrete engineering standards. This conversation reframes compliance as an implementation problem, not a legal one, and explains what “good practice” actually means for teams shipping high-risk AI. * The real work lives below the law Harmonized standards translate vague requirements like fairness and robustness into testable system behaviors with presumption of conformity. * Observability through a risk lens Monitoring shifts from uptime and drift to bias, contestability, and auditability at inference time. * Why black-box models raise the bar Closed APIs limit explainability, pushing regulated teams toward open-weight models and deeper internal traces. Compliance becomes tractable once it is treated as system design rather than paperwork. [https://podcasts.apple.com/us/podcast/cracking-the-black-box-real-time-neuron-monitoring/id1505372978?i=1000746906169](https://podcasts.apple.com/us/podcast/cracking-the-black-box-real-time-neuron-monitoring/id1505372978?i=1000746906169) [https://home.mlops.community/home/videos/cracking-the-black-box-real-time-neuron-monitoring-and-causality-traces](https://home.mlops.community/home/videos/cracking-the-black-box-real-time-neuron-monitoring-and-causality-traces) [https://open.spotify.com/episode/7MA7cjKZvwcmh0cfXunHNM?si=6367408495e4463b](https://open.spotify.com/episode/7MA7cjKZvwcmh0cfXunHNM?si=6367408495e4463b) ## Agent Use for Coding Agents can write code fast, then spend four hours gaslighting your tests into passing. Our January reading group ran a live session on the paper Professional Software Developers Don’t Vibe, They Control: AI Agent Use for Coding in 2025 [https://arxiv.org/abs/2512.14012], using it as a springboard for practical discussion grounded in real workflows. * Control stays with the developer Participants consistently described using agents as junior collaborators - prompts, plans, and context matter, but review, rollback, and judgment never leave the human. * Tests are the fault line Letting agents write or modify tests came up repeatedly as a risk, especially when failures quietly disappeared behind “helpful” changes. * Clear wins, clear limits Agents were seen as effective for repetitive scaffolding and small refactors, and far less reliable for ambiguous architecture, security-sensitive code, or novel domain logic. The session landed on shared patterns rather than prescriptions, shaped by what people had already broken, fixed, and learned the hard way. [Watch here](https://home.mlops.community/home/videos/agent-use-for-coding-in-2025-mlops-reading-group-january-2026) ## How I Reduced AI Token Costs by 91% with Semantic Tool Selection and Redis One platform was sending every tool definition to its LLM on every request, burning tokens even when users only needed two or three tools. Keyword matching couldn’t connect “notify engineering team” with send_slack_message, driving up cost, latency, and noise. * Semantic decomposition instead of tool blobs Each tool is split into description, parameters, examples, and return types, embedded separately in Redis with weighted relevance rather than treated as a single block. * Adaptive selection over fixed quotas The system stops at natural relevance drop-offs instead of returning a preset number of tools, cutting average results from 70 to 2–3. * Redis as the pragmatic choice HNSW vector search plus native metadata and caching replaced multiple databases and dropped input tokens from 7,244 to 198 per query. Precision rose from 72% to 95% while response times fell 31%, showing semantic selection beats brute-force context stuffing. [Read the blog](https://home.mlops.community/home/blogs/how-i-reduced-ai-token-costs-by-91percent-with-semantic-tool-selection-and-redis) ## Testing Myself [https://forms.gle/8EDvXGizxyFVKfwy8](https://forms.gle/8EDvXGizxyFVKfwy8)