Bella
A private AI assistant that knows me, acts on my behalf, and improves its own code over time. Built solo on the Claude API as my own personal assistant, in daily use.
The problem
I wanted an assistant that actually knew me and could do things, not a chat box that forgot the last conversation. Doing that well normally means handing a company your calendar, messages, and private notes. I wanted the capability without giving my own data away, and I wanted it to keep improving without me hand-editing it forever.
Architecture
One Python service behind Cloudflare, reachable from web, a native Android app, Telegram, and voice. Claude runs the reasoning; a local model handles background work.
What she can do: 48 tools
Tools live in one directory and are discovered at startup. Each declares its own trigger keywords, and the agent loop offers the model only the handful relevant to your message rather than all 48 at once, which keeps the prompt small and the model focused. Every call runs under a timeout, and an oversized result is truncated before it reaches the model.
Tasks & habits
To-dos, recurring tasks, reminders, timers, focus sessions, and habit and goal tracking.
Notes & knowledge
Journals, quick notes, a text editor, and read and write access to an Obsidian vault.
Calendar & email
Google Calendar and Gmail: read the day, add events, check unread, look up a contact.
Web & files
Web search, fetch and read a URL, open a link, read an attached file, list the workspace.
Memory
Save a fact on request, or surface facts from the conversation worth keeping.
Everyday utilities
Calculator, date and time, weather, clipboard, and a scripture-library lookup.
Integrations
Bella reaches real services, not a sandbox, all from one local backend behind a Cloudflare tunnel.
- Anthropic Claude for the reasoning, with an optional OpenAI provider behind the same interface, switchable from a setting.
- Google Calendar and Gmail over OAuth, so she can read the day ahead and unread mail and act on them.
- Obsidian as a knowledge base she can search and write back into.
- A local model (Ollama, running qwen3) for background jobs, so routine work never spends cloud tokens and degrades gracefully if it is offline.
- Voice through faster-whisper for speech to text and Kokoro for text to speech.
- Five surfaces: web, a native Android app, real-time voice, Telegram, and a plain API.
The part I'm proudest of: it improves itself, safely
Bella reviews her own weak conversations and proposes fixes to her own code. The whole design is about making that safe: a different model grades the work, a human approves every change, and the quality bar can only get stricter.
# Self-preference bias is real (Panickssery et al., NeurIPS 2024), # so the judge is never Bella's own model. Enforced at import. assert JUDGE_MODEL.family != "anthropic", "judge must be cross-family" verdict = judge.score( change=proposed_diff, rubric=RUBRIC, # SHA-256 pinned: can tighten, never loosen votes=3, # self-consistency, median-aggregated ) if verdict.regresses_gold: # frozen gold conversation set reject(proposed_diff)
Illustrative. The loop opens a PR on a dedicated branch; a human approves every change before merge.
Other decisions worth noting
Prompt caching, to make a personal assistant affordable
A knowledgeable assistant carries a large, mostly-stable system prompt. I split it so the stable half is cached and only the per-turn half is re-sent, cutting input tokens on a typical turn by roughly 80%.
system = [ {"type": "text", "text": PERSONA_AND_TOOLS, "cache_control": {"type": "ephemeral"}}, # stable, cached {"type": "text", "text": todays_context}, # changes each turn ]
Memory that curates itself
Facts live in per-user SQLite with vector embeddings in a local store, so recall blends semantic similarity with keyword search. Each new fact is compared against what she already knows: a near-identical one is dropped, and a close-but-different one is handed to the local model to decide whether it supersedes the old fact and should replace it in place. No managed cloud database, which keeps the data private and the stack cheap.
Voice, end to end
Call mode is a real-time WebSocket session, not record then send. A voice-activity state machine moves through listening, recording, thinking, and speaking. Audio streams in as raw PCM, gets transcribed, runs the same agent loop as a typed message, and streams back as speech. Synthesis starts on the first sentence, so a reply begins out loud within a few hundred milliseconds instead of waiting for the whole answer.
Proactive, not just reactive
A scheduler runs alongside the chat. It pushes a morning briefing, and a gated nudge system watches real signals (hours since we last spoke, overdue reminders, unfinished habits) and only reaches out when a weighted score crosses a threshold, so it stays useful instead of noisy. Every turn also carries a short backlog of what is actually urgent right now (habits still open in the evening, reminders past due, the next calendar event, unread mail), folded into her context so she answers with that in mind.
Outcome
What I'd do differently
I built the speculative voice pipeline before I knew how much I'd actually use voice. It works, but it was the most complex part of the system for the feature I reach for least. Next time I'd prove real demand for a hard feature before building the ambitious version, and put that time into evaluation harnesses earlier, which paid off everywhere I did invest in them.