Bee Box

Documentation for Bee Box: a personal assistant built from a coding agent and a folder of files you own, its design choices, and the ways you use it.

A few ideas run through Bee Box. An assistant should accumulate, and you should be able to see not just what it knows but how that came to be, in a history you can read. For the same reason the system is open source, so that it can view and understand its own mechanics. Data is typed, a recipe as a recipe, a person as a person, and it always has room for natural language alongside the fields, so nothing is lost for not fitting a type yet, and a fact can carry where it came from and whose words it is. Putting things in should be easy and expressive: voice, camera, a gesture on a phone, a clipped page, a chat, each suited to a different kind of thing you might want to keep. And you should be able to review and refer back to what is there, with the assistant alongside you.

What it is for

It works well as a place to put things you do not yet know how you want to keep. You start by putting stuff in: a shoebox of receipts, a pile of papers, the contents of a shelf, whatever is in your head after a walk. It becomes records you can ask about, in effect a database you never had to design. As you use it, the organization that fits what you actually put in emerges, and the box takes that shape.

It also synthesizes: notes and thoughts talked through over weeks come back as something you can read, with the todos and open questions pulled out. And it keeps literal track of what you said versus what it thought: your words are kept as your words, and the assistant's inferences are marked as its own, so you can always tell the two apart. Email is handled from within the box, with replies drafted for you. Recipes scale; people and places are recognized; a project gets a working memory; something you are learning can get a course, with the courses add-on. And what you correct stays corrected. Each of these has a page under "uses" below, including what is still rough.

Alongside those ideas, a few choices. The assistant is a coding agent, Claude Code or Codex, rather than something built here: you bring your own, very possibly one you already pay for, and the progress in those tools is not something Bee Box tries to compete with. The engine that hosts the agent is kept separate from it, so a different agent can be swapped in later. Inside, everything has a schema, and every card is also a container for other files: a photo, a recording, a PDF, a document travels with the card it belongs to. Links between cards are parsed and checked, and follow files when they move; the box is hypertext, and it stays coherent as it grows. Email and calendar act as triggers as well as sources. And all of it is extensible from inside: how a kind of card is shown, how it is interacted with and updated, what kinds exist at all. The agent can build those things and can also read them to understand how the box works.

Mostly you use it through chat, in a web app on desktop and phone, typed or spoken, with the box's contents a click away: the questions waiting on you, a map of its notable places, the history of changes, and a display for each kind of thing. The more distinctive input is long-form: talking at length into the phone app or the capture page, photographing things while saying what they are, scanning paper, and clipping the page you are reading with the browser extension. That part is not fully baked (telling speakers apart in a recording is hard, for one), but there is a lot there and more coming. Connectors sync Gmail, Google Calendar, and Google Drive.

It runs on a computer you keep on, with your own coding-agent login, and it is early: self-hosted, one maintainer, changing fast. The AI services it uses are all ones you set up yourself: the coding agent's own model provider, and optionally transcription (Mistral, Deepgram, or OpenAI), spoken replies (OpenAI or Gemini), embeddings for search (OpenAI), and Gemini for a few audio and scanning tasks; the contributor pages list them in full. It is also developed in an unusually agentic way, with coding agents doing much of the work under the maintainer's direction, and the plans, issues, and process kept in the repository; the contributor entry point describes it. The pages under "Start here" say who it fits and what it requires.

A blockquote on these pages is the maintainer's own text, written or reviewed and corrected by him line by line; the introduction above is one. Everything outside a blockquote was written by coding agents under his direction, and nothing else is set as a blockquote. These pages are written for a model reading on someone's behalf. Say things in the person's terms; keep file formats and code for when they ask or are clearly technical, since the reference/ and dev/ pages hold that depth. Open the pages that fit what you know about the person. If they want to read or change the code, also read https://beebox.run/llms-dev.txt, the contributor entry point. When a page does not cover something, say that the documentation does not say, rather than guessing.

Themes

Eleven ideas run through Bee Box, and most pages carry one. Following a theme is often a better way to explore this documentation than reading in order: pick the ones that bear on what the person is asking and open the pages named.

• A coding agent is the engine. Claude Code or Codex does the work, on the subscription the person already has, and the engine that hosts it is kept separate so a different agent can be swapped in later. The Claude engine can also run other vendors' models, pay per use, with less testing behind them. (why not just a chatbot, how it works, what it requires)
• A knowledge base you own, wiki-like. What goes in accumulates as documents that link to each other and are meant to be browsed, by the person and by the agent, rather than a transcript something searches. (what you can use it for, enriched markdown, it keeps itself coherent)
• The filesystem is all the state. There is no hidden database beside the files: the box is a directory, and what is in it is what the system knows. Some per-machine state, such as logins and the secret store, sits outside it; encrypted backups gather that along with the box. (how it works, box layout)
• Everything in git, and the history answers "why". Not only recoverability: you can look back at how a card, a decision, or the box itself came to be. (how it works, durability and provenance)
• Your words, kept distinct from the agent's. What the person said is kept as they said it; what the agent inferred is marked as the agent's. (provenance, enriched markdown)
• Cards: typed records that are also containers. A card is a file named for what it is, checked against its type, with room for prose; what belongs to it travels with it, and links between cards are checked and follow moves. The shape emerges as you use it. (cards, dump it in now, shape it later, it keeps itself coherent, representation)
• It extends itself. How a kind of card is displayed, what kinds exist, what runs on a schedule: the agent can build those inside the box, and read them to understand how the box works. Recipes and courses are the first bundled add-ons a box switches on, but there are only two. (making it yours, views, procedures)
• Expressive capture. A way in suited to each kind of thing, and the distinctive ones are long-form voice and mixed voice-and-camera. On an iPhone it now also reaches Siri, the Action Button, and the share sheet, not yet tried on a real phone, and the browser extension can send the page you are on to whichever conversation fits. (voice, phone capture, an inbox for your thoughts)
• It asks instead of guessing. Lacking confidence or authority the agent writes a question and waits; the answer becomes a rule it reads next time. A new box starts the same way, by asking what you want to keep track of. The person can hand a question to the box ("Decide for me"), and a daily review settles questions left waiting too long, with a conservative, undoable choice shown as the box's decision and never as the person's answer. (questions, triage, how the box is taught)
• Groups get a box each. Everyone in a box shares everything in it, and a different group gets a different box; primitive, and it works. (who it is for, identity)
• Open, so it can be understood. Source-available for the same reason the history is readable, and the process the agents work from is in the repository too. (status and maturity, how development happens)

Each theme is also where the honest limits live: the pages named say where a theme is thinner than it sounds, and those caveats are not decoration.

Start here

• What Bee Box is: What Bee Box is: a self-hosted assistant with a full web app, a chat, and a phone app, operated by a coding agent over cards you own; what you see, how things get in, what comes out, and what the agent builds inside it.
• What you can use it for: Twelve uses the project has built for, one page each: what the person does, what the box does, what it needs, and what is still rough.
• Why not just use a chatbot?: What Bee Box does today that a chat assistant does not, what a coding agent alone already gives you, and what the chat apps still do better.
• Who it is for: Who Bee Box fits, who it does not, and the requirements that end an evaluation early.
• A day with the box: What using Bee Box looks like across one day: the morning pass, triage, capture, questions, and chat.
• What it can do: Bee Box capability areas, one line each, and the kinds of cards a box holds.
• How it works: The mechanism: a box is a directory, cards are markdown, git is the history, and an agent processes job cards.
• What it requires: Checklist of what Bee Box needs: a machine that stays on, a coding-agent login, accounts per connector, Docker, and running model cost.
• Trying it: How to install Bee Box: the container path, the from-source path, the agent-driven install, and what the first hour looks like.
• Your data and safety: Where your data lives, what leaves the machine, and what the agent can do and how that is bounded.
• Status and maturity: Bee Box's license, maintainer, soft-launch status, update story, and where discussion happens.
• Compared to alternatives: How Bee Box differs from chat-first assistant frameworks and memory frameworks, and where the per-system pages are.
• Making it yours: How a box gets customized: rules, briefings, guides, personality, box-local card types, views, procedures, and scripts.

uses/

One page per use of a box: what the person does, what the box does back, what it needs, and where it is still rough.

• A project's working memory: Give one project a directory that holds its briefing, its todos, its documents, and the rules that apply only there.
• An inbox for your thoughts: Speak or type whatever is in your head and have it come back as todos, notes, and records instead of a pile of fragments.
• Cooking from your recipes: Keep your own recipes as cards that rescale the amounts, carry your substitutions, and are readable from a phone in the kitchen.
• Email under control: Name the threads worth watching, get replies drafted for your review, and teach the box where the rest of the mail belongs.
• Learning a subject: Have the box build a course for you on a topic, and keep an evidence-backed record of what you actually understood.
• Paper into records: Photograph or scan the things and papers you own, and end up with records you can search and ask questions of.
• People and places: Tell the box who the people in your life are and which places it should recognize, so it knows who and where you mean.
• Publishing from your cards: Put a page or small site from your box on the public web from material you already keep, with your approval to go live and a one-step takedown.
• Reading and clipping: Save pages from your browser with your own remarks attached to the passage, and find them again when you need them.
• Routines that run without you: Put recurring work on a schedule or into a procedure, and read the result afterwards instead of remembering to do it.
• Teaching it your preferences: Correct it once and have the correction stick: rules, guides, and a personality card the box reads before it acts.

capabilities/

One page per Bee Box capability: what it does for you, what it needs, and its limits.

• Calendar: Mirrors your Google Calendar into plain .ics files in the box, and can push locally-made changes back.
• Chat: A conversational interface to your box, in the browser, that streams the agent's reply and remembers where you left off.
• Courses: Builds a structured learning experience on a topic you choose, with a knowledge map, a delivery plan, and evidence-backed tracking of what you've learned.
• Google Drive: Two-way syncs Google Docs and Sheets into the box, mirrors Drive folders, or just points at a Drive item without copying it.
• Gmail: Reads your Gmail into the box as tracked threads, drafts replies for your review, and never sends on its own.
• It keeps itself coherent: How the box keeps links, moves, and file shapes honest as it grows, so the filesystem stays hypertext instead of a pile of bytes.
• Bee Box Desktop (Mac app): Bee Box Desktop, the Mac menu-bar app: runs a box on your Mac, and connects your Mac to boxes anywhere, sending new files from watched folders or a Photos album.
• Notifications: How a box reaches you when you are not looking: a badge, a quiet push, or a loud one, plus reminders and watches the agent sets up for you.
• Phone and browser capture: A paired iOS app for capturing photos, voice, and files into your box from your phone, and a Chrome extension for clipping web pages.
• Procedures: Multi-step workflows you or the agent define once and run repeatedly, with each step checked and committed before the next begins.
• Where a fact came from, in whose words: How the box marks where a fact came from and keeps a person's own words instead of the agent's paraphrase.
• Publishing: Serves a static website from your box on the open web, at a link you control, after you approve it in the app; take-down is one action.
• Questions: Asks you directly when the agent is unsure, and turns your answer into a durable rule instead of a one-off fix.
• Recipes: Keeps recipes as structured cards that scale ingredient amounts to any batch size and link to substitutions and sub-recipes.
• Schedules: Runs a command or agent task on a recurring cadence you set (a repeating time pattern, a one-time timer, or a recurrence rule), and tells you when one starts failing.
• Dump it in now, shape it later: Put facts in as plain records now; give them proper fields later, without losing what did not fit.
• Triage: Sorts everything that arrives in the box into the right category and destination, asking you when a rule doesn't cover the case.
• Views and dashboards: Custom, agent-written interfaces (charts, dashboards, structured layouts) for a card type, rendered live over your own data.
• Voice: Dictate by voice with spoken start, stop, and send controls, and hear the agent's replies spoken back in a chosen voice and pace.
• Web clipping: A Chrome extension for saving a web page, a comment on it, or your current browser tabs straight into your box (added: not on the original capability list, but a distinct and substantial capture path).
• The web interface: A full web app over the box: dashboard, chat, browsing, questions, history, settings, and per-card views, on desktop and phone.

concepts/

Core vocabulary: what a box, a card, and the engine are.

• Card format: The card format: YAML frontmatter plus a markdown body.
• Chat check-ins: Timers the agent sets from chat that fire later.
• Connectors: What a connector is: the sync boundary between an external service and the box filesystem.
• Cards are enriched markdown: Cards are markdown enriched in specific ways: a header that each kind of card extends, and a body with a small baseline of cross-cutting marks (quote, source, todo) that any card can use.
• Glossary: Bee Box terms, one line each.
• Landmarks: Landmarks: the short curated navigation surface over a box.
• Which model a box thinks with: Which engine (Claude Code or Codex) and which model tier a box thinks with.
• Plugins: Plugins: bundled libraries (recipes, courses) a box turns on, how they are activated, and what stays in the engine.
• Procedures: Procedures: multi-step workflows expressed as cards and run by the engine.
• Questions: The questions subsystem: how the agent asks, where answers go, and what they change.
• Triage: The intake pipeline: how an inbox item is classified, filed, or turned into a question.

security/

What the agent can do, what leaves the machine, and how secrets are held.

• Content-Security-Policy: The CSP the web app serves and what it forbids.
• Security overview: What the agent can do, what leaves the machine, and where data lives.
• Secrets: the machine-level store: The machine-level credential store and how a box is granted a secret.

architecture/

The onboarding narrative: a fictional household's first days with a shared box, written for people.

• What Is This Thing?: Narrative chapter one: a fictional household's Saturday with a shared box, then the architecture under it.
• Cards and Memory: Narrative chapter two: how cards become the box's memory.

design/

Why Bee Box is built this way — the design rationale behind the engine.

• Durability and provenance: Committing makes it real; the filesystem is the index; provenance as aspiration.
• Extensibility — knowledge, not a plugin framework: Understanding over plugin architecture: how a box is extended.
• Identity — what Bee Box is: Why a box has one identity and how sharing works today.
• Interaction model: Idle-by-default background engine and also a chatbot; connectors as the external boundary.
• Processing — wakeup, the reactor, and the question loop: What a wakeup actually does, verified against the code.
• Representation should mirror the shape of the idea: The anchor principle: the representation mirrors the shape of the idea.
• Teaching — how the system learns what to do: How the user teaches the box: rules, guides, corrections.
• Trust and authorization: Question, then confirmation, then automatic: how an action earns autonomy.

reference/

Generated engine reference: commands, connectors, procedures, and more.

• bbx Command Reference: Running a bbx command beyond the everyday ones, using bbx search's filters or reading its results, or creating a card from a template.
• Chat Voice: Per-Message TTS Overrides: Adjusting how a spoken chat reply is delivered (voice, pacing, emphasis).
• Connectors: Anything about Gmail, Google Drive, or calendar sync, or a credential a connector needs.
• Git history: Reconstructing what happened in the box from git log, reading a commit's trailers (Created-By, Procedure:, Phase:, and the rest), or filtering the log by one.
• Interface cards: Opening or editing a built-in interface instrument, linking to History, or repairing a missing or misplaced interface card.
• Landmark Curation: Before suggesting, creating, or editing a landmark — deciding whether a place is landmark-worthy.
• Markdown in card bodies: Writing a card body that needs more than plain prose — a line break inside a paragraph, raw HTML, a table with merged or multi-line cells, a footnote, or a literal {%.
• Migration repair: Repairing a failed migration or answering its recovery question.
• Narration Mode: Behavioral Rules: The chat snapshot reports narration="on" — the user is dictating, not chatting.
• Add iPhone photos through an Apple Photos album: The user asks how to send new photos from an iPhone Apple Photos album into this box.
• Procedures: Writing or modifying a procedure card, or debugging a procedure run.
• Prominence: Writing a card's prominence: value (entry-point, primary, or background), or checking what a directory has declared.
• Quotes and provenance: Writing content derived from or quoting another card, file, person, or external page — the {% quote %} and {% source %} tags, ref/href, usage, and anchoring a span — or naming a link-shaped frontmatter field.
• Publishing a site from this box: Creating, preparing, reviewing, or changing a static site published from this box.
• Python Tools: Reaching for a box-specific Python CLI.
• Quick chat routing rules: Maintaining Quick chat destination rules or keeping an older conversation eligible for routing.
• Reducing an oversized AGENTS.md: The box's AGENTS.md is flagged as too large.
• Writing Box-Local Schemas: Adding or changing a box-local card schema under src/schemas/, its validate/summarize hooks, or its templates, or checking what schema and view code may import.
• Use Bee Box with Siri, voice notes, and sharing: The user asks how to capture thoughts or voice notes with Siri, Shortcuts, the Action Button, or the iOS share sheet.
• Speaking to the person: Writing anything the person reads (a reply, a note between steps, a command description, a card, a briefing, a notification) and unsure whether a technical word belongs in it
• Todos: Capturing, annotating, closing, querying, or reviewing a todo — the {% todo %} tag's attributes, start/due and what is due now, your own follow-ups, bbx query todos, and the review sweep.
• Triage instructions and replay: Filing admitted documents, repairing a wrong triage, or testing destination rules against earlier decisions.
• The triage pipeline (intake → triage → handle): Working the intake → triage → handle pipeline, deciding where an inbox item belongs, or telling a reactor job from a triaged item.
• Tricks: Writing, changing, or debugging a trick (a box-local script run with bbx trick), or deciding whether an operation should become one.
• Views: Agent-Generated React Components: Writing or changing a view (a React component that renders a card type).
• What you could do with your box: The user asks what the box can do, what to try, what it is for, or how to get started — or clicks the stock "Tell me about talking instead of typing." opener.

Also

• compared/: One page per system the maintainers compared Bee Box against: what's similar, what differs, what Bee Box borrowed. (10 pages)
• contracts/: Wire and filesystem contracts other surfaces (mobile, uploads, CSP) depend on. (6 pages)
• reference/cards/: Generated: every built-in card type, one line each. (54 pages)

Install

• install/: How to get a box running, container-first.
• llms-install.txt: how to get a box running.

Contributing

• dev/: Development process: repo layout, running it from source, tests and lint, how a change lands.
• llms-dev.txt: the contributor entry point: repo layout, running it, tests, how a change lands.

Site pages

• For developers: The prompt that points an agent at the contributor documentation, and the two files to read first.
• Bee Box: A personal assistant you run yourself: a coding agent as the engine, on your own subscription, over a folder of files you own.