Product judgment, built into working software
I execute complex, ambiguous systems to a working outcome.
My professional background is in product leadership across financial infrastructure and institutional banking. I also build and operate private AI systems for real household use, alongside Rook, my AI developer for rapid proofs of concept. I own the product decisions, constraints, and acceptance criteria; AI helps me investigate, implement, and test them.
01 What I Build
Private assistance for daily life. An AI developer for new ideas.
Ellis AI is the private family platform. Its assistants have different purposes, memory permissions, and safety requirements. The personal experience is separate from the engineering tools used to operate it. Rook is my developer: a way to turn a product idea into a concrete prototype that I can inspect, test, and improve.
| Assistant | Purpose | Priority |
|---|---|---|
| Sloan | Executive assistance, research, and thought partnership | Useful answers grounded in evidence |
| Artie | Household logistics, meals, and planning | Practical, dependable help |
| Eunoia | Tutoring, curiosity, and creativity | Age-appropriate help and child safety |
| Ellis | Family financial guidance | Conservative advice and controlled access |
| Rook | AI developer for rapid proofs of concept | Testable software and information security |
Rook: from an idea to a testable POC
I use Rook for quick proofs of concept in F#, Rust, and other languages suited to the problem. The language is a design choice: I want to learn whether an approach works before committing to a larger implementation. A prototype demonstrates a bounded idea; production readiness requires separate evidence.
Rook combines local-model work with frontier-model orchestration and escalation for harder engineering problems. Information security is part of the development process, with specialist review for sensitive work. An implementation recommendation does not grant permission to change credentials, access controls, or production systems.
My role stays human: define the problem, decide what is worth building, set the boundaries, and judge the result. Generated code, a successful build, and a working experience are different kinds of evidence.
02 Seven Decisions Under Constraint
Make the tradeoffs explicit, then test the result.
The work spans product choices, implementation, and the responsibility of operating what ships.
Match the model to the work
The platform uses local inference alongside selected frontier services. Rook has a development-focused model path, with escalation and specialist review where needed. The decision balances capability, latency, cost, and what information may be shared. More expensive inference is a tool to justify against the problem, rather than a default for every request.
Use a prototype to answer a specific question
A quick POC should settle something concrete: whether an interaction makes sense, whether a domain model captures the rules, or whether an integration behaves as expected. F#, Rust, Python, and native or web clients each have different roles and tradeoffs. I choose the scope and stack around the question, then judge the working result before expanding it.
Design privacy into the boundaries
The family platform scopes memory and conversations by user and assistant. Its voice routing considers sensitivity, local availability, and explicit consent for permitted external processing. Restricted content retains a local-only policy. These controls need implementation checks and regression tests; a reassuring prompt alone cannot establish privacy.
Separate capability from permission
An AI developer may be capable of changing a system without being authorized to make that change. Routine work can proceed within approved scope; sensitive actions need the relevant owner approval. Information-security review considers credentials, access, data exposure, and the effects of a change. Passing a code review does not waive those boundaries.
Build recovery into the release path
The family platform's service deployment path includes test gates, backups, health checks, and automatic rollback for supported failure paths. Other projects have their own release requirements. I distinguish code that passed locally, code that passed integration checks, and behavior verified after deployment. A healthy process alone does not prove that a user can complete the intended action.
Measure the whole interaction
Voice involves recording, transcription, inference, synthesis, and playback. The repository includes startup warmup and voice-specific regression checks because a slow first response can come from those stages rather than model quality. The useful measurement is the user's experience across the full path, including failure and recovery.
Require evidence for completion
A generated answer is not a retrieved fact, and an assistant saying “done” is not proof that an action happened. The platform includes evidence checks and action receipts to make that distinction visible. Missing evidence, failed checks, and unfinished validation must remain explicit. Otherwise, downstream decisions inherit confidence that the system has not earned.
03 When It Broke
Historical case study · July 2026The assistant was inventing facts. One symptom had five causes.
An internal review of Sloan conversations from July 9–11, 2026 found fabricated factual claims in 17 of 19 conversations examined. Examples included invented sources and product claims, and the platform's own technical vocabulary presented as if it described the user's professional domain.
This was a small, targeted historical sample. It is not an estimate of the platform's current error rate or the performance of AI systems generally.
Why the first plausible explanation was insufficient
The audit traced the symptoms to five distinct mechanisms. Fixing only one would have left the others in place.
- 1
Vocabulary contamination. Internal platform terms had entered a professional vocabulary prompt, and the model used them in the wrong domain.
- 2
Incomplete retrieval routing. The live-search trigger missed classes of research and product questions, leaving the model without the evidence those answers required.
- 3
A counterproductive response instruction. A mandatory conversational follow-up encouraged repetitive offers and unsupported elaboration; adding more phrasing machinery did not address the original instruction.
- 4
False speech accepted as input. A spurious transcription passed the available checks and became the premise of the conversation.
- 5
A misread intent. A question beginning “remind me” was mistaken for a request to create a reminder, rather than a request for information.
What changed, and what the evidence means
The repository records targeted changes to vocabulary filtering, research routing, response instructions, speech checks, and intent handling, with regression coverage for those paths. It also contains later work on source grounding, memory claims, and action honesty. The investigation became an ongoing engineering discipline rather than a claim that one patch had eliminated fabrication.
Source and test coverage show what has been implemented and what is being checked. They do not establish a current production error rate or prove that unscripted use is free of these failures. That requires fresh runtime evidence against the deployed version.
The lesson: trace a confident wrong answer to its cause, preserve the failure as a regression case, and keep implementation evidence separate from evidence that the user experience actually works.