MediaRaq — Engineering Summary¶
Author: Michael Inman · Updated: 2026-10-28 · Product: https://mediaraq.com · Studio: https://studio.mediaraq.com (sign-in required) · Architecture: Architecture Addendum
MediaRaq is the product. The engineering repository is
AI-Influencer-Factory, and its source code is private. This summary describes the system as of commit5d1094fonmaster(2026-10-26), and every status below is either verified in code and dated validation records or confirmed by the owner. Items marked ⏳ In flight are being built now.
Contents¶
- At a glance
- What MediaRaq is
- Agentic AI capabilities and their status
- How the work was decomposed
- AI-assisted engineering and human oversight
- Evaluation
- Validation and risk control
- Three scenarios
- Limitations and trade-offs
- Summary
At a glance¶
| What it is | A multi-tenant agentic AI platform. AI personas act as tool-using agents: they converse, remember, capture leads, propose appointments, and produce and publish content. |
| Who built it | One engineer (Michael Inman) as architect and operator, directing a fleet of AI coding agents under written, machine-checked rules. |
| Scale of the codebase | 18 backend services, 14 shared libraries, one web Studio; about 6,000 automated tests (5,168 Python, 800 TypeScript, 63 Rust); 269 database migrations; 25 versioned Kafka event topics. |
| Engineering trail | About 3,000 dated documents: about 740 gated handoffs, about 680 validation records and 23 architecture decision records. 3,585 commits since the repository baseline (2026-05-01). |
| Runs on | Docker Compose on a single cloud VM behind nginx and Cloudflare; Studio on Cloudflare; PostgreSQL with pgvector, Kafka, Redis, MinIO; OpenTelemetry, Loki, Tempo, Grafana, GlitchTip. |
| Models and providers | Anthropic Claude, OpenAI, Azure OpenAI, Google Gemini (language); fal.ai, HeyGen, ElevenLabs, Kling, Seedance and others (media). |
1. What MediaRaq is¶
MediaRaq lets a business run AI personas that behave like members of its marketing team. Each persona has a consistent face, voice, personality and history. It can talk to people on a public chat page and messaging channels, remember what they said, answer from the business's knowledge, capture a lead or propose an appointment, and produce short-form video, images and articles that it publishes to social platforms and the web.
The engineering problem underneath is making many agents trustworthy at once:
- each agent needs its own identity, tools, memory and model, without code per agent;
- retrieval has to be grounded, tenant-isolated and consent-aware;
- anything that touches the outside world (a CRM record, a calendar, a publication) needs a human in the loop until it has earned autonomy;
- model choice, cost and quality have to be measurable, so changes are adopted on evidence rather than impressions.
The rest of this document shows how each of those was built, and how far each one is in production.
2. Agentic AI capabilities and their status¶
Status key. Production: running for real tenants, with a dated live validation or production-enabled configuration. Production, approval-gated: live, but every outside action needs an operator's approval by design. Pilot / ⏳ In flight: built and tested, or being built, and not yet proven live end to end.
| Capability | How it works | Status |
|---|---|---|
| Tool-enabled persona agents | A factory compiles each persona into an agent definition: instructions from a compressed persona "kernel", a model-routing policy, a safety policy, per-persona tool grants and a memory scope. On each turn, decision tools run before generation (memory retrieval, context lookup, lead capture, appointment proposal), and their results feed the response. No code is written per persona. | Production |
| Retrieval-augmented generation (RAG) | Hybrid lexical + pgvector retrieval over four memory types: working (the current conversation), episodic (what this specific person said before, per channel), semantic (business knowledge chunks and persona memories) and procedural (how this persona handles situations). Retrieval is itself a tool call, and every result carries its source, revision and consent state, which are re-checked before it reaches the prompt. A background worker embeds new memories continuously. | Production since 2026-10-23; person memory by meaning 2026-10-24 |
| Context engineering | A context budget, rolling conversation summaries, a history policy, a memory manifest with provenance, prompt caching, and a hard per-turn cost ceiling. | Production |
| Model routing | Persona agents can run on Anthropic (default: Claude Haiku 4.5), OpenAI, Azure OpenAI or Gemini through a provider registry and a per-persona routing policy. The Image Prompt Engineer agent runs on Gemini (default), Claude or OpenAI. The operator Copilot plans with Anthropic or OpenAI. Media generation routes across providers using database-backed presets and quality lanes, with explicit, visible fallbacks. | Production |
| Shadow evaluation | The Image Prompt Engineer first ran in shadow: for every generation it recorded the current prompt, the agent's proposed prompt, the adoption decision, its tool trace and the prompting-knowledge sources it cited. After review it was promoted to active (2026-08-29). A reference-image selection policy is running in shadow now. | Production (one promoted, one in shadow) |
| Retrieval evaluation harness | A frozen case set with untouched holdouts, signed captures bound to the deployed commit, and a hard budget cap; lexical, vector and hybrid retrieval scored separately against release gates. See Evaluation. | Run in production |
| Operator Copilot | An agent for the business operator. It discovers 69 tools from MediaRaq's own MCP server, plans with an LLM, classifies write intent, shows a diff and asks for confirmation before any write, and keeps checkpoints and operator memory. | Production |
| Agents that build agents | The same MCP tools let the Copilot, Claude Desktop or ChatGPT create, clone and configure personas. A vision model proposes a persona's visual identity, and canon and profile are edited by tool. Every persona created this way is compiled into a new persona agent. An intake agent classifies imported productions (auto-accepts at ≥ 0.85 confidence, proposes below). | Production |
| CRM: EspoCRM | The persona agent proposes a lead. The operator approves it. MediaRaq then creates the EspoCRM Lead idempotently, with signed requests and a correlation ID. | Production, approval-gated |
| CRM: HighLevel | Per-persona HighLevel destinations; contact creation through HighLevel's MCP server using the Microsoft Agent Framework MCP client, behind an always-on approval gate. | Pilot |
| Appointment setting | The agent produces a structured appointment proposal, checking required fields and using an idempotency key, for operator approval. | Pilot: proposal and approval built; ⏳ calendar booking next |
| Content publishing | Article agent runs publish to WordPress. Instagram publishing has been proven live repeatedly. The YouTube publisher is proven; Google's OAuth app verification is pending. TikTok is in platform audit review. | WordPress and Instagram: production. YouTube and TikTok: gated by platform review. ⏳ HighLevel content publishing and X in flight. |
| Media production | Storyboard-driven Reels (keyframes, per-scene video, narration, composition), talking-head video, persona-consistent images, quality control before publishing. | Production |
3. How the work was decomposed¶
The platform was built in phases, each made of vertical slices that can be deployed and tested on their own:
| Phase | Outcome |
|---|---|
| 1–2 | Persona identity layer: canonical assets, consistency rules |
| 3A–3B | Closed loop: generate → review → publish to Instagram → ingest feedback |
| 4 | Talking-head video with persona continuity and a second provider |
| 5 | Multi-tenant Studio: Clerk identity, tenant memberships and roles, backend enforcement in every API and worker, a second-tenant isolation proof |
| 6–7 | Persona believability and evolution, monetization (billing, affiliate, lead routing), persona agents with memory, CRM and messaging channels |
Every slice follows the same path. A dated handoff describes the work. A script checks that handoff before any agent may start. An AI coding agent implements it in its own git worktree. A dated validation record lists the evidence, including what was not verified. The work is reviewed against the code and the live system before merge, and finally deployed through a guarded release. The trail is in the repository: about 740 handoffs and about 680 validations.
Architecture decisions are written down before big changes: 23 numbered ADRs. Examples are the service boundaries for media extraction, transcription, vision and synthesis; multi-destination lead routing; the storyboard specification; asynchronous scene media jobs with server-sent events; social publishing; and the public web architecture.
4. AI-assisted engineering and human oversight¶
MediaRaq was built by directing AI coding agents (Claude Code and Codex), with Claude also acting as architect and independent verifier. I set direction, made the design calls, ran every destructive or production operation, and reviewed the work before merge.
The control system¶
| Control | What it does |
|---|---|
AGENTS.md (500+ lines) |
The agents' operating rules: worktree safety, a do-not-touch list, secrets handling, an exception and observability standard, a no-polling realtime standard, deployment rules, and an explicit "never fake" rule: no placeholder providers, mocks or simulated success presented as delivered behavior. |
| Gated handoff template and checker | Every work order has fixed sections, from scope, out-of-scope and required discovery through testing, documentation and completion criteria. A script validates it before an agent may start. This is a quality gate on the AI's instructions, not only on its output. |
| Guard scripts | Scripts that fail the build if anyone reintroduces operator polling, silent discarding of agent output, or misconfigured exception reporting; a boundary-test runner. |
| Release gates | Guarded VM deploy (refuses dirty or non-fast-forward state); production capability checks before and after activation; a database image release gate; a verified backup before risky releases. |
| Session logs | Resume notes for every working session, so multi-day work restarts from evidence rather than memory. |
Where human judgment changed the outcome¶
| AI output or situation | What I decided | Why |
|---|---|---|
| Image prompt agent designed, but its output was being discarded by feature gates in some paths | Removed the discard gates; failures now fall back visibly, with provenance | Silent discarding hid whether the agent helped. Every image now records which prompt was used and why. |
| Retrieval looked good on recall alone | Held the public launch when the evaluation gate failed | Hybrid recall was perfect, but the system also "found" matches for questions that should have none. See Evaluation. |
| Two production defects passed unit tests | Acceptance requires real-service verification, not just passing tests | Both bugs got past tests because the test fakes returned the wrong data shape. They were caught in a live shakedown. |
| An agent could write directly to a customer's CRM | Every CRM, contact and appointment action is a proposal that needs approval | Outside writes have to earn autonomy. Approval is the default until there is evidence. |
| An automated deploy re-run with a degraded-provider override | The agent stopped; I approved it explicitly | Overrides on production releases are a human decision. |
Secure AI usage¶
- Secrets never pass through the AI. Agents receive secret names, never values. Credentials live in environment files on the server and in an encrypted vault for per-tenant integrations.
- Destructive operations, such as merges to the main branch, production deploys and data deletion, are run or approved by me.
- Tenant isolation is enforced in the backend from validated identity, never from client input. Retrieval queries are bound to the calling tenant and persona, and that binding is tested.
5. Evaluation¶
One concrete result. I built a retrieval evaluation harness and ran it against production on 2026-10-17:
- 45 frozen test cases, including untouched holdout cases, and approved "no-match" cases where the correct answer is to retrieve nothing.
- Every capture was cryptographically signed and bound to the deployed commit, so results can't be mixed across versions.
- A hard budget cap: the run cost $0.27 of a $5.00 cap.
- Lexical, exact-vector and hybrid retrieval were scored separately.
| Adapter and mode | Recall@5 (should find) | Specificity (should find nothing) |
|---|---|---|
| Knowledge, hybrid | 1.00 | 0.50 |
| Knowledge, exact vector | 0.33 | 1.00 |
| Persona memory, hybrid | 1.00 | 0.33 |
| Persona memory, exact vector | 0.76 | 1.00 |
Decision: no-go for public release. Hybrid retrieval found everything it should, but it also returned matches for questions that should have none, and pure vector search missed too much. I repaired the ranking and the distance cutoff, then verified on production with ten scripted end-to-end conversations: 10/10 passed, with the correct memory ranked first by meaning. In a separate live probe, asking each of three real visitors about their least significant memory ranked that memory first in 3 of 3 cases, which the previous significance-only ordering would not have done.
The evaluation's value was the gate: it stopped a release that looked fine on its headline metric.
6. Validation and risk control¶
Test layers¶
| Layer | What it covers |
|---|---|
| Unit and service tests (about 6,000) | Domain rules, agent decision tools, retrieval, routing, publishing state machines, billing, MCP tools |
| Real-database tests | Request-key races, scheduler end to end, migrations; run against disposable databases, never shared ones |
| Browser smoke tests (Playwright) | Studio flows locally and against the live site |
| Boundary and guard checks | Service boundaries, no-polling rule, no silent discard of agent output, exception backend configuration |
| Release checks | Critical baseline before release; production capability checks before and after activation; post-deploy observability verification |
| Live validations | Dated records of real-service proofs: publishing, CRM delivery, retrieval enablement, messaging channels |
Risk register (public summary)¶
| Risk | Guardrail |
|---|---|
| Cross-tenant data access | Tenant and persona bound from validated identity in every API, worker and retrieval query; a second-tenant isolation proof |
| An agent takes a wrong action in the outside world | All external writes are proposals that need operator approval; idempotency keys prevent duplicates |
| Ungrounded or stale answers | Retrieval re-checks each source's revision and consent; release gates on retrieval quality |
| Runaway model cost | Per-turn cost ceiling, prompt caching, budget-capped evaluations, embedding spend tracked per persona |
| Lost events between services | Transactional outbox with durable relays, dead-letter topics, retry budgets |
| Data loss | Scheduled backups, verified backups before risky releases, and a restore drill |
| Silent failures | Structured exceptions with correlation IDs, exception tracking, traces and logs, operator-visible status instead of polling |
7. Three scenarios¶
Greenfield: memory that understands meaning¶
Request. Personas should remember people and know the business, and recall the right thing even when the question uses different words. Design. Four memory types behind one retrieval tool; hybrid lexical + pgvector; consent and revision checks at retrieval time; a continuous embedding worker with per-tenant grants and a spend cap. Result. In production since 2026-10-23. Embedding the existing corpus cost about $0.0001. The evaluation harness gated the public launch and drove the cutoff repair.
Brownfield: adopting an AI agent without breaking the pipeline¶
Situation. Image generation already worked. An Image Prompt Engineer agent was meant to improve provider-specific prompts, but switching it on blindly risked degrading output. Approach. Run it in shadow first: generate as before, but record the agent's proposed prompt beside the one actually used, along with its reasoning trace and cited knowledge. Review the records, then promote it, with visible fallback and provenance on every image. Result. Active since 2026-08-29. The same shadow pattern is now evaluating a reference-image selection policy.
Ambiguous: "let the personas capture leads and book appointments"¶
The ambiguity. Should an AI write to a client's CRM on its own? Which CRM, since clients use different ones? Resolution. Actions became provider-neutral proposals, which the operator approves and then a provider-specific adapter executes: EspoCRM in production, HighLevel through MCP in pilot. Persona-to-CRM routing is data, not code, so one persona can promote for several clients.
8. Limitations and trade-offs¶
| Limitation | Impact | Path forward |
|---|---|---|
| Single VM for backend services | A host outage stops agents, generation and publishing until the host recovers | Managed PostgreSQL and Kafka, then a second host or container platform |
| Release gates run on the release path, not on every push | Feedback arrives at release time rather than per change | CI on every pull request running the existing baseline and guard scripts |
| Some agent actions are still approval-gated | Operators approve CRM, contact and appointment actions | Graduate actions to autonomy per persona once approval data shows they are reliable |
| Platform reviews gate some channels | YouTube and TikTok wait on Google and TikTok app reviews | Complete the platform verification processes |
| Decision | Benefit | Cost accepted |
|---|---|---|
| Persona agents compiled from data | New agents without code; consistent safety and memory | A compiler and schema to maintain |
| Hybrid retrieval with gates, not vector-only | Better recall with measurable specificity | Two retrieval paths to tune |
| Approval-first external actions | No unreviewed writes to customer systems | Operator time until actions graduate |
| Outbox + Kafka between services | No lost events; retries and replays | Eventual consistency (seconds) |
| Server-sent events instead of polling | Live operator feedback, lower load | Long-lived connections to manage |
9. Summary¶
MediaRaq is a working, multi-tenant agentic AI platform in production. It has tool-enabled persona agents with grounded, tenant-safe memory, model routing across four LLM providers, shadow evaluation before adopting new agents, a budget-capped retrieval evaluation that can stop a release, an operator Copilot built on MCP, and approval-gated actions into CRMs. It was built by one engineer directing AI coding agents through a written, checked and auditable process.
| Described commit | 5d1094f on master (2026-10-26) |
| Product | https://mediaraq.com · Studio https://studio.mediaraq.com |
| Architecture | Architecture Addendum · Roadmap |
| Signed | Michael Inman, 2026-10-28 |