The short version. Horus is a lead-vendor intelligence platform for insurance agencies. Its chat agent answers questions like "which vendor should get more budget next month" in plain English, off live production data, and answers them well enough to spend against. The analysis behind that is genuinely cumbersome: several marketplaces, a different export format from each, costs that change after the fact, and product tiers that are named differently everywhere. Horus does that reconciliation underneath, and every figure is calculated by the platform from a single definition rather than worked out by the model on the spot, so an answer only moves when the data moves and every recommendation comes back with the numbers and reasons behind it. A graded evaluation suite runs the real model before each release to keep it that way.
At a glance
- Sector: insurance lead buying, multi-tenant across agencies
- Problem: agencies buy leads from several marketplaces at once and could not tell which were actually profitable, because the data arrived inconsistent and the analysis needed to drive budget decisions
- Approach: an assisted import that proposes its own column mapping, a reconciled data layer, a deterministic metrics and recommendation engine, and a tool-calling agent on top that answers questions and explains the reasoning
- Tech: Next.js on Vercel, Postgres with row-level security, Vercel AI SDK, Claude Sonnet 5
- Outcome: live in production, multi-tenant across agencies; the analysis that used to be a monthly spreadsheet exercise is available whenever someone asks for it
The challenge: budget decisions made on unreliable arithmetic
An insurance agency buys leads from several marketplaces simultaneously. Each one sends data in its own shape, on its own schedule, with its own definition of what a lead is and what it cost.
That creates a problem that looks like reporting and is actually accounting. Before anyone can ask which vendor performs best, four things have to be true:
- Identity. Leads arrive more than once, across files and across months. Without a stable identity per lead, every re-import inflates the numbers.
- Cost. What a lead cost is not what the export says it cost. Returns, credits and billing lag mean the real figure is only known after the fact.
- Labels. The same product tier appears under different names in different exports, so like is not being compared with like.
- Definitions. Cost per lead is easy. Cost per sold household is the number that decides budget, and it has to mean exactly one thing everywhere it appears.
All four assume the data arrived in a usable shape, and it does not. Every month someone has to pull a fresh export from each marketplace portal, each portal names its columns whatever it likes, those names change between exports, and the person doing the upload is an agency operator rather than a data engineer.
Do this by hand and it is a spreadsheet exercise that takes days, has to be redone every month, and is wrong in a way nobody notices if a single step is skipped. Skip it and the ranking is confidently wrong: a vendor looks cheap because its credits have not landed yet.
Only then does the actual question become answerable: given all that, which vendors should get more money next month and which should be paused?
The people who need that answer are agency owners and their staff. They are not analysts, and they should not have to be.
The approach: solve the analysis once, then let people ask
What it does
Someone asks, in plain English, which vendors are worth more budget next month. They get a ranked answer with the reasoning shown, built on cost per sold household rather than cost per lead, and a straight read on whether the data behind it is complete enough to act on.
The same conversation covers the rest of the analysis an agency would otherwise commission: performance by vendor, by product tier and by lead kind, and geography down to county and ZIP, including where spend is being wasted.
Each answer is bounded honestly. If the current period is partial, or a vendor's data has not fully arrived, the answer says so instead of presenting a thin month as a finished one.
Getting the data in, whatever shape it arrives in
The upload is where the month's numbers are usually lost, because one column mapped to the wrong field makes everything downstream quietly wrong. So Horus does the mapping itself and asks the operator to confirm rather than compose.
The shape is simple: AI proposes the mappings, deterministic validation verifies them, and fallback rules ensure consistency when it cannot. Nothing the model suggests is trusted on its own, and an import that the model has no opinion about still completes correctly.
So the import is quicker and much harder to get wrong, and it stays deterministic. The model is a convenience layer over a strict importer, never in a position to make the import incorrect. The rules operators used to have to remember are in the platform too, including which fields each vendor requires and which exports are live transfers rather than data leads.
The layer underneath the agent
Most of the engineering is below the chat interface, and it is what makes the answers true.
Ingestion and reconciliation. Each marketplace gets its own parser for its own export format, and leads are deduplicated on a stable identity rather than a display field, so re-uploading a file or importing two exports that overlap by a fortnight does not inflate anything. Costs are reconciled before any analytics are calculated, because a single unreconciled source is enough to make a whole vendor look cheaper than it is.
One common taxonomy. Vendor-specific labels are normalised into a common taxonomy, so tiers compare across sources.
One definition per metric. Cost per sold household is calculated in exactly one place, and everything that reports it calls that. The agent's glossary is generated from the same source, so what the model tells a user a number means and what the system actually computed cannot drift apart.
A deterministic recommendation engine. Which vendors to scale and which to pause is decided by code, not model reasoning, evaluating historical performance and data quality. The agent presents that result and is explicitly instructed not to re-derive it. The same inputs produce the same recommendation every time, which is not a guarantee a reasoning model can make on its own.
Where the intelligence sits
Next to all of that, the agent's job is deliberately small: understand the question, pull the relevant parts of the engine, and explain the result in the terms the question was asked in. It performs no arithmetic of its own, so an answer is a property of the platform rather than of the model that happened to serve it, and the model can be upgraded without anything an agency sees changing.
That makes this a different shape from a text-to-SQL agent, which composes its own queries against the warehouse. Both are valid, and the choice follows the question space rather than taste. Here the questions are a known set and the difficulty is underneath them, so the analysis is encoded once and the agent reads it. Where users need to slice data in ways nobody can list in advance, generating SQL is the right call, and that architecture is covered in our text-to-SQL case study.
The agent layer
On top of that sits a single agent loop with a constrained tool set and deterministic guardrails around how it is allowed to answer. It may only cite values a tool returned, it will not recommend action or quote a rate off too little data, and it says so when the current period is thin rather than presenting it as finished. Each of those behaviours has an evaluation case behind it.
Tenant isolation is structural rather than behavioural. Which agency's data a request can see is bound outside anything the model can influence, so there is no parameter for it to get wrong and no injected instruction that can widen it, and the same boundary is enforced independently at the database.
Evaluation: knowing it still works next month
The evaluation suite runs the real production model against a seeded database and grades three things per case: whether the right tools were called in the right order, whether the answer contains what it should and avoids what it should not, and whether a separate model judges the answer grounded in the data actually returned.
That third check catches the failure mode everyone worries about. An answer can call the right tools and still drift into a claim the data does not support; a groundedness judge catches it before a user does.
The suite runs before agent releases, and it is what makes the system safe to change. Prompts get edited, models get upgraded, tools get added. Any one of those can quietly alter an answer that used to be right, and an analytics tool has no natural error signal: a wrong number looks exactly like a right one. We build an evaluation suite into every agentic system we ship for that reason, not just this one.
The same principle covered the rebuild. When the platform migrated from its earlier architecture to this one, the change was locked by byte-identical before-and-after output comparison, so it could be proven not to have altered a single number an agency sees.
What we delivered
Horus runs in production, multi-tenant across agencies, with the chat agent live on the current Claude model.
What changed for an agency is when the analysis exists. It used to be a monthly spreadsheet exercise, done by whoever had the patience for it and out of date within a week. Now it is there whenever someone asks, on data that reconciles itself on the way in, and it arrives with its reasoning attached, so a recommendation can be argued with rather than only accepted or ignored.
Where this approach applies
This generalises to any business where the numbers people act on have to be assembled rather than read: several sources, definitions that need agreeing, and a judgement step in the middle that decides what the data means. Lead buying, media spend, channel performance, supplier or route economics all have the same shape.
What decides whether the result is trusted is mostly not the agent:
- Do the accounting before the AI. If costs, identities and labels are not reconciled, a conversational interface just makes it faster to get the wrong number. The reconciliation is the project; the chat is the last mile.
- Put the model where it saves time, not where it decides. Horus uses a model twice: to propose an import mapping and to answer questions. Neither gets the final word; both are checked by deterministic code. That choice matters more than which model you pick.
- One definition per metric, in one place. Every metric that appears in an answer should resolve to a single implementation, and the model's own glossary should be generated from it rather than written alongside it.
- Encode the judgement. The step that turns numbers into a recommendation is domain logic. Put it in the system where it can be inspected and disagreed with, not in a prompt where it changes shape between answers.
- Be honest about thin data. Knowing when the current period is too partial to act on is worth more than any extra breakdown, because that is where confident, wrong advice comes from.
- Bind the tenant boundary outside the model. Isolation should be structural and enforced again at the database, never a parameter the model fills in.
- Evaluate the real system before every release. Not a spot check on a good day.
Whether the model composes its own queries or selects from a set the platform exposes is a smaller decision than any of these, and it falls out of the question space once the rest is done properly.
Frequently asked questions
What can an AI data analyst actually do for a business? Answer the questions that currently need someone to build a spreadsheet. In Horus that means vendor rankings, cost per sold household, performance by county and ZIP, product-tier breakdowns and budget recommendations, all asked in plain English and answered off current data. The work it removes is not the thinking, it is the assembly: reconciling sources, applying the definitions consistently and redoing it every month.
Can AI help with messy data imports? Yes, and it is one of the better places to use it, as long as it proposes rather than decides. A model can suggest how a file maps onto your fields far faster than a person hunting through a portal export, which is where import mistakes come from. The pattern that makes it safe: AI proposes the mappings, deterministic validation verifies them, and fallback rules ensure consistency when the model is unavailable or wrong. You get the speed without letting the import become non-deterministic.
Can an AI agent be trusted to report business metrics accurately? Yes, if it does not calculate them. Fix every metric definition in code and let the model retrieve the result and explain it. A definition that lives in a prompt will drift, because the model re-reads it fresh every time; one that lives in a query does not. In Horus the model has no arithmetic to do: it selects a query and explains what comes back.
What is the alternative to text-to-SQL for natural-language analytics? Constrained tool calling: the platform exposes a fixed set of parameterized queries and the model chooses which to call and with what filters, instead of composing SQL itself. Which one fits depends on where the difficulty in your data actually is. If it is query construction across a wide, open-ended schema, generate the SQL and guard it properly. If it is reconciliation, definitions and domain judgement, that work has to be built either way, and once it exists the agent is better off reading it than re-deriving it per question.
How do you stop an AI agent leaking one customer's data to another? Do not let the model address the boundary. Bind the tenant scope where the model cannot reach it, rather than passing it as an argument the model fills in, so there is no parameter to get wrong and no prompt injection that can widen it. Then enforce the same boundary independently at the database with row-level security.
How do you know an AI analyst's answers stay right over time? An evaluation suite that runs the real model, not a spot check. Horus grades every case on three axes: the tools called, what the answer does and does not say, and whether a separate model judges it grounded in the data returned. It runs before releases, so a prompt edit or a model upgrade that changes an answer shows up before a user acts on it.
Does an AI analyst replace a data team? No. It removes the queue for questions that already have a defined answer. Someone still has to decide what the metrics mean, reconcile the data, and encode the domain logic. That work does not disappear, it just stops being done one ticket at a time.
Build it properly
If you want people in your business asking questions of your data instead of waiting on a report, the work that decides whether it lands is underneath the chat: getting the sources reconciled, settling what each number means, and encoding the judgement that turns those numbers into a decision. Our guide to AI agent development for business covers how that work is scoped, and an AI roadmap is the cheapest way to find out what your data needs before committing a build budget.
Dot Square Lab builds production AI systems end to end, from ingestion and reconciliation to the agent and the evaluation suite that keeps it honest. Tell us your challenge and we will tell you what your data needs first, including when the answer is that you do not need an agent at all.
