Five AI providers debate one question and hand back a Decision Brief

Ask one AI model a hard question and you get one confident answer. You don't see where a second model would have pushed back, which assumption is carrying the whole argument, or what got left out. For a quick fact, that's fine. For a hiring call, a pricing change, or a build-or-buy decision, the disagreement is the part you most need to see.

The usual workaround is pasting the same question into three chat windows and comparing the answers by hand. That's slow, and it still leaves you doing the reconciling. Salvus founder John McIntosh wanted a tool that did the arguing for you and showed its work.

The AI Council is a web app, live at salvusai.app. You ask once. Models from OpenAI, Anthropic, Google, xAI, and Perplexity each answer independently, then read each other's answers and critique them. A final writer reads the whole exchange and produces a Decision Brief: the verdict, a confidence score, where the models agreed, where they split, the assumptions and risks, what information is missing, an action plan, and the dissenting opinion. A Disagreement Map lays out each contested claim and where every model stands on it.

There are three ways to run it. Open Debate lets the models take the question as themselves. Role Council gives each one a perspective, such as Devil's Advocate, Financial Analyst, or Skeptical Investor. Business Council loads presets for recurring decisions like hiring, product, legal and risk, and local growth. You can attach documents (PDFs, Word files, spreadsheets, CSVs, and more), and every model is told to work from them. Briefs export as a file that matches the on-screen brief exactly. There is also a multi-model chat, a public API, and a connector so other AI tools can call the council directly.

The front end is Next.js on Vercel. The council itself runs as Supabase edge functions, which hold the provider keys so they never reach the browser. Each run moves through four phases: independent answers, critique, synthesis, and a structured pass that builds the map and the brief fields.

Most of the engineering went into keeping that pipeline honest under load. Heavy five-model panels used to fail without a trace, because the time limits on individual calls could add up past what the platform allows. Now each run gets one shared time budget, handed out phase by phase. A model that runs long keeps the text it already wrote instead of erroring, and checkpoints mean a failure leaves partial results and a reason. Runs that come in through the API, where nobody is watching the live stream, ask each provider for a single response instead of a token stream. That cut the server processing each answer costs from about 91 milliseconds to under a tenth of one.

Every run records tokens and cost per model. When a check comparing the model catalog against the price table was finally written, it found nine wrong prices, some too high and some too low. That check now runs automatically on every change. A weekly job pulls the live model lists from four providers and reports what's new and what's been retired, and a scheduled routine acts on the report: it adds new models, retires dead ones with aliases so old runs still open, deploys, and confirms the new seat answers.

Before spending on marketing, John asked for an honest end-to-end review. The verdict: the core product was strong, but it was not ready for paid traffic. Subscription renewals never reset run quotas, picking a second plan could open a second paid subscription, and a refund could be claimed more than once. All of it was fixed the same night, along with private storage for chat attachments, alerting, in-house error tracking, and a checkout that repairs itself if a payment notice from the processor goes missing.

The app now watches itself. A probe checks the site and the council every five minutes, and the owner's dashboard shows uptime, latency, incidents, visitors, and signups, with no third-party analytics involved. Answers from five models arrive in parallel in about 28 seconds. When a provider ships a new model, the next weekly pass flags it and adds the ones worth carrying.

For Salvus clients, this is the same stack and the same habits we bring to their builds: real infrastructure, monitoring that tells the truth, and fixes that go after the root cause instead of the symptom.