A meeting notes bot that joins the right calls on its own

The client wanted the kind of AI notetaker most people have seen join a call, built in-house. Recordings, transcripts, and notes would live in the client's own database, on the same stack as its other internal tools, with access limited to company accounts.

Two things made that harder than it sounds. First, the corporate email scanner opens every link in an inbound message before the person does, so the usual click-to-confirm sign-up email arrives already spent. Second, a notetaker that depends on someone remembering to press record misses the meetings that matter most. The real ask was a bot that shows up on its own, and showing up on its own means reading a calendar full of things that are not really meetings.

Version one took a single session: a web app with three ways to get a meeting in. Record in the browser from the microphone, optionally mixed with a Teams or Zoom tab's audio. Upload any audio or video file up to 500MB. Or import a Teams or Zoom transcript file, with speaker names carried through and no transcription needed.

Deepgram transcribes with speaker separation. Claude then writes structured notes: an overview, key points, decisions, action items with owners and due dates, open questions, topic chapters, and a draft recap email. The meeting page plays the audio alongside a click-to-seek transcript. People rename a speaker once and every line updates, tick off action items, ask questions about the meeting, and search across every meeting they can see.

Sign-up is limited to company email domains and confirmed on the server, so no confirmation link has to survive the email scanner. Recordings are private, every table has row-level security, and playback links expire in an hour.

Version two added a bot that joins Teams, Zoom, or Meet on its own and feeds the same notes pipeline.

A bot that sits in a call for an hour holding a live audio stream is the opposite of serverless, so the work split in two. The bot runs on a small always-on server using Vexa, an open-source meeting bot. Everything downstream stays in the existing app. The server checks in every 30 seconds instead of trusting the bot's own notifications, which are unreliable upstream, and every step is safe to run twice.

Then the calendar turned out to be the hard part.

• Outlook puts a Teams link on almost everything. Lunches, in-person meetings, and focus blocks all carry one, so the bot kept joining events nobody would open. A calendar gate now stands those down before they fire: cancelled, marked free, in-person, or already attended. Retries dropped to three in 20 minutes, because in practice a retry never rescued a real meeting.
• The bot was reading the wrong calendar. Tracing one missed meeting showed the bot was following a different person's feed than the one the app's rules checked. The app now pushes the owner's feed to the bot server directly, and only trusts its own rules when the two agree.
• Bots sent from the calendar were not recording. The bot server's own scheduler sent them with recording switched off by default. The app now sends every bot itself, from two minutes before the start, with recording on.

Then the owner got the final say. An Upcoming page lists the next two weeks of meetings and what the bot will do for each, in plain words: "Bot joins at 3:58 PM," or "Skipped automatically. This is an in-person meeting." Every meeting has a switch that outranks every automatic rule, and a Send now button appears on a meeting already underway.

The bot joins real meetings in production, and its transcripts land in the same notes pipeline as uploads. One meeting the bot attended while calendar recording was still broken was recovered from its live transcript: 387 lines after removing duplicates, rebuilt into notes with 13 key points and 11 action items.

The lesson that shaped the build is that transcription is close to a solved problem. The edge cases live in scheduling, permissions, and email security, and they only gave way once we traced each failure through the actual records instead of retrying it. Transcription was estimated at roughly 26 cents per meeting hour at the time of the build, and the manual transcript import stays in place as the fallback that never needs IT.

Read the full build notes: An in-house meeting notes bot, and why the calendar was the hard part.