LembreMe Copilot
Real time meeting copilot, in a desktop app with an overlay on the call
Lead author, architecture and development
Private repository
About the project
The Copilot follows the meeting as it happens. It transcribes speech live, identifies what deserves attention (questions directed at you, key points, suggestions of what to say), and separates out what turned into a commitment: tasks, reminders, expenses, and decisions. Detected questions can be answered automatically by the model, and an executive summary comes out at the end of the call. It's a desktop app built with Electron, not a browser tab: besides the full panel with history and insights, there's a floating overlay that sits on top of the video call window, in the corner of the screen, so you can follow along without switching context. Detected items can be saved directly to the main LembreMe, closing the loop between what was agreed on in the meeting and the assistant that follows up on it afterward.
How it works
Captured audio is streamed over a dedicated WebSocket to the backend, which keeps a second channel just for broadcasting state and insights to the interface. Transcription uses Deepgram Nova-2, with local faster-whisper as a fallback when the service is unavailable. The text goes through a pipeline of specialized modules: an insight detector, an actionable item detector, a temporal normalizer (which resolves expressions like "next Tuesday" using chrono-node in Portuguese), an intent deduplicator so the same commitment stated in different ways isn't repeated, and a context summarizer that keeps track of the conversation thread. When a question is directed at the user, the answer module queries the model with the accumulated context and returns a suggestion in the overlay. Sessions, insights, and items are stored in Supabase, with optional Redis for caching and a fallback to local memory when it's unavailable.
Architecture
- 01
Electron desktop app with a floating overlay on the call
- 02
React frontend with a dashboard panel, history, insights, and summary
- 03
Node.js and Express backend with two WebSocket channels: audio and state
- 04
AI pipeline: insight detector, item detector, intent dedupe, and summarizer
- 05
Deepgram Nova-2 as primary transcription and faster-whisper as a local fallback
- 06
Supabase for sessions, authentication, and storage
- 07
Optional Redis, with a fallback to in memory cache
Features
- Real time meeting transcription
- Detection of questions directed at you, key points, and speaking suggestions
- Extraction of tasks, reminders, expenses, and decisions with deduplication
- Automatic answers to questions using the conversation's context
- Executive summary generated at the end of the call
- Floating overlay in a desktop app, on top of the video call
- Normalization of dates spoken in Portuguese
- Sending detected items to the main LembreMe
Tech stack
- Frontend
- TypeScript · React · Electron · Vite · Tailwind CSS
- Backend
- Node.js · Express · WebSocket
- AI/ML
- Deepgram Nova-2 · faster-whisper · GPT-4o-mini
- Tools
- chrono-node · Vitest
- Database
- Supabase
- Infrastructure
- Redis