LembreMe
Personal assistant that runs entirely on WhatsApp: finances, tasks, reminders, goals, and routines in natural language
Lead author, architecture and development
Private repository
About the project
LembreMe is a personal assistant that operates entirely through WhatsApp. The user types, sends audio, or photographs a receipt, and the agent interprets, records, and returns what's needed: expenses and income (including fixed and installment ones), tasks, reminders, financial and personal goals, notes, and nearby restaurant searches by GPS or city. The core is a ReAct agent in Python built with LangChain, with tools separated by domain and data access isolated through the Repository Pattern. An independent scheduler sweeps reminders and deadlines every 60 seconds and fires priority based notifications; on the Ultra plan, unfinished critical tasks trigger an automatic phone call through Vonage's voice API. The product has matured into a SaaS with Free, Premium, and Ultra plans and per feature limits. Among the most recent additions are proactive routines, which detect patterns such as spikes in a spending category and act without waiting for a command, and messaging and scheduling to third parties, built with explicit opt-in, an allowlist, and an anti-ban cap.
How it works
The Node.js gateway built with whatsapp-web.js receives the message and normalizes the input: audio is transcribed by Whisper or Google, images go through OCR with Tesseract, and the resulting text is forwarded over HTTP to the Python agent. The FastAPI agent runs the ReAct loop with LangChain on top of GPT-4o-mini, with Gemini 1.5 Flash as a fallback, picks the tools for the right domain (finances, tasks, reminders, goals, notes, restaurants), and persists everything to Supabase, with Redis holding cache, TTL, and state shared across Gunicorn workers. In parallel, the APScheduler based scheduler checks reminders and upcoming deadlines, and weekly financial reports are generated automatically with pandas and matplotlib. Travel searches use custom scrapers built with Playwright and Camoufox over Airbnb, Google Hotels, Google Flights, and Kayak, run in parallel.
Architecture
- 01
Node.js gateway with whatsapp-web.js: channel, audio transcription, and image OCR
- 02
Python FastAPI agent: ReAct loop with LangChain and domain based tools
- 03
Repository Pattern: data access isolated from the agent's logic
- 04
APScheduler based scheduler: reminders, deadlines, and priority notifications
- 05
Redis: cache, TTL, and state shared across Gunicorn workers
- 06
Playwright and Camoufox scrapers: parallel flight and lodging searches
- 07
Vonage Voice: automatic calls for critical tasks on the Ultra plan
- 08
Supabase PostgreSQL and Docker Compose with four containers on AWS EC2
Features
- Full assistant inside WhatsApp, by text, audio, or photo
- Finances with fixed and installment expenses, plus an automatic weekly report
- Tasks and reminders with a continuous scheduler and priority based notifications
- Financial and personal goals with progress tracking and automatic completion
- Proactive routines that react to spending patterns without waiting for a command
- Flight and lodging search through parallel scraping
- Automatic voice call for critical tasks on the Ultra plan
- Messaging to third parties with opt-in, allowlist, and anti-ban cap
- Free, Premium, and Ultra plans with per feature limits
Tech stack
- Backend
- Python · FastAPI · Node.js · whatsapp-web.js
- AI/ML
- LangChain · GPT-4o-mini · Gemini 1.5 Flash · Whisper · Tesseract OCR
- Tools
- Playwright · Vonage Voice
- Database
- Supabase
- Infrastructure
- Redis · Docker · AWS EC2
- Data
- pandas