B2B Lead Generation Crawler — 1,300+ Organizations, Contacts & Reports
1,300+
Nonprofit sites crawled
5
Data points per organization
Auto-synced
Straight to Google Sheets
What I built
Built a lead-generation pipeline that visits each nonprofit's website and pulls out five key details in one pass — organization name, CRA business number, contact email, latest annual report, and latest audited financial statement — ready to use for outreach or research.
Paginated Imagine Canada's public directory of accredited organizations to automatically build the full list of 1,300+ organization websites to crawl, instead of relying on a manually maintained list.
Cracked Cloudflare's email obfuscation trick — many sites hide contact emails behind a scrambled data-cfemail code instead of showing them in plain text — by writing a small decoder that reverses the XOR encoding and recovers the real address.
Layered four fallback strategies for finding a contact email — footer mailto links, decoded Cloudflare-protected emails, a full-page regex scan, and a 'Contact Us' page fallback — so an email is still found even when a site hides or buries it.
Matched each organization's most recent Annual Report and Audited Financial Statement PDFs by keyword and filename, always keeping the newest year (2023–2026) when multiple years were linked, and downloaded every file in safe, memory-friendly chunks with automatic retries.
Wrote every result straight into a shared Google Sheet through the Google Sheets API, with report PDFs saved to disk in folders named after each organization's business number — turning hours of manual research per organization into one unattended batch run across 1,300+ sites.
Stack
Want something like this built for your product?