🚀 Intuit QuickBooks ProAdvisor Data Scraper

A robust, 2-stage hybrid web scraper designed to extract USA QuickBooks ProAdvisor contact information while bypassing complex anti-bot protection (Akamai / reCAPTCHA).

🛠️ Technical Highlights

• Stage 1 (GraphQL API): Queries Intuit's matchmaking API directly using requests for fast base-data extraction (Name, Address, Tier, Search ID).
• Stage 2 (Stealth Playwright Engine): Launches a persistent browser context to extract dynamic fields (Phone numbers, external Websites) hidden behind UI toggles.
• Async CAPTCHA Handling: Uses an asynchronous event loop (asyncio.sleep) to prevent thread freezes, allowing manual/automated reCAPTCHA solving without losing session state or triggering network timeouts.
• Deduplication Engine: Normalizes and parses raw text blocks using Regex to produce clean, CRM-ready phone number strings.

---

⚡ Getting Started

Prerequisites

• Python 3.8+
• Node.js (required for Playwright browser binaries)

🚀 Installation

1. Clone the repository:

git clone https://github.com/MilanGole/proadvisor-intuit-scraper.git
cd proadvisor-intuit-scraper

2. Set up a virtual environment:

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt
playwright install

💻 Usage

Run the main scraper script:

python proadvisor_scraper.py

🔄 Script Execution Flow

1. The script queries the GraphQL API to retrieve profile metadata.

2. It opens a Playwright browser instance to scrape phone numbers and external links.

3. CAPTCHA handling: If reCAPTCHA appears, solve it directly in the open browser window. The script automatically detects completion and resumes scraping.

4. Results are saved automatically to Brooklyn_10_ProAdvisors_Complete.csv.

📦 Output Schema

Data is saved in CSV format with the following structure:

Column	Description
Name	Full name of the ProAdvisor
Company	Business / practice name
Address	Street, locality, or ZIP address
City	City location
State	US State code
Phone	Cleaned and deduplicated phone numbers
Email	Contact email address (N/A if unavailable)
Website	External business website URL
Tier	QuickBooks certifications held (QBO, QBDT, etc.)
Profile Link	Direct link to the ProAdvisor profile