Instalily Project_Vedant: Lead Generation Prototype and MVP

This project automates the process of identifying relevant trade shows, extracting company data, enriching it with key stakeholder information, and generating personalized outreach emails. By integrating multiple APIs (SERP API, OpenAI API, and Apollo AI API), the pipeline produces a series of JSON outputs at various stages, with a final enriched leads file that contains all relevant data, culminating in a personalized email for each lead. The results are displayed in a minimal HTML frontend for the MVP.

Features

• Trade Show Identification: Automatically identifies relevant trade shows and expos to target using the SERP API.
• Company Identification: Uses OpenAI API to parse trade show websites and extract companies attending these events.
• Lead Enrichment: Utilizes Apollo AI API to retrieve relevant stakeholder information for the identified companies (in the code, we run it as an alternative scenario, our priority remains OpenAI for the MVP).
• Personalized Outreach: Generates tailored emails to qualified leads, incorporating details from the enriched data.
• Interactive Dashboard: Displays the results in a local HTML dashboard with a minimal frontend.
• Scalable Design: Future-proof for improvements and further API integrations, such as LinkedIn Sales Navigator.

How It Works

Step-by-Step Execution:

1. Discover Trade Shows (discover_tradeshows.py)

   • The first step in the pipeline is to identify trade shows, expos, and conferences relevant to your product using the SERP API.
   • This generates trade_shows.json, a list of the identified trade shows you should target.

2. Extract Company Data (data_extractor.py)

   • After identifying the trade shows, this step crawls through the websites of each show and extracts relevant company data.
   • This process generates JSON files like exhibitors_signexpo_org.json, exhibitors_sac_ace_ca.json, and others, each containing the companies attending specific trade shows.

3. Combine Extracted Data (combine_json.py)

   • Once the company data has been extracted from all trade show websites, this script combines the data into one unified JSON file, combined_exhibitors.json, which contains the full list of exhibitors.

4. Retrieve Company Websites (json_website.py)

   • This script takes the company names from the combined data and attempts to find their official websites, saving them in websites.json.

5. Extract Qualified Leads (lead_extractor.py)

   • This step identifies the qualified leads for outreach by analyzing the company data and their corresponding websites.
   • It creates the websites_with_leads.json, which contains the leads and their designations within the company.

6. Apollo Lead Fetch (apollo_email_fetch.py) (Optional)

   • This script suggests an alternative route for finding more leads using Apollo AI API by retrieving additional information about decision-makers.

7. Final Aggregation (final_step.py)

   • The final step consolidates all the data, including leads, their designations, and their personalized emails, into enriched_leads.json.

Dashboard (HTML Visualization)

The results are visualized through a minimal HTML dashboard. The dashboard displays:

• Trade Show Information: Details of the trade shows targeted.
• Company Data: Information about companies attending the trade shows.
• Qualified Leads: The final list of qualified leads, their roles, and the generated outreach emails.
• The dashboard is launched by opening the index.html file located in the output folder.

Output Files

1. trade_shows.json: Contains the list of relevant trade shows.
2. combined_exhibitors.json: Combines all extracted data from the trade show websites.
3. websites.json: Contains the websites of the extracted companies.
4. websites_with_leads.json: Contains qualified leads and their designations.
5. enriched_leads.json: The final output that includes all data, including the personalized outreach emails for each lead.

---

Installation and Setup

Clone the Repository

git clone https://github.com/your-repo/Instalily_Project_Vedant.git
cd Instalily_Project_Vedant

Install Required Dependencies

pip install -r requirements.txt

Add API Keys

Ensure you have your Apollo AI API and OpenAI API keys and add them to a .env file in the project root or directly in the script.

---

Running the Pipeline

Step 1: Extract Trade Show Data

python src/discover_tradeshows.py

Step 2: Extract Company Data from Trade Show Websites

python src/data_extractor.py

Step 3: Combine Extracted Data

python src/combine_json.py

Step 4: Retrieve Company Websites

python src/json_website.py

Step 5: Extract Qualified Leads

python src/lead_extractor.py

Step 6: Fetch Leads from Apollo API (Optional)

python src/apollo_email_fetch.py

Step 7: Final Aggregation

python src/final_step.py

Step 8: View the Results

Go to the output folder and click on the index.html file to view the results dashboard in your browser.

---

Insights from the Project

• Data Pipeline: The project utilizes an efficient API-driven pipeline to automatically extract, enrich, and prioritize leads, saving valuable time and reducing manual errors.
• Cost Efficiency: By leveraging the power of the Apollo and OpenAI APIs, the pipeline delivers qualified leads at a cost-efficient rate, which is currently under 50 cents per lead.
• Scalability: The solution is designed to scale, allowing future integrations with additional APIs and incorporating real-time web scraping for continuous updates on trade shows and leads.
• User-Friendly Interface: The minimal HTML dashboard provides an intuitive way to visualize the entire lead generation process, from trade show identification to email generation, helping sales teams make informed decisions.

---

License

This project is private and contains my paid APIs and private tokens.

---

Contact

For any questions or further support, feel free to reach out:

Email: vma2130@columbia.edu

---

If you have any questions or need further clarification on any part of the project, feel free to ask. More than happy to help!