Skip to content

fix: bypass Google cookie consent wall breaking all searches#108

Open
robertojrdev wants to merge 1 commit into
AWeirdDev:devfrom
robertojrdev:fix/google-consent-cookie
Open

fix: bypass Google cookie consent wall breaking all searches#108
robertojrdev wants to merge 1 commit into
AWeirdDev:devfrom
robertojrdev:fix/google-consent-cookie

Conversation

@robertojrdev

@robertojrdev robertojrdev commented May 7, 2026

Copy link
Copy Markdown

Summary

  • Pre-set the SOCS cookie on the primp Client before making the request to Google Flights. Google now shows a "Before you continue" cookie consent page before serving any content, causing all flight searches to fail with "No flights found".
  • Remove stray print(data) debug statement in parser.py that dumped the raw JSON payload to stdout on every search.

Problem

Google changed their consent flow — all requests to google.com/travel/flights now return a cookie consent wall instead of flight results. The primp client receives HTML containing "Before you continue to Google" instead of the expected flight data page. This breaks every search regardless of route, date, or configuration.

Fix

Setting the SOCS cookie tells Google that cookie consent has already been given. The consent wall is skipped and the actual flight results page is returned.

client.set_cookies(URL, {"SOCS": "CAISNQgD..."})

Testing

Before fix:

$ flight-search JFK LAX --date 2026-05-20
Error: No flights found. Check your airports and dates.

After fix:

$ flight-search JFK LAX --date 2026-05-20
Found 30 flights
  American | $235 | 7:04 -> 12:33 | 2 leg(s)
  Delta    | $246 | 8:00 -> 11:10 | 1 leg(s)
  ...

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Improved flight search result retrieval reliability and consistency by enhancing request handling for more dependable performance.
  • Chores

    • Removed debug code.

Google now shows a "Before you continue" cookie consent page before
serving any content. This causes all flight searches to fail with
"No flights found" since the HTML returned is the consent wall,
not flight results.

Fix: pre-set the SOCS cookie on the primp Client before making the
request. This tells Google that consent has been given, and the
actual flight results page is returned.

Also removes a stray `print(data)` debug statement in parser.py
that dumped the raw JSON payload to stdout on every search.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. bug Something isn't working labels May 7, 2026
@coderabbitai

coderabbitai Bot commented May 7, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR contains two targeted improvements to the flight-fetching system: a SOCS cookie is pre-set in the request handler to bypass Google's consent wall that blocks results, and a debug print statement is removed from the parser for cleaner output during extraction.

Changes

Flight Fetching Improvements

Layer / File(s) Summary
Request Cookie Setup
fast_flights/fetcher.py
The fetcher now pre-sets a SOCS cookie on the Google Flights URL before making requests, with explanatory comments about bypassing the consent wall.
Debug Statement Cleanup
fast_flights/parser.py
A print(data) debug statement is removed from parse_js, allowing the function to proceed directly from data extraction to JSON parsing.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 A cookie crumb placed just right,
Bypasses consent with all its might,
Debug prints fade to silent dreams,
Flight data flows in cleaner streams! ✈️

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately describes the main change: adding logic to bypass Google's cookie consent wall that was breaking searches. This directly matches the primary objective and the changes in fetcher.py.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
fast_flights/fetcher.py (1)

86-90: ⚡ Quick win

Harden consent-cookie handling to avoid repeat outages.

The hardcoded SOCS payload is fragile; if Google changes it, searches can silently fail again. Please move this value to a named constant/config override (env var) so it can be rotated without code edits.

Suggested change
+import os
+
 URL = "https://www.google.com/travel/flights"
+DEFAULT_SOCS_COOKIE = "CAISNQgDEitib3FfaWRlbnRpdHlmcm9udGVuZHVpc2VydmVyXzIwMjMwODE1LjA3X3AxGgJlbiACGgYIgJnPpwY"
+SOCS_COOKIE = os.getenv("FAST_FLIGHTS_SOCS_COOKIE", DEFAULT_SOCS_COOKIE)
...
-        client.set_cookies(URL, {"SOCS": "CAISNQgDEitib3FfaWRlbnRpdHlmcm9udGVuZHVpc2VydmVyXzIwMjMwODE1LjA3X3AxGgJlbiACGgYIgJnPpwY"})
+        client.set_cookies(URL, {"SOCS": SOCS_COOKIE})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@fast_flights/fetcher.py` around lines 86 - 90, The hardcoded SOCS cookie in
the call to client.set_cookies with URL is fragile; change it to read from a
named constant that can be overridden via an environment variable (e.g.,
FAST_FLIGHTS_SOCS or SOCS_COOKIE) so operators can rotate the payload without
code edits; update the code around client.set_cookies to load
process.env.FAST_FLIGHTS_SOCS (fall back to the existing literal only if env var
is absent), surface a clear log/warning when falling back, and ensure the
constant name is used wherever SOCS is referenced so the value can be changed
centrally or injected by config.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@fast_flights/fetcher.py`:
- Around line 86-90: The hardcoded SOCS cookie in the call to client.set_cookies
with URL is fragile; change it to read from a named constant that can be
overridden via an environment variable (e.g., FAST_FLIGHTS_SOCS or SOCS_COOKIE)
so operators can rotate the payload without code edits; update the code around
client.set_cookies to load process.env.FAST_FLIGHTS_SOCS (fall back to the
existing literal only if env var is absent), surface a clear log/warning when
falling back, and ensure the constant name is used wherever SOCS is referenced
so the value can be changed centrally or injected by config.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b4eb98b2-e76b-4d0a-a45a-0cdd9c0ae461

📥 Commits

Reviewing files that changed from the base of the PR and between 0138641 and bda1e2c.

📒 Files selected for processing (2)
  • fast_flights/fetcher.py
  • fast_flights/parser.py
💤 Files with no reviewable changes (1)
  • fast_flights/parser.py

thoughtpunch added a commit to thoughtpunch/flights that referenced this pull request May 25, 2026
Google now blocks requests behind a "Before you continue" consent wall,
causing the upstream parser to crash on the ErrorResponse payload it
gets back. Adapted from upstream PRs AWeirdDev#108 and AWeirdDev#105.

- preset SOCS cookie on the Google domain so Google skips the consent wall
- fall back to parsing and submitting the consent "Reject all" form if the
  wall still appears
- raise typed FlightsError on ErrorResponse / missing ds:1 script instead
  of crashing with JSONDecodeError
- drop debug print(data) from parser
- bump example.py date past today so it actually runs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant