fix(search): restore search via the public page's tfs parameter (#223) - #230
fix(search): restore search via the public page's tfs parameter (#223)#230Princeu3 wants to merge 7 commits into
Conversation
…tarani#223) Google's FlightsFrontendService RPC endpoints have required an x-goog-batchexecute-bgr header since early August. The page's own JavaScript signs it over the exact request bytes, so a captured token cannot be replayed against a different body. Both GetShoppingResults and GetCalendarGraph now answer HTTP 200 with a payload-less wrb.fr row carrying error 13, which parse_first_wrb_payload reads as None and the CLI reports as "No flights found" — indistinguishable from a route with no service. Every route returns count: 0. The public /travel/flights page is not gated that way. It embeds the same result payload in an AF_initDataCallback blob keyed ds:1, whose elements [2] and [3] hold exactly the flight rows the RPC returned, so the decoders, models, CLI and MCP surface all keep working unchanged. Only the transport moves. The page takes the tfs protobuf parameter rather than the f.req JSON struct. build_tfs_token already encoded tfs for booking deep links, so its field layout is now shared: encode_tfs_segment and encode_tfs_payload serve both builders, and build_tfs_token stays byte-identical (its captured-token test still passes). Date search loses its one-call grid — the page carries price history, not a forward calendar — so each date is priced by its own page fetch, run concurrently under the existing rate limiter. An eight-day sweep takes about 1.5 seconds. Filters split three ways. Trip type, dates, airports, stop ceiling, cabin and passengers go into the tfs. Airline include/exclude, price cap, max duration and departure window are applied to decoded results, so a filtered search returns fewer options than the RPC did but every option honours the filter. Alliance, emissions, bags and basic-economy have no page equivalent and are logged as ignored rather than silently dropped. Two details cost real debugging time and are covered by tests: - The stop ceiling in field 5 is zero-based, and MaxStops.ANY must omit the field. Writing 0 for "any" pins every search to non-stop. - Selected legs pin a search only in field 4. Google ignores an unknown field number and re-serves the outbound board, so putting them anywhere else makes round-trip expansion pair outbound flights with other outbound flights without erroring. _wire and exceptions also gain SearchRejectedError, so the RPC path that get_booking_options still uses reports a hard block instead of an empty list. Live JFK->LAX checks: 32 one-way options cheapest $204, 38 round-trip itineraries cheapest $357, non-stop filter returns only direct flights, business cabin and EUR pricing both shift as expected. Prices cross-check against an independent Google Flights source. The offline suite passes (433). In tests/search, which hits the live API, no test fails that did not already fail on main, and 32 more pass.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| _iata(segment.departure_airport[0][0]), | ||
| _iata(segment.arrival_airport[0][0]), |
There was a problem hiding this comment.
When a public API or MCP search supplies multiple origin or destination airports, build_tfs serializes only entry zero, causing routes involving every subsequent requested airport to be silently omitted from flight and date results.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: fli/search/_tfs.py
Line: 109-110
Comment:
**Multi-airport choices are truncated**
When a public API or MCP search supplies multiple origin or destination airports, `build_tfs` serializes only entry zero, causing routes involving every subsequent requested airport to be silently omitted from flight and date results.
**Knowledge Base Used:**
- [Google Flights request and result models](https://app.greptile.com/punit-s-org/-/custom-context/knowledge-base/punitarani/fli/-/docs/google-flights-models.md)
- [Flight and date search workflows](https://app.greptile.com/punit-s-org/-/custom-context/knowledge-base/punitarani/fli/-/docs/flight-search-workflows.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
| continue | ||
| if max_price is not None and flight.price and flight.price > max_price: | ||
| continue | ||
| if not _within_window(flight, windows.get(0)): |
There was a problem hiding this comment.
When a round-trip or multi-city search restricts a return or later segment, every expansion is filtered with windows.get(0) instead of that segment's window, causing out-of-window flights to remain and valid flights to be discarded against an unrelated outbound window.
Knowledge Base Used:
Prompt To Fix With AI
This is a comment left during a code review.
Path: fli/search/_tfs.py
Line: 205
Comment:
**Later-segment windows are misapplied**
When a round-trip or multi-city search restricts a return or later segment, every expansion is filtered with `windows.get(0)` instead of that segment's window, causing out-of-window flights to remain and valid flights to be discarded against an unrelated outbound window.
**Knowledge Base Used:**
- [Flight and date search workflows](https://app.greptile.com/punit-s-org/-/custom-context/knowledge-base/punitarani/fli/-/docs/flight-search-workflows.md)
- [Google Flights request and result models](https://app.greptile.com/punit-s-org/-/custom-context/knowledge-base/punitarani/fli/-/docs/google-flights-models.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| """ | ||
|
|
||
|
|
||
| def _sort_key(sort_by: SortBy): |
There was a problem hiding this comment.
_sort_key omits its return annotation, while the new helpers and parameterized test in tests/search/test_tfs.py also leave parameters or return values untyped, preventing type checkers from fully validating these new boundaries.
Context Used: CLAUDE.md (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: fli/search/flights.py
Line: 52
Comment:
**New helpers lack full annotations**
`_sort_key` omits its return annotation, while the new helpers and parameterized test in `tests/search/test_tfs.py` also leave parameters or return values untyped, preventing type checkers from fully validating these new boundaries.
**Context Used:** CLAUDE.md ([source](https://github.com/punitarani/fli/blob/main/CLAUDE.md))
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
I validated this transport live and opened Princeu3#1 with focused fixes for the current review findings:
Live validation returned all four GRU/CGH -> GIG/SDU combinations. The follow-up intentionally does not claim |
Two defects in the page transport, both raised on punitarani#230. Neither raised an error, which is what made them worth tests rather than one-line edits. build_tfs serialized only entry zero of departure_airport and arrival_airport. Fields 13 and 14 are repeated, so a caller asking for BOM,DEL,AMD -> ORD,DTW was silently searching BOM -> ORD alone. The truncated request is still valid and still returns fares, so the missing seventeen city pairs looked like a route with nothing on it. Live, the same query now returns options across five distinct pairs instead of one. apply_client_side_filters read the time window from segment zero for every call. _expand_multi_leg pins a segment by setting selected_flight and re-fetches, so the rows coming back belong to the first segment that is still unpinned. A round trip with a morning outbound window and an evening return window filtered the return candidates against the morning window: valid evening returns dropped, invalid ones kept. The window now follows the segment actually being chosen, falling back to the last segment once everything is pinned. _sort_key gains its return annotation, and the test helpers gain theirs. Tests: three regressions that fail against the previous behaviour and pass against this one, plus a fourth pinning the pre-expansion case so the window fix cannot regress the simple path. The multi-airport case is also checked byte-for-byte against a tfs Google issued for that query. Offline suite 433 passed, search unit tests 130 passed, ruff clean. The browser transport proposed alongside these fixes is deliberately not included: it belongs in its own change, against upstream, without the Chromium --no-sandbox default.
|
Both P1 findings were real and are now fixed in Multi-airport truncation. Later-segment windows. Annotations. Each fix ships with a regression test that fails against the previous behaviour and passes against this one — I verified that by reintroducing both bugs and watching the three tests go red. A fourth test pins the pre-expansion case so the window fix cannot regress the simple path, and the multi-airport encoding is checked byte-for-byte against a Offline suite 433 passed, search unit tests 130 passed, @regiscaio — thank you for Princeu3#1, and for validating the transport independently on GRU/CGH -> GIG/SDU. Your diagnosis on both bugs matched mine and your fix for the window is the same shape I landed. I did not take the branch wholesale, for one reason: it also adds That work deserves its own review rather than riding along inside a search-transport fix. Would you open it against upstream as a separate PR, without the |
|
Validated this branch ( SymptomStraight off the branch, every search failed: Your error message called it correctly. Probing from curl_cffi import requests as r
resp = r.get("https://www.google.com/travel/flights?hl=en&gl=US&curr=USD",
impersonate="chrome", allow_redirects=True)
# status 200, 656,053 chars
# final url: https://consent.google.com/m?continue=...&gl=DE&m=0&pc=flt&cm=2&hl=en&src=1
# 'ds:1' in text -> FalseNote FixA pre-accepted
The legacy Adding it to DEFAULT_HEADERS = {
"content-type": "application/x-www-form-urlencoded;charset=UTF-8",
# EU/EEA IPs get redirected to consent.google.com, which carries no
# ds:1 payload. A pre-accepted SOCS cookie skips the interstitial.
"cookie": os.environ.get("FLI_GOOGLE_COOKIE", "SOCS=CAISNQgQEitib3FfaWRlbnRpdHlmcm9udGVuZHVpc2VydmVyXzIwMjQwMzE3LjA5X3AwGgJlbiADGgYIgLC_rwY"),
}Results from the same three queries that returned
Happy to open this as a PR against your branch if you'd like it in — it's a six-line change, and an env override ( Unrelated: test-suite date bit-rot
|
EU/EEA IPs are redirected to consent.google.com, which returns HTTP 200 with its own AF_initDataCallback payload and no ds:1 blob, so every search fails to parse. Google geolocates the request IP and overrides gl=US, so the locale params cannot avoid it. A pre-accepted SOCS cookie skips the interstitial. The cookie goes in the session jar scoped to .google.com rather than a static Cookie header, so cookies Google sets later still apply. FLI_SOCS_COOKIE overrides the value when Google rotates it; setting it empty sends no cookie. Reported and diagnosed by @stephanhof, who tested from a German IP and established that the legacy CONSENT cookie and a truncated SOCS value are both ignored — only the full-length SOCS value works. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DateSearchFilters.duration is optional and its field validator does not run on the default, so round-trip filters reach the search with it unset. _price_one_date then built the return date from None and raised TypeError: unsupported type for timedelta days component: NoneType, taking the whole sweep down. The gap between the two segments carries the same information — the filter model already requires the two to agree when duration is set — so fall back to it. Regression test fails against the previous behaviour. Also marks the infant-passenger live tests xfail: Google's page inlines no results for some searches carrying infants, while the same query with adults and children returns rows. Non-strict, so a fix on Google's side reports as an unexpected pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
These four filters were listed as unsupported: the search ran without them and only logged a warning, so an alliance-filtered search returned whatever the route offered. Probing Google's own requests shows the segment message carries them: field 6 carrier include, repeated — IATA codes and alliance names field 7 carrier exclude, repeated — same values field 15 layover airport include, repeated field 17 min layover minutes field 18 max layover minutes Verified live: JFK->FRA with STAR_ALLIANCE returns LH/LX/SQ/TK/TP only; field 7 drops those same carriers; BUF->ATH with min 120 moves the shortest layover from 50 to 125 minutes; field 15 confines layovers to the named airports. alliances, alliances_exclude and layover_restrictions leave the unsupported list. Unset filters change no bytes — the captured Google tfs fixtures still match byte-for-byte. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four separate failures, 36 tests in total: - test_search_flights started a patch on the shared client singleton and never stopped it, so every later test that hit the network got its canned "not-a-number" body — 13 unrelated failures, and the count depended on test order. Now uses monkeypatch, which undoes itself. - test_parallel_search and test_booking_options fed the client GetShoppingResults wire bodies; the client reads a ds:1 page blob now. They serve the captured fixtures as pages instead. The date-chunk assertions also still described the calendar RPC's one call per chunk — a sweep costs one page fetch per date now, so they assert that. - Hardcoded travel dates in test_booking_options and test_parallel_search had gone past and tripped the "date cannot be in the past" validator. Both files now build dates relative to today. Reported by @stephanhof. - The booking-options live test is marked xfail: GetBookingResults still needs the browser-signed bgr header (punitarani#223), which this PR does not claim to fix. Suite goes from 742 passed / 36 failed / 4 errors to 785 passed, 5 xfailed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@stephanhof — thank you, this was a precise report and it landed as written. The cookie-variant table saved the work: I would have tried Consent fixIn on if SOCS_COOKIE:
session.cookies.set("SOCS", SOCS_COOKIE, domain=".google.com")A static I could not reproduce the interstitial from a US IP, so the fix is verified structurally, not end-to-end. If you have a minute to re-run your five queries from Germany against the branch, that would close it properly. The 36 failuresYour date note prompted an audit, and the count splits four ways rather than one. 18 were the hardcoded dates you named. 13 were a test bug:
One more gap the live tests exposedAlliance and layover filters were listed as unsupported, so a search set them, logged a warning, and returned whatever the route offered. Google's Left openBoth marked xfail rather than hidden:
Suite is 785 passed, 5 xfailed, ruff clean. |
|
Verified this restores search here (macOS, Python 3.12, editable install of One thing worth fixing before merge: multi-city silently returns the wrong itinerary.
is_one_way=filters.trip_type != TripType.ROUND_TRIP,
For
Opening the exact URL Setting field 19 = 3 does make the browser render the correct multi-city board, but it does not fix the library, because that page carries no rows to read:
So multi-city results look to be fetched client-side through the still-gated RPC, and the page transport cannot serve them at all. Suggested resolution: encode 3 for |
build_tfs encoded multi-city as one-way (field 19 = 2), so Google ignored every segment past the first and served that leg's one-way board. The rows decode cleanly, so a three-leg MNL-DXB-FCO-MNL search came back as nine MNL-DXB fares from $208 — the same payload, byte for byte, as a plain one-way MNL-DXB query. Encoding the correct value (3) is not a fix. The page renders the right multi-city board in a browser, but its ds:1 blob holds no flight rows at all: 6.5 KB against 52 KB, with inner[2] and inner[3] absent, and no other AF_initDataCallback blob on the page carries a parseable row. Multi-city results are fetched client-side through the RPC that has been gated since 2026-08. So build_tfs now raises SearchUnsupportedError for multi-city. That is the one place both SearchFlights and SearchDates route through: SearchFlights.search already failed, but with a confusing shape error from the expansion step, and a multi-city date sweep silently returned first-leg prices. Reported by @edgycoder-ph on punitarani#230.
|
@edgycoder-ph — this is a good catch, and a well-built report. Reproduced every part of it. Fixed on ConfirmedYour three legs, from a US IP:
The first two payloads are the same length — Google drops segments 2 and 3 on the floor and hands back the leg-1 board. Your f19 = 3 figure was right to the byte: 6,591. I also ran every Worth naming that the encoder was wrong on purpose rather than by accident — One correction
Same for a two-segment open jaw. The wrong data is only reachable by calling Your worry is right, through a path neither of us named
Fix
I did not teach field 19 to write 3. Nothing can send it now, so it would be a value with no caller — the reason 3 is not the answer is in a comment above the guard instead, so nobody has to re-run your experiment to find out. Two tests: one pins field 19 to 1 or 2, one pins the refusal. The Thanks also for checking MNL→DXB against the live page. That is the second region on this branch after the German consent report, and route coverage from outside the US is exactly what I cannot test from here. |
GHSA-qw2m-4pqf-rmpp / CVE-2026-33752 (CVSS 8.6 HIGH) covers curl-cffi following HTTP redirects into private/internal address space. The advisory's vulnerable range is < 0.15.0; this branch resolved 0.13.0. Raise the declared floor to >=0.15.0 and relock (curl-cffi 0.13.0 -> 0.16.2, cffi 1.17.1 -> 2.1.1; no new packages — 0.16 moved `rich` to a `cli` extra). The floor alone is not the fix. 0.15.0's remediation is opt-in: `allow_redirects=True` still follows private-IP redirects on every released version, including 0.16.2. What 0.15.0 added is `CurlFollow.SAFE`, reachable from the requests layer as `allow_redirects="safe"`. fli passed an explicit `allow_redirects=True` at all four network call sites, so bumping the dependency would have left runtime behaviour byte-for-byte unchanged. So also: - default `allow_redirects="safe"` in `Client.get`/`Client.post`, alongside the existing `timeout` setdefault, so new call sites inherit the mitigation and cannot silently regress; - drop the explicit `allow_redirects=True` from `SearchFlights.search`, `SearchFlights.get_booking_options`, `SearchDates._price_date` and `scripts/capture_fixtures.py` so the client default applies. Honest scoping: fli builds every request URL internally against hardcoded www.google.com hosts and never accepts a user-supplied URL, so practical exploitability here is effectively nil. This is dependency hygiene plus genuine defence in depth, not the closing of an exploitable hole in fli. User-visible behaviour change: a 3xx hop to an RFC1918/loopback/link-local address now hard-fails as `SearchConnectionError` instead of being followed. The realistic case is a captive portal, which already failed (as `SearchParseError`, one step later). Verified live after the upgrade, since PR punitarani#230's page transport depends on curl-cffi's TLS impersonation and cookie handling: `fli flights LHR PIT --return` returned 11 round-trip options and `fli dates LHR PIT` returned 8 priced dates. Note `impersonate="chrome"` now resolves to chrome150 rather than chrome136; if Google ever treats the newer fingerprint differently, pin an explicit build at the call sites rather than reverting this bump. Tests in tests/search/test_ssrf_redirects.py pin both halves: the runtime capability (`CurlFollow.SAFE == 4`), the declared specifier set, the client defaults, that no call site passes `True`, and an end-to-end proof that a loopback redirect is refused. Credit to @gateway (upstream punitarani#193) for identifying the advisory and proposing the dependency bump. Co-authored-by: Dreaming Computers <519563+gateway@users.noreply.github.com>
`fli flights KJFK KLAX <date>` failed with "Invalid airport code: 'KJFK'". `resolve_airport` now detects a 4-character alpha token, translates it through an ICAO->IATA table, and then runs the existing enum lookup, so `KJFK`/`EGLL`/`VTBS` work anywhere `JFK`/`LHR`/`BKK` do — CLI, MCP tools, and multi-airport lists (`KJFK,LGA`). Three-letter codes take a byte-identical path, error message included. Ports upstream punitarani#111 by Trevin Chow <trevin@trevinchow.com> (closes punitarani#63). The upstream diff no longer applied — this branch's `resolve_airport` grew a keyword-only `label`, and the tests it patched have since been rewritten — so it is reimplemented; only the 281-row mapping payload is carried over verbatim. Four deliberate departures from upstream: 1. Generated module, not a runtime CSV read. Upstream ships `fli/data/icao_to_iata.csv` and parses it lazily behind an unguarded module-global cache. Here `data/icao_to_iata.csv` is the source and `scripts/generate_enums.py` emits `fli/models/icao.py`, matching how `Airport`/`Airline` are already produced. That buys generation-time validation the CSV route cannot have: `_validate_icao_rows` rejects a non-4-alpha key, a duplicate key, or an IATA target absent from `AIRPORT_NAMES`. Upstream would ship a typo'd target happily and then tell the user "Invalid airport code: 'JFX'" for the input `KJFK`. It also keeps the runtime surface at zero — no `csv`, no `importlib.resources`, no first data-file precedent to maintain across wheel/sdist/vendoring. (Both routes are wheel-safe; verified by building the wheel and resolving `KJFK` from an install with no repo checkout on disk.) 2. The error keeps the existing "Invalid <slot> airport code: '<code>'" prefix rather than upstream's new "Unknown ICAO code:" wording, so the CLI and MCP clients that already string-match that prefix keep working, and it threads the `label` this branch added. The explanation is appended after the prefix, and only inside the 4-letter branch — the generic 3-letter message is untouched. 3. Coverage is stated, not implied. 281 airports is a curated subset, so an unmapped 4-letter code says the table is partial and to fall back to IATA, rather than implying the airport does not exist. 4. Beyond upstream: `fli multi` had a second, independent blocker the PR never touched. `LEG_PATTERN` hard-codes `[A-Za-z]{3}`, so `--leg KSEA,VHHH,<date>` died at leg-format validation before ever reaching the parser, with a message about the leg grammar rather than the code. Widened to `{3,4}` — deliberately not `+`, since the comma is the field separator there and a longer token really is a malformed leg. Verified live, not just under mocks: `KJFK->KLAX` and `JFK->LAX` return the same 29 results with byte-identical per-flight `tfs` booking tokens, confirming the punitarani#230 search transport and protobuf encoding see identical input. The MCP `search_flights` executor returns 22 JFK->LHR flights for `KJFK`/`EGLL`. Out of scope, stated so it is not mistaken for an oversight: `find_airports` / `fli airports` still will not match "KJFK" (it needs a new `MatchType`, which changes the MCP output schema), and `fli-js`'s `resolveAirport` stays IATA-only — a deliberate Python-only divergence, noted in the docstring. Co-Authored-By: Trevin Chow <trevin@trevinchow.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`fli dates --round --duration N` could only ask about one trip length, so "cheapest 4-7 night trip" meant running the command four times and merging the tables by hand. Adds `--min-duration`/`--max-duration` to the CLI and `min_duration`/`max_duration` to the `search_dates` MCP tool: with `--round`, every trip length in the range is searched, the results are deduplicated by (departure date, return date) and merged into one list ordered by departure date then price. Ported from upstream punitarani#195 by Roberto Reale (robertoreale2006@gmail.com). The upstream diff no longer applies to this branch, so it was reimplemented with four deliberate departures: 1. REQUEST VOLUME IS BOUNDED. Since the transport moved to the public search page (punitarani#230) each departure date costs its own page fetch, so a sweep costs (trip lengths x departure dates). Upstream defaults the max to the whole date range when only a min is given, which turns `--min-duration 1` into 3,540 requests at CLI defaults and ~93,000 over a 305-day range. Instead min and max must be given together, and the combination count is capped at MAX_DURATION_SWEEP_COMBINATIONS (600, roughly a minute at the client's 10 req/sec limit) before any network call. 2. ONE SHARED RESOLVER, NOT COPY-PASTE. `resolve_duration_sweep` in fli/core/builders.py owns every rule — precedence over `--duration`, the round-trip requirement, the inverted range, the cap — and raises `ParseError`, which both surfaces already funnel into clean user-facing text. Upstream duplicates the logic into the CLI and the MCP server, where it can drift. 3. SEQUENTIAL, WITH NO SLEEP. `_search_chunk` already fans its dates out over the single shared 10-worker pool, so wrapping the duration loop in another `parallel_map` starves that pool and deadlocks once the duration count reaches the worker count. The loop is sequential and the token bucket, not the thread count, sets the wall clock. Upstream's `time.sleep(1)` between durations buys nothing over the rate limiter and is not carried over. 4. FILTERS SURVIVE THE COPY. `SearchDates.search_durations` derives each variant with `model_copy(deep=True)` rather than rebuilding `DateSearchFilters(...)` by hand per duration, which is exactly the pattern that silently drops `passenger_info`, `airlines_exclude`, `alliances`, `alliances_exclude` and `layover_restrictions`. Beyond upstream: `trip_duration` becomes a sentinel (`None`, effective default still 3) so an explicit `--duration 3` is distinguishable from the implicit one and a contradictory combination can be rejected — a visible MCP schema change, covered by tests. MCP sweep responses report `"duration": null` plus `"duration_range"`, and are price-ordered before `max_results` truncation so truncating a durations x days list keeps the cheapest rows rather than an arbitrary slice of sweep order. The CLI price-trend sparkline collapses to the cheapest price per unique departure date, otherwise a sweep renders a sawtooth over repeated departure dates; the table still lists every row. The single-duration path still routes through `SearchDates.search()` unchanged, so existing callers have no regression path. Co-Authored-By: Roberto Reale <robertoreale2006@gmail.com>
--time / departure_window applied the same window to both legs of a round trip, so "morning out, evening back" was impossible without post-filtering. Adds --return-time / -T to `fli flights` and return_departure_window to the search_flights and get_booking_options MCP tools. Ported from upstream punitarani#196 by Roberto Reale. The upstream diff no longer applies to this branch, so this is a reimplementation with four deliberate departures. 1. NO TRI-STATE SENTINEL. Upstream types the builder kwarg `TimeRestrictions | None | bool = False`, using literal False to mean "unset" — a bool in a non-bool type, and a value MCP callers would have to pass through. Here it is `TimeRestrictions | None = None`, where None means "inherit the outbound window". That default is the load-bearing line: anything else silently stops filtering the return leg of every existing round-trip search, and because the window is applied client-side the symptom is MORE results, not an error. Three tests exist solely to pin it (builder, CLI, MCP). Upstream's None-means-"no filter" case is unreachable from any surface — `--return-time 0-23` covers it. 2. AN UNUSABLE WINDOW IS AN ERROR, NOT A NO-OP. A return window without a return date now raises ParseError from build_flight_segments, so the CLI and both MCP tools reject it through one rule rather than accepting a filter nothing will ever read. 3. SCOPE CUT: `dates` / `search_dates` are deliberately excluded. Since punitarani#230 moved the transport to the search page, SearchDates never calls apply_client_side_filters — `--time` on `fli dates` is ALREADY accepted and silently ignored. Adding --return-time there would ship a second dead flag beside the first. Filed separately; upstream's unrelated --min-duration/--max-duration commits are also not carried (already landed here, bounded, as punitarani#195). 4. PROOF THE FILTER REACHES GOOGLE. tfs carries no time field at all — build_tfs is byte-identical with and without windows on either segment, now pinned by a test so a future "encode it in tfs" change fails loudly instead of quietly altering every live query. The window is applied to decoded rows against the ACTIVE segment (the first unpinned one), which is what makes a distinct return window real. Verified live, JFK-LAX 2026-10-15/22: with `--time 6-10` alone every return departs 06:10-09:28; adding `--return-time 18-23` moves all 13 returns to 18:07-23:59 while outbounds stay 06:00-09:59. Co-Authored-By: Roberto Reale <robertoreale2006@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Maps all 16 fork commits to the punitarani/fli PRs they supersede, so a returning maintainer can decide what to take without re-deriving it. Records two findings that matter more than the mapping: punitarani#193 and punitarani#160 both close their issues without fixing them. punitarani#193's curl-cffi bump is inert because 0.15's remediation is opt-in and every call site passed allow_redirects=True; punitarani#160's helper was never regenerated, so 92 IATA codes still resolved to the wrong airport. Both were reproduced, not inferred. Also notes that b932aee..7f03157 are byte-identical to punitarani#230 and should be merged from that PR rather than credited here.
4 participants
Closes #223. Also covers #200, and makes #201 / #205 / #208 / #224 unnecessary for the search path — those surface the error, this removes it.
The break
GetShoppingResultsandGetCalendarGraphhave required anx-goog-batchexecute-bgrheader since early August. The page's own JavaScript signs it over the exact request bytes, so a token captured from a browser cannot be replayed against a different body — @olivierbarbosa and @richardadonnell both isolated this in #223, and their finding matches what I see. Google answers HTTP 200 with a payload-lesswrb.frrow carrying error 13,parse_first_wrb_payloadreturnsNone, and the CLI prints "No flights found". Every route returnscount: 0, and it reads as a route with no service.The fix
The public
/travel/flightspage is not gated that way. It embeds the same result payload in anAF_initDataCallbackblob keyedds:1, andds:1[2]/ds:1[3]hold exactly the flight rows the RPC returned. The decoders, models, CLI and MCP surface are untouched — only the transport moves.The page takes the
tfsprotobuf rather than thef.reqJSON struct.build_tfs_tokenalready encodedtfsfor booking deep links (#190), so the field layout is now shared:encode_tfs_segmentandencode_tfs_payloadserve both builders, andbuild_tfs_tokenstays byte-identical — its captured-token test still passes.Date search loses its one-call grid; the page carries price history, not a forward calendar. Each date is priced by its own page fetch, run concurrently under the existing rate limiter. An eight-day sweep takes about 1.5 seconds.
Filters
tfsThe middle row means a filtered search returns fewer options than the RPC did — Google would have back-filled the list server-side — but every option it returns honours the filter. The bottom row is logged rather than silently dropped, so nobody gets results that quietly ignore what they asked for.
Two details worth flagging
Both cost me real debugging time, and both have tests.
The stop ceiling in field 5 is zero-based, and
MaxStops.ANYhas to omit the field. Writing 0 for "any" pins every search to non-stop.Selected legs pin a search only in field 4. Google ignores an unknown field number and re-serves the outbound board, so putting them anywhere else makes round-trip expansion pair outbound flights with other outbound flights and never error. My first attempt did exactly that and returned 155 plausible-looking itineraries whose cheapest price happened to match a correct search — the direction of the return legs is what gave it away.
Verification
Live JFK → LAX: 32 one-way options, cheapest $204. 38 round-trip itineraries, cheapest $357, return legs correctly LAX → JFK. Non-stop filter returns only direct flights. Business cabin and EUR pricing both shift as expected. Prices cross-check against an independent Google Flights source on the same queries.
The
tfsfixtures intests/search/test_tfs.pyare parameters Google issued for those same queries, so a byte mismatch means the encoder has drifted from what Google accepts.Offline suite: 433 passed, lint clean. In
tests/search, which hits the live API and is flaky under rate limits, no test fails that did not already fail onmain, and 32 more pass. Two tests moved from stubbingSession.posttoSession.getbecause the search path changed verb.Not addressed
get_booking_optionsstill usesGetBookingResults, which is presumably gated the same way. It now raisesSearchRejectedErrornaming the code instead of returning an empty list, so the failure is legible. Restoring it is a separate piece of work.🤖 Generated with Claude Code
Greptile Summary
This PR replaces the gated Google Flights shopping and calendar RPC paths with public-page
tfsrequests while retaining the existing decoders and making rejected legacy RPC calls explicit.tfsprotobuf encoding and extraction of the page'sds:1payload.tfsvalues, page extraction, local filtering, and GET-based error handling.Confidence Score: 3/5
The PR is not safe to merge until multi-airport searches preserve all requested endpoints and time windows are applied to the corresponding return or later segment.
The new transport silently narrows supported multi-airport queries to their first endpoints and can return expanded itinerary legs that violate their segment-specific time restrictions.
Files Needing Attention: fli/search/_tfs.py, fli/search/flights.py, tests/search/test_tfs.py
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Typed search filters] --> B[Build tfs parameter] B --> C[GET public flights page] C --> D[Extract ds:1 payload] D --> E[Existing flight-row decoders] E --> F[Client-side filters and sorting] F --> G[Flight results] A --> H[Concurrent per-date tfs requests] H --> I[Cheapest DatePrice per date]Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "fix(search): restore search via the publ..." | Re-trigger Greptile
Context used (4)