fix(rfc): apply every predicate, and push far more of them to SAP

.gitignore

@@ -85,3 +85,7 @@ python-env/*
 # Experiments
 /examples/experiments/*
 .adt.creds
+
+# Local agent scratch (not part of the extension)
+.codex-delegate/
+.antigravitycli/

API_REFERENCE.md

@@ -1280,6 +1280,7 @@ Notes:
 | `erpl_rfc_persistent_connections` | BOOLEAN | `true` | Cache one RFC connection + function descriptor per column for a `sap_read_table` scan instead of reopening per batch |
 | `erpl_rfc_max_persistent_connections` | UINTEGER | 16 | Upper bound on RFC connections a scan caches concurrently (issue #67); columns past the cap use per-batch open/close |
 | `erpl_rfc_read_table_batch_budget` | UINTEGER | 1310720 | Target max concurrent result rows (projected columns × per-column batch) for `sap_read_table`; bounds peak memory on wide tables (issue #69). Lower = less memory but more RFC round-trips; `0` disables the cap |
+| `erpl_rfc_pushdown_filters` | BOOLEAN | `true` | Translate SQL `WHERE` predicates into `RFC_READ_TABLE`'s `OPTIONS` table so SAP filters the rows instead of sending them all. Turning it off never changes which rows come back, only how many cross the wire. See [Filter Pushdown](#filter-pushdown) |
 | `erpl_rfc_backend` | VARCHAR | `'nwrfc'` | Which implementation serves RFC calls: `'nwrfc'` (SAP's NetWeaver RFC SDK) or `'proto'` (the pure-Rust erpl-proto implementation). Must be set **before the first SAP call**; frozen for the life of the process once resolved. Environment override: `ERPL_RFC_BACKEND` |
 | `erpl_rfc_backend_path` | VARCHAR | `''` | Explicit path to the RFC backend shared library, overriding the search. Empty means: next to the extension, then the loader's library path. Environment override: `ERPL_RFC_BACKEND_PATH` |
 
@@ -1375,6 +1376,33 @@ SELECT * FROM sap_read_table('SFLIGHT', FILTER='CARRID = ''LH''');
 SELECT * FROM sap_read_table('SFLIGHT') WHERE CARRID = 'LH';
 ```
 
+On a large table this is the single biggest lever available: a predicate SAP can
+evaluate turns a multi-million-row transfer into a few thousand rows. What reaches
+the server:
+
+| Predicate | Pushed |
+|---|---|
+| `=`, `<>`, `<`, `>`, `<=`, `>=` | yes |
+| `AND` / `OR` over one column, including `BETWEEN` | yes, all arms or none |
+| `IN (...)` | yes, until the generated clause exceeds ~4000 characters |
+| `IS NULL` / `IS NOT NULL` | no — ABAP has no NULL |
+| Predicates on the client field (`MANDT`, DDIC type `CLNT`) | no — RFC_READ_TABLE rejects a clause naming the client |
+| Literals of type `TIMESTAMP`, `BLOB`, `FLOAT`, `DOUBLE` | no — no unambiguous ABAP spelling |
+
+Literals are rendered the way the DDIC expects them: `DATE` as `YYYYMMDD`, `TIME`
+as `HHMMSS`, character and numeric values quoted with embedded apostrophes
+doubled. A type whose ABAP spelling is not established is not pushed rather than
+guessed at.
+
+**Anything not pushed is still applied** — erpl evaluates it after reading, so the
+result set is identical either way. Only the volume transferred changes.
+
+`erpl_rfc_pushdown_filters = false` disables the translation entirely and makes
+erpl evaluate every predicate itself. It exists as an escape hatch for SAP
+releases that reject the generated syntax, and as a way to check that a filter is
+not the cause of a discrepancy: the same query must return the same rows with it
+on and off.
+
 ### Parallel Reads
 
 Use the `THREADS` parameter on `sap_read_table` and `sap_odp_read_full` for large tables:

CHANGELOG.md

@@ -22,6 +22,44 @@ LOAD erpl;
 
 ---
 
+## Unreleased
+
+### Fixed
+
+- **[rfc]** **`sap_read_table` returned unfiltered rows for any predicate it could not
+  push to SAP.** `filter_pushdown = true` tells DuckDB the scan applies every filter it
+  is handed, and DuckDB removes the filter from the plan accordingly — `EXPLAIN` shows no
+  `FILTER` operator above `SAP_READ_TABLE`. erpl translated what `RFC_READ_TABLE` could
+  express and silently ignored the rest, so ranges, conjunctions and wide `IN` lists were
+  dropped: `WHERE SEATS_MAX > 350` on `/DMO/FLIGHT` returned all 40 rows instead of 14.
+  Predicates that cannot go to the server are now evaluated by erpl instead, so the result
+  is the same whether or not one reaches SAP.
+
+- **[rfc]** Comparison literals were built without escaping, so a value containing an
+  apostrophe closed the ABAP literal early and changed the predicate's meaning. Inequality
+  was emitted as `!=`, which ABAP's dynamic `WHERE` does not accept.
+
+- **[rfc]** The `OPTIONS` line splitter broke inside quoted literals whenever one contained
+  a space, leaving an unbalanced apostrophe on both lines.
+
+### Added
+
+- **[rfc]** **Range predicates, `AND`/`OR` combinations and `IN` lists wider than five
+  values are now pushed to SAP.** Previously only `=` and `IN` with at most five values
+  reached the server, so a date-window predicate transferred the whole table and filtered
+  it locally. `BETWEEN` in particular pushed nothing at all, because DuckDB expresses it as
+  a conjunction. Literals are rendered as the DDIC expects them (`DATE` → `YYYYMMDD`,
+  `TIME` → `HHMMSS`); types with no unambiguous ABAP spelling are left to erpl rather than
+  guessed at. Predicates on the client field are never pushed — `RFC_READ_TABLE` rejects
+  any clause naming it.
+
+- **[rfc]** `erpl_rfc_pushdown_filters` (default `true`) to disable the translation. It
+  changes only where a predicate is evaluated, never which rows come back, which makes it
+  both an escape hatch for SAP releases that reject the generated syntax and a way to rule
+  a filter out as the cause of a discrepancy.
+
+---
+
 ## v2026.08.29 — BEx result sets that fit in memory
 
 ### Fixed

rfc/src/erpl_rfc_extension.cpp

@@ -157,6 +157,10 @@ namespace duckdb {
         SetRfcReadTableBatchBudget(parameter.GetValue<unsigned int>());
     }
 
+    static void OnPushdownFilters(ClientContext &, SetScope, Value &parameter) {
+        SetRfcPushdownFilters(parameter.GetValue<bool>());
+    }
+
     static void OnRfcBackend(ClientContext &, SetScope, Value &parameter) {
         SetRfcBackend(parameter.GetValue<string>());
     }
@@ -262,6 +266,20 @@ namespace duckdb {
             Value::UINTEGER(RfcReadColumnStateMachine::DEFAULT_READ_TABLE_BATCH_BUDGET),
             OnReadTableBatchBudget);
 
+        config.AddExtensionOption(
+            "erpl_rfc_pushdown_filters",
+            "Translate SQL WHERE predicates into RFC_READ_TABLE's OPTIONS table so SAP "
+            "filters the rows instead of sending them all across the wire.  On a large "
+            "table this is the difference between transferring millions of rows and "
+            "thousands, so leave it on unless a specific SAP release rejects the "
+            "generated ABAP syntax.  Comparisons (= <> < > <= >=), AND/OR combinations "
+            "and IN lists are pushed; anything else is evaluated by DuckDB instead.  "
+            "Turning this off never changes which rows you get back -- DuckDB applies "
+            "every filter either way -- it only makes the scan slower.",
+            LogicalType::BOOLEAN,
+            Value::BOOLEAN(true),
+            OnPushdownFilters);
+
         auto provider = make_uniq<RfcEnvironmentCredentialsProvider>(config);
         provider->SetAll();
 

rfc/src/include/sap_rfc.hpp

@@ -1,5 +1,6 @@
 #pragma once
 
+#include <set>
 #include <atomic>
 #include <mutex>
 #include <optional>
@@ -46,10 +47,18 @@ namespace duckdb
 	typedef std::shared_ptr<RfcConnection> (* RfcConnectionFactory_t)(ClientContext &context);
 	std::shared_ptr<RfcConnection> DefaultRfcConnectionFactory(ClientContext &context);
 
+	void SetRfcPushdownFilters(bool enabled);
+	bool GetRfcPushdownFilters();
+
 	class RfcReadTableBindData : public TableFunctionData
     {
 		public: 
-			static const idx_t MAX_OPTION_LEN = 70;
+			static constexpr idx_t MAX_OPTION_LEN = 70;
+			// Budget for the whole generated pushdown clause.  The OPTIONS table holds
+			// 512 lines, but spending all of it on one predicate risks tripping ABAP's
+			// own dynamic-WHERE limits, and a pathological IN list is never worth it --
+			// pushing nothing is always correct, just slower.
+			static constexpr idx_t MAX_PUSHDOWN_CLAUSE_LEN = 4000;
 
 			RfcReadTableBindData(std::string table_name, 
 								 int max_read_threads,
@@ -136,8 +145,14 @@ namespace duckdb
 			ClientContext &client_context;
 			std::vector<std::string> column_names;
 			std::vector<RfcType> column_types;
+			// Columns whose DDIC type is CLNT.  RFC_READ_TABLE's OPTIONS parser rejects
+			// any clause naming the client field, so these are never pushed.
+			std::set<std::string> client_columns;
 			std::vector<RfcReadColumnStateMachine> column_state_machines;
 			std::atomic<unsigned int> persistent_slots_used{0};
+			// Filters that could not be translated into OPTIONS, keyed by projected
+			// column index.  Copied because the TableFilterSet belongs to the plan.
+			std::vector<std::pair<idx_t, duckdb::unique_ptr<TableFilter>>> residual_filters;
 			// Defaults to RfcReadColumnStateMachine::MAX_BATCH_SIZE (that class
 			// is defined later in this header, so we can't name the constant
 			// here); Step() overwrites this with the column-count-aware cap.
@@ -155,7 +170,22 @@ namespace duckdb
 			static std::string TransformLiteral(const Value &val);
 			static std::string TransformBlob(const std::string &val);
 			static std::string CreateExpression(string &column_name, vector<unique_ptr<TableFilter>> &filters, string op);
+			static std::string TransformConjunction(std::string &column_name,
+			                                        vector<unique_ptr<TableFilter>> &children,
+			                                        const std::string &op);
 			static std::string TransformComparision(ExpressionType type);
+
+			// Split a generated WHERE clause into OPTIONS lines.  Never splits inside a
+			// quoted literal: ABAP literals may contain spaces, so the obvious
+			// "back up to the previous whitespace" rule can land in the middle of one
+			// and leave an unbalanced apostrophe on both lines.
+			static std::vector<std::string> ChunkWhereClause(const std::string &where_clause, idx_t max_len);
+
+			// Apply the predicates RFC_READ_TABLE could not express.  Required, not
+			// optional: filter_pushdown = true makes DuckDB drop the filter from the
+			// plan, so anything the scan does not apply is simply not applied.
+			void ApplyResidualFilters(DataChunk &output);
+			bool HasResidualFilters() const { return ! residual_filters.empty(); }
     };
 
 	enum class ReadTableStates {