From 2995d06b66c4f9ed6c84db9759dd875200911c5d Mon Sep 17 00:00:00 2001 From: Joachim Rosskopf Date: Mon, 31 Aug 2026 13:47:35 +0200 Subject: [PATCH] docs: how to parallelise a narrow sap_read_table extract, and what it is worth sap_read_table parallelises per COLUMN -- one concurrent RFC_READ_TABLE call per projected column. That is a good fit for a wide extract and no help at all for a narrow one: a single-column scan issues one call and `threads` has nothing to spread. Nothing said so, so the natural reaction to a slow narrow extract was to raise `threads` and watch it change nothing. Documents the range-partition recipe that works today with no code, now that #128 pushes range predicates to SAP so each branch reads only its slice. States what it is actually worth rather than implying the branch count: measured on a 164,664-row single-column extract, four branches ran in 6.6s against 8.3s for the single scan -- about 1.2x, not 4x. Each branch pays its own connection and metadata round-trip and SAP evaluates each range separately, so splitting into many small branches makes it worse. Even boundaries and an indexed split key matter more than how many branches there are. Also notes that separate processes scale considerably better than either approach, since SAP-side concurrency limits apply per client program -- with the caveat to ask Basis before pointing many parallel readers at production. --- API_REFERENCE.md | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/API_REFERENCE.md b/API_REFERENCE.md index 2045e92..9bbb8d1 100644 --- a/API_REFERENCE.md +++ b/API_REFERENCE.md @@ -1442,6 +1442,48 @@ running a large parallel extract against a production system. `erpl_rfc_fetch_size`. Both remain supported and write the same value — they are two names for one knob, not two knobs. +#### Narrow tables: `threads` will not help + +`sap_read_table`'s parallelism is **per column**: it issues one concurrent +`RFC_READ_TABLE` call per projected column. That is a good fit for a wide extract and +no help at all for a narrow one — a single-column scan runs one call, and `threads` +has nothing to spread. + +If you are extracting few columns from a large table, split it by a key range instead +and let SQL run the pieces: + +```sql +SELECT * FROM sap_read_table('DD02L') WHERE TABNAME < 'D' +UNION ALL +SELECT * FROM sap_read_table('DD02L') WHERE TABNAME >= 'D' AND TABNAME < 'R' +UNION ALL +SELECT * FROM sap_read_table('DD02L') WHERE TABNAME >= 'R' AND TABNAME < 'V' +UNION ALL +SELECT * FROM sap_read_table('DD02L') WHERE TABNAME >= 'V'; +``` + +The range predicates are pushed to SAP (see [Filter Pushdown](#filter-pushdown)), so +each branch reads only its own slice rather than filtering locally. + +**What to expect.** Measured on a 164,664-row single-column extract, four branches ran +in 6.6s against 8.3s for the equivalent single scan — about **1.2x**, not 4x. Each +branch pays its own connection and metadata round-trip, and the SAP system applies each +range filter separately, so the gain is far smaller than the branch count suggests. +Splitting into many small branches makes it worse, not better. + +Two things matter more than the branch count: + +- **Pick boundaries that split the data evenly.** A range that covers most of the table + leaves you with one long branch and three short ones, and the long one sets the wall + time. +- **Split on an indexed key.** If SAP has to scan the whole table to evaluate each + range, you have multiplied the server's work rather than divided it. + +For a genuinely large extract, running several *processes* against SAP scales +considerably better than either approach — concurrency limits on the SAP side apply per +client program, so separate processes are not aggregated the way threads within one are. +Check with your Basis team before pointing many parallel readers at a production system. + ### SSH Tunnel + SAP Connection Complete workflow for connecting through an SSH jump host: