Skip to content

A pre-deployed command driver: sync/replicate without S_DEVELOP #88

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jrosskopf merged 3 commits into from
Sep 1, 2026
Merged

A pre-deployed command driver: sync/replicate without S_DEVELOP #88

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0be92c3
A pre-deployed command driver, so the CLI needs no S_DEVELOP
jrosskopf Sep 1, 2026
466f085
Route every verb through the driver, and prove the zero-auth path
jrosskopf Sep 1, 2026
85312c7
Tests, e2e and docs for the command driver
jrosskopf Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,7 @@ if(TRUE) # tests always build; DuckDB is in-tree
test/test_telemetry.cpp
test/test_abap_codegen.cpp
test/test_cli_common.cpp
test/test_cmd_queue.cpp
test/test_table_render.cpp
test/test_sap_setup.cpp
test/bench_ingest.cpp
Expand Down
10 changes: 7 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,9 +161,13 @@ Add `--print-abap` to any of them to see the ABAP instead of running it, and
`--dry-run` to see the plan. Nothing writes to SAP or DuckDB without a terminal
confirmation or an explicit `--yes`.

> These commands deploy a short-lived class to `$TMP` to carry their parameters,
> because an ADT classrun takes none. That needs **`S_DEVELOP`** — a stronger
> authorisation than running the equivalent report in SE38.
> Parameters reach SAP as *data*: the CLI writes the command into a DuckDB table
> and the pre-deployed `ZCL_ERPL_REV_CLIDRV` executes it, so these commands need
> **no SAP authorisation** and create nothing. `--queue-only` does not contact
> SAP at all — the periodic `ERPL_REV_DELTA` job picks the command up.
>
> Where the driver is not deployed they fall back to generating a temporary
> class, which does need `S_DEVELOP`. `erpl-rev doctor` reports which applies.

## Then set up the SAP side

Expand Down
17 changes: 17 additions & 0 deletions abap/z_erpl_rev_delta.prog.abap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,12 @@ START-OF-SELECTION.

DATA lt_run TYPE zcl_erpl_rev_delta=>tt_run.

" Drain anything the CLI queued before running the due targets. This is what
" lets `erpl-rev sync`/`replicate` work for a caller with no SAP
" authorisation at all: the CLI writes a row into a local DuckDB table and
" this job -- already running on a schedule -- picks it up. See issue #85.
PERFORM drain_cli.

IF p_tgt IS NOT INITIAL.
APPEND zcl_erpl_rev_delta=>run( p_tgt ) TO lt_run.
PERFORM show USING lt_run.
Expand All @@ -123,6 +129,17 @@ START-OF-SELECTION.
PERFORM show USING lt_run.
ENDIF.

*&---------------------------------------------------------------------*
*& Run whatever the CLI queued, and report it in the job log.
*&---------------------------------------------------------------------*
FORM drain_cli.
DATA(lt_cmd) = zcl_erpl_rev_clidrv=>drain( ).
LOOP AT lt_cmd INTO DATA(ls_cmd).
WRITE: / |cli { ls_cmd-cmd_id } { ls_cmd-verb } { ls_cmd-status } | &&
|{ ls_cmd-result }{ ls_cmd-error }|.
ENDLOOP.
ENDFORM.

*&---------------------------------------------------------------------*
*& Show one tick's results as a compact, coloured table.
*&---------------------------------------------------------------------*
Expand Down
281 changes: 281 additions & 0 deletions abap/zcl_erpl_rev_clidrv.abap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,281 @@
"! <p class="shorttext">erpl-rev CLI command driver</p>
"!
"! Executes commands the CLI queued in the DuckDB table `_erpl_rev_cli_cmd`.
"!
"! Why this exists: `erpl-adt object run` executes a class that takes no
"! parameters, so the first version of the CLI generated a class per command
"! with the parameters written into its source. That works, but it needs
"! S_DEVELOP -- a developer authorisation the erpl-rev service user does not
"! have on a production system -- and it makes every value the user types into
"! ABAP source, which is an injection surface that has to be defended.
"!
"! Here the parameters arrive as *data*, in a JSON column, and are only ever
"! read as values. Nothing is generated, nothing is created, nothing is deleted.
"!
"! Two things can drive it, and both end up here:
"! - `object run ZCL_ERPL_REV_CLIDRV`, when the caller may run a classrun;
"! - the periodic Z_ERPL_REV_DELTA heartbeat, which drains the queue on each
"! tick -- needing no ADT call, and so no SAP authorisation, from the CLI.
CLASS zcl_erpl_rev_clidrv DEFINITION PUBLIC FINAL CREATE PUBLIC.
PUBLIC SECTION.
INTERFACES if_oo_adt_classrun.

TYPES: BEGIN OF ty_done,
cmd_id TYPE string,
verb TYPE string,
status TYPE string,
result TYPE string,
error TYPE string,
END OF ty_done.
TYPES tt_done TYPE STANDARD TABLE OF ty_done WITH EMPTY KEY.

"! Run every command currently pending, oldest first. Returns one row per
"! command executed; an empty table means the queue was empty.
"! `iv_max` bounds one drain so a backlog cannot monopolise a job step.
CLASS-METHODS drain
IMPORTING iv_max TYPE i DEFAULT 20
RETURNING VALUE(rt) TYPE tt_done.

PRIVATE SECTION.
"! Claim the oldest pending command, returning its fields as one JSON row.
CLASS-METHODS claim
RETURNING VALUE(rs) TYPE zcl_erpl_rev_util=>ty_query.
CLASS-METHODS execute
IMPORTING iv_verb TYPE string
iv_params TYPE string
EXPORTING ev_result TYPE string
ev_error TYPE string.
CLASS-METHODS finish
IMPORTING iv_id TYPE string
iv_result TYPE string
iv_error TYPE string.
"! Read one string out of a flat JSON object. Values are data, never source.
CLASS-METHODS jstr
IMPORTING iv_json TYPE string
iv_key TYPE string
RETURNING VALUE(rv) TYPE string.
CLASS-METHODS jint
IMPORTING iv_json TYPE string
iv_key TYPE string
iv_def TYPE i DEFAULT 0
RETURNING VALUE(rv) TYPE i.
"! Escape a value for a SQL string literal (doubling the apostrophe). Only
"! ever applied to values we are writing *back*, never to user parameters
"! on the way in -- those are read, not concatenated.
CLASS-METHODS q
IMPORTING iv_in TYPE string
RETURNING VALUE(rv) TYPE string.
ENDCLASS.

CLASS zcl_erpl_rev_clidrv IMPLEMENTATION.

METHOD if_oo_adt_classrun~main.
DATA(lt) = drain( ).
out->write( |ERPL-DRV count={ lines( lt ) }| ).
LOOP AT lt INTO DATA(ls).
out->write( |ERPL-DRV id={ ls-cmd_id };verb={ ls-verb };status={ ls-status }| &&
|;result={ ls-result };error={ ls-error }| ).
ENDLOOP.
ENDMETHOD.

METHOD drain.
" One command per iteration. zcl_erpl_rev_util=>query returns its rows as a
" JSON array *string*, so claiming one at a time avoids parsing an array --
" and it means a command that dumps cannot take the rest of the batch with
" it, since each claim is its own statement.
DO iv_max TIMES.
DATA(ls_q) = claim( ).
IF ls_q-error IS NOT INITIAL.
APPEND VALUE #( status = 'ERROR' error = ls_q-error ) TO rt.
RETURN.
ENDIF.

DATA(lv_id) = jstr( iv_json = ls_q-rows iv_key = 'cmd_id' ).
IF lv_id IS INITIAL.
RETURN. " queue empty
ENDIF.

DATA(lv_verb) = jstr( iv_json = ls_q-rows iv_key = 'verb' ).
DATA(lv_par) = jstr( iv_json = ls_q-rows iv_key = 'params' ).

execute( EXPORTING iv_verb = lv_verb iv_params = lv_par
IMPORTING ev_result = DATA(lv_res) ev_error = DATA(lv_err) ).
finish( iv_id = lv_id iv_result = lv_res iv_error = lv_err ).

APPEND VALUE #( cmd_id = lv_id verb = lv_verb
status = COND string( WHEN lv_err IS INITIAL THEN 'DONE' ELSE 'ERROR' )
result = lv_res error = lv_err ) TO rt.
ENDDO.
ENDMETHOD.

METHOD claim.
" Claim and read in one statement: RETURNING hands back the row we just
" marked, so two drivers racing cannot both take the same command.
rs = zcl_erpl_rev_util=>query(
|UPDATE _erpl_rev_cli_cmd SET status = 'RUNNING', claimed_ts = now() | &&
|WHERE cmd_id = ( SELECT cmd_id FROM _erpl_rev_cli_cmd | &&
| WHERE status = 'PENDING' ORDER BY cmd_id LIMIT 1 ) | &&
|RETURNING cmd_id, verb, params| ).
ENDMETHOD.

METHOD execute.
CLEAR: ev_result, ev_error.

CASE iv_verb.
WHEN 'replicate'.
DATA(ls_r) = zcl_erpl_rev_util=>replicate(
iv_tab = jstr( iv_json = iv_params iv_key = 'table' )
iv_target = jstr( iv_json = iv_params iv_key = 'target' )
iv_columns = jstr( iv_json = iv_params iv_key = 'columns' )
iv_where = jstr( iv_json = iv_params iv_key = 'where' )
iv_params = jstr( iv_json = iv_params iv_key = 'cds_params' )
iv_init = jstr( iv_json = iv_params iv_key = 'init' )
iv_mode = COND string( WHEN jstr( iv_json = iv_params iv_key = 'mode' ) IS INITIAL
THEN 'UPSERT' ELSE jstr( iv_json = iv_params iv_key = 'mode' ) )
iv_batch = COND i( WHEN jint( iv_json = iv_params iv_key = 'batch' ) > 0
THEN jint( iv_json = iv_params iv_key = 'batch' ) ELSE 50000 )
iv_maxrows = jint( iv_json = iv_params iv_key = 'maxrows' )
iv_truncate = COND abap_bool( WHEN jstr( iv_json = iv_params iv_key = 'truncate' ) = 'false'
THEN abap_false ELSE abap_true ) ).
ev_error = ls_r-error.
ev_result = |rows={ ls_r-rows_affected };seconds={ ls_r-seconds }|.

WHEN 'sync_register'.
ev_error = zcl_erpl_rev_delta=>register( VALUE #(
target = jstr( iv_json = iv_params iv_key = 'target' )
method = jstr( iv_json = iv_params iv_key = 'method' )
source_from = jstr( iv_json = iv_params iv_key = 'source_from' )
keys = jstr( iv_json = iv_params iv_key = 'keys' )
chg_col = jstr( iv_json = iv_params iv_key = 'chg_col' )
wm_kind = jstr( iv_json = iv_params iv_key = 'wm_kind' )
wm_value = jstr( iv_json = iv_params iv_key = 'wm_value' )
safety_secs = COND i( WHEN jint( iv_json = iv_params iv_key = 'safety_secs' ) > 0
THEN jint( iv_json = iv_params iv_key = 'safety_secs' ) ELSE 120 )
cadence = jstr( iv_json = iv_params iv_key = 'cadence' )
extra = jstr( iv_json = iv_params iv_key = 'extra' ) ) ).
IF ev_error IS INITIAL.
ev_result = |registered { jstr( iv_json = iv_params iv_key = 'target' ) }|.
ENDIF.

WHEN 'sync_run'.
DATA(lv_tgt) = jstr( iv_json = iv_params iv_key = 'target' ).
DATA lt_run TYPE zcl_erpl_rev_delta=>tt_run.
IF lv_tgt IS INITIAL.
lt_run = zcl_erpl_rev_delta=>run_due( ).
ELSE.
APPEND zcl_erpl_rev_delta=>run( lv_tgt ) TO lt_run.
ENDIF.
LOOP AT lt_run INTO DATA(ls_run).
ev_result = |{ ev_result }{ ls_run-target }:rows={ ls_run-rows },| &&
|ins={ ls_run-ins },upd={ ls_run-upd },del={ ls_run-del };|.
IF ls_run-error IS NOT INITIAL.
ev_error = |{ ev_error }{ ls_run-target }: { ls_run-error }; |.
ENDIF.
ENDLOOP.
IF lt_run IS INITIAL.
ev_result = 'nothing due'.
ENDIF.

WHEN 'schedule'.
ev_result = zcl_erpl_rev_delta=>schedule(
iv_minutes = COND i( WHEN jint( iv_json = iv_params iv_key = 'minutes' ) > 0
THEN jint( iv_json = iv_params iv_key = 'minutes' ) ELSE 1 )
iv_remove = COND abap_bool( WHEN jstr( iv_json = iv_params iv_key = 'remove' ) = 'true'
THEN abap_true ELSE abap_false ) ).
IF ev_result CS 'ERROR:'.
ev_error = ev_result.
ENDIF.

WHEN OTHERS.
ev_error = |unknown verb '{ iv_verb }'|.
ENDCASE.
ENDMETHOD.

METHOD finish.
" Newlines would break the one-line result contract the CLI parses.
DATA(lv_res) = replace( val = iv_result sub = cl_abap_char_utilities=>newline
with = ` ` occ = 0 ).
DATA(lv_err) = replace( val = iv_error sub = cl_abap_char_utilities=>newline
with = ` ` occ = 0 ).
zcl_erpl_rev_util=>query(
|UPDATE _erpl_rev_cli_cmd SET | &&
|status = '{ COND string( WHEN lv_err IS INITIAL THEN 'DONE' ELSE 'ERROR' ) }', | &&
|finished_ts = now(), result = '{ q( lv_res ) }', error = '{ q( lv_err ) }' | &&
|WHERE cmd_id = { iv_id }| ).
ENDMETHOD.

METHOD jstr.
" A deliberately small reader for the flat JSON the CLI writes: no nesting,
" no arrays. /ui2/cl_json would drag a structure definition per verb into
" this class for no benefit.
DATA(lv_needle) = |"{ iv_key }":|.
DATA(lv_off) = find( val = iv_json sub = lv_needle ).
IF lv_off < 0.
RETURN.
ENDIF.
DATA(lv_p) = lv_off + strlen( lv_needle ).
WHILE lv_p < strlen( iv_json ) AND iv_json+lv_p(1) = ` `.
lv_p = lv_p + 1.
ENDWHILE.
IF lv_p >= strlen( iv_json ).
RETURN.
ENDIF.
IF iv_json+lv_p(1) <> '"'.
" An unquoted scalar: a number, true/false or null. cmd_id arrives this
" way, and reading only quoted values made the driver silently claim a
" command and then decide there was nothing to run.
WHILE lv_p < strlen( iv_json ).
DATA(lv_u) = iv_json+lv_p(1).
IF lv_u = ',' OR lv_u = '}' OR lv_u = ' '.
EXIT.
ENDIF.
rv = rv && lv_u.
lv_p = lv_p + 1.
ENDWHILE.
IF rv = 'null'.
CLEAR rv.
ENDIF.
RETURN.
ENDIF.
lv_p = lv_p + 1.
WHILE lv_p < strlen( iv_json ).
DATA(lv_c) = iv_json+lv_p(1).
IF lv_c = '\'.
lv_p = lv_p + 1.
IF lv_p < strlen( iv_json ).
DATA(lv_e) = iv_json+lv_p(1).
CASE lv_e.
WHEN 'n'. rv = rv && cl_abap_char_utilities=>newline.
WHEN 't'. rv = rv && cl_abap_char_utilities=>horizontal_tab.
WHEN OTHERS. rv = rv && lv_e.
ENDCASE.
lv_p = lv_p + 1.
ENDIF.
CONTINUE.
ENDIF.
IF lv_c = '"'.
EXIT.
ENDIF.
rv = rv && lv_c.
lv_p = lv_p + 1.
ENDWHILE.
ENDMETHOD.

METHOD jint.
DATA(lv_s) = jstr( iv_json = iv_json iv_key = iv_key ).
IF lv_s IS INITIAL.
rv = iv_def.
RETURN.
ENDIF.
TRY.
rv = CONV i( lv_s ).
CATCH cx_sy_conversion_error.
rv = iv_def.
ENDTRY.
ENDMETHOD.

METHOD q.
rv = replace( val = iv_in sub = `'` with = `''` occ = 0 ).
ENDMETHOD.

ENDCLASS.
1 change: 1 addition & 0 deletions cmake/embed_abap.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ set(ERPL_ABAP_ASSETS
"zcl_erpl_rev_mkfm.abap|ZCL_ERPL_REV_MKFM|CLAS/OC||create the Z_DUCKDB_* RFC FMs"
"zcl_erpl_rev_setup.abap|ZCL_ERPL_REV_SETUP|CLAS/OC||create the registered destination"
"zcl_erpl_rev_diag.abap|ZCL_ERPL_REV_DIAG|CLAS/OC||round-trip probe (STFC_CONNECTION)"
"zcl_erpl_rev_clidrv.abap|ZCL_ERPL_REV_CLIDRV|CLAS/OC||CLI command driver (queue in DuckDB)"
"z_erpl_rev_repl_worker.prog.abap|Z_ERPL_REV_REPL_WORKER|PROG/P||parallel-replication worker"
"z_erpl_rev_replicate.prog.abap|Z_ERPL_REV_REPLICATE|PROG/P||replicate SAP table -> DuckDB"
"z_erpl_rev_sql.prog.abap|Z_ERPL_REV_SQL|PROG/P||DuckDB SQL console"
Expand Down
13 changes: 8 additions & 5 deletions docs/INSTALL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,14 @@ Two parts: (1) get the **ABAP objects** into the SAP system, (2) install the
> ```
>
> **`setup` needs `S_DEVELOP`** (OBJTYPE=CLAS, ACTVT 01 and 02): it creates and
> activates ABAP objects, and `sync`/`replicate` generate a temporary class to
> carry their parameters. That is a developer authorisation and is normally
> absent on production. `doctor` checks it and says so; if it is missing, import
> the transport below instead and use `--print-abap` to get ABAP you can run from
> SE38 as someone who has the rights.
> activates ABAP objects. That is a developer authorisation and is normally
> absent on production, so on production import the transport below instead.
> `doctor` checks it and says so.
>
> The `sync` and `replicate` subcommands do **not** need it once setup (or the
> transport) has deployed `ZCL_ERPL_REV_CLIDRV`: they pass their parameters as
> data through a queue the driver reads. `--queue-only` goes further and does not
> contact SAP at all.
>
> `setup` deploys the production ABAP objects over ADT, creates the function group,
> the type-T destination and the eight `Z_DUCKDB_*` modules, and writes
Expand Down
18 changes: 12 additions & 6 deletions docs/security.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,18 @@ S_RFC: ACTVT=16, RFC_TYPE=FUGR, RFC_NAME=ZERPL_REV
- The **RFC service user** the running server connects as needs only `S_RFC`
(`ACTVT=16`, `RFC_TYPE=FUGR`, `RFC_NAME=ZERPL_REV`) — the eight `Z_DUCKDB_*`
modules and nothing else. It needs **no** developer rights.
- The user who runs **`erpl-rev setup`**, or the `sync`/`replicate` subcommands,
needs **`S_DEVELOP`** (`OBJTYPE=CLAS`, `ACTVT` 01 and 02, plus PROG/INTF/TABL
for the initial deploy), because those create and activate ABAP. This is a
developer authorisation; do not grant it to the service user to make the CLI
work. On a production system, import the transport and drive the reports from
SE38 instead — `--print-abap` prints exactly what to run.
- The user who runs **`erpl-rev setup`** needs **`S_DEVELOP`** (`OBJTYPE=CLAS`,
`ACTVT` 01 and 02, plus PROG/INTF/TABL), because setup creates and activates
ABAP. This is a developer authorisation; do not grant it to the service user.
On a production system, import the transport instead (docs/INSTALL.md) and
never run setup there at all.
- The `sync` and `replicate` subcommands need **no SAP authorisation** once
`ZCL_ERPL_REV_CLIDRV` is deployed. They write the command into a DuckDB table
and the driver executes it, so the parameters travel as data and nothing is
created in SAP. With `--queue-only` the CLI does not contact SAP at all: the
periodic `ERPL_REV_DELTA` job drains the queue. Without the driver they fall
back to generating a temporary class, which does need `S_DEVELOP` — `doctor`
reports which of the two applies.
- `erpl-rev doctor` needs neither: it only reads, and it reports whether
`S_DEVELOP` is present so the gap is visible before anyone tries to deploy.

Expand Down
Loading
Loading