update automation workflows to work on v2 with new api

[aleksa-krolls]

Background, context, and business value

Search the ConSoSci v1 jobs for "auto" to view the workflow. See docs for background on how the solution works. Step A4 is a job that creates another job automatically via the openfn api.

The specific request, in as few words as possible

1. Reconfigure/migrate the existing v1 automation workflow to run on app.openfn.org v2 - in this v2 test project
2. Update job "A4" (see L422) for the part of the job that needs to be updated to use the new v2 openfn api

Input

Use this as an input to job A2, which will trigger job A4: 02b-syncToOpenFn.js (which needs to be edited)

{
  "formId": "aJTDVjyPKssq7ukpvJdRKG",
  "formUpdate": true,
  "lastModified": "2024-11-27T14:02:10.908997Z",
  "prefix1": "TT",
  "prefix2": "",
  "tableId": "SurvivalMonitoring",
  "tag": "Survival Monitoring Bismarck (Full inventory)",
  "url": "https://kf.kobotoolbox.org/api/v2/assets/aJTDVjyPKssq7ukpvJdRKG/?format=json"
}

Credentials

1. Job A2 - use the below raw json cred and see LP Trillion Trees Kobo

{"openfnInboxUrl":"{v1-inbox-url}","password":"{kobo-pwd}","type":"basic","username":"{kobo-UN}"}

2. Job A4 - you will need to authorize with the openfn api... not sure what v2 requires (probably an api token), so discuss with TD if not clear.

Don't worry about running Job A3 for now - you can disable this step and focus on Jobs A2 and A4.

adaptor

openfn v2
see slack where i'm trying to find the latest docs

jobs

Job to be updated (labeled step A4 - see consosci project): 02b-syncToOpenFn.js

Testing notes

You should have a workflow in V2 that has at least 2 steps - A2 and A4. (Don't worry about running Job A3 for now - you can disable this step and focus on Jobs A2 and A4. )

1. Run the above Input payload for job A2...this should trigger job A4.
2. A4 should use the output of A2 to generate a job expression, and then create a new workflow in the same project.
3. If you re-try the workflow using the same input, then it should upsert the existing workflow created after the first test.

[aleksa-krolls]

hey @mtuchi this is ready go and see here for new v2 staging project... if you'd like, you can connect it to a staging branch on ConSoSci, or hold off on Github sync for now - whatever is easiest for you. Once this workflow is tested, we'll move it to the prod social-sciences project.

Keep me posted on any questions. Also, it might help to go GET a workflow from v2 to see what the payload looks like in order to determine how to build the payload to POST or PUT the workflow to create/update in step A4.

[aleksa-krolls]

@mtuchi just chatted with Taylor... you should feel free to make changes to the openfn adaptor to work with the new api, as needed. And if you have any questions or feedback on how the new openfn api workflows - then please speak up!

[mtuchi]

@aleksa-krolls what's the toggl project for this ticket ?

[aleksa-krolls]

@mtuchi Lightning Migration

[mtuchi]

Hoooray 🙌🏽 , I am done updating A4 Job which updates/creates a workflow , See latest run here. @aleksa-krolls you can proceed with testing.

Notes

• Because in v2 job names can not contain special characters, I have changed the job name to start with auto- instead of auto/
• Configuration schema for A4 job, { "baseUrl": "https://app.openfn.org", "projectId": "<project id for upserting workflow>", "access_token": "<your openfn personal access token>"}

[mtuchi]

On other front, when i was studying the auto-generated jobs i ended up creating a v2 spec.yaml file for wcs project. So when we are ready to migrate the whole project to v2 you can you use this spec.yaml here

[mtuchi]

@aleksa-krolls do you need help testing ?, I can record a video to show you how to test this