tap-salesforce

Singer tap that extracts data from a Salesforce database and produces JSON-formatted data following the Singer spec.

$ mkvirtualenv -p python3 tap-salesforce
$ pip install tap-salesforce
$ tap-salesforce --config config.json --discover
$ tap-salesforce --config config.json --properties properties.json --state state.json

Quickstart

Install the tap

> pip install tap-salesforce

Create a Config file

{
  "client_id": "secret_client_id",
  "client_secret": "secret_client_secret",
  "instance_url": "https://test.my.salesforce.com",
  "start_date": "2017-11-02T00:00:00Z",
  "api_type": "BULK",
  "select_fields_by_default": true,
  "lookback_window": 10
}

The client_id and client_secret keys are your ECA(External Client App)'s secrets. Find details here on how to create External Client App.

When creating the ECA, enable the following under API (Enable OAuth Settings):

• OAuth Scopes: Add Manage user data via APIs (api) — this is the only scope required
• Enable Client Credentials Flow — required for server-to-server authentication without user interaction
• Run As: Set to a dedicated integration user whose profile has API Enabled

The instance_url is My Domain URL which you can find at My Domain → My Domain Settings → My Domain Details → Current My Domain URL

Existing authorization-code configurations remain supported. Provide refresh_token instead of instance_url, and set is_sandbox to true when authenticating to a sandbox:

{
  "client_id": "secret_client_id",
  "client_secret": "secret_client_secret",
  "refresh_token": "secret_refresh_token",
  "is_sandbox": "false",
  "start_date": "2017-11-02T00:00:00Z",
  "api_type": "BULK",
  "select_fields_by_default": true
}

The start_date is used by the tap as a bound on SOQL queries when searching for records. This should be an RFC3339 formatted date-time, like "2018-01-08T00:00:00Z". For more details, see the Singer best practices for dates.

The api_type is used to switch the behavior of the tap between using Salesforce's "REST" and "BULK" APIs. When new fields are discovered in Salesforce objects, the select_fields_by_default key describes whether or not the tap will select those fields by default.

The lookback_window (in seconds) subtracts the desired amount of seconds from the bookmark to sync past data. Recommended value: 10 seconds.

Run Discovery

To run discovery mode, execute the tap with the config file.

> tap-salesforce --config config.json --discover > properties.json

Sync Data

To sync data, select fields in the properties.json output and run the tap.

> tap-salesforce --config config.json --properties properties.json [--state state.json]

Copyright © 2017 Stitch