docs(audiences): document Salesforce export full-load behavior, API monitoring, and Last Enriched timestamp - #2356
Open
george-dilthey wants to merge 3 commits into
Open
Conversation
…onitoring, and Last Enriched timestamp pattern - Clarify that the first Audiences→Salesforce export is a full load (all records), explaining why it takes longer at high volume - Remove wrong '~10,000 records per batch' export batch size claim (verifier: FAIL — exports batch by 100 MB file size, not record count) - Add API usage card monitoring info for tracking export progress and daily Bulk API quota - Add new FAQ: 'How do I keep a Clay Last Enriched date current in Salesforce?' with formula column timestamp pattern - Fix 'Do the API math' best practice to remove wrong '10,000 for export' estimate Trigger: Intercom conversation #215475628567527 — customer asked about sync duration, progress monitoring, and Last Enriched timestamp; Fin could not answer any of these.
Contributor
george-dilthey
marked this pull request as ready for review
August 27, 2026 00:20
This was referenced Aug 27, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Added three pieces of missing information to
docs/audiences.mdabout Audiences→Salesforce export sync, plus corrected a wrong batch-size claim already in the docs.1. First export is a full load: Expanded the "First export" bullet in "Export sync behavior" to explicitly state the first export processes all records as a full load — not just recently changed ones — explaining why it takes longer at high volume. Previously the docs said only "Subsequent syncs: Incremental," implying but never stating the first is a full load.
2. API usage card for monitoring: Added a new "Monitoring" bullet describing the API usage card on the Salesforce source settings page — shows daily Bulk API quota and calls in the last 24 hours. Also replaced the wrong "To estimate API calls, divide record count by 10,000" line with guidance to use the API usage card. Previously absent from the docs entirely.
3. New FAQ — "How do I keep a 'Clay Last Enriched' date current in Salesforce?": Describes the formula-column timestamp pattern: add a formula column returning the current timestamp (DateTime:
new Date()?.toISOString?.()or Date:new Date()?.toISOString?.()?.split?.("T")?.[0]), map with Always write. Includes the caveat that the timestamp updates on every re-run even when enrichment data hasn't changed.4. Corrected wrong batch-size claim (existing bug): The existing doc stated "Export batch size: ~10,000 records per batch" and the "Do the API math" best practice said "or by 10,000 for export." Verifier returned FAIL — exports batch by 100 MB file size, not record count. The 10,000 figure is the Salesforce import prefetch limit, not the export batch size. Removed the wrong bullet and updated the API math best practice accordingly.
Why
Intercom conversation #215475628567527: Wendy Lac (Anrok, Enterprise) asked three questions about Audiences→Salesforce sync that Fin could not answer, requiring escalation to a human agent (Bo Morin). All three were legitimate operational questions about documented behavior that simply wasn't documented:
Source link(s)
services/audiences/src/materialization/services/export-handler.ts:3307-3309—logger.info('No previous exports found, performing full export', ...)whenlastExportis null; incremental filterf.updated_at > lastExportCreatedAton subsequent runsapps/frontend/src/components/Audiences/AudienceFieldMappingForms/SalesforceApiUsageCard.tsx— renders "API usage", "Last 24 hours", "Calls used: X / Y"libs/api-contract/src/audiences/lib/import-types.ts:881-882;services/scheduler/src/bull/queues/audience-export-queue.ts:13export-handler.ts:349buildAlwaysWriteFieldColumn;import-types.ts:49MappingRule.AlwaysWriteservices/big-source-sync/src/services/domain/salesforce/salesforce-bulk-export.ts:113—chunkSizeMB: 100(file-size based, not record-count based)Uncertainties
None — all changes verified against product code. Audiences overall requires
EnableAudiencesGAper-workspace flag (default false), but this is an entitlement gate for the whole Audiences product, not a flag for any individual behavior documented here.Note on batch-size correction: the human support agent cited "roughly 10,000 records per batch" — a plausible estimate that presumably reflects typical record sizes relative to the 100 MB chunk limit. The code unambiguously uses file-size chunking, not record-count chunking, so the existing claim was removed rather than corrected with a potentially misleading approximation.
Related observations
Many other PRs are currently open on
docs/audiences.md(e.g., #2179, #2184, #2192, #2202–#2205, #2210, #2215, #2224–#2225, #2231–#2232, #2245, #2251, #2261, #2272, #2280, #2288–#2291, #2295, #2303, #2305, #2321, #2323, #2345, #2348, #2349). All cover separate, non-overlapping topics — verified. This PR's changes (export sync behavior, API monitoring card, "Clay Last Enriched" FAQ, batch-size correction) appear in none of them. Reviewers should expect merge conflicts given the high concurrent edit volume on this file.PR #2303 adds a formula-column date pattern for a different use case (timed segment entry via Upsert Audiences Record). The new FAQ in this PR addresses a different question: keeping a Salesforce timestamp current via the native Audiences export sync. Different mechanism (export mapping vs Upsert), different customer question.
Closes DOC-2110