fix(core): set 1.201s default HTTP agent timeout, disable keep-alive for AI Core requests by InjunPark-sap · Pull Request #2266 · SAP/ai-sdk-js

InjunPark-sap · 2026-09-07T15:38:29Z

Context

Closes SAP/ai-sdk-js-backlog#623.

What this PR does and why it is needed

The SDK inherited the SAP Cloud SDK's 5s HTTP socket timeout, which is too short for long-running chat and streaming completions. This PR sets a 12-minute (720000 ms) default socket timeout on the service-binding-derived AI Core destination (covering both client-secret and mTLS auth).

Keep-alive is disabled on these requests: with the old 5s timeout, idle pooled sockets were evicted before any load balancer closed them; the longer timeout would otherwise keep stale sockets around long enough to be reused and trigger ECONNRESET. Disabling pooling avoids stale-socket reuse while the long timeout still covers slow responses. Rationale and the researched load-balancer idle timeouts are in the decision record.

Both defaults apply only to the SDK-managed destination and can be overridden via a custom destination's agentOptions or per-request CustomRequestConfig.httpsAgent.

Open for reviewers: confirm disabling keep-alive is acceptable vs. adding agentkeepalive to preserve pooling (would need SAP BTP's actual LB idle timeout to tune). Docs handled in a separate SAP/ai-sdk PR — link to follow.

Definition of Done

Code is tested (Unit, E2E)
Error handling created / updated & covered by the tests above
Documentation updated
- Only Public APIs are allowed to be used in documentation/tutorials/sample code
(Optional) Aligned changes with the Java SDK
(Optional) Release notes updated

…or AI Core requests (#623)

hyperspace-pr-bot · 2026-09-07T15:55:57Z

Summary

The following content is AI-generated and provides a summary of the pull request:

Context

Closes SAP/ai-sdk-js-backlog#623.

What this PR does and why it is needed

Sets a 12-minute (720,000 ms) default HTTP socket timeout for AI Core requests, overriding the inherited SAP Cloud SDK default of 5 seconds, which is too short for long-running chat and streaming completions.

Keep-alive is also disabled on SDK-managed destinations. With the previous 5s timeout, stale pooled sockets were evicted quickly; the longer timeout would otherwise keep those sockets alive past the load balancer's idle timeout, causing ECONNRESET errors on reuse. Disabling connection pooling avoids this class of stale-socket failures.

Changes

packages/core/src/context.ts: Introduces two constants (DEFAULT_AGENT_TIMEOUT = 720_000 and DEFAULT_AGENT_KEEP_ALIVE = false) and merges them into the agentOptions of the service-binding-derived destination. Pre-existing agentOptions on the destination take precedence (spread order ensures user values win).
packages/core/src/context.test.ts: Adds test coverage for:
- Default timeout injection for client-secret and mTLS-shaped destinations
- No injection when a user-provided destination is passed directly
- Pre-existing agentOptions are not overwritten
.changeset/default-agent-timeout.md: Patch-level changelog entry for @sap-ai-sdk/core.

ℹ️ Both defaults apply only to the SDK-managed service-binding destination and can be overridden via a custom destination's agentOptions or per-request CustomRequestConfig.httpsAgent.

Category: 🐛 Bug Fix

🔄 Regenerate and Update Summary
✏️ Insert as PR Description (deletes this comment)
🗑️ Delete comment

PR Bot Information

Version: 1.31.14

Output Template: Repository PR Template
Correlation ID: 946bafb0-aad4-11f1-8f9d-23576be03029
Event Trigger: pull_request.ready_for_review
File Content Strategy: Full file content
Summary Prompt: Default Prompt
LLM: anthropic--claude-4.6-sonnet
GithubContextProvider: chore: Document the correct Azure OpenAI spec version #623

davidkna-sap · 2026-09-08T11:18:29Z

+// Disabled so the long timeout above does not keep stale pooled sockets alive past
+// a load balancer's idle timeout, which would cause ECONNRESET on reuse.
+const DEFAULT_AGENT_KEEP_ALIVE = false;


I would prefer enabling this by default. Please investigate if increasing the timeout actually leads to issues and if the keep alive timeout can be configured in isolation.

As far as I know, this is also the default.

Node core can't isolate idle vs active timeout — one timeout governs both (keepAliveMsecs is only the TCP probe delay). Isolation needs agentkeepalive(new dep).

12min + keepAlive:true has a documented risk (ECONNRESET while doing http 1.1 keep alive requests and server closes the connections nodejs/node#47130): if the LB drops idle conns first, the reused half-open socket → ECONNRESET, no core fix. The old 5s timeout avoided this implicitly.

So it hinges on the AI Core LB idle timeout. If it is more than 12min, keepAlive:true + 1200s is safe and I'll switch; if shorter, we add agentkeepalive or keep keepAlive:false. WDYT?

Sounds good, sadly axios does not appear to handle this, please add a comment here to re-visit this if we do switch to a different HTTP library like unidici.
The fetch adapter would be an option for non-mTLS connections, but that could also lead to issues with any custom HTTP middleware users might have.

Done. Added a comment to revisit keepAlive if we switch HTTP library (undici / fetch adapter).

Please add the TODO: string to make it easier to find.

Done — prefixed the revisit comment with TODO:.

davidkna-sap · 2026-09-08T11:23:43Z

    }
  )) as HttpDestination;
-  return aiCoreDestination;
+  return {


[req] Ensure the other axios options mentioned in the documentation are added; I think we might already set timeout: 0 somewhere.

timeout: 0 is the axios request-level timeout in Cloud SDK default — no request deadline, deliberate for streaming.

Separate layer from the agent socket timeouthere, no conflict. CustomRequestConfig already exposes httpsAgent/httpAgent/proxy/signal/middleware; axios timeout isn't picked.

Do you want it to be added, or other options?

https://github.com/cline/cline/blob/cdcaa744223465a9bf5beb4d5b5a5316d4a4ea65/apps/vscode/src/shared/net.ts#L185-L186

Added maxContentLength/maxBodyLength: Infinity to the default request config.

Note it's a no-op on the current http adapter (axios defaults both to -1 = unlimited), but makes the intent explicit and future-proofs a switch to undici/fetch.

timeout: 0 is already set by the Cloud SDK (https://github.com/SAP/cloud-sdk-js/blob/052efd32b668687051e9f0f91883a241ecd6a5f4/packages/http-client/src/http-client.ts#L494).

Co-authored-by: David Knaack <david.knaack@sap.com>

InjunPark-sap self-assigned this Sep 7, 2026

fix(core): set 12min default HTTP agent timeout, disable keep-alive f…

438b5fc

…or AI Core requests (#623)

InjunPark-sap force-pushed the feat/default-agent-timeout-623 branch from 12c0cbe to 438b5fc Compare September 7, 2026 15:48

InjunPark-sap marked this pull request as ready for review September 7, 2026 15:55

InjunPark-sap requested a review from davidkna-sap September 7, 2026 15:55

InjunPark-sap mentioned this pull request Sep 8, 2026

docs: document default http agent timeout for AI Core requests (#623) SAP/ai-sdk#721

Draft

davidkna-sap requested changes Sep 8, 2026

View reviewed changes

Apply suggestion from @davidkna-sap

93fd2f9

Co-authored-by: David Knaack <david.knaack@sap.com>

InjunPark-sap requested a review from KavithaSiva September 8, 2026 12:50

InjunPark-sap added 2 commits September 8, 2026 15:05

test(core): update agent timeout assertions to 1201s

5212189

fix(core): default unlimited request/response size for AI Core requests

be0cd9b

davidkna-sap changed the title ~~fix(core): set 12min default HTTP agent timeout, disable keep-alive for AI Core requests~~ fix(core): set 1.201s default HTTP agent timeout, disable keep-alive for AI Core requests Sep 8, 2026

fix from feedback

5a15857

Navigation Menu

Uh oh!

fix(core): set 1.201s default HTTP agent timeout, disable keep-alive for AI Core requests - #2266

fix(core): set 1.201s default HTTP agent timeout, disable keep-alive for AI Core requests#2266
InjunPark-sap wants to merge 5 commits into
mainfrom
feat/default-agent-timeout-623

InjunPark-sap commented Sep 7, 2026

Uh oh!

hyperspace-pr-bot Bot commented Sep 7, 2026

Uh oh!

Uh oh!

davidkna-sap Sep 8, 2026

Uh oh!

InjunPark-sap Sep 8, 2026

Uh oh!

davidkna-sap Sep 8, 2026

Uh oh!

InjunPark-sap Sep 8, 2026

Uh oh!

davidkna-sap Sep 8, 2026

Uh oh!

InjunPark-sap Sep 9, 2026

Uh oh!

davidkna-sap Sep 8, 2026

Uh oh!

InjunPark-sap Sep 8, 2026

Uh oh!

davidkna-sap Sep 8, 2026

Uh oh!

InjunPark-sap Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

InjunPark-sap commented Sep 7, 2026

Context

What this PR does and why it is needed

Definition of Done

Uh oh!

hyperspace-pr-bot Bot commented Sep 7, 2026

Summary

Context

What this PR does and why it is needed

Changes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants