Skip to content

rewrite orchestrion in javascript - #41

Merged
bizob2828 merged 39 commits into
nodejs:mainfrom
rochdev:js-rewrite
Mar 25, 2026
Merged

rewrite orchestrion in javascript#41
bizob2828 merged 39 commits into
nodejs:mainfrom
rochdev:js-rewrite

Conversation

@rochdev

@rochdev rochdev commented Mar 10, 2026

Copy link
Copy Markdown
Contributor

[RFC] Rewrite Orchestrion in JavaScript

A few features were also carried over from the original iteration of the rewrite:

  • Supports an astQuery field to filter AST nodes with an esquery query. This
    is mostly meant to be used when experimenting or if what needs to be queried
    is not a function.
  • Supports replacing methods of child class instances in the base constructor.
  • Supports tracing iterator (sync/async) returning functions (sync/async) will be added in a future PR

@jsumners-nr jsumners-nr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't reviewed any of the functionality at this time. Just some maintenance type suggestions.

I'd like to see jsdoc blocks on all functions, classes, and methods. This will reduce the need to follow all paths to understand what shapes parameters are meant to be and clarify intentions.

Comment thread yarn.lock Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread index.js Outdated
@rochdev
rochdev marked this pull request as ready for review March 12, 2026 16:42
@rochdev rochdev changed the title rewrite in js except parsing that uses oxc with meriyah as fallback rewrite orchestrion in javascript Mar 12, 2026

@bizob2828 bizob2828 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my first pass and didn't get very far because i have a few failures with 2 of our migrated libraries. My biggest comment is why tracing channel api isn't getting used directly to tracePromise, traceCallback, and traceSync. I suspect this nuance is leading to my test failures

Comment thread package.json
Comment thread package.json
Comment thread lib/matcher.js
}

/** Releases all cached transformers, freeing any associated resources. */
free () {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need free methods or was this just to ensure the API was consistent with the rust/wasm one?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's to avoid a breaking change, but they could technically be removed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are still at v0.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I think it's safe to remove. The tracing-hooks package can get updated when we up orchestrion to get rid of calling free on the transformer

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do the break in another PR? I prefer to isolate breaking changes whenever possible instead of just sneaking them in a huge PR where it's less obvious where the breakage happened.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea that's fair

Comment thread package.json Outdated
Comment thread lib/transforms.js Outdated

const options = { module: moduleType === 'esm' }
const index = node.body.findIndex(child => child.directive === 'use strict')
const dcPolyfill = moduleType === 'esm'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this called polyfil when you could pass in an alternate dcModule but not always

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I figured the point was to allow passing dc-polyfill in the first place, but you're right that it could be anything, I'll rename it.

I would personally probably prefer to add an actual dependency on the polyfill and deprecate this option entirely, it's not as trivial as it looks to pass a module by name depending on how the caller is installed (for example mounted as a Docker volume instead of installed in node_modules) and allowing any name to be passed pushes the logic to figure out the actual install path (and that it can be imported properly in both CJS and ESM) to the caller.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't make a dep. We use this package but don't support node 18 so we just rely on the diagnostics channel stdlib

Comment thread lib/transforms.js
Comment thread lib/transforms.js
Comment thread lib/transformer.js Outdated
*/
#visit (state, ...args) {
const transform = this.#customTransforms[state.operator] ?? transforms[state.operator]
const { index } = state.functionQuery

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we had a loosely defined functionQuery where it was only method name. It turns out there were multiple classes in the same file with the method name. The previous iteration of this worked because it defaulted to the 1st index. now it appears to wrap every method in the file. We can def fix this in our function query but just calling out that this behavior is different now

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan of having to explicitly set an option to null when it's a number. I think it's more natural to have it undefined by default, and then any number passed is the value you want. That is a breaking change though, so I will restore the previous behaviour for now and might open another PR if you think that makes sense.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think defaulting to locating the first makes sense. I worry if this changes we'll have to update function queries to specify the index: 0 for most of the cases. I can't imagine a use case where you want to rewrite the same query multiple times for a file but i could be wrong.

Comment thread index.d.ts Outdated
Comment thread lib/transformer.js
Comment thread lib/transforms.js Outdated
Comment thread lib/transformer.js

@bizob2828 bizob2828 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to submit some comments

Comment thread lib/matcher.js
* @param {string} name - Operator name (e.g. `'traceSync'`).
* @param {Function} fn - Transform function `(state, node, parent, ancestry) => void`.
*/
addTransform (name, fn) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't in the rust version. can you explain how to use? Do you think this should be in a follow up?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main reason I put it right away is because we need it for iterator support. My original PR had the support built-in but since the change in Node core was more contentious than expected I shelved it, so we need a way to be able to add support externally. Open to other suggestions, and we could always continue using our internal implementation for a little while instead but I was hoping to switch back to Orchestrion proper as soon as possible.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see

Comment thread lib/transformer.js
ast = parse(code, { ...options, module: !options.module })
}

if (moduleType === 'esm') { // TODO: cjs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is there to do for cjs here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as ESM, so for example:

function t () {}

module.exports.test = t

With the current code, patching test will not work because it's not able to find t from the test export. This was implemented in Rust for ESM but not for CJS, so I did the same here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok. Maybe this is a nuance of ESM because you can still wrap t by locating it by functionName

Comment thread lib/transformer.js Outdated
Comment thread lib/transformer.js
queries.push(
`[id.name="${className}"]`,
`[id.name="${className}"] > ClassExpression`,
`[id.name="${className}"] > ClassBody > [key.name="${method}"][key.type=${type}] > [async]`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's been a while since i've used estree selectors but isn't this only matching methods on classes that are async? I don't see any sync versions of this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All function nodes have an async property, and this only checks for the presence of the property and not its value. It was basically just a hacky way to get the function node without specifying exactly which type since there are multiple.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still unclear. can't it just be omitted?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to ensure that the child is a function without it?

Comment thread lib/matcher.js
}

/** Releases all cached transformers, freeing any associated resources. */
free () {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I think it's safe to remove. The tracing-hooks package can get updated when we up orchestrion to get rid of calling free on the transformer

Comment thread lib/transforms.js
Comment thread lib/transforms.js

@bizob2828 bizob2828 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this works as expected. Thanks for rewriting @rochdev 🙏🏻 . It will be nice to not have such a large binary getting loaded via wasm now.

Comment thread lib/transformer.js
ast = parse(code, { ...options, module: !options.module })
}

if (moduleType === 'esm') { // TODO: cjs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok. Maybe this is a nuance of ESM because you can still wrap t by locating it by functionName

Comment thread lib/transformer.js
queries.push(
`[id.name="${className}"]`,
`[id.name="${className}"] > ClassExpression`,
`[id.name="${className}"] > ClassBody > [key.name="${method}"][key.type=${type}] > [async]`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still unclear. can't it just be omitted?

Comment thread lib/matcher.js
}

/** Releases all cached transformers, freeing any associated resources. */
free () {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea that's fair

Comment thread lib/transforms.js Outdated

const options = { module: moduleType === 'esm' }
const index = node.body.findIndex(child => child.directive === 'use strict')
const dcPolyfill = moduleType === 'esm'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't make a dep. We use this package but don't support node 18 so we just rely on the diagnostics channel stdlib

Comment thread lib/transformer.js Outdated
*/
#visit (state, ...args) {
const transform = this.#customTransforms[state.operator] ?? transforms[state.operator]
const { index } = state.functionQuery

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think defaulting to locating the first makes sense. I worry if this changes we'll have to update function queries to specify the index: 0 for most of the cases. I can't imagine a use case where you want to rewrite the same query multiple times for a file but i could be wrong.

Comment thread lib/matcher.js
* @param {string} name - Operator name (e.g. `'traceSync'`).
* @param {Function} fn - Transform function `(state, node, parent, ancestry) => void`.
*/
addTransform (name, fn) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see

Comment thread lib/transforms.js

return ${channelVariable}.start.runStores(__apm$ctx, () => {
try {
let promise = __apm$traced();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had hoped we could remove shim code we had in our agent for libraries like ioredis and openai but we still need it. Now that this is js, I hope to provide a PR in the future to make it work for libraries that mess up promises

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants