1376 bulk editor collect ai prompt content client side via yoastseo instead of the php mirror - #23524
Open
FAMarfuaty wants to merge 9 commits into
Conversation
…dmin Lets the bulk editor reuse the loop and the 300/150 token budgets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…service So field re-scoring and prompt content collection share one instance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Lets the parser strip shortcode delimiters but keep the text they enclose, matching the post editor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Parses a post's raw content on the main thread via ensureTree, so the bulk editor collects the same prompt content as the in-editor generator. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds helpers and constants to the existing window.yoast.bulkEditor bridge, so Premium needs no yoastseo import and the token budgets stay defined once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Kept out of the posts route so the table payload stays lean. Inaccessible IDs are omitted rather than failing the batch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Coverage Report for CI Build 0Coverage increased (+0.02%) to 55.677%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
The same helper the in-editor AI tip measures with, so Premium's bulk AI judges "limited content" identically instead of measuring server-side. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FAMarfuaty
added
changelog: non-user-facing
FAMarfuaty
marked this pull request as ready for review
July 30, 2026 13:37
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aligns Premium’s bulk-editor AI prompt-content generation with the in-editor AI generator by moving token-budgeted sentence accumulation and visible-content measurement into shared JS helpers, and by exposing bulk-editor-side prompt preparation + constants via the existing window.yoast.bulkEditor bridge. It also introduces a dedicated REST route to fetch raw post_content client-side (plus localized shortcode tags) so the browser can parse content using the analysis engine rather than a server-side mirror.
Changes:
- Add a new Bulk Editor REST route (
/yoast/v1/bulk_editor/posts_content) to fetch raw post content in batches, omitting inaccessible IDs. - Introduce shared-admin JS helpers for prompt-content accumulation and visible-content length, and wire them into both the in-editor AI generator and the bulk editor.
- Expose
preparePromptContent,getVisibleContentLength, and token-budget constants onwindow.yoast.bulkEditorfor Premium consumption, plus localize shortcode tags to the bulk editor page.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Unit/Bulk_Editor/User_Interface/Posts_Content_Route/Register_Routes_Test.php | Adds unit coverage for REST route registration arguments. |
| tests/Unit/Bulk_Editor/User_Interface/Posts_Content_Route/Get_Posts_Content_Test.php | Adds unit coverage for content retrieval behavior (raw content, omissions, dedupe, casting). |
| tests/Unit/Bulk_Editor/User_Interface/Posts_Content_Route/Check_Permissions_Test.php | Adds unit coverage for route permission callback. |
| tests/Unit/Bulk_Editor/User_Interface/Posts_Content_Route/Abstract_Posts_Content_Route_Test.php | Introduces shared test setup for the new route unit tests. |
| tests/Unit/Bulk_Editor/User_Interface/Bulk_Editor_Integration/Enqueue_Assets_Test.php | Updates localized bulk editor script data expectations to include shortcode tags. |
| src/bulk-editor/user-interface/posts-content-route.php | Implements the new REST route returning raw post_content for requested IDs. |
| src/bulk-editor/user-interface/bulk-editor-integration.php | Localizes registered shortcode tags into bulk editor analysis config. |
| src/bulk-editor/infrastructure/endpoints/posts-content-endpoint.php | Adds an endpoint descriptor for the new posts-content route (for localized endpoints list). |
| packages/js/tests/shared-admin/helpers/prompt-content.test.js | Adds tests for shared prompt-content accumulation logic and budgets. |
| packages/js/tests/shared-admin/helpers/get-visible-content-length.test.js | Updates tests to the new shared-admin getVisibleContentLength location. |
| packages/js/tests/bulk-editor/services/prompt-content.test.js | Adds tests for bulk-editor prompt-content preparation via main-thread parsing. |
| packages/js/tests/bulk-editor/initialize.test.js | Ensures the bulk-editor bridge exposes helpers/constants to Premium without clobbering existing entries. |
| packages/js/src/shared-admin/helpers/prompt-content.js | Introduces shared token-budgeted prompt-content accumulator and constants. |
| packages/js/src/shared-admin/helpers/index.js | Re-exports new shared helpers/constants from the shared-admin barrel. |
| packages/js/src/shared-admin/helpers/get-visible-content-length.js | Moves/introduces visible-content length measurement using analysis sanitization. |
| packages/js/src/bulk-editor/services/researcher.js | Extracts shared researcher singleton construction for bulk-editor main-thread analysis use. |
| packages/js/src/bulk-editor/services/prompt-content.js | Implements main-thread prompt-content preparation using ensureTree() + shared accumulator. |
| packages/js/src/bulk-editor/services/field-scores.js | Refactors field scoring to reuse the extracted researcher singleton. |
| packages/js/src/bulk-editor/initialize.js | Extends the Premium bridge with prompt-content helpers and token budgets. |
| packages/js/src/ai-generator/helpers/prepare-prompt-content.js | Switches in-editor accumulation to the shared prompt-content helper (worker still supplies paragraphs). |
| packages/js/src/ai-generator/helpers/index.js | Stops exporting getVisibleContentLength from the AI generator helpers barrel (moved). |
| packages/js/src/ai-generator/constants/index.js | Removes duplicated token-budget constants in favor of the shared helper. |
| packages/js/src/ai-generator/components/tip-notification.js | Updates visible-content length import to the shared-admin helper module. |
| * @return WP_REST_Response The posts and their raw content. | ||
| */ | ||
| public function get_posts_content( WP_REST_Request $request ): WP_REST_Response { | ||
| $post_ids = \array_unique( \array_map( '\intval', (array) $request->get_param( 'ids' ) ) ); |
| continue; | ||
| } | ||
|
|
||
| $post = \get_post( $post_id ); |
Comment on lines
+43
to
+65
| export const collectPromptContent = ( paragraphs, maxTokens ) => { | ||
| let promptContent = ""; | ||
| let tokenCount = 0; | ||
|
|
||
| ( paragraphs || [] ).forEach( ( paragraph ) => { | ||
| ( paragraph.sentences || [] ).forEach( ( sentence ) => { | ||
| tokenCount += sentence.tokens.length; | ||
| // Stop when the sentences so far exceed the maximum allowed number of tokens. | ||
| if ( tokenCount > maxTokens ) { | ||
| return; | ||
| } | ||
|
|
||
| promptContent += sanitizeText( sentence.text ); | ||
| } ); | ||
|
|
||
| // Add a space between paragraphs, which counts as a token itself. | ||
| promptContent += " "; | ||
| tokenCount += 1; | ||
| } ); | ||
|
|
||
| // To prevent a completely empty prompt content, fall back to a single full stop. | ||
| return promptContent.trimEnd() || "."; | ||
| }; |
Comment on lines
5
to
+7
| export { getModalNotificationPosition } from "./get-modal-notification-position"; | ||
| export { collectPromptContent, MAX_TOKENS_DEFAULT, MAX_TOKENS_IRREGULAR } from "./prompt-content"; | ||
| export { getVisibleContentLength } from "./get-visible-content-length"; |
Comment on lines
+31
to
+35
| // The registered shortcode tags are read straight off the WordPress global. | ||
| $GLOBALS['shortcode_tags'] = [ | ||
| 'gallery' => 'gallery_shortcode', | ||
| 'caption' => 'caption_shortcode', | ||
| ]; |
Comment on lines
+11
to
+20
| class Posts_Content_Endpoint implements Bulk_Editor_Endpoint_Interface { | ||
|
|
||
| /** | ||
| * Gets the name. | ||
| * | ||
| * @return string | ||
| */ | ||
| public function get_name(): string { | ||
| return 'posts_content'; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
yoastseotokenizer that had already drifted from the in-editor AI generator (mid-sentence cuts, a flat token cap, a duplicated tokenizer). Part of Yoast/reserved-tasks#1376.Summary
This PR can be summarized in the following changelog entry:
Relevant technical choices:
collectPromptContent()and the 300/150 token budgets moved toshared-admin/helpersso the in-editor generator and the bulk editor share one definition.prepare-prompt-content.jskeeps using the analysis worker to obtain paragraphs — only the accumulation is shared, so the in-editor flow is behaviourally untouched and its existing tests guard that.ensureTree()instead of spinning up an analysis worker. The worker'srunResearchdoes nothing more than this rebind-and-call, and a worker round trip per post would cost more than the 2–3 ms the parse actually takes.window.yoast.bulkEditorbridge rather than a newyoastseoexport. That keeps the surface out of the package's public API, and keeps the analysis package out of Premium's bundle entirely — Premium imports nothing fromyoastseo.posts_contentis a separate route rather than a flag on the posts route: the table listing never needs content, so this keeps it out of every page load. Inaccessible IDs are omitted rather than failing the request, so one locked post cannot cost the caller its whole batch.strip_shortcodes()deletes the enclosed text, so without this a post whose body sits inside a[caption]would have sent nothing at all.getVisibleContentLengthalso moved toshared-adminand onto the bridge, so Premium measures "limited content" with the same helper as the in-editor tip rather than a server-side approximation that strips markup differently.Test instructions
Test instructions for the acceptance test before the PR gets merged
Prerequisites
A. The new content route
A1. On the bulk editor page, grab a nonce from the console with
window.wpseoBulkEditorData.nonce, then request:/wp-json/yoast/v1/bulk_editor/posts_content?ids=<id1>,<id2>{ posts: [ { id, content } ] }contentpost_content, block comments and shortcodes intactA2. Confirm the table page itself does not request content: reload the bulk editor and check that
posts_contentis only called when Premium generates, never on page load.B. The bridge exposed to Premium
B1. In the console on the bulk editor page, check:
helpers/constantsabsent; no shortcode list localisedcomponents/hooksbridge entries still thereB2. Confirm the shortcode list makes the parser keep enclosed text:
Expected:
"Wrapped wording stays."— the text without the brackets. Before this PR there was nothing to call.C. Unhappy paths
C1. No permission. As an Author, request
posts_contentfor a post owned by someone else. That ID must be absent frompostswhile the others are returned — it must not fail the whole request.C2. Not logged in / insufficient capability. Request the route while logged out. Expect a
401/403, not data.C3. Deleted or invalid IDs. Request with a non-existent ID (e.g.
999999) mixed with a valid one. The valid one comes back, the missing one is simply absent.C4. Schema limits. Request with 21 IDs and confirm a
400(max 20). Request with noidsand confirm a400(required).C5. Unparseable content. Call
preparePromptContentwith junk, e.g."<p>unclosed <div>",""andnull. It must resolve to a string and never throw; an empty result comes back as".".D. Regressions to watch
D1. In-editor AI generator. Open a post with a focus keyphrase, click Generate meta description, and confirm suggestions still appear and are based on the opening sentences. Its accumulation loop and the 300/150 budgets moved to shared code, so this is the main thing to re-check.
D2. In-editor AI tip. Confirm the "not enough content" tip still appears for a very short post and not for a long one —
getVisibleContentLengthmoved modules.D3. Bulk editor field re-scoring. Edit an SEO title or meta description in the bulk editor and confirm its score still updates. The researcher singleton it uses was extracted into its own module and is now shared with the prompt-content service.
D4. Products and terms. In the post editor for a product (with WooCommerce) and for a term, confirm the in-editor generator still applies the lower 150-token budget.
Relevant test scenarios
The console is needed for the bridge checks (B) and because the prompt-content service parses on the main thread — a parser error surfaces there rather than in the UI. Content types matter for D4, where products and terms take a different token budget.
Test instructions for QA when the code is in the RC
QA can test this PR by following these steps:
Impact check
This PR affects the following parts of the plugin, which may require extra testing:
getVisibleContentLengthmoved modules. Same function, new location.Other environments
[shopify-seo], added test instructions for Shopify and attached theShopifylabel to this PR.[yoast-doc-extension], added test instructions for Yoast SEO for Google Docs and attached theGoogle Docs Add-onlabel to this PR.Documentation
The
window.yoast.bulkEditorbridge contract is documented where it is exposed inbulk-editor/initialize.js, including that it is unversioned and that Free and Premium ship in lockstep.Quality assurance
grunt build:imagesand committed the results, if my PR introduces or edits images or SVGs.Not behind a feature flag; no images or SVGs touched. Verified in a real install:
helpers,constants) and the localised shortcode list are all present and correct on the page, and the existingcomponents/hooksentries still are.。; with the site in Indonesian,Buku-buku itutokenizes as["Buku-buku", " ", "itu"].Innovation
innovationlabel.Part of Yoast/reserved-tasks#1376 (closed by Yoast/wordpress-seo-premium#5044).