feat: add support for the embedding batch task - #516
Conversation
Register the tigerflow-ml embed task and add a launcher entry (text and image/PDF inputs, encode mode, normalize, truncate dimension). Default embed output to .npy: the task rejects any other suffix, so an unset default would make tigerflow write .out and fail every file. Expose encode mode as one select rather than two booleans, since the task raises when use_encode_document and use_encode_query are both true. Closes #463.
|
Review Nicely scoped change — task registry, launcher UI, and docs are all updated together, and the Possible bug:
So a user-entered This may be fine if the tigerflow-ml task itself does its own Minor / non-blocking
Everything else looked solid
|
|
`encode_kwargs` — false positive on the parsing concern. Verified against `tigerflow_ml.multimodal.embed._base.Params` at v0.2.0: ```python The field is declared as `str` on purpose. Line 105 hands it to `parse_kwargs`, which tries `json.loads` first, then Python literal syntax. So the string form is expected, and — notably — the help text example (`{'prompt':'query: '}`) is Python-literal, not JSON. Adding client-side `JSON.parse` in Blackfish would be stricter than what tigerflow-ml accepts and would reject the very syntax the help text advertises. Leaving it as-is; malformed input surfaces as a clear tigerflow-ml error at task setup. `chat` row in CLI docs table — fixed. Added in ac42a00 while we were already editing that table. embed row misalignment — skipping. Cosmetic; pipe tables render fine. |
Summary
Test plan
Notes
Closes #463