Skip to content

add support for OCR/Vision Prompting #51

Description

@koushal2018

To add vision/OCR support, you'd need to modify at minimum:

  1. BedrockConverseHandler._get_messages() — Support image content blocks alongside text:

Bedrock Converse API expects this for images:

{"image": {"format": "png", "source": {"bytes": <image_bytes>}}}
2. PromptAdapter — Add a way to specify that a template variable should be populated with image data
rather than text.
3. DatasetAdapter — Handle dataset rows where an input column contains image file paths, loading the
bytes at inference time.
4. InferenceAdapter — Thread image data through to the Bedrock call.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions