Skip to content

Handle content_filter errors during document indexing #15

Description

@Allob

DescriptionRetirever indexing uses gpt-4o-mini to get image descriptions. Some documents may trigger content filter error which will look like that in the stream:

{'error': {'inner_error': {'code': 'ResponsibleAIPolicyViolation', 'content_filter_results': {'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': True, 'severity': 'high'}, 'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}}}, 'code': 'content_filter', 'message': "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: \r\nhttps://go.microsoft.com/fwlink/?linkid=2198766.", 'param': 'prompt', 'type': None}}

The main indicator is code=content_filter.

Currently, such error will be reported to the UI as the following message:

I'm sorry, but I can't process the documents because of the following errors:
<document>: Internal error

Dial RAG should be able to detect such errors in the stream and show some user-friendly message.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions