GPT-4o API 帧上传数量限制问题

[wang2q]

Hello, in the experiment section, can GPT-4o handle uploading 120 frames at once? Why can I only upload up to 50 frames when I call the API?

[JUNJIE99]

In our experiments, we resized the resolution of all frames to 512×512, which greatly reduces the number of visual tokens. We did not find any specific limitation on the number of frames for GPT-4o in our experiments—as long as the total number of tokens stays within the context window, it works fine. (Since our experiments were conducted in mid-2024, we are not sure if there are any new restrictions in the current API, but you can try compressing the frame resolution first.)

[wang2q]

I am wondering if your code can be open-sourced. On my side, I am using the OpenAI SDK to upload images like this:

base64_images = [encode_image(img_path) for img_path in image_paths] 
image_dicts = [
    {
        "type": "image_url",
        "image_url": {
            "url": f"data:image/png;base64,{x}"
        }
    }
    for x in base64_images
]
messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": prompt},
            *image_dicts
        ],
    }
]

Even after reducing the resolution, the same issue persists. How do you handle image uploads on your side?Thank you very much for your guidance!

[JUNJIE99]

Due to a server migration, we’re unable to locate the original GPT-4o invocation code from that period. However, I recall that as long as the images are properly resized, the model can process up to around 500 frames. For details on OpenAI’s token calculation method, please refer to: https://platform.openai.com/docs/guides/images-vision#gpt-4o-gpt-4-1-gpt-4o-mini-cua-and-o-series-except-o4-mini.

From our experience and reports from other groups, it appears that GPT-4o does not have a strict 50-frame limit. However, since our experiments were conducted about a year and a half ago, we cannot confirm whether OpenAI has since introduced any new restrictions to their API.