Bug when inference with custom video

[tringuyen98]

After runing about 8 batch, it returns bug

Fail to perform transform [<paddlevideo.loader.pipelines.sample_one_file.OneFileSampler object at 0x7f5ab4d1f3a0>] with error: index 333 is out of bounds for axis 0 with size 327 and stack:
Traceback (most recent call last):
File "/content/Soccernet-features/paddlevideo/loader/pipelines/compose.py", line 69, in call
data = p(data)
File "/content/Soccernet-features/paddlevideo/loader/pipelines/sample_one_file.py", line 171, in call
return self._get(frames_idx, results)
File "/content/Soccernet-features/paddlevideo/loader/pipelines/sample_one_file.py", line 126, in _get
imgbuf = frames[idx]
IndexError: index 333 is out of bounds for axis 0 with size 327

[paulxinzhou]

Hi, thank you for submitting the issue. Here's the steps I suggest to dig deeper.

First, you can focus on running the file that produces this error. The indices are computed from start seconds, end seconds and fps. One of them is inconsistent with the input file hence reading the frame is out of bound for the index.

Let me know if this helps you.

[Daanfb]

Hi, I have the same issue. I tried it with a a video of duration 6:53 so I typed in the json this one ("fps": 25, "length_secs": 413) and I got this error after 15 batches

12/02 12:31:41] fail to perform transform [<paddlevideo.loader.pipelines.sample_one_file.OneFileSampler object at 0x7ee2eebae740>] with error: index 516 is out of bounds for axis 0 with size 513 and stack: Traceback (most recent call last): File "/content/Soccernet-features/paddlevideo/loader/pipelines/compose.py", line 69, in call data = p(data) File "/content/Soccernet-features/paddlevideo/loader/pipelines/sample_one_file.py", line 171, in call return self._get(frames_idx, results) File "/content/Soccernet-features/paddlevideo/loader/pipelines/sample_one_file.py", line 126, in _get imgbuf = frames[idx] IndexError: index 516 is out of bounds for axis 0 with size 513

[paulxinzhou]

Hi @Daanfb, it's a simple io error. We can work through this.

One file failed with number of frames index out of bound. Do you know which file?

You can just run that one file and check the number of frames read in in line 95 for example. I would also check if number of seconds is slightly less than 413. Like actually 412.xx but rounded up?

[Daanfb]

Hi @paulxinzhou. The number of seconds is 413.88, so I wrote 413 in the json.

I printed out the length of the frames variable from File "/content/Soccernet features/paddlevideo/loader/pipelines/sample_one_file.py" and I got 513. I don't know why this is the length when the video is 413 seconds long and 30 fps. Shouldn't it be 413*30 frames?

[paulxinzhou]

Hi @Daanfb, right, 513 sounds wrong, and it should be 413 * 30.

One of the worries I have is having a long-ish video with a high fps make you run out of RAM pretty quickly. There is a possibility you may be hitting the memory limit of your machine and something failed along the way.

I would suggest trying a couple of things. 1. Try with another video that worked to see if len(frames) make sense. 2. Convert this video to a lower fps and see if that can be read correctly.

[Daanfb]

Hi @paulxinzhou. After several tests I came to the following conclusions:

For videos with mp4 format, the maximum duration allowed is the duration in seconds * the number of frames indicated in the .json file < 513. These are several examples in which it would work:

• Duration 3:30 (30 fps), but 2 fps is set in the json file. When printing frames_len it would output 513, but it works.
• Duration 0:15 (30 fps), and 30 fps is set in the json file. frames_len = 461 (correct because actually the duration was 15.38 seconds)

On the other hand, for mkv format files it seems to work fine. I have performed the following tests:

• Duration 4:42 (25 fps), and 25 fps is set in the json file.
  frames_len = 6635. WORKS
• Duration 9:23 (25 fps), and 25 fps is set in the json file.
  After 'opened path' is printed I get the error message Exit_code -9. Maybe it may be because it requires too much RAM.
• Duration 7:32 (25 fps), and 25 fps is set in the file.
  frames_len = 11300. WORKS.

In this last test the System RAM is between 8-11 GB and my system has 12.7GB.

[paulxinzhou]

@Daanfb , that's a great observation. Re mp4 vs mkv, it must be that mkv is more memory efficient. The system memory is used up while decoding mkv / mp4 into numpy. That was my hypothesis and I think it still holds with the new data.