Hi, I have the following bug when running inference on a folder of videos over long time:
The setup:
- I have a camera that saves videos every 1minute and use the script inside docker to send videos to the Module 1 or 2:
export FILE_TYPE=video
export SOURCE_ID=${VIDEO_NAME}
export LOCATION=/inputs/${RELATIVE_VIDEO_PATH}
export ZMQ_ENDPOINT=dealer+connect:ipc:///tmp/zmq-sockets/input-video-1.ipc
bash /opt/savant/adapters/gst/sources/media_files.sh
- The module has following environments
module_1: &base_module
privileged: true
build:
context: .
dockerfile: docker/modules/Dockerfile.l4t
volumes:
- zmq_sockets:/tmp/zmq-sockets
- ./checkpoint:/models
- ${INPUT_PATH}:/inputs
- ${OUTPUT_PATH}:/outputs
- ./projects:/opt/savant/projects
environment:
- WEBSERVER_PORT=8888
- MODEL_PATH=/models
- ZMQ_SRC_ENDPOINT=router+bind:ipc:///tmp/zmq-sockets/input-video-1.ipc
- ZMQ_SINK_ENDPOINT=pub+bind:ipc:///tmp/zmq-sockets/output-video-1.ipc
- I have a docker compose file that specifies a video source, and 2 modules. The modules are identical, I just spawn 2 instances so that 2 videos can be processed in parallel. The video source will send a video file to either of module based on its ZMQ_ENDPOINT
// Video source
// either
export ZMQ_ENDPOINT=dealer+connect:ipc:///tmp/zmq-sockets/input-video-1.ipc
// or
export ZMQ_ENDPOINT=dealer+connect:ipc:///tmp/zmq-sockets/input-video-2.ipc
and for the modules
// module 1
ZMQ_SRC_ENDPOINT=router+bind:ipc:///tmp/zmq-sockets/input-video-1.ipc
// module 2
ZMQ_SRC_ENDPOINT=router+bind:ipc:///tmp/zmq-sockets/input-video-2.ipc
The problem:
The inference runs fine after a certain of time, and return the following error shown in the attached file
bug-log.txt
- I confirmed that the video files have no problem, as I ran inference again on these video files without error.
- The problem happens randomly, at different video files, often after a certain amount of time.
Thank you
Hi, I have the following bug when running inference on a folder of videos over long time:
The setup:
and for the modules
The problem:
The inference runs fine after a certain of time, and return the following error shown in the attached file
bug-log.txt
Thank you