Depends on #8.
Running the results of Object Detection through GPT for translating the detected object names into OpenStreetMap tags might prove useful as a method of geolocating a video without metadata or hints from audio (#3).
Once we've got applicable OSM tags, we can search through OpenStreetMap to find locations that contain all these elements, like done in this project: https://github.com/bellingcat/osm-search
There's already a panel for this in the frontend, with its own message to the backend. Matching on "osm" here would grab the request:
|
# might be cached |
|
match data["messageType"]: |
|
case "metadata": |
|
task_id: str = get_metadata.apply_async((data["videoId"], )).id # ignore the error on this line |
|
case "transcription": |
|
task_id: str = get_transcription.apply_async((data["videoId"], )).id # ignore the error on this line |
|
case "map": |
|
task_id: str = get_coordinates.apply_async((data["videoId"], )).id # ignore the error on this line |
|
case "keyframes": |
|
task_id: str = get_keyframes.apply_async((data["videoId"], )).id # ignore the error on this line |
|
case _: |
|
raise NotImplementedError(f"The messageType {data['messageType']} is not yet implemented") |
We would then create a task in the Celery application, dependent on the existence of object detection results in the jobs cache, to do the actual work. Because of this dependency, tasks started from the main request handler (above) would most likely always exit without a result. Realistically, most results are produced from running the job on completion of the Object Detection task from here:
|
# TODO: handle dependent jobs |
|
match message_type: |
|
case "transcription": |
|
pass # TODO: rerun map task |
|
case "keyframes": |
|
pass # TODO: rerun object detection |
|
case "objectdetection": |
|
pass # TODO: rerun osm |
In the frontend, the appropriate message response type should be created, and the panel should show the extracted tag names.
Depends on #8.
Running the results of Object Detection through GPT for translating the detected object names into OpenStreetMap tags might prove useful as a method of geolocating a video without metadata or hints from audio (#3).
Once we've got applicable OSM tags, we can search through OpenStreetMap to find locations that contain all these elements, like done in this project: https://github.com/bellingcat/osm-search
There's already a panel for this in the frontend, with its own message to the backend. Matching on "osm" here would grab the request:
veridash/backend/veridash_backend/webserver/actions.py
Lines 53 to 64 in daec03a
We would then create a task in the Celery application, dependent on the existence of object detection results in the jobs cache, to do the actual work. Because of this dependency, tasks started from the main request handler (above) would most likely always exit without a result. Realistically, most results are produced from running the job on completion of the Object Detection task from here:
veridash/backend/veridash_backend/webserver/app.py
Lines 91 to 98 in daec03a
In the frontend, the appropriate message response type should be created, and the panel should show the extracted tag names.