Skip to content

OSM tag extraction #9

Description

@skivdal

Depends on #8.

Running the results of Object Detection through GPT for translating the detected object names into OpenStreetMap tags might prove useful as a method of geolocating a video without metadata or hints from audio (#3).

Once we've got applicable OSM tags, we can search through OpenStreetMap to find locations that contain all these elements, like done in this project: https://github.com/bellingcat/osm-search

There's already a panel for this in the frontend, with its own message to the backend. Matching on "osm" here would grab the request:

# might be cached
match data["messageType"]:
case "metadata":
task_id: str = get_metadata.apply_async((data["videoId"], )).id # ignore the error on this line
case "transcription":
task_id: str = get_transcription.apply_async((data["videoId"], )).id # ignore the error on this line
case "map":
task_id: str = get_coordinates.apply_async((data["videoId"], )).id # ignore the error on this line
case "keyframes":
task_id: str = get_keyframes.apply_async((data["videoId"], )).id # ignore the error on this line
case _:
raise NotImplementedError(f"The messageType {data['messageType']} is not yet implemented")

We would then create a task in the Celery application, dependent on the existence of object detection results in the jobs cache, to do the actual work. Because of this dependency, tasks started from the main request handler (above) would most likely always exit without a result. Realistically, most results are produced from running the job on completion of the Object Detection task from here:

# TODO: handle dependent jobs
match message_type:
case "transcription":
pass # TODO: rerun map task
case "keyframes":
pass # TODO: rerun object detection
case "objectdetection":
pass # TODO: rerun osm

In the frontend, the appropriate message response type should be created, and the panel should show the extracted tag names.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions