Weather Estimation using Clothing

Group Name: Bears

Group Members: Logan Lucas, Eric Chavez, Ron Pham, Vidhi Patel

Info

About

This project analyzes video footage to detect what kind of clothing is most common in a given space. We then use this data to:

• Estimate weather conditions outside (specifically temperature)
• Suggest recommendations for ideal clothing
• Help building managers keep indoor temperatures comfortable

Documentation

• System Architecture
• Final Report
• Presentation

Instructions

Important

The machine running the backend and the machine running the frontend must be able to make requests to each other. For our implementation with an NVIDIA Jetson Thor and our local machines, we used Tailscale.

Backend

Note

The current Dockerfile setup is using an r39.1 PyTorch image for Jetson devices. If this does not match your version, either pull a prebuilt image for your version here or build a custom image for your version from the jetson-containers GitHub repository.

1. Set up your environment variables using the following steps:
   1. Create a copy of .env.sample named .env in the backend folder.
   2. Replace the templated values in your newly created .env file with your actual values.
2. From the root directory of this repository, run the command docker compose up --build in the terminal.

Frontend

1. Set up your environment variables using the following steps:
   1. Create a copy of .env.sample named .env.development in the frontend folder.
   2. Replace the templated values in your newly created .env file with your actual value
2. Run the command pnpm install in the terminal from the frontend directory.
3. Run the command pnpm run dev in the terminal from the frontend directory.

Metrics

Bandwidth:

• Within a period of around 5 hours in the middle of a weekday, around 6.5 MB worth of detection and temperature data was sent to be stored in the Supabase cloud database according to our benchmark script. If we used the cloud only for this task, assuming a 720p video stream at 30 FPS, we would need a constant bandwidth in the range of 2-4 Mbps (1/4-1/2 MB/s) according to Vimeo. Over 5 hours, this would end up being around 4500-9000 MB of total bandwidth used.
  • Total Savings: around 0.1% of bandwidth to the cloud compared to streaming live video to the cloud

Data Sources

We are using live camera footage from a single camera in the EVL space in UIC's CDRLC building. We also have sample data within backend/data for testing purposes.

AI Use Log

According to Section 3.12 in the CS 494 syllabus, the use of AI tools is allowed, but must be disclosed. Here is how LLMs were used during this project:

1. Proposing a design for streaming endpoint
   • Took suggestion on using an MPEG stream for live video feed
2. Debugging error messages during Docker setup
   • Accepted minor changes including adding an extra index URL and setting --retries to 0 for pip install steps
3. Suggested how to persist CLIP in the cache so a complete reinstall was not needed every time docker compose up --build was ran
4. Comparing different tools for clothing classification with their tradeoffs
   • Gave advice on tradeoffs between using OpenCV vs CLIP. These tradeoffs then influenced our final decision to choose CLIP
5. Suggesting frontend libraries to use for graphs
   • Accepted suggestion to use Recharts for visuals on dashboard page
6. Used for debugging issues with the GPU as well as CUDA errors
   • Advised on what versions of Docker, YOLO as well as versions of CUDA to use in order to make the models compatible

The tasks above were completed using Claude Sonnet 4.6.