Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 3 additions & 18 deletions api/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@
FROM python:3.12-bullseye
FROM python:3.12-slim-bookworm

# Ensure all system packages are up to date to reduce vulnerabilities
RUN apt-get update && apt-get upgrade -y && apt-get clean

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

# remove requirements-dev installation for deployments
COPY requirements-dev.txt requirements-dev.txt
RUN pip install -r requirements-dev.txt


# https://www.digicert.com/kb/digicert-root-certificates.htm
# Get the .pem file from digicert and add it to the bundle used by certifi
Expand All @@ -23,24 +19,13 @@ RUN wget -O /usr/lib/ssl/certs/GeoTrustTLSRSACAG1.crt.pem https://cacerts.digice
update-ca-certificates && \
cat /usr/lib/ssl/certs/GeoTrustTLSRSACAG1.crt.pem >> $(python -c "import requests; print(requests.certs.where())")

# Install system dependencies (required for Google Cloud SDK)
RUN apt-get update && apt-get install -y \
curl \
&& rm -rf /var/lib/apt/lists/*

#RUN python -m wget https://dl.min.io/client/mc/release/linux-amd64/mc
#RUN chmod +x mc
#RUN mv mc /usr/local/bin/mc
RUN curl -fsSL https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/local/bin/mc && \
chmod +x /usr/local/bin/mc

COPY ./com_res /com_res
# (Optional) For local testing with a service account key:
# COPY key.json /app/key.json
# ENV GOOGLE_APPLICATION_CREDENTIALS="path/to/your/service-account-key.json"

ENV PYTHONPATH "/com_res/:${PYTHONPATH}"
ENV PYTHONPATH=/com_res/

EXPOSE 8000

CMD uvicorn --host 0.0.0.0 --port 8000 --proxy-headers main:app
CMD ["uvicorn", "--host", "0.0.0.0", "--port", "8000", "--proxy-headers", "main:app"]
38 changes: 24 additions & 14 deletions api/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,35 @@
# com_res FastAPI
# FloodSavvy API

A python FastAPI application
FastAPI service for FloodSavvy data endpoints.

The Dockerfile declares a python base image and installs the dependencies declared in `requirements.txt` and `requirements-dev.txt` and starts up the FastApi application at port 8000.
## Active Routers

`com_res/main.py` is the entrypoint to the FastAPI application, configures the routers. The file also contains a startup event hook that initialized the mongodb database with [beanie ODM](https://beanie-odm.dev/). The startup event hook also sets up a minio client for the [CUAHSI MinIO instance](https://console.minio.cuahsi.io). The minio client is used for synchronizing user specific access policies and keys/secrets.
The deployment now includes only the routers mounted in `com_res/main.py`:

API documentation is rendered at https://com_res-api-jbzfw6l52q-uc.a.run.app/redoc (This will be updated to https://api.com_res.cuahsi.io/redocs pending certificate creation). OpenAPI spec documentation is generated from the code defining the api endpoints (FastAPI) and input/output models (Pydantic).
- `timeseries` (`/timeseries/*`): National Water Model historical/forecast utilities.
- `fim` (`/fim` and `/historical-quantiles`): flood inundation mapping and quantile lookups.

User authentication is achieved by configuring the [fastapi_users](https://github.com/fastapi-users/fastapi-users) module with [CUAHSI SSO](https://auth.cuahsi.org/) using the `OpenID Connect` protocol. On registration a S3 bucket is created for the user on [CUAHSI MinIO](https://console.minio.cuahsi.io) (TODO: create a default quota of 5 GB). An admin may increase the quota on a case by case basis.
## Runtime Configuration

The com_res API is divided into 4 routers defined at `com_res/app/routers/`.
Required environment variables:

## Routers
### Access Control Router
The `access_control` router contains prototyped synchronization of view/edit access to paths on MinIO that have a HydroShare resource that references a path on the CUAHSI MinIO instance. In the [mongo_discovery-access-control](https://github.com/hydroshare/hydroshare/compare/develop...mongo-discovery-access-control) HydroShare branch, event hooks are created for exporting Resource and User access to a mongo database. This mongo database is accessed to look up the resources which a user has view/edit privileges and generates the view/edit policies that are assigned to the user on CUAHSI MinIO storage. This means a path in a user's bucket may be registered on HydroShare and enjoy the same access control capabilities of a HydroShare Composite Resource.
- `VITE_APP_API_URL`
- `ALLOW_ORIGINS`
- `NWM_BIGQUERY_KEY`
- `NWM_BIGQUERY_URL`
- `BIGQUERY_PROJECT_ID` (optional, defaults to `com-res`)
- `CLOUD_RUN_REGION` (optional, defaults to `us-central1`)
- `CLOUD_RUN_JOB_NAME` (optional, defaults to `fimserv`)
- `GCS_BUCKET_NAME` (optional, defaults to `com_res_fim_output`)
- `GOOGLE_APPLICATION_CREDENTIALS_PATH` (optional, for explicit service account key path)

### Discovery Router
A copy of the IGUIDE discovery router that includes endpoints for searching resource metadata. The com_res workflows run the hydroshare metadata extraction tool to extract metadata the same metadata that a HydroShare composite resource will extract from recognized file formats. The resulting metadata can then be written to the Discovery database on Atlas. TODO: collect the metadata extracted from com_res outputs into a discovery database.
## Local Run

### Storage Router
Contins the endpoints to generate presigned urls for PUT and GET of objects on S3. This is not currently used but could be used to create a resource landing page for resources stored on S3 equivalent to a resource on HydroShare.
From this `api` directory:

```bash
pip install -r requirements.txt
uvicorn --host 0.0.0.0 --port 8000 --proxy-headers main:app
```

Or use the repository-level `docker-compose.yml` to run the API container (non-debug runtime).
100 changes: 0 additions & 100 deletions api/com_res/app/db.py

This file was deleted.

54 changes: 0 additions & 54 deletions api/com_res/app/models.py

This file was deleted.

Loading