FTS Refactor/migration

[frankinspace]

Migrate feature translation service from JPL Enterprise GH to public github.com

[nikki-t]

Top-level tasks needed to migrate FTS repos

1. Create public GitHub repo for FTS.
2. Move API and database repos into a single repo.
3. Review/remove sensitive data in API and database codebases.
4. Project file updates: README, CONTRIBUTING, LICENSE, PR templates.
5. Push new single repo up to public GitHub repo.
6. Create GitHub action workflows for CI/CD.
7. Documentation for use.

Questions/items to track

1. Do we want to use a single Docker container like Hydrocron or host two separate containers as GitHub packages and in Amazon ECR?
2. Decide how to organize database code. Either include all files at the root of ‘db’ directory or split our by HUC and SWORD operations.
3. Automate and move testing under GitHub actions. Should we continue to use AWS CodeBuild? https://github.com/aws-actions/aws-codebuild-run-build
   a. How much does CodeBuild cost?
4. Host API and database documentation on GitHub pages like Hydrocron?
5. Create a CMR service record?

[nikki-t]

Tentative repo structure: FTS

• .github/
  • workflows/
    • build.yml
    • release-create.yml
    • build-pages.yml
    • pull_request_template.md
• cmr/
  • fts_umm_s.json
  • ops_associations.txt
  • sit_associations.txt
  • uat_associations.txt
• docker/
  • Readme.md
  • ftsdb.fargate.Dockerfile
  • ftsapi.Dockerfile
• docs/
  • … include docs for API and Database.
  • examples/
    • HUC Examples/ (API)
    • Jupyter Notebooks/ (API)
    • SWOT Examples/ (API)
• fts/
  • api/
    • controllers/
      • fts_controller.py
  • db/
    • huc/
      • README.md
      • init.py
      • create_huc_dataset.py
      • create_swot_dataset.py
      • download_data.py
      • remove_multipolygons.py
      • simplify_huc.py
    • sword/
      • setup_sword.py
      • sword_fargate.py
• terraform/
  • api_specification_templates/
    • fts_aws_api.yml
  • bin/
    • deploy.sh
    • destroy.sh
    • setup_databases.sh (DB)
  • resources/
    • DBuserData.sh (DB)
  • tfvars/
    • ops.tfvars
    • sit.tfvars
    • uat.tfvars
  • feature-translation-service-db-iam.tf (DB)
  • feature-translation-service-db.tf (DB)
  • feature-translation-service-fargate-iam.tf (DB)
  • feature-translation-service-fargate.tf (DB)
  • feature-translation-service-lambda-iam.tf (DB)
  • feature-translation-service-lambda.tf (DB)
  • feature-translation-service-lambda-iam.tf (API)
  • feature-translation-service-main.tf (API)
  • main.tf (DB)
  • variables.tf (DB)
  • versions.tf
• tests/
  • data/internal/SWORD/Reaches_Nodes_Test/shp/AF/ (DB)
  • test_setup_sword.py (DB)
  • FTS.postman_collection.json (API)
  • buildspec.yml (API)
  • test_huc_lambda.py (API)
  • test_rivers_lambda.py (API)
• CHANGELOG.md
• CONTRIBUTING.md
• LICENSE
• README.md
• poetry.lock
• pyproject.toml

[nikki-t]

Initial migration is complete: https://github.com/podaac/feature-translation-service

Pending items:

• Attempt SIT deployment to fix any bugs or issues created from migration.
  • poetry.lock has not been not created yet as the single FTS codebase needs to be tested and deployed.
• Create Docker container images.
• Create GitHub action workflow for CI/CD
  • Do we want to use CodeBuild for API tests?
• Host API and database documentation on GitHub pages

[nikki-t]

Moving pending items to separate GitHub issues in the new repository. See this milestone for work: https://github.com/podaac/feature-translation-service/milestone/1