Skip to content

Asap 215 document audit export - #284

Merged
lkacenja merged 23 commits into
devfrom
asap-215-document-audit-export
Sep 3, 2025
Merged

Asap 215 document audit export#284
lkacenja merged 23 commits into
devfrom
asap-215-document-audit-export

Conversation

@lkacenja

@lkacenja lkacenja commented Aug 25, 2025

Copy link
Copy Markdown
Contributor

We want to create a page for document audit export options. We think this will be handy for stakeholders. We want to support a CSV export and guide folks toward our API. Our API documentation is not visible or up to date.

This PR adds the following:

  • A new tab on the site for export related activity
  • An S3 integrated CSV exporter
  • Refactor of our API to include access control that matches the rest of the site
  • Remove API endpoints we don't want to or aren't ready to support
  • Controller and view for displaying API documentation /api-docs
  • What additional steps are required to test this branch locally?

Requires a docker compose build --no-cache to run the revised bucket setup script.

Also requires a yarn install to add the new Swagger UI package.

  • Are there any areas you would like extra review?

Would love some help testing out the API endpoints.

  • Are there any rake tasks to run on production?

No

@lkacenja
lkacenja marked this pull request as ready for review August 27, 2025 15:36

before_action :ensure_user_site_admin

# This form is only for local development.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces a move to storing more items in Rails config. These secret names are now in config/environments/development.

@filters_for_sorts = query_params [:sort, :direction, :page]
end

def insights

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the insights (dashboard) endpoint here to documents, because it nests under documents in the URL and views directory. I thought this would make more sense.

<pre data-prefix="$"><code>curl -X 'GET' \
'<%= request.base_url %>/api/sites/1/documents?page=0&items_per_page=100' \
-H 'accept: application/json' \
-H 'authorization: Basic [Your base64 encoded credentials]'</code></pre>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about actually including the user's authorization header here, but it seems more secure not to.

@4dh 4dh Sep 2, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, activity exporting looks good, but having some auth issues when testing the API via curl in terminal-
Screenshot 2025-09-02 at 7 58 57 PM

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also adding this interesting error that we saw when generating AI exception:
Screenshot 2025-09-02 at 5 10 02 PM

@lkacenja lkacenja Sep 3, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, activity exporting looks good, but having some auth issues when testing the API via curl in terminal-

When you get back, maybe we could have a look at the command you are running. Curl requests work for me with the following process:

  1. Base64 encode credentials: echo -n '<email>:<password>' | base64
  2. Add the encoded value to the authorization header.
curl -X 'GET' \
'http://localhost:3000/api/sites/1/documents?page=0&items_per_page=100' \
-H 'accept: application/json' \
-H 'authorization: Basic <encoded auth goes here>'

@lkacenja lkacenja Sep 3, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for documenting the LocalStack error. I've encountered this a few times as well. When we see this error, I think the LocalStack Lambda service has crashed. I'm not really sure why this happens. To fix it, restart the docker compose and potentially the docker daemon. Usually, rebuilding the images is not required.

@lkacenja lkacenja self-assigned this Aug 27, 2025
@lkacenja
lkacenja requested a review from 4dh August 27, 2025 15:46
@lkacenja

lkacenja commented Sep 3, 2025

Copy link
Copy Markdown
Contributor Author

Given that the API seems to be working ok for me and that the LocalStack error is not related to this branch, I'd like to go ahead and merge this work.

@lkacenja
lkacenja merged commit eba2ba2 into dev Sep 3, 2025
2 of 3 checks passed
lkacenja added a commit that referenced this pull request Sep 3, 2025
* Asap 208 feedback UI (#265)

* Add feedback item model.

* Add feedback controller.

* Add static markup for feedback widget.

* Functional feedback UI.

* Add deletion route and make feedback user-based.

* Remove checkboxes.

* Use auto timestamps.

* Move hidden fields out of additional comments.

* Wrap interface in form tag.

* Use buttons for feedback.

* Add a test for the feedback UI.

* Clean up sass.

* Specify fallback versioning.

* Run accessibility scan on feedback form.

* Fix linting issues.

* Asap 216 collect more llm information (#286)

* Return token usage and model name from document inference.

* Set token usage and model name on API calls.

* Update schema to incldue token usage and model name.

* Get rid of pypdf.

* Return commented out portion.

* Remove pypdf.

* Add model name to inference UIs.

* Add llm author to tests.

* Remove scratch script.

* Fix up linting issues.

* Fix tests.

* Asap 215 document audit export (#284)

* Stashing initial work on background job.

* Stash progress on exporting.

* Mostly function audit export page.

* Clean up and make localstack erros more obvious. Move secret names to config.

* Remove secret name constants.

* Use actual bucket name.

* Move S3 permissions to the correct role.

* Allow backend to handle errors.

* Refactor and simplify API.

* Add swagger UI to the app.

* Fix API paths and example.

* Handle errors more gracefully.

* Use staging bucket name (default).

* Improve access and responses as revealed by testing.

* Refactor tests for new endpoints.

* Fix up linting issues.

* Remove future fields from factory.

* Fix tests by matching new routes and using required fields.

* Add a light test for exports.

* Dep: 2025-09-03 (#294)

* Bump pg from 1.6.1 to 1.6.2 (#293)

Bumps [pg](https://github.com/ged/ruby-pg) from 1.6.1 to 1.6.2.
- [Changelog](https://github.com/ged/ruby-pg/blob/master/CHANGELOG.md)
- [Commits](ged/ruby-pg@v1.6.1...v1.6.2)

---
updated-dependencies:
- dependency-name: pg
  dependency-version: 1.6.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Leo Kacenjar <lkacenja@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump aws-sdk-ses from 1.88.0 to 1.90.0 (#292)

Bumps [aws-sdk-ses](https://github.com/aws/aws-sdk-ruby) from 1.88.0 to 1.90.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/version-3/gems/aws-sdk-ses/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/commits)

---
updated-dependencies:
- dependency-name: aws-sdk-ses
  dependency-version: 1.90.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Leo Kacenjar <lkacenja@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump aws-sdk-lambda from 1.158.0 to 1.160.0 (#291)

* Bump aws-sdk-lambda from 1.158.0 to 1.160.0

Bumps [aws-sdk-lambda](https://github.com/aws/aws-sdk-ruby) from 1.158.0 to 1.160.0.
- [Release notes](https://github.com/aws/aws-sdk-ruby/releases)
- [Changelog](https://github.com/aws/aws-sdk-ruby/blob/version-3/gems/aws-sdk-lambda/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-ruby/commits)

---
updated-dependencies:
- dependency-name: aws-sdk-lambda
  dependency-version: 1.160.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Reroll lock file.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Leo Kacenjar <lkacenja@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Hide llm author for inferences without one. (#295)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants