Use memoizer with TTL to hugely speed up response latency

[rm-hull]

At the expense of increased memory usage, 100x increase in performance (real world performance will be different..)

Before

$ wrk -d 30s 'http://localhost:8080/v1/postcode/polygons?bbox=431667,455097,432504,455847'
Running 30s test @ http://localhost:8080/v1/postcode/polygons?bbox=431667,455097,432504,455847
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    94.44ms   11.87ms 155.32ms   73.29%
    Req/Sec    53.02      8.69    80.00     82.32%
  3182 requests in 30.08s, 273.35MB read
Requests/sec:    105.77
Transfer/sec:      9.09MB

After

$ wrk -d 30s 'http://localhost:8080/v1/postcode/polygons?bbox=431667,455097,432504,455847'
Running 30s test @ http://localhost:8080/v1/postcode/polygons?bbox=431667,455097,432504,455847
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.21ms    2.91ms  47.46ms   96.75%
    Req/Sec     6.34k     0.89k    7.44k    82.67%
  378309 requests in 30.00s, 31.74GB read
Requests/sec:  12608.51
Transfer/sec:      1.06GB

[gemini-code-assist]

Summary of Changes

Hello @rm-hull, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on a critical performance enhancement for the API's polygon search functionality. By implementing an in-memory cache for frequently accessed GeoJSON data, I've managed to drastically reduce response times for the /v1/postcode/polygons endpoint, making the service much more responsive under load.

Highlights

• Performance Optimization: I've introduced a memoization (caching) layer with a Time-To-Live (TTL) to significantly speed up responses for the /v1/postcode/polygons endpoint. This change aims to improve latency by caching the results of GeoJSON file decompression.
• Dependency Addition: A new Go module, github.com/kofalt/go-memoize, has been added to facilitate the caching mechanism. This library provides the core functionality for memoizing function calls.
• Impact on Latency: As demonstrated by the wrk benchmarks, this change results in approximately a 100x increase in requests per second and a drastic reduction in average latency for the affected endpoint, at the expense of increased memory usage.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

	Tests	Passed ❌️	Skipped	Failed
JUnit Test Report	0 ran	0 passed	0 skipped	0 failed

Test	Result
No test annotations available

• View JUnit Test Report

[gemini-code-assist]

Code Review

The pull request introduces a memoizer with TTL to improve response latency. The changes involve adding the go-memoize library, initializing a memoizer instance, and using it to cache the results of the DecompressFeatureCollection function. The changes significantly improve performance at the cost of increased memory usage.

[coveralls]

Pull Request Test Coverage Report for Build 15913481100

Details

• 0 of 7 (0.0%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 0.0%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
cmd/api_server.go	0	3	0.0%
routes/search.go	0	4	0.0%

Totals
Change from base Build 15913179295:	0.0%
Covered Lines:	0
Relevant Lines:	454

---

💛 - Coveralls