docs(technical-design): global database - #11472
Conversation
github-actions
Bot
added
the
tb-team
|
✅ Validation Passed: All report and feature-flag labels are correctly set. |
wmontwe
added
pr: stacked
wmontwe
changed the title
| - **The global mail database** owns the legacy-compatible mail schema, migration metadata, and derived data. | ||
| It is one physical database for the application, not one database per account. | ||
| - **The attachment store** remains file-backed for content that legacy already held on disk, which is every body part | ||
| above a size threshold. Parts at or below it stay `message_parts` BLOBs, as in legacy. Database records and attachment |
There was a problem hiding this comment.
Parts at or below it stay
message_partsBLOBs, as in legacy.
I wonder what the benefits are of having two sources for attachments. Why not store everything as files on disk instead of storing them as BLOBs in the database?
There was a problem hiding this comment.
I wanted to keep the existing behavior, if it's not too much work, the migration could change that too, but adds complexity.
| - Technical design: [Global Database](../technical-designs/0003-global-database.md) | ||
| - Repository pattern: [ADR 0010 proposal](https://github.com/thunderbird/thunderbird-android/pull/11452) | ||
| - Portable data format: [RFC 0008: Portable Profile Data Format](0008-portable-profile-data-format.md) | ||
| - Status: **Proposed** |
There was a problem hiding this comment.
Since this is a stacked PR, and this PR will only be merged once the #11471 is reviewed and accepted, you could change the Status here to accepted.
| downloaded attachments. Legacy keeps a body part on disk above a size threshold and as a `message_parts` BLOB at or | ||
| below it. The migration preserves that split rather than changing where content lives. | ||
|
|
||
| ## Migration |
There was a problem hiding this comment.
We need to ensure that the migration process can take place even when the app is running in the background. Users with large email accounts may require minutes or even hours to migrate all their data.
It's important to recognize that we cannot rely on users to keep the app in the foreground while the migration is occurring.
I recommend starting the migration process as a data synchronization foreground service. This will notify the user when the migration is complete, and we can keep the progress updated through a persistent notification.
If the user remains in the app while the migration is ongoing, the screen should continuously update to reflect the migration's progress. However, if the user switches to the background or closes the app, we must ensure that the migration continues to run until it is fully completed.
There was a problem hiding this comment.
I added a section for that.
There was a problem hiding this comment.
Along with that, I'd say we want to figure out when we'll tell users to update. Perhaps we can run a few tests of our own and figure out, in general, how long a transfer might take on a midrange device. Obviously it would be hard to predict this with any accuracy unless we test on a great deal of different hardware and DB sizes, but we may be able to tell them something like, "This migration could take an hour, would you prefer to be reminded tonight?"
The goal would be that users never notice the down time. They tap the notification we may send them at a chosen time, go to bed, and wake up to their app using the new database with the report and success message.
I don't think this needs to be detailed here, necessarily, but we should consider a possible stretch goal of scheduling the upgrade so it can happen when the user isn't going to be using the app.
a7b52e1 to
89217a9
Compare
wmontwe
removed
the
pr: stacked
… model (cherry picked from commit db3649d9706e78f6d2c982f4e63f7e6c6070a70c)
…covery, and cleanup procedures for migration (cherry picked from commit b54d7166bbcc4e63e180b75d319b3c38ece713ae)
…l database migration (cherry picked from commit 65b1fb8058e6c4242155d4bd50d8c41094b8a1ab)
…and headroom validation
89217a9 to
a787446
Compare
| downloaded attachments. Legacy keeps a body part on disk above a size threshold and as a `message_parts` BLOB at or | ||
| below it. The migration preserves that split rather than changing where content lives. | ||
|
|
||
| ## Migration |
There was a problem hiding this comment.
Along with that, I'd say we want to figure out when we'll tell users to update. Perhaps we can run a few tests of our own and figure out, in general, how long a transfer might take on a midrange device. Obviously it would be hard to predict this with any accuracy unless we test on a great deal of different hardware and DB sizes, but we may be able to tell them something like, "This migration could take an hour, would you prefer to be reminded tonight?"
The goal would be that users never notice the down time. They tap the notification we may send them at a chosen time, go to bed, and wake up to their app using the new database with the report and success message.
I don't think this needs to be detailed here, necessarily, but we should consider a possible stretch goal of scheduling the upgrade so it can happen when the user isn't going to be using the app.
| data, SQLite transaction and temporary-file growth, and a safety margin. If an RFC 0008 archive is written to the same | ||
| volume, its estimated size is also included. | ||
|
|
||
| Migration starts only when available space on the target volume meets the calculated headroom. If it does not, the |
There was a problem hiding this comment.
We may want to inform users who can't do the update not to upgrade to a newer version of the app and turn off auto-update until they can. I doubt we'd have many users who could not transfer their database over but also have auto update turned on, a feature that could increase the storage space used, but it's still perhaps worth mentioning to users who can't update the DB due to storage space.
3 participants
Contribution Summary
Linked Issue/Ticket: resolves #11293
RFC / Technical Design (if applicable): #11104
Description
Technical design for the Global Database
AI Disclosure
Select one of the following (mandatory)