Add RocksDB-backed persistence#17
Open
piotrbar250 wants to merge 47 commits into
Open
Conversation
also, added the instructions.md for setup instructions
Database schema correctly saving and retrieving
Co-authored-by: Piotr Baranowski <piotr.baranowski250@gmail.com>
when there are many column families in a table, all the data in this table gets multiplicated as many times as there are CFs in this table
…llStream::Next though only kCell/default mode is used
3 tables and 3 CFs
…amily this aligns with the logic of per-cf stream in the emulator code; now readrows works well, without duplicated values
(TODO: delete column families; for that, we probably need to embed table name in cf name)
this required putting the table id in the CF id and adapting some places where we still need the bare cf name
previously it seemingly worked, because the new schema without the deleted column family was saved; but the data from this cf still stayed in RocksDB this commit fixes that
also, fixed stuff in normal script
(from a single row, as in the original emulator)
maybe will be useful created it, because it was supposed to be able to delete cfs from specific rows
this is tested by the python client (there is no cbt command that does this) returning errors properly if a CF/row doesn't exist, written functions in Storage for that
…g rows (works, even after emulator restart)
feat: Implement filters for persistent filtered column family stream
It tests: - Table Create() persists schema and manifest entry. - Prefixed column-family IDs are normalized to bare IDs in persisted schema. - ModifyColumnFamilies create operation is persisted. - Update() with field mask persists deletion_protection. - Repeated Create() for same table does not duplicate manifest entry.
Mirror DeleteValue rollback into RocksDB by deleting the specific persisted cell. Carry family name through undo entries and add a regression test for failed MutateRow rollback persistence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds persistence to the emulator by using RocksDB to store table data and emulator metadata.
Implementation details and design notes are documented in persistence.md.
Authors:
Piotr Głowacki
Piotr Baranowski
Dominik Seredyn
This change is