backport: dumptxoutset and loadtxoutset RPCs - #11
Open
edtubbs wants to merge 32 commits into
Open
Conversation
(cherry picked from commit 344a2c4)
(cherry picked from commit e6756ad)
This allows estimating the in-memory size of a LevelDB batch. (cherry picked from commit e66dbde)
The error() function unconditionally reports an error. It should only be used for actually exception situations, and not for the type of inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal with. This also makes a subtle semantics change: in ApplyTxInUndo, when a record with metadata is encountered (indicating it is the last spend from a tx), don't wipe the CCoins record if it wasn't empty at that point. This makes sure that UTXO operations never affect any other UTXOs (including those from the same tx). (cherry picked from commit f54580e)
This is necessary later, when we drop the nVersion field from the undo data. At that point deserializing and reserializing the data won't roundtrip anymore, and thus that approach can't be used to verify checksums anymore. With this CHashVerifier approach, we can deserialize while hashing the exact serialized form that was used. This is both more efficient and more correct in that case. (cherry picked from commit e484652)
We'll need a version of SipHash for tuples of 256 bits and 32 bits data, when CCoinsViewCache switches from using txids to COutPoints as keys. (cherry picked from commit 7e00322)
This makes the following changes: * In undo data and the chainstate database, the transaction nVersion field is removed from the data structures, always written as 0, and ignored when reading. * The definition of hash_serialized in gettxoutsetinfo is changed to no longer incude the nVersion field. It is renamed to hash_serialized_2 to avoid confusion. The new definition also includes transaction height and coinbase information, as this information was missing before. This depends on having a CHashVerifier-based undo data checksum verifier. Apart from changing the definition of serialized_hash, downgrading after using this patch is supported, as no release ever used the value of nVersion field in UTXO entries. (cherry picked from commit d342424)
(cherry picked from commit c3aa0c1)
Previously, transaction metadata (height, coinbase or not, and before the previous commit also nVersion) was only stored for undo records that correspond to the last output of a transaction being spent. This only saves 2 bytes per undo record. Change this to storing this information for every undo record, and stop complaining for having it in non-last output spends. This means that undo dat written with this patch won't be readable by older versions anymore. (cherry picked from commit 7d991b5)
(cherry picked from commit 422634e)
The earlier CTxInUndo class now holds the same information as the Coin class. Instead of duplicating functionality, replace CTxInUndo with a serialization adapter for Coin. (cherry picked from commit cb2c7fd)
This avoids a prevector copy in ApplyTxInUndo. (cherry picked from commit bd83111)
The new functions are: * CCoinsViewCache::AddCoin: Add a single COutPoint/Coin pair. * CCoinsViewCache::SpendCoin: Remove a single COutPoint. * AddCoins: utility function that invokes CCoinsViewCache::AddCoin for each output in a CTransaction. * AccessByTxid: utility function that searches for any output with a given txid. * CCoinsViewCache::AccessCoin: retrieve the Coin for a COutPoint. * CCoinsViewCache::HaveCoins: check whether a non-empty Coin exists for a given COutPoint. The AddCoin and SpendCoin methods will eventually replace ModifyCoins and ModifyNewCoins, AddCoins will replace CCoins::FromTx, and the new AccessCoins and HaveCoins functions will replace their per-txid counterparts. Note that AccessCoin for now returns a copy of the Coin object. In a later commit it will be change to returning a const reference (which keeps working in all call sites). (cherry picked from commit 0003911)
(cherry picked from commit f68cdfe)
This clarifies a bit more the ways in which the new script execution cache could break consensus in the future if additional data from the CCoins object were to be used as a part of script execution. After this change, any such consensus breaks should be very visible to reviewers, hopefully ensuring no such changes can be made. (cherry picked from commit c87b957)
(cherry picked from commit 8b3868c)
(cherry picked from commit 961e483)
(cherry picked from commit 05293f3)
(cherry picked from commit 13870b5)
(cherry picked from commit 4ec0d9e)
This patch makes several related changes: * Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...) to be COutPoint/Coin-based rather than txid/CCoins-based. * Changes the chainstate db to a new incompatible format that is also COutPoint/Coin based. * Implements reconstruction code for hash_serialized_2. * Adapts the coins_tests unit tests (thanks to Russell Yanofsky). A side effect of the new CCoinsView model is that we can no longer use the (unreliable) test for transaction outputs in the UTXO set to determine whether we already have a particular transaction. (cherry picked from commit 5083079)
- Rename GetCoins→GetCoin, remove HaveCoins override in CCoinsViewTest so it properly overrides the virtual base method - Add HaveCoin testing to coins_cache_simulation_test (per 21180ff) - Update CheckAccessCoin(PRUNED,ABSENT) to expect ABSENT/NO_ENTRY now that GetCoin returns false for spent coins in child caches
(cherry picked from commit 707fde7)
Also changes existing CCoinsStats attributes to be C++11 initialized. (cherry picked from commit 92fafb3)
Allows the creation of a UTXO snapshot to disk. (cherry picked from commit 57cf74c)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backports the
dumptxoutsetandloadtxoutsetRPC endpoints to export and import UTXO set snapshots. Includes snapshot validation and chainstate loading logic invalidation.cpp, per-txoutCoinUTXO database refactoring and integration tests.