6d625af283
7378f27b4ftest: run utxo-to-sqlite script test with spk/txid format option combinations (Sebastian Falbesoner)b30fca7498contrib: utxo_to_sqlite.py: add options to store txid/spk as BLOBs (Sebastian Falbesoner) Pull request description: This PR is a late follow-up to https://github.com/bitcoin/bitcoin/pull/27432, introducing an option for the utxo-to-sqlite script to store the txid/scriptPubKey columns as bytes (= `BLOB` storage class in sqlite, see e.g. https://www.sqlite.org/datatype3.html in sqlite) rather than hex strings. This was proposed in earlier reviews (https://github.com/bitcoin/bitcoin/pull/27432#issuecomment-1516857024, https://github.com/bitcoin/bitcoin/pull/27432#issuecomment-1653739351) and has the obvious advantage of a significantly smaller size of the resulting database (and with that, faster conversion) and the avoidance of hex-to-bytes conversion for further processing of the data [1]. The rationale on why hex strings were chosen back then (and still stays the default, if only for compatibility reasons) is laid out in https://github.com/bitcoin/bitcoin/pull/27432#issuecomment-1516922824 [2]. The approach taken is introducing new parameters `--spk` and `--txid` which can either have the values "hex", "raw" (for scriptpubkey) and "hex", "raw", "rawle" (for txid). Thanks to ajtowns for providing this suggestion. Happy to take further inputs on naming and thoughts on future extensibility etc. [1] For a concrete example, I found that having these columns as bytes would be nice while working on a SwiftSync hints generator tool (https://github.com/theStack/swiftsync-hints-gen), which takes the result of the utxo-to-sqlite tool as input. [2] note that in contrast what I wrote back then, I think there is no ambiguity on byte-string-serialization of txids; they are ultimately just hash results and hence, they should be stored as such, and adding a big/little endian knob wouldn't make much sense. The drawback of not being able to immediately show txid-strings (as one would need to do the bytes-reversal step first, which is not possible in sqlite, see e.g. https://github.com/bitcoin/bitcoin/pull/24952#issuecomment-1165499803) still remains though. ACKs for top commit: ajtowns: ACK7378f27b4fw0xlt: reACK7378f27b4fsedited: ACK7378f27b4fTree-SHA512: 265991a1f00e3d69e06dd9adc34684720affd416042789db2d76226e4b31cf20adc433a74d14140f17739707dee57e6703f72c20bd0f8dd08b6d383d3f28b450
Repository Tools
Developer tools
Specific tools for developers working on this repository.
Additional tools, including the github-merge.py script, are available in the maintainer-tools repository.
Verify-Commits
Tool to verify that every merge commit was signed by a developer using the github-merge.py script.
Linearize
Construct a linear, no-fork, best version of the blockchain.
Qos
A Linux bash script that will set up traffic control (tc) to limit the outgoing bandwidth for connections to the Bitcoin network. This means one can have an always-on bitcoind instance running, and another local bitcoind/bitcoin-qt instance which connects to this node and receives blocks from it.
Seeds
Utility to generate the pnSeed[] array that is compiled into the client.
Build Tools and Keys
Packaging
The Debian subfolder contains the copyright file.
All other packaging related files can be found in the bitcoin-core/packaging repository.
MacDeploy
Scripts and notes for Mac builds.
Test and Verify Tools
TestGen
Utilities to generate test vectors for the data-driven Bitcoin tests.
Verify-Binaries
This script attempts to download and verify the signature file SHA256SUMS.asc from bitcoin.org.
Command Line Tools
Completions
Shell completions for bash and fish.
UTXO Set Tools
UTXO-to-SQLite
This script converts a compact-serialized UTXO set (as generated by Bitcoin Core with dumptxoutset)
to a SQLite3 database. For more details like e.g. the created table name and schema, refer to the
module docstring on top of the script, which is also contained in the command's --help output.