Mastering bitcoin
A brief introduction.
A combination of software engineering, cryptography and economics.
There are two basic issues that need to be addressed with digital currency:
- Whether the money is authentic or not
- How does the money identify ownership
Bitcoin composition:
- A decentralized peer-to-peer network (Bitcoin protocol)
- A public ledger of transactions (blockchain)
- A decentralized mathematical and deterministic currency issue (distributed mining)
- A decentralized transaction verification system (transaction script)
Bitcoin client:
Full client: A full client, or “full node,” that stores the real transaction history of all Bitcoin transactions (every transaction completed by every user, every transaction ever owned), can manage the wallet, and can start transactions directly on the Bitcoin network.
Lightweight client: Stores the user’s wallet, but relies on third-party servers for bitcoin transactions and access to the Bitcoin network. Complete copies of all transactions are not kept.
Online client: Access and store the user’s wallet on a third-party server through a web browser
Choice of different clients: the complete client manages the wallet itself, and the responsibility of wallet backup and security is transferred to the user. Others require consideration of third party risks.
Two. Basic operation principle
A key point of operation: trust, not just one person trusting another, but all participants in the area giving their approval. How did ** do it? ** Blockchain technology is a public transaction ledger that everyone can see and copy, but the information inside cannot be modified. Only new information can be added as corrections on the basis of previous information. ** How do you make information on the blockchain unmodifiable? ** The “can’t” is relative, one area connects to another, and the next one stores the hash value of the previous one. If one area is modified, the chain reaction will occur, and the information in the next area needs to be modified, and the next area needs to be modified, so the cost of modification will be very high, so no one does it.
** Who does the block hash? ** does not have a centralized server, ** who will provide such services? ** This gives rise to miners, who provide the computing power to calculate blocks and record transactions on the chain, so how do miners work? An incentive is created, and it’s the simplest and most violent of all: pay to work. The miner receives a share of the proceeds for providing computing power to the trading chain. How this revenue is calculated and who ultimately gets it will be discussed later.
With this push, the system is basically set up. ** So how is the deal structured? How do you make data valuable? **utxo—-unspent transacation output ** unspent transacation output **utxo—-unspent transacation output The basic information of recording is input and output. Input is the expenditure of this account, and output is the income of this account or other accounts (including mining). The record of how much wealth you have is that an account inputs coins to you, and you can spend this coin to form a new record. The noun that marks your account is the address, that is the wallet address, this can be made public, the address that everybody transfers money is filled in this address. This address is through the private key elliptic encryption algorithm (one-way) to get the public key, the public key through the double hash algorithm (one-way) to get the address, so the most important is the private key, his security is maintained through two encryption algorithms, if the day this encryption algorithm was cracked, then it will gg.
3. Use of the client
Compilation and use of Bitcoin client.
-
1. Install Bitcoin.org directly from the official website
-
2. Download the source code on Github and compile it
Key, address, wallet, generation and use
- 1. Ownership of bitcoin: established by digital key, Bitcoin address and digital signature.
Wallet: The digital key is generated by the user and stored in a file or simple database, not in the network, called a wallet.
Digital keys stored in users are completely independent of the Bitcoin protocol and can be generated and managed by user wallet software without the need for a blockchain or network connection.
This chapter will place the key in the wallet. Understand how keys are generated, stored, and managed. Various encoding formats for private and public keys, addresses, and script addresses. Special uses for keys: generating signatures, proving ownership, and creating Bitcoin addresses and paper wallets.
- 2. Private key === Elliptic curve multiplication (one-way) === “Public key === Hash function (one-way) ===” Bitcoin address
How do I obtain a secure private key?
The value range of the private key is 1- (n-1), n=1.158*10^77. To generate a relatively safe private key, a 256 – bit number, randomly check whether he is less than n, from the perspective of programming, typically by in a random password security source out a long string of random bytes, for operational use sha256 hash algorithm, you can get a 256 – bit number, check whether meet the scope.
How do I obtain a public key?
Use the private key to compute elliptic curve multiplication, get the public key, I don’t know. Secp256k1 standard is used.
How do I get a Bitcoin address?
Bitcoin addresses often appear as the recipient of transactions. The bitcoin address can be obtained from a public key through a one-way cryptographic hash algorithm. Hashing algorithms are widely used in bitcoin: bitcoin addresses, script addresses, and proof-of-work algorithms in mining. The hash algorithm used to generate a Bitcoin address from a public key is: Secure Hash Algorithm(SHA) and the RACE Integrity Primitives Evaluation Message Digest(RIPEMD), especially SHA256 and RIPEMD160.
Public key ==== “SHA256===” RIPEMD160=== “Public key hash ====” Base58check encoding (prefix version bit 0x00) === “Bitcoin address (Base58check encoding public key hash)
You know base58 and Base58Check encoding
To represent long strings of numbers more succinctly and conveniently, use a larger than decimal representation consisting of numbers and digits. For example, hexadecimal. Base64 uses 26 lowercase letters, 26 uppercase letters, 10 numbers, and two symbols (such as “+” and “/”). Base58 is a text-based binary encoding format used in Bitcoin and other cryptocurrencies. This encoding format not only achieves data compression, but also maintains legibility. Base58 does not contain 0, digits, zeros, O letters, L lowercase letters, and I uppercase letters.
Base58Check adds error check codes to check for errors in data transcribed. The verification code is 4 bytes long and is added to the end of the data to be encoded.
Data ==== Version prefix + array ==Hash (version + data) Extracts the first four bytes as the parity code == Version + Data + Parity code ==Base58 Encoding ==Base58 encoding data
Format of public key
Can be divided into uncompressed format or compressed format two forms. A public key is a point on an elliptic curve consisting of a pair of coordinates (x, y). We need to determine the positive and negative values of y, because we need to compress the positive and negative information of y. The positive and negative values of y are expressed as: when we calculate the elliptic curve in the finite field of prime order P using binary arithmetic, y may be odd or even, corresponding to the positive and negative signs of y values. Y is even: 02x, y is odd: 03x
Scenario: When a grabbag application imports the private key of another wallet application, it needs to scan the blockchain and find all transactions related to the imported private key. So which Bitcoin address should the wallet scan? (Because there are two types of public key, compressed and uncompressed, you get different bitcoin addresses.)
Solution: When exporting the private key, add a suffix to indicate whether the public key can be compressed. Add the suffix 01 to indicate that the private key is from a relatively new wallet and can only be used to generate the compressed public key. Form a specification.
WIF (Wallet Import Format) : The private key is displayed in Base58 checksum encoding Format.
The concept of bitcoin wallets
A wallet is a container for private keys, usually through an ordered file or a simple database.
Another way to generate a private key is to use the original private key, generate each new private key through a one-way hash function, and join the newly generated keys in order.
Different private key generation methods and wallet structure.
- 1. Nondeterministic (random) wallets
Nondeterministic wallet is also called zero nondeterministic wallet. Generate as many private keys as possible from the beginning and use each key only once. It’s hard to manage, back up, and import because you generate a lot and you have to keep copies of all of them. It is now being replaced by the certainty wallet.
- 2. Certainty (seed) wallet
A deterministic wallet contains a private key that can be generated from a common seed by using monomial discrete equations. The seeds here are randomly generated numbers. This number also contains the index number or “chain code” that generates the private key. With seeds, you don’t have to make a backup of every private key, just a simple backup when the accident is created.
- 3. Mnemonic vocabulary
A mnemonic vocabulary is a sequence of English words representing (encoding) random numbers used as seeds corresponding to a deterministic wallet. It’s a deterministic wallet that uses mnemonic words as seeds. One advantage is that it is much easier to read and copy correctly than a random sequence of numbers.
BIP0039 defines the mnemonic and seed creation process as follows:
- 1. Create a random order of 128 to 256 bits (entropy); - 2. The first few bits of SHA256 hash can create a checksum of a random sequence; - 3. Load the checksum after the random order. - 4. Decompose the order into different sets of 11 bits, and use these sets to correspond to a predefined dictionary of 2048 words; - 5. Generate a mnemonic code of 12 to 24 words.Copy the code
- 4. Layered Deterministic Wallet (BIP0032/BIP0044)
the hierarchical deterministic wallet or HD wallet defined
HD Wallet offers random (uncertain) keys with two major advantages,
- 1. Trees can be used to express additional organizational meaning. For example, when a subkey of a particular branch is used to receive transaction revenue and another subkey is used to pay the call fee.
- 2. It allows the user to establish a sequence of public keys without accessing the corresponding private key.
Create HD wallets from seeds
Cryptographically secure
Pseudo random number generator
| | | | | master private key m (256 bits) \ / | | = = = = = "HMAC - root seeds SHA512 = = = = = = | main public key m (264 bits)Copy the code
(128, 256 or 512 bits) output (512 bits) | ChanXiangHa and function
The main chain of the | | | coding (256 bits) | | \ / mnemonic codeCopy the code
Why so many bitcoin addresses, public keys?
V. Transaction process
The essence of bitcoin trading is data structures that contain information about the value transfer of bitcoin participants.
Any other part of the system is designed to ensure that bitcoin transactions can be generated, propagated and verified across the bitcoin network, and eventually added to the global Ledger of Bitcoin transactions (the Bitcoin blockchain).
Bitcoin transactions themselves do not contain sensitive information, so they can be sent to the bitcoin network over an unencrypted network (such as wifi, Bluetooth, NFC, ChirP, bar code, or copy-pasted to a web form).
A bitcoin transaction is sent to any node connected to the Bitcoin network, and the bitcoin is verified by that node. If the verification is valid, the node propagates to the other nodes that receive you, with a return message indicating that the transaction was successful. If the transaction is validated as invalid, the node rejects the transaction and returns a message to the originator indicating that the transaction was rejected.
So bitcoin’s transaction verification engine relies on two types of scripts to verify bitcoin transactions:
One is the lock script and unlock script. The lock script carries a public key; The unlock script is provided by the user to resolve the “locking” of the lock script.
This script is a stack-based execution language based on inverse Polish notation. It is deliberately designed to be important — flow control without loops or other conditions other than complex flow control functions. Makes the scripting language Turing incomplete. A restricted language prevents transaction activation mechanisms from being exploited as weak environments.
The basic unit of a transaction is an unused transaction output, or UTXO for short.
What is UTXO?
Unspent Transaction Output No transaction output was used. It is a certain amount of Bitcoin money that can no longer be divided, locked up by the owner, or recorded in the blockchain and recognized as a unit of currency by the entire network. A user’s Bitcoin balance is calculated by scanning the blockchain and aggregating all UTXOs belonging to the user, which is spread across hundreds of transactions and hundreds of blockchains.
The transaction output consists of two parts:
-
1. A certain amount of bitcoin, named “cong”, is the smallest bitcoin unit;
-
2. A lock script that presents the conditions that the payment output must meet to “lock” the total.
The transaction fee generated by the transaction is the mining fee. It is based on the size of the transaction, measured in kilobytes.
The standard trading
The five standard scripts are P2PKH, P2PK, MS, P2SH and OP_Return.
- 1.P2PKH(Pay-to-Public-Key-Hash)
This type of transaction script is characterized by the need to type the public key and the digital signature of the corresponding private key to unlock a locking script with a public key hash implementation.
- 2.P2PK(Pay-to-Public-Key)
The difference between P2PKH and P2PK is that the public key itself is already stored in the lock script.
- 3.MS (Multiple signature)
The multi-signature script sets a condition that if the number of public keys recorded in the script is N, at least M of them must be unlocked. This is called an M-N combination. N is the total number of public keys recorded in the script, and M is the public key threshold (minimum number) for multiple signatures to take effect.
- 4. Data output (OP_RETURN)
Bitcoin’s distribution and time-stamp account mechanism, also known as blockchain, has potential applications that go far beyond payments. Take advantage of the security and recoverability of its transaction scripting language and apply it to e-justice services, securities authentication and smart contract protocols. But its use in non-payment areas has divided developers, as it would burden all blockchain nodes with the task of storing such data at the cost of consuming disk storage space. The compromise is achieved by using the OP_RETURN operator to avoid memory bloat in UTXO sets.
- 5. P2SH (Pay – to – the Script – the Hash)
P2SH is designed to address the complex operations that MS multiple signatures produce, making it as easy to use complex scripts as paying bitcoin addresses.
Vi. Network construction and communication
P2P means that each computer in the same network is peer to peer, and each node provides network services together, without any “special” node.
Although the nodes in the network are equivalent to each other, they may have different roles based on the functions provided. Each Bitcoin node is a functional set of routing, blockchain database, mining, and wallet services.
Each node participates in the routing function of the whole network and may also contain other functions. Each node participates in validating and disseminating transaction and block information, discovering and maintaining connections to peers. Contains different functions can be divided into the following nodes:
Bitcoin core client: routing + blockchain database + mining + wallet service
Full node: Maintains a complete, up-to-date copy of the blockchain and is able to charm all transactions autonomously without any external reference. Routing + blockchain database + wallet service
SPV node (lightweight node) : Transaction verification is done through a “simple payment verification SPV”. Routing + wallet service
Complete blockchain node: Routing + blockchain database
Standalone miner node: Routing + with a copy of the full blockchain database + mining
Mining pool protocol server: Connects nodes running other protocols (such as mining nodes and Stratum nodes) to a NETWORK management router on a P2P network. Pool server, Stratum server.
Mining node: includes mining functions that do not have blockchain but have Straum protocol nodes or other mining pool mining protocol nodes.
Lightweight (SPV) wallet: Stratum wallet, Straum protocol running, without a blockchain.
Communication between nodes When a connection is established with a known peer node, a version message of basic authentication content is sent to start the handshake communication process:
-
PROTOCOL_VERSION: constant. It defines the version of the bitcoin P2P protocol used by the client.
-
NLocalServices: a list of local services supported by the node. Currently, only NODE_NETWORK is supported
-
NTime: indicates the current time
-
AddrYou: indicates the IP address of the remote node visible to the current node
-
AddrMe: local IP address discovered by the local node
-
Subver: indicates the subversion number of the software type running on the current node
-
BaseHeight: Block height of the current node blockchain
How does the new node discover its peers in the network?
-
1. The client maintains a list of nodes that run stably for a long time. Such nodes are called seed nodes. Seed nodes help new nodes quickly discover other nodes in the network.
-
2. Initially provide the IP address of at least one Bitcoin node to the node being launched (this node does not contain any information about the composition of the bitcoin network).
After one or more connections are established, a new node can push information containing its OWN IP address to its neighbor node. The neighbor can then push the information to its neighbor node to ensure that the information of the new node is received by multiple nodes and the connection is more stable. The newly involved node can also send a request to its neighbors to pull down the list of known IP addresses of the neighbors.
In real networks, nodes join and leave randomly, and communication paths are unreliable. Actual solution: 1. Discover a new node when an existing connection is lost (how?) And help other nodes when they start up. A node that is connected to a large number of peers wastes network resources. 2. After the startup is complete, the node remembers its most recently connected peer (saved in the database?). When it reboots, it can quickly re-establish a connection with the previous peer network. If the previous network does not respond, the node can be restarted using the seed node.
Once the connection is successful, the first thing to do is to see the full blockchain in hindsight. If it is a brand new node, then it has only one genesis block – statically embedded in the client software. When two nodes communicate with each other, when sending version, the BestHeight field contained in the message indicates the current blockchain height (quantity) of a node. Both parties have a basic interconnection, version, and number of blocks. You can send a message asking getBlock to exchange the top blockchain hash of their local blockchain, depending on where the hash is locally to determine whose block is long and which blocks need to be sent to complement it. He propagates the block hash using an INV (Inventory) message, and nodes that lack these blocks can request the whole block information via their respective sent GetData message.
For example, suppose a node has only creation blocks. It will then receive an INV message from its peers containing hashes for the next 500 blocks in the chain. It will start requesting blocks from all of its connected peers, spreading the load and ensuring that it does not overwhelm any of its peers with requests. The node keeps track of the number of blocks “in transfer” per peer connection, that is, blocks requested but not yet received, and checks whether it has not exceeded the limit (MAX_BLOCKS_IN_TRANSIT_PER_PEER). This way, if a large number of blocks are required, new blocks are requested only when previous requests are satisfied, allowing peers to control the speed of updates without overwhelming the network. As each block is received, it will be added to the blockchain. As the local blockchain builds up, more blocks are requested and received, and the process continues until the nodes catch up with the rest of the network.
The verification method for nodes with complete block information is to construct a verification chain, which consists of thousands of blocks and transactions traced back to Trands block in reverse chronological order along the block chain, establish a complete UTXO database, and verify the validity of the transaction by confirming whether THE UTXO has not been paid.
Simple Payment Verification (SPV) node
Instead of downloading the transaction information contained in each block, the SPV node only needs to download the block header. Verify the transaction by checking the block above it and pressing it to the depth below. Privacy risks arise because SPV nodes need to read specific transactions to selectively verify them, and SPV nodes’ requests for specific data may inadvertently reveal the address information in the wallet. Bloom filters to address privacy risks associated with SPV nodes. Probabilistic rather than fixed mode filtering allows SPV nodes to receive only a subset of the transaction information without disclosing exactly which addresses they are interested in.
How does the Bloom filter do that?
Bloom filters are a probability-based filtering method that allows users to describe specific combinations of keywords without having to be precise. It allows users to effectively search for keywords while protecting their privacy. In the SPV node, it is used to send a transaction query request to the peer without exposing the transaction address.
Bloom filter concrete implementation
trading
A temporary list of unconfirmed transactions maintained by almost every node. He is the “material” that miners prepare in advance to start the next block. When trading, will be broadcast in the network, is stored in the transaction in the pool, when a new block to competition, will be validated from trading in the pool, added to the block and broadcast to the network, begin to complete the proof of the work, the problem solving trading on utxo pool right now, because he had passed the verification, but the deal is not complete, haven’t block on the chain, If the block containing the transaction fails to compete on the chain, the transaction is still in the UTXO pool and the transaction is considered completed.
7. Blockchain technology
Block structure: Consists of a block header containing metadata followed by a long list of transactions by a subsequent block body.
Block header composition: three groups of block metadata composition. First a set of data that references the hash value of the parent block. This set of metadata is used to connect the block to the previous block in the blockchain. The second set of metadata, namely difficulty, timestamp, and Nonce, is related to mining competition. The third set of metadata is the Merkle root, a data structure used to effectively summarize all transactions in a block.
Block identifiers: block header hash and block height
A digital fingerprint obtained by a secondary hash of the block header.
The head of the child block stores the head hash of the parent block, and the head hash of the child block is calculated as the “upper block bulk hash” of the head of the child block.
Merkle trees are hash binary trees used as data structures to quickly summarize and verify the integrity of large-scale data. This binary tree contains cryptographic hashes. In the Bitcoin network, it is used to summarize all transactions in a block, generate a digital fingerprint of the entire set of transactions, and provide an efficient way to verify the presence of a transaction in the block.
Merkle tree generation: You need to recursively hash the pair of hash nodes and insert the newly generated hash node into the Merkle tree until there is only one hash node left, which is the root of the Merkle tree.
It is necessary to prove whether the transaction K exists in the block === “to prove whether the Merkle tree has Hk (double hash value of k transaction) ===”, a Merkle path to the root === “is needed to compute the four times hash value for authentication comparison.
How do I get Merkle paths?
Merkle tree combined with Simple Payment Verification (SPV). An SPV node that wants to know that a bitcoin address in its wallet is due for payment sets up bloom filters on the communication links between the nodes, limiting the acceptance of transactions that only contain the targeted bitcoin address. When the node detects a transaction that matches the Bloom filter, it sends the block as a Merkle block message. The Merkle block message contains the block header and a Merkle path linking the target transaction to the Merkle root. The SPV node can use the path to find the block related to the transaction, and then verify whether the transaction exists in the corresponding block.
Consensus and incentive drive, mining
Mining: Computationally complete a mathematical problem based on a hash algorithm. The answers to these learning problems are included in the new block as proof of the miner’s computational workload, known as proof of work.
Rewards: There are two types: rewards for creating a new block, and a transaction fee for completing a computational puzzle.
Mining (consensus and incentive drive) === underpins the decentralization of bitcoin security
Each node verifies each transaction against a long standard list. If it is invalid, it will be discarded at the first node, otherwise it will propagate across the network. If the number of transactions received by the node is too large to be fully verified, a pool will be created for valid new transactions in the corresponding order when received, that is, the transaction pool.
The race to mine ends with the spread of new blocks. How do I know? The need to maintain a block chain complete records, and continuously monitored online trading, make sure you are the latest state of chain blocks, and preparing for the new block of mining) each node mining after the new block transmission (inception) announced the winners, new new blocks, copy a locally as a candidate for the block, (if able to untie top work proof solution then become block), record the New Deal, perfect the block header, and solve the “difficult” workload of the block has been proved that for the next new new new blocks collect good deal in trading pool to prepare in advance (be careful not to deal with blocks in front of the repeat, Ensure that any remaining transactions in the memory pool are unacknowledged, waiting to be recorded in the new block).
Because blocks are limited in size, miners need to assign a priority to each transaction in the memory pool and select high-priority transaction records to build candidate blocks.
Transaction priority:
If the transaction input value is high, the transaction with a large block age has a higher priority. If there is enough space in the block, high-priority transactions do not require miners’ fees.
Priority = Sum(Value of inout * Input) / Transaction Size;
The value entered for a transaction is expressed in bitcoin units “cong”, the UTXO block age is the number of blocks that the UTXO has experienced until it is recorded on the blockchain, and the size of the transaction record is expressed in bytes. Prioritize the transactions with high miner fees to fill the remaining blocks, with a block size upper limit of MAX_BLOCK_SIZE. The rest, according to “kilobyte miner’s rate”
When the transaction is stored in the transaction pool, it is not authenticated, so the user is in the unauthenticated state, stored in the memory pool, it may be restarted, the transaction will be erased, or because of the priority, it has been placed in the memory pool, too long unprocessed, it will disappear. At this point, the wallet software should resend the transaction or repay a higher miner fee, increasing the priority.
Crecoin, or Coinbase, is used to reward miners for creating new blocks. The number of rewards will be cut in half with every 210,000 blocks produced.
When the information in the head of the block is filled in, mining can begin. The mining target finds a Nonce with a block header hash value less than the difficulty target. Mining nodes usually need to try billions of different Nonce values until they find a nonce value that meets the condition.
Nonce count (0,1,2,3… Count up) is appended to a list of numbers to compute its hash value. When the computed hash value is less than this target, it proves that a certain amount of work has been achieved.
With the improvement of computing power, a random value ascending scheme is developed to solve the basic structure limitation of block header. To solve the problem of insufficient random number variation: After one round of calculation, the nonce reaches the bottom, and then a backdated timestamp is taken to recalculate the value for target. However, as the difficulty increases, the backdated timestamp will invalidate the block because it is too old, so the 2-100 bytes that can be stored using the Coinbase script, Source of additional random values.
So how do you set this target value?
Through the difficulty target, the value of the marker is stored as a coefficient/exponent, with the first two hexadecimal digits as powers, followed by a six-digit coefficient. For example, 0x1903A30c, the formula for calculating the difficulty target is:
target = coefficient * 2^(8 * (exponent – 3));
Want to keep the mining speed in the Bitcoin network at about ten minutes per block
For every 2016 blocks discovered, a difficulty adjustment is made based on the actual time of use of the previous 2016 blocks.
When the mining of the proof of work is completed, it will be transmitted to its neighboring nodes in the network. After the neighboring nodes verify, it will also be transmitted to its neighboring nodes and spread to the network. To ensure that the node propagated is valid.
How do you validate a new block?
The validation criteria are obtained in the CheckBlock and CheckBlockHead functions.
Aggregates blocks into the chain with the maximum proof of work
Once a node validates a new block, it will attempt to connect the new block to the existing blockchain, assembling them together.
So which chain will the new block be connected to?
Three types of blocks are maintained on each node: 1. Blocks that are connected to the main chain, 2. Blocks that branch from the main chain (alternate chains), and 3. The main chain is the block chain with the most accumulative difficulty. The main chain is opposite to the standby chain. When the accumulative difficulty is greater, it will become the new main chain and the rest will be the standby chain. If a node receives a block, but its parent block cannot be found, the block is called a “lone block”. The orphan block is kept in the orphan block pool until the parent block is found by the node.
The fork of blockchain
Inevitably there will be two nodes (set as A, B) on the same parent block (set as P), opportunity to complete the new block of work proof at the same time, to broadcast to the network, then there will be two block fork, there will be P — “A, P –” B, two different chains, these two chains are legal, At this point, the two chains in the network are immediately put into new acquisition, in order to extend their own length, competing to become the main chain. The other one will be the backup chain. At this point, these nodes adopt a new and longer chain, changing their view of the blockchain, which is called chain reconsensus. The result is that the candidate block on the backup chain becomes a lone block, because its parent block is on the backup chain and the main chain is not found, and the miners who used the backup chain all stop and use the chain that became the main chain.