--- title: P2P Network description: "## Creating A [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/)[https://developer.bitcoin.org/examples/p2p_networking.html#creating-a-bloom-filter](https://developer.bitcoin.org/examples/p2p_networking.html#creating-a-bloom-filter) In this section, we’ll use variable names that correspond to the field names in the [“filterload” message documentation](https://developer.bitcoin.org/reference/p2p_networking.html#filterload). Each code [Block](https://bitcoindevelopers.org/encyclopedia/block/) precedes the paragraph describing it. ``` #!/usr/bin/env python BYTES_MAX = 36000 FUNCS_MAX = 50 nFlags…" url: "https://bitcoindevelopers.org/docs/examples/p2p-network-example/" updated: "2025-05-14" category: Examples --- # P2P Network ## Creating A [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/)[https://developer.bitcoin.org/examples/p2p_networking.html#creating-a-bloom-filter](https://developer.bitcoin.org/examples/p2p_networking.html#creating-a-bloom-filter) In this section, we’ll use variable names that correspond to the field names in the [“filterload” message documentation](https://developer.bitcoin.org/reference/p2p_networking.html#filterload). Each code [Block](https://bitcoindevelopers.org/encyclopedia/block/) precedes the paragraph describing it. ``` #!/usr/bin/env python BYTES_MAX = 36000 FUNCS_MAX = 50 nFlags = 0 ``` We start by setting some maximum values defined in [BIP37](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki): the maximum number of bytes allowed in a filter and the maximum number of hash functions used to hash each piece of data. We also set nFlags to zero, indicating we don’t want the remote [Node](https://bitcoindevelopers.org/encyclopedia/node/) to update the filter for us. (We won’t use nFlags again in the sample program, but real programs will need to use it.) ``` n = 1 p = 0.0001 ``` We define the number (n) of elements we plan to insert into the filter and the false positive rate (p) we want to help protect our privacy. For this example, we will set *n* to one element and *p* to a rate of 1-in-10,000 to produce a small and precise filter for illustration purposes. In actual use, your filters will probably be much larger. ``` from math import log nFilterBytes = int(min((-1 / log(2)**2 * n * log(p)) / 8, BYTES_MAX)) nHashFuncs = int(min(nFilterBytes * 8 / n * log(2), FUNCS_MAX)) from bitarray import bitarray # from pypi.python.org/pypi/bitarray vData = nFilterBytes * 8 * bitarray('0', endian="little") ``` Using the formula described in [BIP37](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki), we calculate the ideal size of the filter (in bytes) and the ideal number of hash functions to use. Both are truncated down to the nearest whole number and both are also constrained to the maximum values we defined earlier. The results of this particular fixed computation are 2 filter bytes and 11 hash functions. We then use *nFilterBytes* to create a little-endian bit array of the appropriate size. ``` nTweak = 0 ``` We also should choose a value for *nTweak*. In this case, we’ll simply use zero. ``` import pyhash # from https://github.com/flier/pyfasthash murmur3 = pyhash.murmur3_32() def bloom_hash(nHashNum, data): seed = (nHashNum * 0xfba4c795 + nTweak) & 0xffffffff return( murmur3(data, seed=seed) % (nFilterBytes * 8) ) ``` We setup our hash function template using the formula and 0xfba4c795 constant set in [BIP37](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki). Note that we limit the size of the seed to four bytes and that we’re returning the result of the hash modulo the size of the filter in bits. ``` data_to_hash = "019f5b01d4195ecbc9398fbf3c3b1fa9" \ + "bb3183301d7a1fb3bd174fcfa40a2b65" data_to_hash = data_to_hash.decode("hex") ``` For the data to add to the filter, we’re adding a [Txid](https://bitcoindevelopers.org/encyclopedia/txid/). Note that the [Txid](https://bitcoindevelopers.org/encyclopedia/txid/) is in [Internal byte order](https://bitcoindevelopers.org/encyclopedia/internal-byte-order/). ``` print " Filter (As Bits)" print "nHashNum nIndex Filter 0123456789abcdef" print "~~~~~~~~ ~~~~~~ ~~~~~~ ~~~~~~~~~~~~~~~~" for nHashNum in range(nHashFuncs): nIndex = bloom_hash(nHashNum, data_to_hash) ## Set the bit at nIndex to 1 vData[nIndex] = True ## Debug: print current state print ' {0:2} {1:2} {2} {3}'.format( nHashNum, hex(int(nIndex)), vData.tobytes().encode("hex"), vData.to01() ) print print "Bloom filter:", vData.tobytes().encode("hex") ``` Now we use the hash function template to run a slightly different hash function for *nHashFuncs* times. The result of each function being run on the transaction is used as an index number: the bit at that index is set to 1. We can see this in the printed debugging [Output](https://bitcoindevelopers.org/encyclopedia/output/): ``` Filter (As Bits) nHashNum nIndex Filter 0123456789abcdef ~~~~~~~~ ~~~~~~ ~~~~~~ ~~~~~~~~~~~~~~~~ 0 0x7 8000 0000000100000000 1 0x9 8002 0000000101000000 2 0xa 8006 0000000101100000 3 0x2 8406 0010000101100000 4 0xb 840e 0010000101110000 5 0x5 a40e 0010010101110000 6 0x0 a50e 1010010101110000 7 0x8 a50f 1010010111110000 8 0x5 a50f 1010010111110000 9 0x8 a50f 1010010111110000 10 0x4 b50f 1010110111110000 Bloom filter: b50f ``` Notice that in iterations 8 and 9, the filter did not change because the corresponding bit was already set in a previous iteration (5 and 7, respectively). This is a normal part of [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/) operation. We only added one element to the filter above, but we could repeat the process with additional elements and continue to add them to the same filter. (To maintain the same false-positive rate, you would need a larger filter size as computed earlier.) Note: for a more optimized Python implementation with fewer external dependencies, see [python-bitcoinlib’s](https://github.com/petertodd/python-bitcoinlib) [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/) module which is based directly on Bitcoin Core’s C++ implementation. Using the [“filterload” message](https://developer.bitcoin.org/reference/p2p_networking.html#filterload) format, the complete filter created above would be the binary form of the annotated hexdump shown below: ``` 02 ......... Filter bytes: 2 b50f ....... Filter: 1010 1101 1111 0000 0b000000 ... nHashFuncs: 11 00000000 ... nTweak: 0/none 00 ......... nFlags: BLOOM_UPDATE_NONE ``` ## Evaluating A [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/)[https://developer.bitcoin.org/examples/p2p_networking.html#evaluating-a-bloom-filter](https://developer.bitcoin.org/examples/p2p_networking.html#evaluating-a-bloom-filter) Using a [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/) to find matching data is nearly identical to constructing a [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/)—except that at each step we check to see if the calculated index bit is set in the existing filter. ``` vData = bitarray(endian='little') vData.frombytes("b50f".decode("hex")) nHashFuncs = 11 nTweak = 0 nFlags = 0 ``` Using the [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/) created above, we import its various parameters. Note, as indicated in the section above, we won’t actually use *nFlags* to update the filter. ``` def contains(nHashFuncs, data_to_hash): for nHashNum in range(nHashFuncs): ## bloom_hash as defined in previous section nIndex = bloom_hash(nHashNum, data_to_hash) if vData[nIndex] != True: print "MATCH FAILURE: Index {0} not set in {1}".format( hex(int(nIndex)), vData.to01() ) return False ``` We define a function to check an element against the provided filter. When checking whether the filter might contain an element, we test to see whether a particular bit in the filter is already set to 1 (if it isn’t, the match fails). ``` ## Test 1: Same Txid as previously added to filter data_to_hash = "019f5b01d4195ecbc9398fbf3c3b1fa9" \ + "bb3183301d7a1fb3bd174fcfa40a2b65" data_to_hash = data_to_hash.decode("hex") contains(nHashFuncs, data_to_hash) ``` Testing the filter against the data element we previously added, we get no [Output](https://bitcoindevelopers.org/encyclopedia/output/) (indicating a possible match). Recall that bloom filters have a zero false negative rate—so they should always match the inserted elements. ``` ## Test 2: Arbitrary string data_to_hash = "1/10,000 chance this ASCII string will match" contains(nHashFuncs, data_to_hash) ``` Testing the filter against an arbitrary element, we get the failure [Output](https://bitcoindevelopers.org/encyclopedia/output/) below. Note: we created the filter with a 1-in-10,000 false positive rate (which was rounded up somewhat when we truncated), so it was possible this arbitrary string would’ve matched the filter anyway. It is not possible to set a [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/) to a false positive rate of zero, so your program will always have to deal with false positives. The [Output](https://bitcoindevelopers.org/encyclopedia/output/) below shows us that one of the hash functions returned an index number of 0x06, but that bit wasn’t set in the filter, causing the match failure: ``` MATCH FAILURE: Index 0x6 not set in 1010110111110000 ``` ## Retrieving A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#retrieving-a-merkleblock](https://developer.bitcoin.org/examples/p2p_networking.html#retrieving-a-merkleblock) For the [“merkleblock” message](https://developer.bitcoin.org/reference/p2p_networking.html#merkleblock) documentation on the reference page, an actual [Merkle block](https://bitcoindevelopers.org/encyclopedia/merkle-block/) was retrieved from the [network](https://developer.bitcoin.org/devguide/p2p_network.html) and manually processed. This section walks through each step of the process, demonstrating basic [network](https://developer.bitcoin.org/devguide/p2p_network.html) communication and [Merkle block](https://bitcoindevelopers.org/encyclopedia/merkle-block/) processing. ``` #!/usr/bin/env python from time import sleep from hashlib import sha256 import struct import sys network_string = "f9beb4d9".decode("hex") # Mainnet def send(msg,payload): ## Command is ASCII text, null padded to 12 bytes command = msg + ( ( 12 - len(msg) ) * "\00" ) ## Payload length is a uint32_t payload_raw = payload.decode("hex") payload_len = struct.pack("I", len(payload_raw)) ## Checksum is first 4 bytes of SHA256(SHA256()) checksum = sha256(sha256(payload_raw).digest()).digest()[:4] sys.stdout.write( network_string + command + payload_len + checksum + payload_raw ) sys.stdout.flush() ``` To connect to the P2P [network](https://developer.bitcoin.org/devguide/p2p_network.html), the trivial Python function above was developed to compute [Message](https://bitcoindevelopers.org/encyclopedia/message/) headers and send payloads decoded from hex. ``` ## Create a version Message send("version", "71110100" # ........................ Protocol Version: 70001 + "0000000000000000" # ................ Services: Headers Only (SPV) + "c6925e5400000000" # ................ Time: 1415484102 + "00000000000000000000000000000000" + "0000ffff7f000001208d" # ............ Receiver IP Address/Port + "00000000000000000000000000000000" + "0000ffff7f000001208d" # ............ Sender IP Address/Port + "0000000000000000" # ................ Nonce (not used here) + "1b" # .............................. Bytes in version string + "2f426974636f696e2e6f726720457861" + "6d706c653a302e392e332f" # .......... Version string + "93050500" # ........................ Starting Block height: 329107 + "00" # .............................. Relay transactions: false ) ``` Peers on the [network](https://developer.bitcoin.org/devguide/p2p_network.html) will not accept any requests until you send them a [“version” message](https://developer.bitcoin.org/reference/p2p_networking.html#version). The receiving [Node](https://bitcoindevelopers.org/encyclopedia/node/) will reply with their [“version” message](https://developer.bitcoin.org/reference/p2p_networking.html#version) and a [“verack” message](https://developer.bitcoin.org/reference/p2p_networking.html#verack). ``` sleep(1) send("verack", "") ``` We’re not going to validate their [“version” message](https://developer.bitcoin.org/reference/p2p_networking.html#version) with this simple script, but we will sleep a short bit and send back our own [“verack” message](https://developer.bitcoin.org/reference/p2p_networking.html#verack) as if we had accepted their [“version” message](https://developer.bitcoin.org/reference/p2p_networking.html#version). ``` send("filterload", "02" # ........ Filter bytes: 2 + "b50f" # ....... Filter: 1010 1101 1111 0000 + "0b000000" # ... nHashFuncs: 11 + "00000000" # ... nTweak: 0/none + "00" # ......... nFlags: BLOOM_UPDATE_NONE ) ``` We set a [Bloom filter](https://bitcoindevelopers.org/encyclopedia/bloom-filter/) with the [“filterload” message](https://developer.bitcoin.org/reference/p2p_networking.html#filterload). This filter is described in the two preceeding sections. ``` send("getdata", "01" # ................................. Number of inventories: 1 + "03000000" # ........................... Inventory type: filtered Block + "a4deb66c0d726b0aefb03ed51be407fb" + "ad7331c6e8f9eef231b7000000000000" # ... Block Header hash ) ``` We request a [Merkle block](https://bitcoindevelopers.org/encyclopedia/merkle-block/) for transactions matching our filter, completing our script. To run the script, we simply pipe it to the Unix ``netcat` command <[https://en.wikipedia.org/wiki/Netcat](https://en.wikipedia.org/wiki/Netcat)>\`__ or one of its many clones, one of which is available for practically any platform. For example, with the original netcat and using hexdump (`hd`) to display the [Output](https://bitcoindevelopers.org/encyclopedia/output/): ``` ## Connect to the Bitcoin Core Peer running on localhost python get-merkle.py | nc localhost 8333 | hd ``` Part of the response is shown in the section below. ## Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#parsing-a-merkleblock](https://developer.bitcoin.org/examples/p2p_networking.html#parsing-a-merkleblock) In the section above, we retrieved a [Merkle block](https://bitcoindevelopers.org/encyclopedia/merkle-block/) from the [network](https://developer.bitcoin.org/devguide/p2p_network.html); now we will parse it. Most of the [Block](https://bitcoindevelopers.org/encyclopedia/block/) [Header](https://bitcoindevelopers.org/encyclopedia/header/) has been omitted. For a more complete hexdump, see the example in the ``MerkleBlock` [Message](https://bitcoindevelopers.org/encyclopedia/message/) section <../reference/p2p_networking.html#[MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)>\`__. ``` 7f16c5962e8bd963659c793ce370d95f 093bc7e367117b3c30c1f8fdd0d97287 ... Merkle root 07000000 ........................... Transaction count: 7 04 ................................. Hash count: 4 3612262624047ee87660be1a707519a4 43b1c1ce3d248cbfc6c15870f6c5daa2 ... Hash #1 019f5b01d4195ecbc9398fbf3c3b1fa9 bb3183301d7a1fb3bd174fcfa40a2b65 ... Hash #2 41ed70551dd7e841883ab8f0b16bf041 76b7d1480e4f0af9f3d4c3595768d068 ... Hash #3 20d2a7bc994987302e5b1ac80fc425fe 25f8b63169ea78e68fbaaefa59379bbf ... Hash #4 01 ................................. Flag bytes: 1 1d ................................. Flags: 1 0 1 1 1 0 0 0 ``` We parse the above [“merkleblock” message](https://developer.bitcoin.org/reference/p2p_networking.html#merkleblock) using the following instructions. Each illustration is described in the paragraph below it. ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-001.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id1](https://developer.bitcoin.org/examples/p2p_networking.html#id1) We start by building the structure of a [Merkle tree](https://bitcoindevelopers.org/encyclopedia/merkle-tree/) based on the number of transactions in the [Block](https://bitcoindevelopers.org/encyclopedia/block/). ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-002.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id2](https://developer.bitcoin.org/examples/p2p_networking.html#id2) The first flag is a 1 and the [Merkle root](https://bitcoindevelopers.org/encyclopedia/merkle-root/) is (as always) a non-[Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/), so we will need to compute the hash later based on this [Node](https://bitcoindevelopers.org/encyclopedia/node/)’s children. Accordingly, we descend into the [Merkle root](https://bitcoindevelopers.org/encyclopedia/merkle-root/)’s left child and look at the next flag for instructions. ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-003.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id3](https://developer.bitcoin.org/examples/p2p_networking.html#id3) The next flag in the example is a 0 and this is also a non-[Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/), so we apply the first hash from the [“merkleblock” message](https://developer.bitcoin.org/reference/p2p_networking.html#merkleblock) to this [Node](https://bitcoindevelopers.org/encyclopedia/node/). We also don’t process any child nodes—according to the [Peer](https://bitcoindevelopers.org/encyclopedia/peer/) which created the [“merkleblock” message](https://developer.bitcoin.org/reference/p2p_networking.html#merkleblock), none of those nodes will lead to TXIDs of transactions that match our filter, so we don’t need them. We go back up to the [Merkle root](https://bitcoindevelopers.org/encyclopedia/merkle-root/) and then descend into its right child and look at the next (third) flag for instructions. ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-004.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id4](https://developer.bitcoin.org/examples/p2p_networking.html#id4) The third flag in the example is another 1 on another non-[Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/), so we descend into its left child. ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-005.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id5](https://developer.bitcoin.org/examples/p2p_networking.html#id5) The fourth flag is also a 1 on another non-[Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/), so we descend again—we will always continue descending until we reach a [Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/) or a non-[Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/) with a 0 flag (or we finish filling out the tree). ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-006.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id6](https://developer.bitcoin.org/examples/p2p_networking.html#id6) Finally, on the fifth flag in the example (a 1), we reach a [Txid](https://bitcoindevelopers.org/encyclopedia/txid/) [Node](https://bitcoindevelopers.org/encyclopedia/node/). The 1 flag indicates this [Txid](https://bitcoindevelopers.org/encyclopedia/txid/)’s transaction matches our filter and that we should take the next (second) hash and use it as this [Node](https://bitcoindevelopers.org/encyclopedia/node/)’s [Txid](https://bitcoindevelopers.org/encyclopedia/txid/). ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-007.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id7](https://developer.bitcoin.org/examples/p2p_networking.html#id7) The sixth flag also applies to a [Txid](https://bitcoindevelopers.org/encyclopedia/txid/), but it’s a 0 flag, so this [Txid](https://bitcoindevelopers.org/encyclopedia/txid/)’s transaction doesn’t match our filter; still, we take the next (third) hash and use it as this [Node](https://bitcoindevelopers.org/encyclopedia/node/)’s [Txid](https://bitcoindevelopers.org/encyclopedia/txid/). ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-008.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id8](https://developer.bitcoin.org/examples/p2p_networking.html#id8) We now have enough information to compute the hash for the fourth [Node](https://bitcoindevelopers.org/encyclopedia/node/) we encountered—it’s the hash of the concatenated hashes of the two TXIDs we filled out. ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-009.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id9](https://developer.bitcoin.org/examples/p2p_networking.html#id9) Moving to the right child of the third [Node](https://bitcoindevelopers.org/encyclopedia/node/) we encountered, we fill it out using the seventh flag and final hash—and discover there are no more child nodes to process. ![Parsing A MerkleBlock](https://developer.bitcoin.org/_images/en-merkleblock-parsing-011.svg) Parsing A [MerkleBlock](https://bitcoindevelopers.org/encyclopedia/merkleblock/)[https://developer.bitcoin.org/examples/p2p_networking.html#id10](https://developer.bitcoin.org/examples/p2p_networking.html#id10) We hash as appropriate to fill out the tree. Note that the eighth flag is not used—this is acceptable as it was required to pad out a flag byte. The final steps would be to ensure the computed [Merkle root](https://bitcoindevelopers.org/encyclopedia/merkle-root/) is identical to the [Merkle root](https://bitcoindevelopers.org/encyclopedia/merkle-root/) in the [Header](https://bitcoindevelopers.org/encyclopedia/header/) and check the other steps of the parsing checklist in the [“merkleblock” message](https://developer.bitcoin.org/reference/p2p_networking.html#merkleblock) section.