Module address: github.com/netwarps/li…

In rust-libp2p, when the protocol wants to obtain the address corresponding to peer_id, it needs to implement the addresses_of_peer method of NetworkBehaviour. In contrast, go-libp2p uses peerStore to store the relationship between peer_id and address, so we can refer to it to implement a rust version of PeerStore.

Realize the idea

Firstly, since our startup core is Swarm, peerStore will be one of the properties. Secondly, in go-libp2-peerstore, peerstore mainly stores three pieces of data: AddrBook of address information, KeyBook of public and private key information, and ProtoBook of protocol information. We can combine all three to form a new struct called PeerRecord in a Hashmap with peer_id as key.

Garbage collection mechanism

The purpose of GC is to prevent hashMaps from overexpanding. As the network status changes all the time, the connections between peers may also change accordingly. The PeerStore plays an important role in storing the address information of peers. If invalid peer information is not cleared, the working efficiency of PeerStore will be affected.

Swarm > task::spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn spawn Clear the current address; Also, if the current address set for peer_id does not have any address information, it is removed from the Hashmap.

Pinned

The GC mechanism, though, helps the system by keeping Hashmap from expanding indefinitely. However, there are still some shortcomings. Consider the following situation:

For THE KAD protocol, the peer needs to iteratively query the known nodes and fill its KBucket step by step, which is a time-consuming process. If the address information queried earlier is removed from peerStore during GC, then iterative query needs to be enabled again to obtain the address, so we add a bool value pinned to PeerRecord. The GC determines the category of the record and if the pinned value is true, the cleanup step will be skipped.

Serialization and persistence

For each peer, when the peer needs to go offline or stop for some reason, the node information stored in peerStore should not be discarded. Data that needs to be persisted also needs to be serialized for easy storage.

In libp2-RS, the call to the main loop is also started with Task ::spawn. When Swarm receives a close message, it will exit the event processing loop and send a close() event to the GC thread running PeerStore, ending the GC process. The save_data() method of PeerStore is then called to serialize the data to JSON format using serde and to a TXT file in the root directory using STD :: IO.

Methods to analyze

Parse by GC:

  1. Swarm main spawn runs tasks and triggers a select every ten minutes.
  2. The Hashmap is wrapped by Arc and can be retrieved by lock() for concurrency security.
  3. If the peer information was not obtained through KAD, call retain to filter the addresses that did not exceed the TTL.
  4. If the address data of the current peer has been cleared, remove the peer from the HashMap.
// swarm/lib.rs // The GC task is to remove all expired addresses from the peer store task::spawn(async move { log::info! ("starting Peerstore GC..." ); loop { let either = future::select(rx.next(), task::sleep(PEERSTORE_GC_PURGE_INTERVAL).boxed()).await; match either { Either::Left((_, _)) => break, Either::Right((_, _)) => peer_store.remove_expired_addrs(), } } log::info! ("quitting Peerstore GC..." ); }); // core/peerstore.rs /// Removes all expired address. pub fn remove_expired_addrs(&self) { let mut to_remove = vec! []; let mut guard = self.inner.lock().unwrap(); for (peer, pr) in guard.iter_mut() { if ! pr.pinned { log::debug! ("GC attempt for {:? }", peer); pr.addrs.retain(|record| record.expiry.elapsed() < record.ttl); // delete this peer if no addr at all if pr.addrs.is_empty() { log::debug! ("remove {:? } from peerstore", peer); to_remove.push(peer.clone()); } } } for peer in to_remove { guard.remove(&peer); }}Copy the code

Netwarps is composed of a senior cloud computing and distributed technology development team in China, which has rich experience in the financial, power, communication and Internet industries. Netwarps has set up research and development centers in Shenzhen and Beijing, with a team size of 30+, most of which are technical personnel with more than 10 years of development experience, respectively from the Internet, finance, cloud computing, blockchain and scientific research institutions and other professional fields. Netwarps focuses on the development and application of secure storage technology products, the main products are decentralized file system (DFS), decentralized computing platform (DCP), is committed to providing distributed storage and distributed computing platform based on decentralized network technology, with high availability, low power consumption and low network technical characteristics. Applicable to scenarios such as the Internet of Things and industrial Internet. Public account: Netwarps