Implementing a Blockchain in Rust

This article provides a step-by-step breakdown of implementing a basic blockchain in Rust, from the initial setup of the block structure, including unique identifiers and cryptographic hashes, to block creation, mining, and validation, laying the groundwork. It explains the rationale behind each function, offering insights into the mechanics of blockchain technology, such as proof of work, nonce calculation, and maintaining the integrity and continuity of the blockchain.

Disclaimer: Remember that this is a bare-bones and basic Blockchain implementation for learning purposes only and is not intended for production environments! :)

Let’s dive right in, fellow Rustaceans!

The Groundwork

Here are the core concepts and key elements in handling P2P network interactions in blockchain:

Node and Peer Discovery

• Node Identity: Each node in the blockchain network has a unique identifier, often derived from a cryptographic key pair.
• Peer Discovery: Nodes must discover each other to form a network. This can be achieved through various methods, including static configuration (predefined peers), DNS-based discovery, or using protocols like mDNS for local network discovery.
• Bootstrap Nodes: New nodes often connect to known, reliable nodes (bootstrap nodes) to quickly integrate into the network.

Network Protocols

• Protocol Stacks: Blockchain P2P networks use specific protocol stacks for communication. Commonly used protocols include TCP/IP for basic transmission and cryptographic protocols (like TLS or Noise) for secure communication.
• Messaging Protocols: Protocols like libp2p's Floodsub or Gossipsub are used for message broadcasting and propagation across the network.

Data Propagation and Synchronization

• Broadcasting: Nodes broadcast transactions and newly mined blocks to the network, ensuring that all participants receive the latest data.
• Chain Synchronization: Nodes synchronize their blockchain copies to the longest chain (commonly accepted as the valid one) to maintain consistency in the network.
• Consensus Mechanisms: Consensus algorithms like Proof of Work (PoW) or Proof of Stake (PoS) are used to agree on the state of the blockchain, especially for validating and adding new blocks.

Once these key concepts are understood, let’s now jump to the code!

cargo.toml — Add the following dependencies

[dependencies]
chrono = "0.4"
sha2 = "0.9.8"
serde = {version = "1.0", features = ["derive"] }
serde_json = "1.0"
libp2p = { version = "0.40.0", features = ["tcp-tokio", "mdns"] }
tokio = { version = "1.0", features = ["io-util", "io-std", "macros", "rt", "rt-multi-thread", "sync", "time"] }
hex = "0.4"
once_cell = "1.5"
log = "0.4"
pretty_env_logger = "0.4"

We won’t cover every lib in detail to not take much time, but it’s worth mentioning a key library for P2P implementation — the libp2p.

I wrote a full article about the libp2p here where you can get familiar with this rich Rust crate.

blockchain.rs — Event Processing and P2P Communication

Let’s create a new file named ‘blockchain.rs’ in which we will implement our event processing and p2p methods, as below:

Basic Setup

• KEYS, PEER_ID, CHAIN_TOPIC, BLOCK_TOPIC: These static variables initialize cryptographic keys for identity, the peer ID for the network node, and topics for the Floodsub protocol to handle chain and block-related messages.

use super::{App, Block};
use libp2p::{
    floodsub::{Floodsub, FloodsubEvent, Topic},
    identity,
    mdns::{Mdns, MdnsEvent},
    swarm::{NetworkBehaviourEventProcess, Swarm},
    NetworkBehaviour, PeerId,
};
use log::{error, info};
use once_cell::sync::Lazy;
use serde::{Deserialize, Serialize};
use std::collections::HashSet;
use tokio::sync::mpsc;

pub static KEYS: Lazy<identity::Keypair> = Lazy::new(identity::Keypair::generate_ed25519);
pub static PEER_ID: Lazy<PeerId> = Lazy::new(|| PeerId::from(KEYS.public()));
pub static CHAIN_TOPIC: Lazy<Topic> = Lazy::new(|| Topic::new("chains"));
pub static BLOCK_TOPIC: Lazy<Topic> = Lazy::new(|| Topic::new("blocks"));

Event Types

• ChainResponse, LocalChainRequest, EventType: These data structures define the types of events and messages that nodes can send and receive. ChainResponse and LocalChainRequest are for responding to chain requests and requesting the local chain state.

#[derive(Debug, Serialize, Deserialize)]
pub struct ChainResponse {
    pub blocks: Vec<Block>,
    pub receiver: String,
}

#[derive(Debug, Serialize, Deserialize)]
pub struct LocalChainRequest {
    pub from_peer_id: String,
}

pub enum EventType {
    LocalChainResponse(ChainResponse),
    Input(String),
    Init,
}

AppBehaviour

• NetworkBehaviour Implementation (AppBehaviour): This struct implements the NetworkBehaviour trait, combining different behaviours like Floodsub (for pub/sub messaging) and mDNS (for local network peer discovery). It also holds channels for sending responses and initializing events and an instance of the App struct which contains the blockchain logic.

#[derive(NetworkBehaviour)]
#[behaviour(out_event="Event")]
pub struct AppBehaviour {
    pub floodsub: Floodsub,
    pub mdns: Mdns,
    #[behaviour(ignore)]
    pub response_sender: mpsc::UnboundedSender<ChainResponse>,
    #[behaviour(ignore)]
    pub init_sender: mpsc::UnboundedSender<bool>,
    #[behaviour(ignore)]
    pub app: App,
}

#[derive(Debug)]
pub enum Event {
    ChainResponse(ChainResponse),
    Floodsub(FloodsubEvent),
    Mdns(MdnsEvent),
    Input(String),
    Init,
}

impl From<FloodsubEvent> for Event {
    fn from(event: FloodsubEvent) -> Self {
        Self::Floodsub(event)
    }
}

impl From<MdnsEvent> for Event {
    fn from(event: MdnsEvent) -> Self {
        Self::Mdns(event)
    }
}

impl AppBehaviour {
    pub async fn new(
        app: App,
        response_sender: mpsc::UnboundedSender<ChainResponse>,
        init_sender: mpsc::UnboundedSender<bool>,
    ) -> Self {
        let mut behaviour = Self {
            app,
            floodsub: Floodsub::new(*PEER_ID),
            mdns: Mdns::new(Default::default())
                .await
                .expect("can create mdns"),
            response_sender,
            init_sender,
        };

        behaviour.floodsub.subscribe(CHAIN_TOPIC.clone());
        behaviour.floodsub.subscribe(BLOCK_TOPIC.clone());

        behaviour
    }
}

Event Handling

• NetworkBehaviourEventProcess for FloodsubEvent and MdnsEvent: These implementations define how the application reacts to different network events.
• Floodsub Events: Handle incoming blockchain-related messages, such as new blocks, chain responses, or local chain requests. For example, when a new block is received, it is added to the blockchain through try_add_block.
• mDNS Events: Handle discovery of new peers or loss of existing ones in the local network. It updates the peer list in the Floodsub protocol accordingly.

// incoming event handler
impl NetworkBehaviourEventProcess<FloodsubEvent> for AppBehaviour {
    fn inject_event(&mut self, event: FloodsubEvent) {
        if let FloodsubEvent::Message(msg) = event {
            if let Ok(resp) = serde_json::from_slice::<ChainResponse>(&msg.data) {
                if resp.receiver == PEER_ID.to_string() {
                    info!("Response from {}:", msg.source);
                    resp.blocks.iter().for_each(|r| info!("{:?}", r));

                    self.app.blocks = self.app.choose_chain(self.app.blocks.clone(), resp.blocks);
                }
            } else if let Ok(resp) = serde_json::from_slice::<LocalChainRequest>(&msg.data) {
                info!("sending local chain to {}", msg.source.to_string());
                let peer_id = resp.from_peer_id;
                if PEER_ID.to_string() == peer_id {
                    if let Err(e) = self.response_sender.send(ChainResponse {
                        blocks: self.app.blocks.clone(),
                        receiver: msg.source.to_string(),
                    }) {
                        error!("error sending response via channel, {}", e);
                    }
                }
            } else if let Ok(block) = serde_json::from_slice::<Block>(&msg.data) {
                info!("received new block from {}", msg.source.to_string());
                self.app.try_add_block(block);
            }
        }
    }
}

impl NetworkBehaviourEventProcess<MdnsEvent> for AppBehaviour {
    fn inject_event(&mut self, event: MdnsEvent) {
        match event {
            MdnsEvent::Discovered(discovered_list) => {
                for (peer, _addr) in discovered_list {
                    self.floodsub.add_node_to_partial_view(peer);
                }
            }
            MdnsEvent::Expired(expired_list) => {
                for (peer, _addr) in expired_list {
                    if !self.mdns.has_node(&peer) {
                        self.floodsub.remove_node_from_partial_view(&peer);
                    }
                }
            }
        }
    }
}

Utility Functions

• get_list_peers: Returns a list of discovered peers in the network.
• handle_print_peers: Logs the list of peers to the console.
• handle_print_chain: Logs the local blockchain state, providing a visual representation of the blockchain.
• handle_create_block: Handles user input to create a new block. It creates a new block with the provided data, updates the local blockchain, and broadcasts the new block to peers using Floodsub.

pub fn get_list_peers(swarm: &Swarm<AppBehaviour>) -> Vec<String> {
    info!("Discovered Peers:");
    let nodes = swarm.behaviour().mdns.discovered_nodes();
    let mut unique_peers = HashSet::new();
    for peer in nodes {
        unique_peers.insert(peer);
    }
    unique_peers.iter().map(|p| p.to_string()).collect()
}

pub fn handle_print_peers(swarm: &Swarm<AppBehaviour>) {
    let peers = get_list_peers(swarm);
    peers.iter().for_each(|p| info!("{}", p));
}

pub fn handle_print_chain(swarm: &Swarm<AppBehaviour>) {
    info!("Local Blockchain:");
    let pretty_json =
        serde_json::to_string_pretty(&swarm.behaviour().app.blocks).expect("can jsonify blocks");
    info!("{}", pretty_json);
}

pub fn handle_create_block(cmd: &str, swarm: &mut Swarm<AppBehaviour>) {
    if let Some(data) = cmd.strip_prefix("create b") {
        let behaviour = swarm.behaviour_mut();
        let latest_block = behaviour
            .app
            .blocks
            .last()
            .expect("there is at least one block");
        let block = Block::new(
            latest_block.id + 1,
            latest_block.hash.clone(),
            data.to_owned(),
        );
        let json = serde_json::to_string(&block).expect("can jsonify request");
        behaviour.app.blocks.push(block);
        info!("broadcasting new block");
        behaviour
            .floodsub
            .publish(BLOCK_TOPIC.clone(), json.as_bytes());
    }
}

main.rs — Main Loop and Block Mining

Let’s now create a new file named ‘main.rs’ which will take care of the main loop and block mining, the core logic, as below:

Block Structure

The Block struct defines the structure of a block in the blockchain. It includes:

• id: A unique identifier for the block.
• hash: The block's hash.
• previous_hash: The hash of the previous block in the chain.
• timestamp: The creation time of the block.
• data: The data or payload of the block.
• nonce: A value used during the mining process.

use std::io::{Read, Write};
use chrono::prelude::*;
use libp2p::{
    core::upgrade,
    futures::StreamExt,
    mplex,
    noise::{Keypair, NoiseConfig, X25519Spec},
    swarm::{Swarm, SwarmBuilder},
    tcp::TokioTcpConfig,
    Transport,
};
use log::{error, info, warn};
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use std::time::Duration;
use tokio::{
    io::{stdin, AsyncBufReadExt, BufReader},
    select, spawn,
    sync::mpsc,
    time::sleep,
};

const DIFFICULTY_PREFIX: &str = "00";

mod blockchain;
pub struct App {
    pub blocks: Vec<Block>,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Block {
    pub id: u64,
    pub hash: String,
    pub previous_hash: String,
    pub timestamp: i64,
    pub data: String,
    pub nonce: u64,
}

Block Creation (new Function)

The new function in the Block impl block creates a new block. It generates the block's hash and nonce by calling the mine_block function. It sets the other properties based on the inputs and the current time.

impl Block {
    pub fn new(id: u64, previous_hash: String, data: String) -> Self {
        let now = Utc::now();
        let (nonce, hash) = mine_block(id, now.timestamp(), &previous_hash, &data);
        Self {
            id,
            hash,
            timestamp: now.timestamp(),
            previous_hash,
            data,
            nonce,
        }
    }
}

Hash Calculation (calculate_hash Function)

calculate_hash generates the hash of a block. It creates a JSON object from the block's properties and uses the SHA-256 algorithm to hash this data. This function is critical for ensuring the integrity of the blockchain.

fn calculate_hash(id: u64, timestamp: i64, previous_hash: &str, data: &str, nonce: u64) -> Vec<u8> {
    let data = serde_json::json!({
        "id": id,
        "previous_hash": previous_hash,
        "data": data,
        "timestamp": timestamp,
        "nonce": nonce
    });
    let mut hasher = Sha256::new();
    hasher.update(data.to_string().as_bytes());
    hasher.finalize().as_slice().to_owned()
}

Mining (mine_block Function)

mine_block is where the Proof of Work algorithm is implemented. It attempts to find a hash that starts with a specific prefix (defined by DIFFICULTY_PREFIX) by iterating over the nonce value. Once a valid hash is found, the function returns the nonce and the hash.

fn mine_block(id: u64, timestamp: i64, previous_hash: &str, data: &str) -> (u64, String) {
    info!("mining block...");
    let mut nonce = 0;

    loop {
        if nonce % 100000 == 0 {
            info!("nonce: {}", nonce);
        }
        let hash = calculate_hash(id, timestamp, previous_hash, data, nonce);
        let binary_hash = hash_to_binary_representation(&hash);
        if binary_hash.starts_with(DIFFICULTY_PREFIX) {
            info!(
                "mined! nonce: {}, hash: {}, binary hash: {}",
                nonce,
                hex::encode(&hash),
                binary_hash
            );
            return (nonce, hex::encode(hash));
        }
        nonce += 1;
    }
}

Binary Hash Representation (hash_to_binary_representation Function)

This function converts a hash into its binary representation, which is used in the mining process to check if the hash meets the difficulty criteria.

fn hash_to_binary_representation(hash: &[u8]) -> String {
    let mut res: String = String::default();
    for c in hash {
        res.push_str(&format!("{:b}", c));
    }
    res
}

Genesis Block (genesis Function)

This function creates the first block in the blockchain, known as the genesis block. It’s manually added to the blockchain.

impl App {
    fn new() -> Self {
        Self { blocks: vec![] }
    }

    fn genesis(&mut self) {
        let genesis_block = Block {
            id: 0,
            timestamp: Utc::now().timestamp(),
            previous_hash: String::from("genesis"),
            data: String::from("genesis!"),
            nonce: 2836,
            hash: "0000f816a87f806bb0073dcf026a64fb40c946b5abee2573702828694d5b4c43".to_string(),
        };
        self.blocks.push(genesis_block);
    }

Adding a Block (try_add_block Function)

try_add_block attempts to add a new block to the blockchain. It first validates the new block against the latest block in the chain. If the new block is valid, it is appended to the chain.

    fn try_add_block(&mut self, block: Block) {
        let latest_block = self.blocks.last().expect("there is at least one block");
        if self.is_block_valid(&block, latest_block) {
            self.blocks.push(block);
        } else {
            error!("could not add block - invalid");
        }
    }

Block Validation (is_block_valid Function)

This function validates a block against a previous block. It checks:

• The previous_hash field matches the hash of the previous block.
• The hash difficulty level is correct.
• The block’s ID follows the previous block’s ID.
• The block’s hash is valid.

    fn is_block_valid(&self, block: &Block, previous_block: &Block) -> bool {
        if block.previous_hash != previous_block.hash {
            warn!("block with id: {} has wrong previous hash", block.id);
            return false;
        } else if !hash_to_binary_representation(
            &hex::decode(&block.hash).expect("can decode from hex"),
        )
            .starts_with(DIFFICULTY_PREFIX)
        {
            warn!("block with id: {} has invalid difficulty", block.id);
            return false;
        } else if block.id != previous_block.id + 1 {
            warn!(
                "block with id: {} is not the next block after the latest: {}",
                block.id, previous_block.id
            );
            return false;
        } else if hex::encode(calculate_hash(
            block.id,
            block.timestamp,
            &block.previous_hash,
            &block.data,
            block.nonce,
        )) != block.hash
        {
            warn!("block with id: {} has invalid hash", block.id);
            return false;
        }
        true
    }

Chain Validation (is_chain_valid Function)

is_chain_valid checks the validity of an entire chain of blocks. It's used to ensure that the blockchain remains in a consistent and valid state.

    fn is_chain_valid(&self, chain: &[Block]) -> bool {
        for i in 0..chain.len() {
            if i == 0 {
                continue;
            }
            let first = chain.get(i - 1).expect("has to exist");
            let second = chain.get(i).expect("has to exist");
            if !self.is_block_valid(second, first) {
                return false;
            }
        }
        true
    }

Chain Selection (choose_chain Function)

This function is used to resolve conflicts when there are multiple versions of the blockchain. It always chooses the longest valid chain.

    // We always choose the longest valid chain
    fn choose_chain(&mut self, local: Vec<Block>, remote: Vec<Block>) -> Vec<Block> {
        let is_local_valid = self.is_chain_valid(&local);
        let is_remote_valid = self.is_chain_valid(&remote);

        if is_local_valid && is_remote_valid {
            if local.len() >= remote.len() {
                local
            } else {
                remote
            }
        } else if is_remote_valid && !is_local_valid {
            remote
        } else if !is_remote_valid && is_local_valid {
            local
        } else {
            panic!("local and remote chains are both invalid");
        }
    }

Main Function and Blockchain Networking

The main function sets up the networking and event-handling part of the blockchain application. It uses the tokio and libp2p libraries to manage peer-to-peer interactions, responding to events like chain requests and handling user inputs.

#[tokio::main]
async fn main() {
    pretty_env_logger::init();

    info!("Peer Id: {}", blockchain::PEER_ID.clone());
    let (response_sender, mut response_rcv) = mpsc::unbounded_channel();
    let (init_sender, mut init_rcv) = mpsc::unbounded_channel();

    let auth_keys = Keypair::<X25519Spec>::new()
        .into_authentic(&blockchain::KEYS)
        .expect("can create auth keys");

    let transp = TokioTcpConfig::new()
        .upgrade(upgrade::Version::V1)
        .authenticate(NoiseConfig::xx(auth_keys).into_authenticated())
        .multiplex(mplex::MplexConfig::new())
        .boxed();

    let behaviour = blockchain::AppBehaviour::new(App::new(), response_sender, init_sender.clone()).await;

    let mut swarm = SwarmBuilder::new(transp, behaviour, *blockchain::PEER_ID)
        .executor(Box::new(|fut| {
            spawn(fut);
        }))
        .build();

    let mut stdin = BufReader::new(stdin()).lines();

    Swarm::listen_on(
        &mut swarm,
        "/ip4/0.0.0.0/tcp/0"
            .parse()
            .expect("can get a local socket"),
    )
        .expect("swarm can be started");

    spawn(async move {
        sleep(Duration::from_secs(1)).await;
        info!("sending init event");
        init_sender.send(true).expect("can send init event");
    });

    loop {
        let evt = {
            select! {
                line = stdin.next_line() => Some(blockchain::EventType::Input(line.expect("can get line").expect("can read line from stdin"))),
                response = response_rcv.recv() => {
                    Some(blockchain::EventType::LocalChainResponse(response.expect("response exists")))
                },
                _init = init_rcv.recv() => {
                    Some(blockchain::EventType::Init)
                }
                event = swarm.select_next_some() => {
                    info!("Unhandled Swarm Event: {:?}", event);
                    None
                },
            }
        };

        if let Some(event) = evt {
            match event {
                blockchain::EventType::Init => {
                    let peers = blockchain::get_list_peers(&swarm);
                    swarm.behaviour_mut().app.genesis();

                    info!("connected nodes: {}", peers.len());
                    if !peers.is_empty() {
                        let req = blockchain::LocalChainRequest {
                            from_peer_id: peers
                                .iter()
                                .last()
                                .expect("at least one peer")
                                .to_string(),
                        };

                        let json = serde_json::to_string(&req).expect("can jsonify request");
                        swarm
                            .behaviour_mut()
                            .floodsub
                            .publish(blockchain::CHAIN_TOPIC.clone(), json.as_bytes());
                    }
                }
                blockchain::EventType::LocalChainResponse(resp) => {
                    let json = serde_json::to_string(&resp).expect("can jsonify response");
                    swarm
                        .behaviour_mut()
                        .floodsub
                        .publish(blockchain::CHAIN_TOPIC.clone(), json.as_bytes());
                }
                blockchain::EventType::Input(line) => match line.as_str() {
                    "ls p" => blockchain::handle_print_peers(&swarm),
                    cmd if cmd.starts_with("ls c") => blockchain::handle_print_chain(&swarm),
                    cmd if cmd.starts_with("create b") => blockchain::handle_create_block(cmd, &mut swarm),
                    _ => error!("unknown command"),
                },
            }
        }
    }
}

Wow, that’s a lot!

You can find the complete implementation in my GitHub repository: https://github.com/luishsr/rustychain.

Testing the Blockchain

To use and test the provided blockchain implementation in Rust, you’ll need to follow a series of steps to set up your environment, launch nodes, and interact with them. Here’s a quick guide to get you started:

Running a Single Node

1. Compile and Run: Navigate to the directory containing your Rust blockchain code and compile the project using cargo build. After successful compilation, run the node using cargo run.
2. Initial Testing: Initially, test with a single node to ensure that it launches correctly and the genesis block is created. Use commands (if implemented) to display the current chain or the status of the node.

Running Multiple Nodes

To simulate a real blockchain network, you’ll want to run several nodes simultaneously.

1. Open Multiple Terminals: Open several terminal windows or tabs. Each will represent a separate node in your network.
2. Run Nodes Independently: In each terminal, navigate to your project directory and run cargo run. Each instance will act as a separate node in the blockchain network.

Interacting with Nodes

To test the functionality of your blockchain, you’ll want to interact with the nodes.

1. Create New Blocks: Use the implemented commands (e.g., create b <data>) to create new blocks. This will simulate transactions or data additions to your blockchain.
2. Broadcasting Blocks: Once a new block is created on one node, it should broadcast this block to other nodes. Verify that other nodes receive and validate this new block, adding it to their version of the blockchain.
3. View Blockchain State: Regularly use the command to print the current state of the blockchain on each node. It should be consistent across nodes, reflecting the latest valid blocks.
4. Test Chain Conflicts: Simulate chain conflicts by creating different blocks on different nodes simultaneously. Observe how your implementation resolves these conflicts (usually by choosing the longest valid chain).