What You Will Accomplish
- Compute a Merkle root from a batch of off-chain records
- Anchor that Merkle root on HCS using
ConsensusSubmitMessage - Verify the batch (and a single record) using the mirror node
Prerequisites
- Node.js
- A Hedera testnet account (see Hedera Portal)
Table of Contents
- Setup and Installation
- Understand the dataset
- Create a topic for batch anchoring and verification
- Compute the Merkle root
- Anchor the Merkle root on HCS
- Verify the batch via Mirror Node
- Verify a single record (Proof)
- Next steps
1. Setup and Installation
1a. Clone and Install
Clone the repository and install dependencies:1b. Configure Environment
Copy the example environment file:.env and fill in your Testnet credentials:
OPERATOR_ID: Your Account ID (e.g.0.0.12345)OPERATOR_KEY: Your HEX-encoded ECDSA Private Key (e.g.0x8ccf...)HEDERA_NETWORK:testnetMIRROR_NODE_BASE_URL: Leave ashttps://testnet.mirrornode.hedera.com
2. Understand the Dataset You Will Anchor
When you rannpm install in Step 1, a postinstall script automatically executed scripts/generate-data-internal.js. This script populated the data/ directory with the sample datasets (batch-10.json and batch-100.json) and their corresponding Merkle proofs.
Each record in the generated JSON files looks like this:
Canonicalization
To ensure the hash is deterministic (always the same for the same data), we “canonicalize” the record before hashing. This means:- Sorting the object keys alphabetically.
- Removing all whitespace.
- Encoding as UTF-8.
{ "a": 1, "b": 2 } and { "b": 2, "a": 1 } result in the exact same hash.
Canonicalization is the process of converting data into a standard, unique format. It is essential because different representations of the same logical data (like different key orders or whitespace) would produce different hashes, making verification difficult.
Terminology: Batch Hash vs. Merkle Root
- Batch Hash: Usually
hash(record1 + record2 + ...). This approach is simple, but makes it hard to verify any single record. - Merkle Root:
hash(hash(r1) + hash(r2) + ...). This approach allows efficient batch verification and single-record proofs. This example uses Merkle Roots.
3. Create a Topic for Batch Anchoring and Verification
Run the setup script to create a new HCS topic:View the code
View the code
The
01-create-topic.js script initializes the Hedera client and calls the createTopic helper function to mint a new topic ID.TOPIC_ID into your .env file.
4. Compute the Merkle Root (Local)
Before anchoring on-chain, calculate the Merkle root locally for the dataset you want to anchor. This example uses the dataset indata/batch-100.json. Run scripts/02-compute-root.js as shown below:
- Load Dataset: Reads the JSON file from the
data/directory. - Canonicalize: Standardizes each record to ensure a deterministic hash.
- Hash: Computes the SHA-256 hash of each canonicalized record (the leaves of the tree).
- Compute Root: Recursively pairs and hashes leaves using
computeRootuntil a single root hash remains.
View the code
View the code
5. Anchor the Merkle Root on HCS
Now that you have the root hash, proceed to anchor it on Hedera. This step recomputes the root for safety and then submits a message to HCS. While you could manually use the root hash from the previous step, recomputing it immediately before submission is a best practice. This ensures the anchor reflects the current state of your local dataset and serves as a final integrity check before committing the hash to the public ledger.View the code
View the code
6. Verify the Anchored Root Using the Mirror Node
With the Merkle root hash on the public ledger, anyone can verify the batch integrity. Runningscripts/04-verify-batch.js confirms this by completing the following steps:
- Recompute Root: Loads the local dataset and calculates the Merkle root from your local
data/batch-100.jsonexactly as before usingcomputeRoot. - Fetch Message: Queries the Mirror Node REST API for the latest message on the topic using
getLatestTopicMessage. - Compare: Decode the message and verify that the on-chain root matches the locally computed root.
Hedera operates a free/public mirror node for testing and development. Production applications should use commercial-grade mirror node services provided by third-party vendors.
View the code
View the code
7. Verify a Single Record Using a Merkle Proof
A powerful feature of Merkle trees is that they enable proving one item is in the batch without revealing the other items. For simplicity, in this tutorial we use pre-generated proofs indata/proofs-100.json. The script takes the single record’s hash and combines it with “siblings” from the pre-generated proof until it reaches the root. If the calculated root matches the trusted root, the record is proven content.
Running scripts/05-verify-single-record.js demonstrates Merkle proofs with the following steps:
- Load Proof: Reads the pre-generated Merkle proof for the specific record.
- Trusted Root: In a real scenario, this comes from HCS (as in step 6). Here we simulate it with a manifest (
data/manifest.json). - Verify: Use the
verifyProoffunction to hash the record with its sibling hashes up the tree. If the final hash matches the trusted root, the record is proven.
View the code
View the code
Code Check ✅
It’s time to try this example yourself! Get the code from the GitHub repository: tutorial-hcs-batching-hashing-verifying-jsNotes on Message Limits
Message Limits
- HCS Message Size: 1024 bytes (1 KB).
- HCS Transaction Size: 6 KB (includes signatures and keys).
Chunking
If your anchor message exceeds 1 KB (e.g., if you added a lot of metadata), you must use HCS Chunking. The SDK handles this automatically if you configure it:Next steps
- Hedera Developer Playground: Try sending messages and creating topics in the browser.
- GitHub Repo: Explore the full source code and data generation scripts.
- Next Tutorial: Query Messages with Mirror Node - Learn how to filter and retrieve specific messages like an audit log.
Have a question?