Record and Event Stream File Formats

Search…
Overview 
The Hedera network produces record stream files and event stream files that captures chronological information about transactions that took place on the network. A record stream file is a file that contains a series of transactions in chronological order that occurred within a two second interval. The record stream file (.rcd) contains a transaction record for each transaction in that file. For each record stream file there is a corresponding signature file (rcd_sig) that includes the signature generated by the node. The event stream file(.evts) contains information about an event. Event information includes event hash data, event unhashed data, and consensus data. Each event stream files are created in 5 second intervals. Each event stream file has a corresponding signature file (.evts_sig) that includes the node signature.  
The record stream file and event stream file formats will both be updated to version 5 in a future mainnet release. 
Version Migration 
File Type 
Current Version 
Next Version 
Record Stream File 
​2 
​5 
Record Stream Signature File 
​No version number ​
​5 ​
Event Stream File 
​3 
​5 
Event Stream Signature File 
​No version number 
​5 
Note: Current record signature files (rcd_sig) and event signature (evts_sig) file formats do not have a version number. The first byte's value is 4, which denotes a marker. For maintaining backwards compatibility, we made the first byte in the new version stream signature file be 5, which denotes the version. Thus, we use version 5 as the next version number. 
Version 2 Record Stream File Format 
Record Stream Files Names 
A record stream file name is made up of a string representation of the Instant of consensus timestamp of the first transaction in the file using ISO-8601 representation, with colons converted to underscores for Windows compatibility. The nano-of-second outputs zero, three, six or nine digits as necessary. 
Examples of a record stream file name with corresponding signature file: 
Example 1: 
Record Stream File: 2020-10-19T21_35_39Z.rcd 
Record Stream Signature File: 2020-10-19T21_35_39Z.rcd_sig 
Example 2: 
Record Stream File: 2020-10-19T21_35_39.454265Z.rcd 
Record Stream Signature File: 2020-10-19T21_35_39.454265Z.rcd_sig 
Record Stream File Format (.rcd) 
The below table describes the content that can be parsed from a record file. 
Name 
Type (Bytes) 
Description 
Record Stream File Format Version 
int (4) 
Record stream file format version,  
Value: 2 
Hapi Version 
int (4) 
HAPI protocol version,  
Value: 3 
Prev File Hash Marker 
byte 
Value: 1 
Prev File Hash 
byte[48] 
SHA384 hash of previous file, if not exist then all zeroes 
Record Marker 
byte 
Value: 2 
Length of Transaction 
int (4) 
Byte size of the following Transaction message 
Transaction 
byte[] 
Serialized Transaction message bytes 
Length of TransactionRecord 
int (4) 
Byte size of the following TransactionRecord message 
TransactionRecord 
byte[] 
Serialized TransactionRecord message bytes 
Record Marker 
byte 
Value: 2 
... 
... 
... 
The signature file *.rcd_sig signs the hash of corresponding *.rcd file; 
 
Record Stream Signature File Format (.rcd_sig)
Note: Event stream signature file format does not have a version number 
The below table describes the content that can be parsed from a record signature file. 
Name 
Type (Bytes) 
Description 
File Hash Marker 
byte 
Value: 4 
File Hash 
byte[48] 
SHA384 hash of corresponding *.rcd file 
Signature Marker 
byte 
Value: 3 
Length of Signature 
int (4) 
Byte size of the following signature bytes 
Signature  
byte[] 
Signature bytes 
File Hash Calculation 
The record file hash is calculated using the following formula. 
h[i] = hash(p[i-1] || h[i-1] || hash(c[i-1])) 
|| denotes concatenation 
h[i] is the hash of the file i 
p[i-1] is the contents in the file before the PREV_FILE_HASH 
h[i-1] is the hash of file i-1, i.e., PREV_FILE_HASH 
c[i-1] is the contents of the file after PREV_FILE_HASH 
Version 5 Record Stream File Format 
Generated using functionality provided by swirlds-common in the swirlds-platform sdk for serializing objects to record files (.rcd), and deserializing objects from record file (.rcd). In the next release, swirlds-common package will contain functionalities for parsing and verifying version 5 record stream files, which can be used by both services and mirror node.  
Record Stream Files Names 
A record file name is created using a string representation of the Instant of consensus timestamp of the first transaction in the file using ISO-8601 representation, with colons converted to underscores for Windows compatibility.  
The nano-of-second always outputs nine digits with padding when necessary, to ensure same length filenames and proper sorting.  
Examples of a record stream file name with corresponding signature file: 
Example 1: 
Record File (.rcd): 2020-10-19T21_35_39.000000000Z.rcd 
Record Signature File(.rcd_sig): 2020-10-19T21_35_39.000000000Z.rcd_sig 
Example 2: 
Record File (.rcd): 2020-10-19T21_35_39.454265000Z.rcd 
Record Signature File(.rcd_sig): 2020-10-19T21_35_39.454265000Z.rcd_sig 
Record Stream File Format (.rcd)
MD – Meta Data 
The below tables describe the content that can be parsed from a record file. 
Name 
Type (Bytes) 
Description 
MD 
Record Stream File Format Version 
int (4) 
Record stream file format version,  
Value: 5 
y 
Hapi Proto Major Version 
int (4) 
Matches NetworkGetVersionInfo.hapiProtoVersion 
Major.Minor.Patch 
Major version: 0;  
Minor version: 9; 
Patch version: 0 
y 
Hapi Proto Minor Version 
int (4) 
​
y
Hapi Proto Patch Version 
int (4) 
​
y
Object stream version 
int (4) 
Value: 1  
This defines the format of the remainder of the file. This version number is used when parsing a stream file with methods defined in swirlds-common package 
y 
Start Object Running Hash 
byte[] 
Running Hash of all RecordStreamObject before writing this file. 
See `Hash object` table below for details  
y 
1st RecordStreamObject 
byte[] 
Serialized RecordStreamObject bytes. 
See ` RecordStreamObject ` table below for details 
n 
2nd RecordStreamObject 
byte[] 
... 
n 
... 
... 
... 
n 
End Object Running Hash 
byte[] 
Running Hash of all RecordStreamObject before closing this file. 
y 
Hash Object 
Name 
Type (Bytes) 
Description 
Class ID 
long (8) 
Value: 0xf422da83a251741e 
Class Version 
int (4) 
Value: 1 
This version number will be updated each time when the serialization format of a Hash object is changed 
Digest Type 
int (4) 
Value: 0x58ff811b  
denotes SHA-384 
Length of Hash 
int (4) 
Value: 48 for SHA-384 
Hash Bytes 
byte[] 
Serialized Hash bytes 
Record Stream Object 
Name 
Type (Bytes) 
Description 
Class ID 
long (8) 
Value: 0xe370929ba5429d8b 
Class Version 
int (4) 
Value: 1 
Length of TransactionRecord 
int (4) 
Byte size of the following TransactionRecord message 
TransactionRecord 
byte[] 
Serialized TransactionRecord message bytes 
Length of Transaction 
int (4) 
Byte size of the following Transaction message 
Transaction 
byte[] 
Serialized Transaction message bytes 
Record Stream Signature File Format (.rcd_sig)
In version 5, the record stream signature file format is the same as event stream signature file format. The below tables describe the content that can be parsed from a record signature file. 
Name 
Type (Bytes) 
Description 
Signature File Format Version 
byte 
Value: 5 
Object Stream Signature Version 
int (4) 
Value: 1   
This defines the format of the remainder of the signature file. This version number is used when parsing a signature file with methods defined in swirlds-common package 
Entire Hash of the corresponding stream file 
byte[] 
Hash of the entire corresponding stream file 
Signature on hash bytes of  Entire Hash 
byte[] 
A signature object generated by signing the hash bytes of Entire Hash. See ` Signature ` table below for details 
Metadata Hash of the corresponding stream file 
byte[] 
Metadata Hash of the corresponding stream file 
Signature on hash bytes of Metadata Hash 
byte[] 
A signature object generated by signing the hash bytes of Metadata Hash 
Signature Object 
Name 
Type (Bytes) 
Description 
Class ID 
long (8) 
Value: 0x13dc4b399b245c69 
Class Version 
int (4) 
Value: 1 
SignatureType 
int (4) 
Value: 1 
Denotes SHA384withRSA 
Length of Signature 
int (4) 
Byte size of the signature bytes 
CheckSum
int (4)
101 - length of signature bytes
Signature bytes 
byte[] 
Serialized Signature bytes 
File Hash Calculation 
In version 5 there are 3 different hashes being calculated: 
Object Running Hash 
Object Running hash is calculated based on hash of each object, which allows us to remove contents of an object from a stream file while maintaining an unbroken chain of hashes.  
The object running hash will be saved in state, so that reconnect nodes will be able to continue generating stream files identical to those generated by other nodes after reconnecting.  
Object running hash is calculated as the following:

hash(Object Running Hash || hash(OBJECT)) 

|| denotes concatenation 
In the event stream file OBJECT will be each event; 
In the record stream file OBJECT will be the Record Stream Object; 
Entire .rcd Hash 
This hash is calculated on all the bytes of a .rcd file. 
With the entire hash, mirror node can download valid .rcd files whose entire hash are agreed on by valid signatures of at least 1/3 of nodes. 
Suppose the content of a .rcd file with version 5’s content is as following:
 
f[i] = head[i] || startHash[i] || contents[i] || endHash[i] 

|| denotes concatenation
f[i] denotes the whole content of a .rcd file;
head[i] denotes the bytes before the start Object Running Hash; 
startHash[i] denotes the Object Running Hash before writing this file;
contents[i] denotes the content between the start Object Running Hash and the end    Object Running Hash;
endHash [i] denotes the Object Running Hash before closing this file;
Entire hash is calculated as following: 
     entireHash[i] = hash(head[i] || startHash[i] || contents[i] || endHash[i]) 
Metadata .rcd Hash 
This hash is calculated on metadata bytes in the file, denoted in the table. 
If some contents of an object were removed from a file, the metadata hash would still be valid. 
Metadata hash is calculated as following: 
   metaHash[i] = hash(head[i] || startHash[i] || endHash[i])   

Migration fo the record stream from v2 to v5
Hash Bytes in Start Object Running Hash in the first v5 .rcd file is the same as File Hash in the last v2 .rcd_sig file, which is also the hash of the last v2 .rcd file.
Version 3 Event Stream File Format 
Event Stream Files Names 
A string representation of the Instant of consensus timestamp of the first transaction in the file using ISO-8601 representation, with colons converted to underscores for Windows compatibility. The nano-of-second outputs zero, three, six or nine digits as necessary. 
Example of an event stream file name with corresponding signature file: 
Event Stream File (.evts): 2020-10-19T21_35_39.454265Z.evts 
Event Stream Signature File (.evts_sig): 2020-10-19T21_35_39.454265Z.evts_sig 
Event Stream File Format (.evts) 
The below table describes the content that can be parsed from an event stream file. 
Name 
Type (Bytes) 
Description 
Event Stream File Format Version 
int (4) 
Value: 3 
Prev File Hash Marker 
byte 
Value: 1 
Prev File Hash 
byte[48] 
SHA384 hash of previous .evts file, if not exist then all zeroes 
Event Marker 
byte 
Value: 90 
Event Format Version 
int (4) 
Value: 3 
Event 
byte[] 
serialized Event bytes 
see ` ConsensusEvent` table for details 
Event's Hash 
byte[] 
This Event’s Hash 
Event Marker 
byte 
Value: 90 
... 
… 
… 
Event Stream Signature File Format (.evts_sig)
Note: Event stream signature file format does not have a version number 
The below table describes the content that can be parsed from an event stream signature file. 
Name 
Type (Bytes) 
Description 
File Hash Marker 
byte 
Value: 4 
File Hash 
byte[48] 
SHA384 hash of corresponding *.evts file 
Signature Marker 
byte 
Value: 3 
Length of Signature 
int (4) 
Byte size of the following signature bytes 
Signature  
byte[] 
Signature bytes 
Version 5 Event Stream File Format 
Generated using functionality provided by swirlds-common for serializing objects to .evts file, and deserializing objects from .evts file. In the next release, swirlds-common will contain functionalities for parsing and verifying version 5 event stream files, which can be used by both services and mirror node. 
Event Stream Files Names 
A string representation of the Instant of consensus timestamp of the first transaction in the file using ISO-8601 representation, with colons converted to underscores for Windows compatibility.  
The nano-of-second always outputs nine digits with padding when necessary, to ensure same length filenames and proper sorting.  
Example of an event stream file name with corresponding signature file. 
Example 1: 
Event Stream File (.evts): 2020-10-19T21_35_39.000000000Z.evts 
Event Stream Signature File (.evts_sig): 2020-10-19T21_35_39.000000000Z.evts_sig 
Example 2: 
Event Stream File (.evts): 2020-10-19T21_35_39.454265000Z.evts 
Event Stream Signature File (.evts_sig): 2020-10-19T21_35_39.454265000Z.evts_sig 
Event Stream File Format (.evts) 
The below tables describe the information that can be parsed from an event stream file. 
MD – Meta Data 
Name 
Type (Bytes) 
Description 
MD 
Event Stream File Format Version 
int (4) 
Event stream file format version,  
Value: 5 
y 
Object Stream Version 
int (4) 
Value: 1  
This defines the format of the remainder of the file. This version number is used when parsing a stream file with methods defined in swirlds-common package 
y 
Start Object Running Hash 
byte[] 
Running Hash of all Events reached consensus before writing this file 
y 
1st ConsensusEvent 
byte[] 
Serialized ConsensusEvent bytes. 
The following ` ConsensusEvent ` table contains detail of a ConsensusEvent 
n 
2nd ConsensusEvent 
byte[] 
... 
n 
... 
... 
... 
n 
End Object Running Hash 
byte[] 
Running Hash of all Events reached consensus before closing this file 
y 
ConsensusEvent 
Name 
Type (Bytes) 
Description 
Class ID 
long (8) 
Value: 0xe250a9fbdcc4b1ba 
Class version 
Int 
Value: 1 
BaseEventHashedData 
byte[] 
Serialized BaseEventHashedData bytes 
BaseEventUnhashedData 
byte[] 
Serialized BaseEventUnhashedData bytes 
ConsensusData 
byte[] 
Serialized ConsensusData bytes 
BaseEventHashedData 
Name 
Type (Bytes) 
Description 
Class Version 
int (4) 
Value: 1 
Creator’s Id 
long (8) 
ID of this event's creator 
SelfParent's Generation 
long (8) 
the generation for the event’s self parent 
OtherParent's Generation 
long (8) 
the generation for the event’s other parent 
SelfParent's Hash  
Class version 
int (4) 
Value: 1 
SelfParent's Hash 
Digest Type 
int (4) 
Value: 0x58ff811b  
denotes SHA-384 
SelfParent's Hash 
length 
int (4) 
Value: 48 for SHA-384 
SelfParent's 
Hash bytes 
byte[] 
SelfParent's Hash bytes 
OtherParent's Hash  
Class version 
int (4) 
Value: 1 
OtherParent's Hash 
Digest Type 
int (4) 
Value: 0x58ff811b  
denotes SHA-384 
OtherParent's Hash 
length 
int (4) 
Value: 48 for SHA-384 
OtherParent's Hash 
bytes 
byte[] 
OtherParent's Hash bytes 
Time Created 
byte[] 
Creation time of this event 
See `Instant` table for details 
Transactions 
byte[] 
A serialized array of Transactions, 
Details is shown in the following table 
Transactions: serialized form of an array of transactions 
​
Name 
Type (Bytes) 
Description 
​
Size of the Transaction Array 
int (4) 
Number of transactions in this array. 
-1 denotes null. 
0 denotes empty array. 
Stop deserialization when size is –1 or 0. 
​
All Same Class 
boolean (1 bit) 
Value: true 
1st  
Transaction 
isNull 
boolean (1 bit) 
Whether this transaction is null 
Transaction Class Version 
int (4) 
Value: 1 
​
Length of Contents 
int (4) 
Length of contents in this transaction 
​
Checksum 
int (4) 
277 – contents.length 
​
System 
boolean (1 bit) 
Whether this transaction was originated by the application or the platform 
​
Contents 
byte[] 
Contents bytes 
​
Size of Signatures List 
int (4) 
Value: 0 
signatures are not serialized, so the size is 0  
​
CheckSum 
int (4) 
Value: 353 
​
2nd  
Transaction 
... 
... 
... 
... 
... 
... 
... 
Instant 
Name 
Type (Bytes) 
Description 
Epoch Second 
long (8) 
If this instant is null, Epoch Second will be Long.MIN_VALUE 
Nanos 
long (8) 
If the instant is null, Nanos will not be written 
BaseEventUnhashedData 
Name 
Type (Bytes) 
Description 
Class Version 
int (4) 
Value: 1 
Creator Sequence 
long (8) 
Sequence number for this by its creator (0 is first) 
OtherParent’s ID 
long (8) 
ID of otherParent 
OtherParent’s Sequence 
long (8) 
Sequence number for otherParent event 
Length of Signature 
int (4) 
Length of signature bytes 
Signature bytes 
byte[] 
Creator's signature bytes for this event 
ConsensusData 
Name 
Type (Bytes) 
Description 
Class Version 
int (4) 
Value: 2 
Generation 
long (8) 
Generation (which is 1 plus max of parents' generations) 
Round Created 
long (8) 
The created round of this event 
isStale 
boolean (1 bit) 
Is there a consensus that this event is stale (no order, transactions ignored) 
Last Event in Round Received 
boolean (1 bit) 
Is this event the last in consensus order of all those with the same received round 
Consensus Timestamp 
byte[] 
The community's consensus timestamp for this event 
See `Instant` table for details 
Round Received 
long (8) 
Round where >=1/2 famous see this event 
Consensus Order 
long (8) 
 The event’s consensus order in history (0 first) 
Event Stream Signature File Format (.evts_sig)
In version 5, the event stream signature file format is the same as record stream signature file format. The below table describes the content that can be parsed from an event stream signature file.  
Name 
Type (Bytes) 
Description 
Signature File Format Version 
byte 
Value: 5 
Object Stream Signature Version 
int (4) 
Value: 1   
This defines the format of the remainder of the signature file. This version number is used when parsing a signature file with methods defined in swirlds-common package 
Entire Hash of the corresponding stream file 
byte[] 
Hash of the entire corresponding stream file 
Signature on hash bytes of Entire Hash 
byte[] 
A signature object generated by signing the hash bytes of Entire Hash. See ` Signature ` table below for details 
Metadata Hash of the corresponding stream file 
byte[] 
Metadata Hash of the corresponding stream file 
Signature on hash bytes of Metadata Hash 
byte[] 
A signature object generated by signing the hash bytes of Metadata Hash 