Block types
These data structures are constructed of mostly User Elements defined by the Federated servers.
Directory Block
A Directory Block consists of a header and a body. The body is a series of pairs of ChainIDs and Entry Block Merkle Roots. The Body and BodyMR are derived from iterating the process lists generated by the Federated servers over the last 10 minutes.
data | Field Name | Description |
Header | ||
varInt_F | Version | Describes the protocol version that this block is made under. Only valid value is 0. Can safely be coded using 1 byte for the first 127 versions. |
4 bytes | NetworkID | This is a magic number identifying the main Factom network. The value for MainNet Directory Blocks is 0xFA92E5A2. TestNet is 0xFA92E5A3. |
32 bytes | BodyMR | This is the Merkle root of the body data which accompanies this block. It is calculated with SHA256. |
32 bytes | PrevKeyMR | Key Merkle root of previous block. It is the value which is used as a key into databases holding the Directory Block. It is calculated with SHA256. |
32 bytes | PrevFullHash | This is a SHA256 checksum of the previous Directory Block. It is calculated by hashing the serialized block from the beginning of the header through the end of the body. It is included to allow simplified client verification without building a Merkle tree and to doublecheck the previous block if SHA2 is weakened in the future. |
4 bytes | Timestamp | This the time when the block is opened. Blocks start on 10 minute marks based on UTC (ie 12:00, 12:10, 12:20). The data in this field is POSIX time, counting the number of minutes since epoch in 1970. |
4 bytes | DB Height | The Directory Block height is the sequence it appears in the blockchain. Starts at zero. |
4 bytes | Block Count | This is the number of Entry Blocks that were updated in this block. It is a count of the ChainID:Key pairs. Inclusive of the special blocks. Big endian. |
Body | ||
32 bytes | Admin Block ChainID | Indication the next item is the serial hash of the Admin Block. |
32 bytes | Admin Block LookupHash | This is the LookupHash of the Admin Block generated during this time period. |
32 bytes | Entry Credit Block ChainID | Indication the next item belongs to the Entry Credit Block. |
32 bytes | Entry Credit Block HeaderHash | This is the serial hash of the Entry Credit Block Header generated during this time period. |
32 bytes | Factoid Block ChainID | Indication the next item belongs to the Factoid Block. |
32 bytes | Factoid Block KeyMR | This is the KeyMR of the Factoid Block generated during this time period. |
32 bytes | ChainID 0 | This is the ChainID of one Entry Block which was updated during this block time. These ChainID:KeyMR pairs are sorted numerically based on the ChainID. |
32 bytes | KeyMR 0 | This is the Key Merkle Root of the Entry Block with ChainID 0 which was created during this Directory Block. |
32 bytes | ChainID N | Nth Entry Block ChainID. |
32 bytes | KeyMR N | Nth Entry Block KeyMR. |
Note about the Timestamp: This timestamp differs from Bitcoin, as it signifies the opening rather than the closing of a block. All the lower level blocks have minute markers in them, signifying how many full minutes after opening that an Entry was seen. If block creation starts in the middle of a 10 minute window, it will still show being opened a the beginning of the window, but all the Entries will have minute markers indicating they were seen at the correct time.
Administrative Block
This is a special block which accompanies this Directory Block. It contains the signatures and organizational data needed to validate previous and future Directory Blocks. It has a LookupHash, which is a SHA256 of the entire block. The LookupHash is included in the directory block body paired with the Admin ChainID.
data | Field Name | Description |
Header | ||
32 bytes | Admin ChainID | The Admin ChainID is predefined as 0x000000000000000000000000000000000000000000000000000000000000000a. |
32 bytes | BackReferenceHash | This is the top 256 bits of a SHA512 checksum (SHA512[:256]) of the previous Admin Block. It is calculated by hashing the previous serialized Admin block. It is included to doublecheck the previous block if SHA2 is weakened in the future. First block has a BackReferenceHash of 0. |
4 bytes | DB Height | This is the Directory Block height which this Admin Block is located in. Big endian. |
varInt_F | Header Expansion Size | This is the number bytes taken up by the Header Expansion area. Set at zero for now. |
Variable | Header Expansion Area | This is a field which can be defined and expanded in the future. It is good for things that can be derived deterministically by all the Federated servers when iterating the process lists. One planned feature to go in this field is a Chain Head Commitment. This would be a Merkle root of ChainIDs with their current heads. This would allow a peer to demonstrate to a light client that the Chain head being offered is the current chain head as defined by the Federated servers. |
4 bytes | Message Count | This is the number of Admin Messages that the body of this block contains. Big endian. |
4 bytes | Body Size | This is the number of bytes the body of this block contains. Big endian. |
Body | ||
variable | All objects | A series of variable sized objects. Each object is prepended with an AdminID byte. |
Header Expansion Area
There can be several distinct items in the expansion area. They are identified by a type byte. The length of the serialized data is also specified. This allows older versions of software to skip data types they don't understand. The length descriptor is a varInt_F.
Type Byte | Name | Data Bytes | Description |
0x01 | Balances Hash | 32 bytes | The Balances Hash (BH) is a 32 byte hash of the serialized sorted FCT and EC balances in the previous block. |
Here is a serialization example:
... | Header Expansion Size varInt_F | BH type byte | Length of BH | 32 bytes of BH | Future type byte | length of Future object | Future object (8 bytes for example) | ...
... 2C 01 20 c76c73de94826eefa4e485469922ee3d98f8f3aec8ddb45b101a2012772a9cad 02 08 0123456789ABCDEF ...
AdminID Bytes
Administrative Identifier (AdminID) bytes are single bytes which specify how to interpret the following data. It specifies the type, and the type determines how to interpret subsequent bytes.
Binary | Name | Data Bytes | Description |
0x00 | Minute Number | 1 byte | (Depricated in Milestone2) The preceding data was acknowledged before the minute specified. 1 byte follows the Minute Number. |
0x01 | DB Signature | 128 bytes | The following data is a signature of the preceding Directory Block header. The signature consists of the servers 32 byte identity ChainID, a 32 byte Ed25519 public key in that identity and a 64 byte signature of the previous Directory Block's header. |
0x02 | Reveal Matryoshka Hash | 64 bytes | This is the latest M-hash reveal to be considered for determining server priority in subsequent blocks. Following this byte are 32 bytes specifying the identity ChainID and 32 bytes for the M-hash reveal itself. |
0x03 | Add/Replace Matryoshka Hash | 64 bytes | This is a command which adds or replaces the current M-hash for the specified identity with this new M-hash. Following this byte are 32 bytes specifying the identity ChainID and 32 bytes for the new M-hash itself. This data is replicated from the server's identity chain. |
0x04 | Increase Server Count | 1 byte | The server count is incremented by the amount encoded in a following single byte. |
0x05 | Add Federated Server | 36 bytes | The first 32 bytes are the ChainID of the Federated server which is added to the pool. The next 4 are the Directory Block height that it takes effect. |
0x06 | Add Audit Server | 36 bytes | The first 32 bytes are the ChainID of the Audit server which is added to the pool. The next 4 are the Directory Block height that it takes effect. |
0x07 | Remove Federated Server | 36 bytes | The following 32 bytes are the ChainID of the Federated server which is removed from the pool. The next 4 are the Directory Block height that it takes effect. All public keys associated with it are removed as well. |
0x08 | Add Federated Server Signing Key | 64 bytes | This adds an Ed25519 public key to the authority set. First 32 bytes are the server's identity ChainID. Next 32 bytes are the public key itself. If the specified key for this server already exists, this replaces the old one. |
0x09 | Add Federated Server Bitcoin Anchor Key | 66 bytes | This adds a Bitcoin public key hash to the authority set. First 32 bytes are the server's identity ChainID. Next byte is the key priority. Next byte is 0=P2PKH 1=P2SH. Next 20 bytes are the HASH160 of ECDSA public key. If the specified priority for the server already exists, this replaces the old one. |
0x0A | Server Fault Handoff | This holds a rollup of all the messages which were sent out by the federated servers which authorize the removal of one server and the promotion of another server. This is not currently serialized into the blockchain. | |
0x0B | Coinbase Descriptor | 10232 bytes max | This is a field which is used to specify a future genesis transaction. This field is only present every 25 blocks, on blocks divisible by 25. The coinbase transation which is created by the info in this transaction included 1000 blocks after this Admin block. This delay is to allow 7 days to respond to software bugs, etc. See the Coinbase Descriptor section for more details. |
0x0C | Coinbase Descriptor Cancel | 8 bytes max | This cancels a specific output index in an earlier Coinbase Descriptor. It is only effective in a block between when the Coinbase Descriptor is created and when the coinbase is included in the Factoid block (non-inclusive). See the Coinbase Descriptor Cancel section for more details. |
0x0D | Add Authority Factoid Address | 65 bytes | This sets a Factoid address to be used in the Coinbase Descriptor. The first byte is a length descriptor varint (always 64). The next 32 bytes are the server's identity ChainID. Next 32 bytes are the Factoid address (RCD Hash). If the specified address for this identity already exists, this replaces the old one. |
0x0E | Add Authority Efficiency | 35 bytes | This sets what percentage of the Factoid rewards for the specified server are yeilded to the Grant Pool. The first byte is a length descriptor varint (always 34). next 32 bytes are the server's identity ChainID. Next 2 bytes a big endian representation of the percentage with 2 fixed decimals. This overrides the previous efficiency settings for this identity. |
> 0x0E | Forward Compatible Type | unspecified | All types above 0x09 are prefixed with a Varint_F specifying how many of the following bytes are part of this message. |
Coinbase Descriptor
The coinbase descriptor is an entry in the Admin block which specifies a number of factoid output addresses and amounts. These outputs are used to deterministically generate a coinbase transaction 1000 blocks (about 7 days) later. It is included in each block height that is divisible by 25. There can only be one per Admin block.
The Coinbase Descriptor cannot be larger than 10233 bytes (1 AdminID byte + 2 varint length bytes + (10 KiB max FCT tx - 10 header bytes of coinbase))
data | Field Name | Description |
1 byte | AdminID byte | is 0x0B for this type |
varInt_F | AdminID Message Size | This value specifies how many of the following bytes relate to this Admin message. |
varInt_F | Value | (Output 0) This is how much the Output 0 Factoshi balance will be increased by. |
32 bytes | Factoid Address | (Output 0) This is an RCD hash which will have its balance increased. |
varInt_F | Value | (Output X) This is how much the Output X Factoshi balance will be increased by. |
32 bytes | Factoid Address | (Output X) This is an RCD hash which will have its balance increased. |
Coinbase Descriptor Cancel
The Coinbase Descriptor Cancel (CDC) is an entry in the Admin block which reverses an output in a Coinbase Descriptor. When a coinbase is made in the future, any outputs that have a CDC pointed at them will be missing. The output will not appear in the coinbase, and there will be fewer outputs in the coinbase than in the Coinbase Descriptor. The Factoids which aren't created will be retained as potential Factoids in the Grant Pool. This allows the grant process to correct an accident that was prevented by the CDC.
The Coinbase Descriptor cannot be larger than 8 bytes (on a 100 year timeline) (1 AdminID byte + 1 varint length byte + 4 bytes for height + 2 bytes for max index)
data | Field Name | Description |
1 byte | AdminID byte | is 0x0C for this type |
varInt_F | AdminID Message Size | This value specifies how many of the following bytes relate to this Admin message. |
varInt_F | Descriptor Height | An output the Descriptor at this height will not be created. |
varInt_F | Descriptor Index | This index into the specified descriptor will not be created. |
Entry Credit Block
An Entry Credit (EC) Block is a datastructure which packages Chain Commits, Entry Commits, and EC balance increases over a 10 minute period. The Entries are ordered in the Entry Block in the order that they were received by each Federated server. All the Federated servers contribute to the building of the EC Block.
The Entry Credit Block consists of a header and a body. The body is composed of primarily Commits and balance increases with minute markers and server markers distributed throughout the body.
data | Field Name | Description |
Header | ||
32 bytes | EC ChainID | The EC ChainID is predefined as 0x000000000000000000000000000000000000000000000000000000000000000c. |
32 bytes | BodyHash | This is the SHA256 hash of the serialized body data which accompanies this block. |
32 bytes | PrevHeaderHash | This is the key to the previous block. It is a SHA256 hash of the serialized header of the previous block. This is the value of the previous EC Block's key which was placed in the previous Directory Block. It is the value which is used as a key into databases holding the EC Block. |
32 bytes | PrevFullHash | This is a SHA256 checksum of the entire previous Entry Block. It is calculated by hashing the serialized block from the beginning of the header through the end of the body. It is included to doublecheck the previous block if SHA2 is weakened in the future. Genesis block has a PrevFullHash of 0. |
4 bytes | DB Height | This the Directory Block height which this block is located in. Big endian. |
varInt_F | Header Expansion Size | This is the number bytes taken up by the Header Expansion area. Set at zero for now. |
Variable | Header Expansion Area | This is a field which can be defined and expanded in the future. It is good for things that can be derived deterministically by all the Federated servers when iterating the process lists. Two features are planned. One is SegmentsMR, which will allow the Body to be chopped into small pieces for parallel download and validation. Another is Balance Commitment, where there is a Merkle root of pairing each public key to a balance. |
8 bytes | Object Count | This is the number of objects this block contains. Big endian. |
8 bytes | Body Size | This is the number of bytes the body of this block contains. Big endian. |
Body | ||
variable | All objects | A series of variable sized objects arranged in chronological order. Each object is prepended with an ECID byte. |
ECID Bytes
Entry Credit Identifier (ECID) bytes are single bytes which specify how to interpret the following data. It specifies the type, and the type determines how to interpret subsequent bytes.
Binary | Name | Description |
0x00 | Server Index Number | The following data was acknowledged by the server with the specified Index. This byte is followed by another byte which signifies the server's order. |
0x01 | Minute Number | The preceding data was acknowledged before the minute specified. 1 byte follows the Minute Number. |
0x02 | Chain Commit | The following data is a Chain Commit. The following 200 bytes are a Chain Commit. |
0x03 | Entry Commit | The following data is an Entry Commit. The following 136 bytes are an Entry Commit. |
0x04 | Balance Increase | The following data is a balance increase. The following 66 - 82 bytes are a Balance Increase. |
Factoid Block
Factoid Block is a datastructure which packages Factoid transactions over a 10 minute period. The Factoid transactions are ordered in the Block in the order that they were received by the Federated server.
The Factoid Block consists of a header and a body. The body is composed of serialized Factoid transactions with minute markers distributed throughout the body. The minute markers consist of a single byte 0x00. There are 10 of them, each being placed at a minute boundary, and the 10th marker being the last item in the block. Factoid transactions begin with a version number above zero, which is how they can be differentiated from transactions. The minute markers are included in both the body and ledger Merkle roots.
data | Field Name | Description |
Header | ||
32 bytes | Factoid ChainID | The Factoid ChainID is predefined as 0x000000000000000000000000000000000000000000000000000000000000000f. |
32 bytes | BodyMR | This is the Merkle root of the Factoid transactions which accompany this block. The leaves of the Merkle tree are the full Factoid transacion, from the version through the last signature, inclusive. Minute markers are also leaves of this tree. It is calculated with SHA256. |
32 bytes | PrevKeyMR | Key Merkle root of previous block. This is the value of the Factoid Block's previous Key Merkle root which was placed in the Directory Block. It is the value which is used as a key into the Directory Block. It is calculated with SHA256. |
32 bytes | PrevLedgerKeyMR | This is a data structure which allows proofs of only the value transfers. This approach allows the confirmation and tracking of a transaction while ignoring signatures. A Merkle tree is constructed with the leaves being individual Factoid transactions hashed from the Version through the Entry Credit Purchase section. Neither the RCD reveals nor the signatures are included in this Merkle root. This allows a future client to prove they recieved all the data documenting value movement, without needing to download signatures or all the public keys needed. The Merkle root of last block's value transfers is hashed with the last block's header hash to make a KeyMR, called PrevLedgerKeyMR. This differs from regular KeyMRs, in that the LedgerMR comes before the previous header hash. First block has a PrevLedgerKeyMR of 0. The minute markers are included in the tree. |
8 bytes | EC Exchange Rate | This the number of Factoshis required to purchase 1 Entry Credit, and set the minimum fees. This is the exchange rate currently in force for this block. The initial value will be about 700000, but will be re-targeted based on the factoid/$ exchange rate. It is an integer, because it is always expected that ECs will cost more than a single Factoshi. Big endian. |
4 bytes | DB Height | This the Directory Block height which this Factoid Block is located in. Big endian. |
varInt_F | Header Expansion Size | This is the number bytes taken up by the Header Expansion area. Set at zero for now. |
Variable | Header Expansion Area | This is a field which can be defined and expanded in the future. It is good for things that can be derived deterministically by all the Federated servers when iterating the process lists. One planned feature is Balance Commitments, with a Merkle root of public keys paired with balances. |
4 bytes | Transaction Count | This is the number of Factoid transaction included in this block. Big endian. |
4 bytes | Body Size | This is the number of bytes the body of this block contains. Big endian. |
Body | ||
variable | All objects | A series of variable sized objects arranged in chronological order. |
Entry Block
An Entry Block is a datastructure which packages references to Entries all sharing a ChainID over a 10 minute period. The Entries are ordered in the Entry Block in the order that they were received by the Federated server. The Entry Blocks form a blockchain for a specific ChainID.
The Entry Block consists of a header and a body. The body is composed of primarily Entry Hashes with 10 one minute markers distributed throughout the body.
data | Field Name | Description |
Header | ||
32 bytes | ChainID | All the Entries in this Entry Block have this ChainID |
32 bytes | BodyMR | This is the Merkle root of the body data which accompanies this block. It is calculated with SHA256. |
32 bytes | PrevKeyMR | Key Merkle root of previous block. This is the value of this ChainID's previous Entry Block Merkle root which was placed in the Directory Block. It is the value which is used as a key into databases holding the Entry Block. It is calculated with SHA256. |
32 bytes | PrevFullHash | This is a SHA256 checksum of the previous Entry Block of this ChainID. It is calculated by hashing the serialized block from the beginning of the header through the end of the body. It is included to doublecheck the previous block if SHA2 is weakened in the future. First block has a PrevFullHash of 0. |
4 bytes | EB Sequence | This is the sequence which this block is in for this ChainID. This number increments by 1 for every new EB with this chain ID. First block is height 0. Big endian. |
4 bytes | DB Height | This the Directory Block height which this Entry Block is located in. Big endian. |
4 bytes | Entry Count | This is the number of Entry Hashes and time delimiters that the body of this block contains. Big endian. |
Body | ||
32 bytes | All objects | A series of 32 byte sized objects arranged in chronological order as received by the Federated server for that minute. |
Time delimiters are 32 byte big endian objects between 1 and 10 (inclusive). They are inserted in into the Entry Block when a new Federated server yields control of the Chain and an Entry has been acknowledged prior to the handoff. They are not needed if there is not an Entry to include that minute. Note, there can be duplicate Entries included in an Entry Block. If an Entry is paid for twice, it is included twice. The times are organized when the Federated server saw and acknowledged the Entry.
Body With 1 Entry at 0:10 into block
Body With 1 Entry at 1:10 into block
Body With 1 Entry at 5:50 into block
Body With 1 Entry at 9:50 into block
Body With 2 Entries: 2:20 and 5:30 into block
Body With 3 Entries: 2:20, 2:40 and 5:30 into block
Body With 8 Entries: 0:05, 3:40, 3:59, 4:00, 4:01, 4:30, 5:30, 9:59 into block with a duplicate
Components
These are some custom datastructures for Factom
KeyMR
A Key Merkle Root is a datastructure which allows fast validation of a header and also allows Merkle proofs to be built to the body data elements.
First a Merkle tree is constructed of all the body elements. It is called the BodyMR. This is very similar to how all Bitcoin transactions can be proven with a Merkle root in the header.
The BodyMR is included in the header, among other things. The serialized header is then hashed. The hashed header is combined with the BodyMR and hashed. This creates the KeyMR. With only the KeyMR, when a header is produced by a peer, the header can be validated with 2 hashes.
Note: the Factoid's LedgerKeyMR is constructed differently than the other KeyMRs. It concatenates the LedgerMR first, then the header hash. This is the opposite of other KeyMRs, which put the header hash first. The rationale for this is to have different hashes for the Factoid's PrevKeyMR and PrevLedgerKeyMR when there is only a coinbase, which increases the security level to that of other objects in this case.
Variable Integers (varInt_F)
Integers can be serialized in blocks in a compact form to save space when stored indefinitely, like in a blockchain. Factom's varInt_F is modeled after the Protobuff's variable length integer, but is big-endian compared to Protobuff's little endian, and operates on the byte level rather then 4 bytes.
Values 127 and below can be represented in a single byte. Only positive integers are supported. The highest number is 2^64, which takes 10 bytes.
Larger numbers are represented by a sequence of bytes. The sequence length is indicated by the Most Significant Bit of each byte. If the MSB is set to one, then the next byte is considered part of the number. The number itself is held in the lower 7 bits. The LSB of the number is held in the LSB of the last byte. Bits higher than the 7th are held in earlier bytes.
This is the algorithm to create the stream: Convert the value to big endian. Count the number of bits between the LSB and the most significant 1 bit, inclusive. Divide this number by 7 and take the cieling of the remainder. This is the byte count M. Create a byte sequence with M bytes. Take the least significant 7 bits of the number and place them in the Mth byte. Set the highest bit of the Mth byte to zero. Take the bits 13 through 7 and add them to the byte M-1. Set the highest bit of byte M-1 to one. Continue until all the M bytes have been filled with with data.
Here are some examples:
Base 10 value | Binary Value | varInt_F Serialization |
0 | 00000000 | 00000000 |
3 | 00000011 | 00000011 |
127 | 01111111 | 01111111 |
128 | 10000000 | 10000001 00000000 |
130 | 10000010 | 10000001 00000010 |
2^16 - 1 | 11111111 11111111 | 10000011 11111111 01111111 |
2^16 | 00000001 00000000 00000000 | 10000100 10000000 00000000 |
2^32 - 1 | 11111111 11111111 11111111 11111111 | 10001111 11111111 11111111 11111111 01111111 |
2^32 | 00000001 00000000 00000000 00000000 00000000 | 10010000 10000000 10000000 10000000 00000000 |
2^63 - 1 | 01111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111 | 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111 01111111 |
2^64 - 1 | 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111 | 10000001 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111 01111111 |
Last updated