HIP-794: Sunsetting Balance File

Abstract

This HIP proposes sunsetting the account balance file. This stream file is generated every 15 minutes by consensus nodes and contains every active account’s hbar balance as well as its token balances.

Motivation

The ongoing success of the Hedera Network has caused an enormous growth of the data stored in consensus nodes. To ensure the continuous delivery of stable and responsive services, functionality that requires reading large amounts of data has to be disabled on consensus nodes and moved to other systems.

Rationale

Calculating and providing balance files requires iterating over all accounts. This causes many disk reads, which can interfere with normal operations.

User stories

• As a user, I want a stable and responsive system that does not degrade as the usage scales out.

• As a mirror node operator, I want to provide accurate balance information to users.

Specification

Producing and making the balance file available will be stopped completely.

Backwards Compatibility

Systems that depend on balance file availability must adapt and change to generate balance information from record files. It is an open discussion if balance files should be provided in the future using other techniques.

Security Implications

Mirror node operators who don’t update their software to generate balance information from record files may return stale balance information to their users for a period of time. This can cause users to be concerned that their transactions do not appear to have had the expected effect on their account or token.

How to Teach This

Any references to balance files in Hedera documentation should be de-emphasized or removed. Mirror node operators should be aware of how their software uses balance files and taught how to generate similar information from the record files.

Reference Implementation

The Hedera Mirror Node will release version 0.88 with support for generating the balance snapshot from the record file. Mirror node operators using the open source mirror node software only need to upgrade to this version before the balance file sunset date to ensure no impact to users.

For operators not using the open source software and relying upon the balance file for the current or historical balance information, the changes necessary will vary per codebase. Below is pseudo-code for keeping the current balance up to date where the domain model and repositories would be provided by the app. To generate a historical balance snapshot of this information would just require a cron process that runs every 15 minutes that duplicates the balance data from the operator’s account and token relationship tables.

    public void updateBalances(RecordStreamFile recordFile) {
        recordFile.getRecordStreamItemsList().forEach(recordItem -> {
            recordItem.getRecord().getTransferList().getAccountAmountsList().forEach(accountAmount -> {
                var account = accountRepository.find(accountAmount.getAccountID());
                account.setBalance(account.getBalance() + accountAmount.getAmount());
                accountRepository.save(account);
            });

            recordItem.getRecord().getTokenTransferListsList().forEach(t -> {
                t.getTransfersList().forEach(accountAmount -> {
                    var tokenRelationship = tokenRelationshipRepository.find(t.getToken(), accountAmount.getAccountID());
                    tokenRelationship.setBalance(tokenRelationship.getBalance() + accountAmount.getAmount());
                    tokenRelationshipRepository.save(tokenRelationship);
                });
            });
        });
    }

Rejected Ideas

All ideas that produced the balance file on consensus nodes were rejected because they would not solve the original problem of vast numbers of file reads.

Open Issues

It is under discussion if balance files can be provided with a process that does not involve analyzing the data on consensus nodes.

References

• Account Balance File Protobuf
• Hedera Mirror Node

Copyright/license

This document is licensed under the Apache License, Version 2.0 – see LICENSE or (https://www.apache.org/licenses/LICENSE-2.0)

Citation

Please cite this document as: