GitHub Logo HIP-980: Enhance mirror node entity rest APIs with timestamp query parameter

Author Ivan Kavaldzhiev
Working Group Steven Sheehy, Nana Essilfie-Conduah
Requested By Solo Labs
Discussions-To https://github.com/hashgraph/hedera-improvement-proposal/discussions/981
Status Accepted
Needs Council Approval Yes
Review period ends Tue, 23 Jul 2024 07:00:00 +0000
Type Standards Track
Category Mirror
Created 2024-06-03
Updated 2024-10-09

Abstract

This HIP enhances part of the mirror node REST APIs to support querying by timestamp. This will allow users to query for the historical state of any entity info on the network that currently misses this functionality.

Motivation

To enhance user experience and allow users or developers to query the state of an entity at a specific point in time we will add a timestamp query parameter to all of the entity-based REST APIs that currently don’t have one.

Rationale

Users may want to run historical transactions on a local mirror node instance using public state data. To make this possible, some of the existing REST APIs should be enhanced with a timestamp query parameter, so that historical public data can be fetched and used for the simulations.

User stories

  • As a user of mirror node, I want to be able to query for the historical state of nfts.
  • As a user of mirror node, I want to be able to query for the historical state of token relationships for an account.
  • As a user of mirror node, I want to be able to query for the historical state of allowances for an account.

Specification

The proposed enhancement involves adding a query parameter timestamp to the existing routes:

  • /api/v1/accounts/{idOrAliasOrEvmAddress}/tokens to return correct historical information about an account and their associated fungible tokens for a specific timestamp
  • /api/v1/accounts/{idOrAliasOrEvmAddress}/nfts to return correct historical information about an account and their associated nfts for a specific timestamp
  • /api/v1/accounts/{idOrAliasOrEvmAddress}/allowances/crypto to return correct historical information about an account’s crypto allowances for a specific timestamp
  • /api/v1/accounts/{idOrAliasOrEvmAddress}/allowances/nfts to return correct historical information about an account’s NFT allowances for a specific timestamp
  • /api/v1/accounts/{idOrAliasOrEvmAddress}/allowances/tokens to return correct historical information about an account’s token allowances for a specific timestamp
  • /api/v1/tokens/{tokenId}/nfts/{serialNumber} to return correct historical information about an NFT for a specific timestamp

Timestamp Query Parameter

A new timestamp query parameter will be added to search for all entities and entity related information that are missing its support. It will be with the format of array[string]. This query parameter will represent a single or multiple consensus timestamps in a Unix format in seconds.nanoseconds format and will have an optional comparison operator. That is, it will search for results that are equal, greater or less than the timestamp itself or its seconds component.

The exact names of the operators will be eq, gt, gte, lt and lte. The user doesn’t need to know the exact timestamp at which an entity was created or updated for it to function. The API will return the results that match the timestamp or the closest one to it. For example, if we have an asc ordering of the results, then ?timestamp=X, timestamp=eq:X, or timestamp=lte:X will have the same behaviour and vice versa, where X is the desired timestamp.

The timestamp query parameter will support having a range of timestamps. For example, timestamp=gt:X&timestamp=lt:Y will return a single result with the entity or entity related info in the most recent state within the range that is greater than X and less than Y, where X and Y are different timestamps.

Here are the affected APIs with the query parameter added:

/api/v1/accounts/{idOrAliasOrEvmAddress}/tokens?timestamp={value}
/api/v1/accounts/{idOrAliasOrEvmAddress}/nfts?timestamp={value}
/api/v1/accounts/{idOrAliasOrEvmAddress}/allowances/crypto?timestamp={value}
/api/v1/accounts/{idOrAliasOrEvmAddress}/allowances/nfts?timestamp={value}
/api/v1/accounts/{idOrAliasOrEvmAddress}/allowances/tokens?timestamp={value}
/api/v1/tokens/{tokenId}/nfts/{serialNumber}?timestamp={value}

Note: There is an existing API with a timestamp query parameter, which is /api/v1/accounts/{idOrAliasOrEvmAddress}?timestamp={value}. While parameter changes won’t be applied for it, it’s behaviour will reflect to actually return modification changes of all account fields. Currently, the timestamp filter is applied only to the transaction list. The balance info section will continue to have granularity of 15 minutes.

Request

/api/v1/accounts/{idOrAliasOrEvmAddress}/tokens?timestamp={value}

Example Request

GET api/v1/accounts/4408244/tokens?order=asc&timestamp=lte:1725334849.603493001

Note that the following two requests will return the same response as the query above:

GET api/v1/accounts/4408244/tokens?order=asc&timestamp=1705334449.803393003
GET api/v1/accounts/4408244/tokens?order=asc&timestamp=eq:1705334449.803393003

Response

The response format of all the mentioned APIs will be enhanced with additional timestamp range with from and to fields, reflecting the period in which the returned data state was valid. The to field will be null if the result represent the latest updated state.

{
  "tokens": [
    {
      "automatic_association": true,
      "balance": 1,
      "created_timestamp": "1705334449.803393003",
      "decimals": 0,
      "token_id": "0.0.4328145",
      "freeze_status": "NOT_APPLICABLE",
      "kyc_status": "NOT_APPLICABLE",
      "timestamp": {
        "from": "1705334449.803393003",
        "to": null
      }
    }
  ],
  "links": {
    "next": null
  }
}

Backwards Compatibility

The change is backwards compatible as it is simply adding a query parameter to an existing route. The data required to filter by timestamp is already stored in the mirror node.

If a timestamp query parameter is not provided, the API will use the latest state of the entity as it does currently.

Security Implications

Historical timestamp filtering can slow down some of the queries, especially if the timestamp is not indexed or the data set is very large.

Rejected Ideas

Initially, the HIP aimed to allow the grounding steps for supporting shadow fork on a local mirror node. This means users could run a local instance of the mirror node and use a public state of the network, including historical one. However, this idea was rejected, since the implementation of this feature would have meant having a lot of REST API requests against the public mirror nodes, which would have generated a heavy load on the network. Additionally, it would have been impractical for many users accustomed to rely on a public service, having this functionality (e.g. call eth_call or eth_debugTraceTransaction with a specific block number against a public service, which may be paid), to spend additional efforts of spinning their own local mirror node and sending their call against it.

As a summary, this idea was rejected in favour of a public mirror node provider to allow this functionality as a service.

References

https://mainnet-public.mirrornode.hedera.com/api/v1/docs/

Copyright/license

This document is licensed under the Apache License, Version 2.0 – see LICENSE or (https://www.apache.org/licenses/LICENSE-2.0)

Citation

Please cite this document as: