HIP-694: Real-time events in JSON-RPC Relay

Author	Ivo Yankov
Working Group	Nana Essilfie-Conduah, Georgi Lazarov, Alfredo Gutierrez, Eric Badiere
Discussions-To	https://github.com/hashgraph/hedera-improvement-proposal/discussions/695
Status	Accepted ⓘ
Needs Council Approval	No ⓘ
Review period ends ⓘ	Mon, 10 Apr 2023 07:00:00 +0000
Type	Standards Track ⓘ
Category	Application ⓘ
Created	2023-03-13
Updated	2023-04-11

Abstract

Provide a way for users to receive information for specific events as soon as it is available.

Motivation

The JSON-RPC Relay enables clients to query for particular events, but there is no way to receive real-time notifications whenever an event of interest occurs.

Currently, in order to retrieve newly created contract logs developers need to constantly query the eth_getLogs method. Every developer that requires data in real-time has to create custom logic on their side, which results in poor user experience.

By keeping an open a WebSocket connection it would be possible for the Relay to send data to the clients as soon as it is available, without them having to constantly query the Relay. Users would also be able to use existing tools created for this exact purpose, like WebSocketProvider from ethers.js.

Rationale

This HIP proposes the implementation of an additional method in the Relay: eth_subscribe, which allows users to subscribe for particular events and receive the data in near real-time. Additionally, an eth_unsubscribe method will be implemented to cancel an existing subscription. These methods are part of the industry standard and are present in other JSON-RPC Relays.

An additional WebSocket API will be developed for the Relay. It will handle connections on a separate port. This API will be handled by a different NodeJS process and can be controlled by an environment variable. This would provide flexibility when running several instances of the Relay - if required it can be configured that only a subset of the several Relay instances will start the WebSocket Server process, and thus optimize the resources used.

The Relay will implement a class that queries the Mirror node for every unique filter combination at equal time intervals. The exact interval will be configurable, but it should be at the most once every 2 seconds. A smaller interval would help with redundancy - in the case of a network error this would make sure that no data is skipped.

Another option is for the Relay to query the Mirror node for all new contract logs at every new block and filtered on an application level in the Relay. This idea was rejected because it would result in more data being transferred over the network.

If at some point the Mirror Node provides a WebSocket API the Relay can directly connect to that, without the need to constantly pinging the REST API.

Every live WebSocket connection will use up resources, so a limiting mechanism will need to be implemented.

User stories

As a developer building applications on top of the Hedera Network, I want to be able to listen for certain contract events and be notified of their occurrence as soon as possible.
As a developer I want to be able to use existing tools to listen for contract events so that I don’t have to write code that constantly sends requests to the Relay.

Specification

Subscribing to specific contract events is done by connecting to the Relay via WebSocket and creating a subscription by calling the eth_subscribe method. When a subscription is created a unique subscriptionId is generated and returned to the client. The subscriptionId is then used to distinguish data from different subscriptions.

At that point the Relay will start polling the Mirror node for Contract Logs that match the filters specified by the clients. At any time when new data is received from the Mirror node it will be sent back to the client. Subscriptions are closed by calling the eth_unsubscribe method with a subscriptionId.

Users will be able to subscribe for new contract logs filtered by a single contract address, multiple addresses and/or topics.

Users will be able to subscribe to events by sending a JSON-RPC formatted message via WebSocket to the eth_subscribe method and with an array of params. The first element from the params specifies the event name, possible values are:

newHaeds - Returns data when a new block is created.
logs - Returns data when a new contract log is created.

Initially the Relay needs to support only the logs event.

The second element of the params array are the additional filters. The logs event accepts address and topics filters.

eth_subscribe request:

{
  "jsonrpc":"2.0",
  "id": 1,
  "method": "eth_subscribe",
  "params": [
    "logs",
    {"address":"0x8320fe7702b96808f7bbc0d4a888ed1468216cfd", "topics": ["0xd78a0cb8bb633d06981248b816e7bd33c2a35a6089241d099fa519e361cab902"]}
  ]
}

eth_subscribe success response:

{
    "id": 1,
    "jsonrpc": "2.0",
    "result": "0x9cef478923ff08bf67fde6c64013158d"
}

eth_unsubscribe request:

{"jsonrpc":"2.0", "id": 1, "method": "eth_subscribe", "params": ["0x9cef478923ff08bf67fde6c64013158d"]}

eth_unsubscribe success response:

{
"id": 1,
"jsonrpc": "2.0",
"result": true
}

eth_subscribe and eth_unsubscribe error response:

{
  "id": 1,
  "jsonrpc": "2.0",
  "error": {
    "code": -32602,
    "message": "Invalid params"
  }
}

Data sent to client when a new Contract Log is retrieved from the Mirror node

{
    "method":"eth_subscription",
    "params": {
        "subscription":"0x4a8a4c0517381924f9838102c5a4dcb7",
        "result": {
            "address":"0x8320fe7702b96808f7bbc0d4a888ed1468216cfd","blockHash":"0x61cdb2a09ab99abf791d474f20c2ea89bf8de2923a2d42bb49944c8c993cbf04",
            "blockNumber":"0x29e87","data":"0x00000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000003",
            "logIndex":"0x0",
            "topics":["0xd78a0cb8bb633d06981248b816e7bd33c2a35a6089241d099fa519e361cab902"],"transactionHash":"0xe044554a0a55067caafd07f8020ab9f2af60bdfe337e395ecd84b4877a3d1ab4",
            "transactionIndex":"0x0"
        }
    }
}

Connection Limits

There will be several types of limits. All values will be configurable via environment variables:

global - Limits the total connections that the Relay will accept. Any further connections will be rejected and an error will be thrown.
IP based - There will be a limit on the maximum subscriptions from a single IP address. Any further subscriptions will be rejected and an error will be thrown.
A time-to-live mechanism will be implemented on every subscription. After the TTL runs out the subscription will be automatically closed.

Backwards Compatibility

There are no sources of backward incompatibility as this HIP simply introduces new functionality to the JSON-RPC Relay.

Security Implications

No security implications noted.

How to Teach This

Describe the new methods in the JSON-RPC Relay documentation.

Reference Implementation

Rejected Ideas

Open Issues

None.

References

Copyright/license

This document is licensed under the Apache License, Version 2.0 – see LICENSE or (https://www.apache.org/licenses/LICENSE-2.0)

Citation

Please cite this document as: