HIP-513: Smart Contract Traceability Extension

Abstract

This HIP overrides HIP-260 by moving traceability info from ContractFunctionResult to sidecar (HIP-435).

Extends traceability with new sidecar records ContractActions and ContractBytecode.

Proposes additional data placed in sidecar transaction records to enable full replay of transactions by mirror nodes and downstream users of Hedera.

Motivation

Enabling mirror nodes to download traceability info only if they need or want to process it.

Enabling full replay of smart contract service transactions will improve the usability, auditability, and debuggability of smart contract service transactions. Current facilities are insufficient to inspect “internal” transactions executed across contracts, and provide insufficient error messages when correcting errors during smart contract development.

Enabling estimations like gas to be consumed.

Rationale

HIP-435 allows the creation of sidecar record files containing detailed information about transaction executions without bloating the main transaction record stream. This allows us to externalise verbose debug information about smart contract transactions that was previously expensive to download and process every time.

User stories

Аs a mirror node operator, I want to have the option to download verbose contract operations-related data like contract state changes, contract actions, or contract bytecode.

As a client to the mirror node I want to have the option to see the state and traces associated with a previous contract execution and be able to execute a contract against a mirror node with a contract’s actual state without modifying state or charging gas.

Specification

We will introduce a couple of sidecar record types, one containing the minimum needed data that allows a user to fully replay the transaction and derive the same set of results, and the other containing an entire operation-by-operation trace.

Add ContractStateChanges sidecar record type

Add and populate a new sidecar record of type ContractStateChanges. It will have a single field - repeated ContractStateChange, which will be populated whenever a transaction causes contract storage states to change. This field will include the changes across all contract storage and not just the initial contract in the transaction.

message ContractStateChanges {
  repeated ContractStateChange contract_state_changes = 1;
}

For each slot read or written by the smart contract transaction a ContactStateChange record should be added to the ContractStateChanges.

Add ContractAction sidecar record type

Add and populate a new sidecar record of type ContractActions. It will have a single field - repeated ContractAction with fine-grained information about transaction. Which will improve debugging and auditing.

/**
 * The type of action described by the action proto.
 */
enum ContractActionType {
    /**
     * default non-value.
     */
    NO_ACTION = 0;
    
    /**
     * Most CALL, CALLCODE, DELEGATECALL, and STATICCALL, and first action of ContractCall/ContractCallLocal to deployed
     * contracts. This does not include calls to system or precompiled contracts.
     */
    CALL = 1;

    /**
     * CREATE, CREATE2, and first action of ContractCreate.
     */
    CREATE = 2;

    /**
     * like Call, but to precompiled contracts (0x1 to 0x9 as of Berlin)
     */
    PRECOMPILE = 3;

    /**
     * Call, but to system contract like HTS or ERC20 facades over Token accounts
     */
    SYSTEM = 4;
}

/**
 * The specific operation type of a call. The OP prefix has been added to avoid name collisions for
 * the CALL and CREATE operation types since both ContractActionType and CallOperationType enums are
 * used in ContractAction
 */
enum CallOperationType {
    OP_UNKNOWN = 0;
    OP_CALL = 1;
    OP_CALLCODE = 2;
    OP_DELEGATECALL = 3;
    OP_STATICCALL = 4;
    OP_CREATE = 5;
    OP_CREATE2 = 6;
}

message ContractActions {
    repeated ContractAction contract_actions = 1;
}

/**
 * A finer grained action with a function result. Sometimes called "internal transactions." The function call itself
 * will be the first action in a list, followed by sub-action in the order they were executed.
 */
message ContractAction {

    /**
     * The type of this action.
     */
    ContractActionType call_type = 1;

    /**
     * Only the first action can come from an account, the rest will come from contracts.  Because of DELEGATECALL
     * and CALLCODE the caller of actions whose parent is an account may also be an account.
     */
    oneof caller {
        /**
         * If the caller was a regular account, the AccountID.
         */
        AccountID calling_account = 2;

        /**
         * If the caller was a smart contract account, the ContractID.
         */
        ContractID calling_contract = 3;
    }

    /**
     * The upper limit of gas this action can spend.
     */
    int64 gas = 4;

    /**
     * Bytes passed in as input data to this action.
     */
    bytes input = 5;

    /**
     * Who this action is directed to.
     */
    oneof recipient {
        /**
         * The AccountID of the recipient if the recipient is an account. Only HBars will be transferred, no other side
         * effects should be expected.
         */
        AccountID recipient_account = 6;

        /**
         * The ContractID of the recipient if the recipient is a smart contract.
         */
        ContractID recipient_contract = 7;

        /**
         * If the action was directed at an invalid solidity address, what that address was.
         */
        bytes invalid_solidity_address = 8;
    }

    /**
     * The value (in tinybars) that is associated with this action.
     */
    int64 value = 9;

    /**
     * The actual gas spent by this action.
     */
    int64 gas_used = 10;

    /**
     * The result data of the action.
     */
    oneof result_data {

        /**
         * If successful, the output bytes of the action.
         */
        bytes output = 11;

        /**
         * The contract itself caused the transaction to fail via the `REVERT` operation
         */
        bytes revert_reason = 12;

        /**
         * The transaction itself failed without an explicit `REVERT`
         */
        bytes error = 13;
    }

    /**
     * The nesting depth of this call. The original action is at depth=0.
     */
    int32 call_depth = 14;

    /**
     * The call operation type
     */
    CallOperationType call_operation_type = 15;
}

For every smart contract transaction a ContractAction record should be added to the sidecar record.

Add ContractBytecode sidecar record type

Add and populate a new sidecar record of type ContractBytecode. It will have three fields - contract_id, initcode and runtime_bytecode. Fields will allow tracking smart contracts. initcode field will be populated only if ContractCreationTransaction.fileID is populated or we have child contract create. If we have child contract create TransactionSidecarRecord.consensus_timestamp will be associated with it.

message ContractBytecode {
    /**
    * The contract to which the bytecodes apply to
    */
    ContractID contract_id = 1;

    /**
    * Contract bytecode during deployment
    */
    bytes initcode = 2;
    
    /**
    * Contract bytecode after deployment
    */
    bytes runtime_bytecode = 3;
}

For every contract create ContractBytecode should be added to the sidecar record.

Remove ContractStateChange from ContractFunctionResult

ContractStateChange will be removed from ContractFunctionResult. It will be part of sidecar transaction record.

// existing protobuf
message ContractFunctionResult {
    // existing fields
    ContractID contractID = 1;
    bytes contractCallResult = 2;
    string errorMessage = 3;
    bytes bloom = 4;
    uint64 gasUsed = 5;
    repeated ContractLoginfo logInfo = 6;
    repeated ContractID createdContractIDs = 7;
    
    // field to remove
    //repeated ContractStateChange stateChanges = 8;
    reserved 8;
    
    google.protobuf.BytesValue evm_address = 9;
    int64 gas = 10;
    int64 amount = 11;
    bytes functionParameters = 12;
    AccountID sender_id = 13;
}

Add new sidecar records and migration field to the TransactionSidecarRecord

Add and populate new sidecar records to the TransactionSidecarRecord - ContractActions and ContractBytecode. Add new field migration to differentiate between migration and non migration data.

message TransactionSidecarRecord {
  Timestamp consensus_timestamp = 1;
  
  // new field
  bool migration = 2;
  
  oneof sidecar_records {
    ContractStateChanges state_changes = 3;
    
    //new sidecar records
    ContractActions actions = 4;
    ContractBytecode bytecode = 5;
  }
}

Add new sidecar types tо SidecarType

Add new sidecar types to the SidecarType - CONTRACT_ACTION and CONTRACT_BYTECODE.

enum SidecarType {
  SIDECAR_TYPE_UNKNOWN = 0;
  CONTRACT_STATE_CHANGE = 1;
  
  //new sidecar types
  CONTRACT_ACTION = 2;
  CONTRACT_BYTECODE = 3;
}

Migration of smart contract storage and bytecode

Run the following algorithm in order to migrate all existing contract storage:

1. Iterate through all existing accounts.
2. Filter smart contract accounts.
3. Populate ContractStateChanges value read fields for each smart contract storage slot.
4. Populate ContractBytecode.runtime_bytecode for each smart contract.
5. Create sidecar files which will have the timestamp of the current top-level transaction.

Mirror Node

The mirror node will be updated to be able to conditionally detect and download sidecar files and ingest the data into its database. This information will then be made available via new and existing REST APIs.

Contracts REST API

The existing /api/v1/contracts/{id} REST API will be enhanced to add the bytecode information that it receives from the sidecar file. This API already has the bytecode field which contains the contract initcode. A new runtime_bytecode field encoded as a hex string with a 0x prefix will be added to contain the contract’s runtime bytecode.

Contract Actions REST API

A new /api/v1/contracts/results/{transactionIdOrHash}/actions contract actions REST API will be added to show the flat traces associated with a smart contract execution. Either a transaction ID in the form ${payerAccount}-${validStartSeconds}-${validStartNanos} or a 32 byte hex-encoded transaction hash can be used as the unique identifier in the URL path.

{
  "actions": [{
    "call_depth": 3,
    "call_operation_type": "OP_DELEGATECALL",
    "call_type": "CALL",
    "caller": "0.0.101",
    "caller_type": "CONTRACT",
    "from": "0x0000000000000000000000000000000000000065",
    "gas": 1000,
    "gas_used": 500,
    "index": 0,
    "input": "0xabcd",
    "recipient": "0.0.100",
    "recipient_type": "ACCOUNT",
    "result_data": "0xabcd",
    "result_data_type": "OUTPUT",
    "timestamp": "123.456",
    "to": "0x0000000000000000000000000000000000000064",
    "value": 1000
  }],
  "links": {
    "next": "/api/v1/contracts/results/0.0.1000-123-456/actions?index=gt:99"
  }
}

Note: from and to are just caller and recipient converted to the 20 byte hex encoded EVM address format

Supported query parameters:

• index - The position of the action within the ordered list of actions, starting at zero. Used as the primary key in pagination. Supports eq, gt, gte, lt, lte operators.
• limit - The maximum number of results to return. Defaults to 25 and max 100 allowed.
• order - Sort by the action index. Value of asc or desc with a default of asc.

Contract Results REST API

The existing /api/v1/contracts/{id}/results/{timestamp} and /api/v1/contracts/results/{transactionIdOrHash} already contain a list of contract state changes in the state_changes field in its response. This information was added when ContractStateChange was added to ContractFunctionResult but never populated. With the implementation of the sidecar, this information will now be filled in for mirror nodes that enable sidecar functionality. Below is an example of this state change information returned in the API response:

{
  "contracts": {
    "...": "Other fields omitted for brevity",
    "state_changes": [
      {
        "address": "0000000000000000000000000000000000001f41",
        "contract_id": "0.1.2",
        "slot": "0x00000000000000000000000000000000000000000000000000000000000000fa",
        "value_read": "0x97c1fc0a6ed5551bc831571325e9bdb365d06803100dc20648640ba24ce69750",
        "value_written": "0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925"
      }
    ],
    "transaction_index": 1,
    "type": 2,
    "v": 1
  }
}

Contract State REST API

While the state changes above shows how particular slot values change as a result of a contract invocation, it does not provide a complete list of all slots and their values. For that, we add an /api/v1/contracts/{idOrAddress}/state REST API where id is either the shard.realm.num, realm.num, num, or a hex encoded EVM address of the contract.

{
  "state": [{
    "address": "0x0000000000000000000000000000000000001f41",
    "contract_id": "0.0.100",
    "slot": "0x",
    "timestamp": "10000000.000000001",
    "value": "0x"
  }],
  "links": {
    "next": null
  }
}

• Supported query parameters:
  • limit - The maximum number of results to return. Defaults to 25 and max 100 allowed.
  • order - Sort by the slot. Value of asc or desc with a default of asc.
  • slot - The slot value of the contract storage. Used as the primary key in pagination. Supports eq, gt, gte, lt, lte operators.

Backwards Compatibility

This HIP changes ContractFunctionResult protobuf by removing ContractStateChange from it. Though this is a breaking change, the information was never populated, so it shouldn’t break any client logic.

Security Implications

None.

How to Teach This

Respective documentation will be added.

Reference Implementation

• For protobufs changes, please follow this issue.
• For traceability info removal from ContractFunctionResult, please follow this issue.
• For creation of new sidecar files, please follow this issue.

Rejected Ideas

None.

Open Issues

None.

References

• HIP-435 - Record Stream V6
• HIP-260 - Smart Contract Traceability
• HIP-206 - Hedera Token Service Precompiled Contract for Hedera SmartContract Service

Copyright/license

This document is licensed under the Apache License, Version 2.0 – see LICENSE or (https://www.apache.org/licenses/LICENSE-2.0)

Citation

Please cite this document as: