HIP-1037: Protocol Buffer API Specification
| Author | Joseph Sinclair |
|---|---|
| Requested By | Richard Bair <@rbair> |
| Discussions-To | https://github.com/hashgraph/hedera-improvement-proposal/discussions/1037 |
| Status | Last Call ⓘ |
| Type | Standards Track ⓘ |
| Category | Service ⓘ |
| Created | 2024-08-09 |
| Updated | 2024-01-08 |
Table of Contents
Abstract
The current protocol buffer API definitions are technically correct, but lack usable or useful specification. We should add, in comments compatible with protoc-gen-doc, specification text that clearly states both requirement and expectation for every message, file, and field. This documentation should be automatically transformed to markdown documentation that is published to the Hedera API site. In the ideal case this specification will be sufficient for any qualified team to implement a completely independent consensus node that is fully interoperable with the current consensus node software.
Motivation
The existing protocol buffer API documentation is brief, entirely descriptive, and often inaccurate. This produces two negative consequences. First, it is not possible to produce a compatible independent implementation of the Hedera consensus node software without extensive reference to the existing source code. Second, developers are often unclear how best to interact with the Hedera API, and independent SDK developers, in particular, may struggle to correctly implement these API calls.
As part of expanding the Hedera ecosystem we MUST reduce the difficulty and hurdles for independent developers when building systems that interact with the Hedera network. Clear and accurate API specifications are one element required to accomplish this goal.
Rationale
This HIP improves the developer experience and enhances decentralization by making it easier to design distributed applications or alternative implementations of core network systems.
The use of a markdown format that is automatically generated from the functional protocol buffer definition files encourages more frequent updates that are more closely tied to the functional updates to the API documents, and reduces the inevitable “drift” between functional API and API specification.
User stories
- As a dApp developer I want to have clear and accurate specification for all Hedera APIs so that I can confidently design my application to call those APIs.
- As an independent implementor I want to have clear and accurate specification for all Hedera APIs so that I can design an independent, correct, and compatible implementation of Hedera network nodes, including a consensus node.
- As an Hedera software contributor I want to have clear and accurate specification for all Hedera APIs so that I can confidently implement changes to Hedera-led software.
- As a dApp developer that receives bytes or is expected to parse, produce, or process bytes representing a HAPI protobuf message I want to have a clear specification of the structure and implications of the corresponding proto messages to understand how to correctly parse them.
Specification
The core of this HIP is
the guidelines
recommended for this, and all future Hedera protocol buffer specifications.
This HIP, however, also implements sweeping changes to specification text for
the existing API documents. These changes affect over 20,000 lines of text
across almost 150 files in order to add full specification text that complies
with the guidelines expressed in this improvement proposal. The full reference
implementation, therefore, is only linked here for brevity and clarity.
The implementation of this HIP is detailed in github as a
pull request linked later in this document.
Backwards Compatibility
This HIP does not change any functional characteristic, and documents the state of the API as-is. This does not introduce any change in compatibility.
Security Implications
This HIP updates and clarifies specification and expectation for the Hedera API. This does not materially impact the security of the system.
How to Teach This
The Hedera API is a critical component of the design and interaction for the Hedera ecosystem. Improvements to the formality and completeness of the API specification help everyone involved to develop distributed applications more quickly and with greater confidence.
Reference Implementation
We have a pull request that implements these guidelines across the full hedera-protobufs repository.
Rejected Ideas
- Use of a highly formal “language” for specification
- This was deemed excessive. Protocol Buffers are already a highly formal language, and adding specification as markdown comments, using the light weight RFC/HIP language is sufficient while remaining approachable for the largest practical number of community members.
- Addition of specialized “tag” elements to specification comments.
- This would have required building a complete custom documentation processor and would have been non-standard. Rather than do so we chose to adhere closely to existing standard approaches for protocol buffers.
Open Issues
None.
References
- Protocol Buffer Document Generator protoc-gen-doc.
Copyright/license
This document is licensed under the Apache License, Version 2.0 – see LICENSE or (https://www.apache.org/licenses/LICENSE-2.0)
Citation
Please cite this document as: