feat(api): support int64_as_string parameter for GET requests #6699
feat(api): support int64_as_string parameter for GET requests #6699lvs0075 merged 2 commits intotronprotocol:developfrom
Conversation
b2ce599 to
0587166
Compare
…otocol#6568) Add an opt-in `int64_as_string` query parameter on TRON HTTP GET endpoints. When set, int64/uint64 protobuf fields in the response are serialized as quoted JSON strings to avoid precision loss in clients whose native number type cannot safely represent integers above 2^53 - 1 (e.g. JavaScript). Scope: GET only. POST is intentionally unsupported because reading the request body in a centralized location (RateLimiterServlet.service or a Filter) would consume request.getReader() and break downstream servlets that read the body themselves. Most TRON query endpoints support both GET and POST, so clients that need precision can use the GET form. POST- only write endpoints return Transaction proto whose int64 fields would break round-trip JsonFormat.merge if quoted, so they should not enable this flag in the first place. - JsonFormat: add INT64_AS_STRING ThreadLocal + setInt64AsString / clearInt64AsString / isInt64AsString helpers; split printFieldValue INT64/SINT64/SFIXED64 and UINT64/FIXED64 branches so they emit quoted strings only when the flag is set. - Util: add INT64_AS_STRING constant + getInt64AsString (URL query, mirrors getVisible). - RateLimiterServlet.service: set ThreadLocal from URL query on GET only; clear in finally so reused Tomcat threads do not leak state across requests. - GetBurnTrx / GetPendingSize / GetTransactionCountByBlockNum: emit quoted int64 in their hand-built JSON responses when isInt64AsString is true. - JsonFormatInt64AsStringTest: covers default behavior, int64 / uint64 quoting, non-int64 fields unaffected, nested / map / boundary values (2^53 +/- 1, Long.MAX/MIN, -1), state cleanup (normal close, after exception, explicit clear), thread isolation, thread-reuse anti-pollution. Backward compatibility: requests without int64_as_string=true produce byte-identical responses to develop -- the new code paths are gated entirely on the new flag. Closes tronprotocol#6568.
0587166 to
cb77f36
Compare
|
@waynercheung It’s a bit unfortunate that POST request parameters are not supported, as the functionality feels somewhat incomplete. Do you have plans to support this in the future? This PR — java-tron PR #6728 — introduced CachedBodyRequestWrapper and added request validation. |
@317787106 Thanks for raising this. A few reasons this PR stays GET-only:
If POST support becomes a clear need, I'd rather do it properly as a follow-up once #6728 lands and the spec issue is resolved -- happy to file a tracking issue at that point so it doesn't get lost. |
Three small adjustments per review on PR tronprotocol#6699: - GetTransactionCountByBlockNumServlet: add trailing newline at end of file to satisfy checkstyle. - Util.getInt64AsString: align control flow with the existing Util.getVisible (single-return via local boolean, Boolean.valueOf instead of Boolean.parseBoolean). Functionally identical -- both return true only when the parameter value is "true" (case-insensitive). - Util.INT64_AS_STRING -> Util.INT64_AS_STRING_PARAM: rename the public parameter-name constant to avoid potential confusion with the unrelated private ThreadLocal field of the same simple name in JsonFormat. The user-facing query parameter remains "int64_as_string" -- only the Java identifier changes.
What does this PR do?
Adds an opt-in
int64_as_stringquery parameter on TRON HTTP GET endpoints. When set, int64 / uint64 protobuf fields in the response are serialized as quoted JSON strings (e.g."123456789012345"instead of123456789012345), preventing precision loss in clients whose native number type cannot safely represent integers above 2^53 - 1 (e.g. JavaScript).Scope: GET only, by design.
POST is intentionally unsupported. Reading the body in a centralized location (
RateLimiterServlet.serviceor a Servlet Filter) would consumerequest.getReader(), which TRON's existing 25 servlets already consume themselves for business logic. Centralizing POST would require introducing aCachingHttpServletRequestWrapperto make the body re-readable -- significant infrastructure (~150 lines), runtime overhead (every POST body byte-copied to memory), and edge cases (multipart, encoding, large bodies). The simpler and lower-risk approach is to limit this flag to GET.Most TRON query endpoints support both GET and POST. JS clients that need precision can use the GET form. POST-only write endpoints (Transfer / Trigger / Proposal*) return Transaction proto whose int64 fields would break round-trip
JsonFormat.mergeif quoted -- those clients should not enable this flag in the first place.Backward compatibility: when the parameter is absent or false, every response is byte-identical to current
develop-- same JSON shape, same error messages, same log output, same exception paths.Why are these changes required?
JavaScript
Numberis IEEE 754 double-precision with 53 bits of mantissa. Any int64 / uint64 value larger than2^53 - 1loses precision when consumed as a JS number -- for example, an accountbalanceof9007199254740993becomes9007199254740992. Many TRON HTTP fields routinely exceed this threshold (balance,frozen.amount, energy/bandwidth limits, transactionfee_limit, blocktimestampin microseconds, etc.).The standard mitigation across protobuf JSON ecosystems (proto3 JSON spec, Google's
protobuf-java JsonFormat) is to emit 64-bit integers as quoted strings. This PR brings TRON's HTTP API in line with that convention while keeping the new behavior fully opt-in.Closes #6568.
This PR has been tested by:
Unit Tests -- 19 tests across 2 files, all passing locally:
JsonFormatInt64AsStringTest(13 tests): default behavior, int64 / uint64 quoting, non-int64 fields unaffected, nested / map / boundary values (2^53 ± 1,Long.MAX/MIN, -1), state cleanup (normal close, after exception, explicit clear), thread isolation, thread-reuse anti-pollution.RateLimiterServletInt64Test(6 tests): end-to-end integration pinning four contracts:?int64_as_string=trueproduces quoted int64 fields.service()'sfinallyblock clears the ThreadLocal (defends against silent leak across reused Tomcat threads).isInt64AsString() ? quoted : unquotedternary -- mutation-tested against ternary inversion.Manual Testing -- recommended pre-merge: hit a few endpoints with and without the flag, diff the JSON shape, confirm no regression on the default path.
Follow up
Transaction/Block/TransactionExtention/TransactionSignWeight/TransactionApprovedListproto structures cannot be re-parsed byJsonFormat.merge(the parser does not accept quoted int64 input). Clients that need precision and only consume the response should enable the flag; clients that need round-trip should leave it off.CachingHttpServletRequestWrapper. The trade-offs of that approach (body memory caching, multipart edge cases) deserve their own scoped review./wallet/getnodeinfoand/monitor/getstatsinfoserialize their POJO responses (NodeInfo/MetricsInfo) directly via fastjson, soint64_as_stringhas no effect there. The long fields they expose (memory sizes, network counters, sync numbers, sampling intervals) are well within JS safe integer range in any realistic deployment, so this is a contract-consistency note rather than a precision concern. Can be addressed in a follow-up by passingSerializerFeature.BrowserCompatibletoJSON.toJSONStringconditionally.Extra details
Files changed (9 total).
Core machinery:
JsonFormat.java:INT64_AS_STRINGThreadLocal +setInt64AsString/clearInt64AsString/isInt64AsStringhelpers; split INT64 and UINT64 branches inprintFieldValueto emit quoted strings only when the flag is set. When the flag is unset, the code path is byte-identical todevelop.Util.java:INT64_AS_STRINGconstant +getInt64AsString(URL query, mirrorsgetVisible).RateLimiterServlet.service: set ThreadLocal from URL query on GET only; clear infinallyso reused Tomcat threads do not leak state across requests. Thefinallyclear has a defensive comment because removing it silently breaks request isolation.Hand-built JSON responses (4 files) -- these emit JSON literals manually instead of going through
JsonFormat.printToString, so they readJsonFormat.isInt64AsString()to choose between quoted and unquoted format:GetBurnTrxServlet:burnTrxAmountfieldGetPendingSizeServlet:pendingSizefieldGetTransactionCountByBlockNumServlet:countfieldGetRewardServlet:rewardfieldTest coverage:
JsonFormatInt64AsStringTest(mechanism level)RateLimiterServletInt64Test(end-to-end integration, including critical defense against ThreadLocal leakage)~50 other read-only query servlets automatically pick up the flag through the centralized
RateLimiterServlet.serviceGET path with zero changes at the call site.Sample contract for clients