Skip to content

fix: write too large integers and NaN as canonical xsd:double values - #373

Merged
mielvds merged 21 commits into
digitalbazaar:masterfrom
jdsika:fix/to-rdf-large-integers
Nov 12, 2026
Merged

mielvds merged 21 commits into
digitalbazaar:masterfrom
jdsika:fix/to-rdf-large-integers

Conversation

@jdsika

@jdsika jdsika commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This fixes how two special xsd:double values are produced.

1. Integers too large for a double. jsonld.expand(), jsonld.to_rdf() and the other operations (compact, flatten, frame, normalize) fail with an OverflowError for an integer too large for a double, such as 10**309 (flatten() wraps it in a JsonLdError). Python's json module reads such a number as an exact integer, while JavaScript reads it as Infinity.

from pyld import jsonld

doc = {"@id": "http://example.org/s", "http://example.org/p": 10**309}
print(jsonld.to_rdf(doc, {"format": "application/n-quads"}))
# master:  OverflowError: int too large to convert to float
# this PR: <http://example.org/s> <http://example.org/p> "INF"^^<http://www.w3.org/2001/XMLSchema#double>  .

2. NaN. to_rdf() writes the xsd:double NaN as NAN, while the XSD canonical form is NaN. Such a value can come from a typed string {"@value": "NaN", "@type": "xsd:double"}, or as a Python float('nan') (e.g. json.loads('NaN')).

doc = {"@id": "http://example.org/s", "http://example.org/p": {"@value": "NaN", "@type": "http://www.w3.org/2001/XMLSchema#double"}}
print(jsonld.to_rdf(doc, {"format": "application/n-quads"}))
# master:  ... "NAN"^^<http://www.w3.org/2001/XMLSchema#double>  .
# this PR: ... "NaN"^^<http://www.w3.org/2001/XMLSchema#double>  .

Cause: three places convert the integer to a float: _is_numeric() and the xsd:double branch of _object_to_rdf() call float(), and _canonicalize_double() formats it with f'{value:1.15E}'.

For NaN, _canonicalize_double() uses Python's E format, which writes NAN.

Fix:

  • A small helper, _to_double(), converts a number to a double and gives ±infinity when it does not fit.
  • _canonicalize_double() writes the special values explicitly as INF, -INF and NaN. Object to RDF Conversion (JSON-LD 1.1 API §8.2.2, step 10) makes a number with an absolute value of 10^21 or more an xsd:double, and the canonical xsd:double form of infinity is INF. With processingMode json-ld-1.0, the integer is still written exactly as an xsd:integer.

Details

Tests

  • tests/jsonld/test_to_rdf.py:
    • 10**309 and -(10**309) as plain numbers;
    • 10**309 typed as xsd:double;
    • 10**309 in json-ld-1.0 mode, which stays an exact xsd:integer;
    • NaN, INF and -INF as numbers and as typed strings.
  • tests/jsonld/test_expand.py: expand() keeps the integer as it is.

All five tests fail on master. The full suite passes: 1913 passed, 27 skipped, 1 xfailed, and a per-test comparison over the W3C manifests shows no other change.

Found while cross-checking RDFLib's JSON-LD parser with PyLD.

jdsika added 13 commits November 4, 2026 14:23
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…ze_double

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

@mielvds mielvds left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jdsika. I have some style requests, but looks fair otherwise. Please propagate this to the relevant test suites as this is clearly not being picked up by the current manifests.

Comment thread lib/pyld/jsonld.py Resolved
Comment thread lib/pyld/jsonld.py Resolved
Comment thread tests/jsonld/test_to_rdf.py Resolved
Comment thread tests/jsonld/test_to_rdf.py Resolved
Comment thread CHANGELOG.md Resolved
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…l double to_rdf tests

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…lpers

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
… changelog entry

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…flatten wraps

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
@jdsika

jdsika commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @mielvds, I pushed the changes: the JavaScript references are gone, the docstrings link the XSD and JSON-LD specs, and the new test docstrings describe the expected behaviour. I also made a few small changes after another pass: typed xsd:double strings are now converted once instead of twice (5f1efef), and the changelog now says that negative integers too large for a double become -INF, that inside @json literals to_rdf() and normalize() still raise OverflowError for such numbers (c4cbabf), and that the operations no longer fail with an OverflowError (57c1988; on master, flatten() wraps it in a JsonLdError).

On the test suites: the toRdf manifest has no test for these values, as you noted. I opened w3c/json-ld-api#740 proposing tests for numbers beyond the double range and for the typed strings "INF", "-INF" and "NaN". The JSON-LD spec's own sample code writes Infinity for such numbers, so I asked the group to confirm INF first. For RDFC-1.0, see my reply on the changelog.

@mielvds
mielvds merged commit 874c56e into digitalbazaar:master Nov 12, 2026
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants