Repository navigation
fix: write too large integers and NaN as canonical xsd:double values - #373
Conversation
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…ze_double Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…l double to_rdf tests Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…lpers Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
… changelog entry Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…flatten wraps Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
|
Thanks @mielvds, I pushed the changes: the JavaScript references are gone, the docstrings link the XSD and JSON-LD specs, and the new test docstrings describe the expected behaviour. I also made a few small changes after another pass: typed On the test suites: the toRdf manifest has no test for these values, as you noted. I opened w3c/json-ld-api#740 proposing tests for numbers beyond the double range and for the typed strings |
Summary
This fixes how two special
xsd:doublevalues are produced.1. Integers too large for a double.
jsonld.expand(),jsonld.to_rdf()and the other operations (compact,flatten,frame,normalize) fail with anOverflowErrorfor an integer too large for a double, such as10**309(flatten()wraps it in aJsonLdError). Python'sjsonmodule reads such a number as an exact integer, while JavaScript reads it asInfinity.2. NaN.
to_rdf()writes thexsd:doubleNaN asNAN, while the XSD canonical form isNaN. Such a value can come from a typed string{"@value": "NaN", "@type": "xsd:double"}, or as a Pythonfloat('nan')(e.g.json.loads('NaN')).Cause: three places convert the integer to a float:
_is_numeric()and thexsd:doublebranch of_object_to_rdf()callfloat(), and_canonicalize_double()formats it withf'{value:1.15E}'.For NaN,
_canonicalize_double()uses Python'sEformat, which writesNAN.Fix:
_to_double(), converts a number to a double and gives ±infinity when it does not fit._canonicalize_double()writes the special values explicitly asINF,-INFandNaN. Object to RDF Conversion (JSON-LD 1.1 API §8.2.2, step 10) makes a number with an absolute value of 10^21 or more anxsd:double, and the canonicalxsd:doubleform of infinity isINF. WithprocessingModejson-ld-1.0, the integer is still written exactly as anxsd:integer.Details
Infinityand writes"Infinity"^^xsd:double, which is not a validxsd:doublelexical form. PyLD writes the spec'sINF, as it already does for a float infinity. RDFC hashes of such documents therefore differ between the two until fix: write special xsd:double values in their XSD canonical forms jsonld.js#591 is merged.@jsonliterals still reject such a number. The vendored canonicalizer refuses non-finite numbers, as RFC 8785 does, so that is left as it is.INF,+INF,-INFandNaNfor the special values.doubleLexicalMaprounds withfloatingPointRound, which returns ±infinity on overflow, andspecialRepCanonicalMapwrites'INF','-INF'and'NaN'.xsd:doublestrings into the canonical form, e.g."1"to"1.0E0". That goes beyond §8.2.2, which converts numbers only, and is left as it is. The typed strings in the new test are already canonical.normalize(): output changes for documents with a NaNxsd:double, and so do RDFC-1.0 hashes. Typed"NaN"now matches jsonld.js.1e309as"Infinity"and turns typed"INF"into"NaN". The matching fix there is fix: write special xsd:double values in their XSD canonical forms jsonld.js#591.CHANGELOG.mdsection, so whichever of the three lands later needs a trivial rebase. The code doesn't overlap.Tests
tests/jsonld/test_to_rdf.py:10**309and-(10**309)as plain numbers;10**309typed asxsd:double;10**309injson-ld-1.0mode, which stays an exactxsd:integer;INFand-INFas numbers and as typed strings.tests/jsonld/test_expand.py:expand()keeps the integer as it is.All five tests fail on
master. The full suite passes: 1913 passed, 27 skipped, 1 xfailed, and a per-test comparison over the W3C manifests shows no other change.Found while cross-checking RDFLib's JSON-LD parser with PyLD.