Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/performance/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Performance benchmarks

[Jelly-JVM]({{ jvm_link() }}) 4.0.0 was benchmarked against the serialization formats built into Apache Jena and Eclipse RDF4J:
[Jelly-JVM]({{ jvm_link() }}) 4.0.0 was benchmarked against the serialization formats built into Apache Jena and Eclipse RDF4J, and [pyjelly]({{ python_link() }}) 0.9.0 against the formats built into RDFLib:

- **[Jelly-RDF](rdf.md)** – RDF triples and quads, compared with N-Triples / N-Quads, Turtle / TriG, RDF/XML, JSON-LD, RDF/JSON, Jena's binary formats, and RDF4J Binary.
- **[Jelly-SPARQL](sparql.md)** – SPARQL query results, compared with the XML, JSON, and TSV results formats, Jena's binary formats, and RDF4J Binary.
- **[Jelly-RDF 1.1 vs 1.2](rdf-1-1-vs-1-2.md)** – Jelly-RDF 1.2 (the default in Jelly-JVM 4.0.0) compared with Jelly-RDF 1.1.
- **[Jelly-RDF](rdf.md)** – RDF triples and quads, compared with the formats in Apache Jena, Eclipse RDF4J, and RDFLib.
- **[Jelly-SPARQL](sparql.md)** – SPARQL query results, compared with the formats in Apache Jena, Eclipse RDF4J, and RDFLib.
- **[Jelly-RDF 1.1 vs 1.2](rdf-1-1-vs-1-2.md)** – Jelly-RDF 1.2 (the default in Jelly-JVM 4.0.0 and pyjelly 0.9.0) compared with Jelly-RDF 1.1.
77 changes: 70 additions & 7 deletions docs/performance/rdf-1-1-vs-1-2.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
# Jelly-RDF 1.1 vs 1.2

Jelly-RDF 1.2 features a new column layout for stream frames, which improves throughput and reduces serialized size (see the [changes from Jelly-RDF 1.1](../specification/serialization.md#changes-from-jelly-rdf-11)). The first version of Jelly-JVM to support it is 4.0.0, which also includes other performance improvements in the implementation itself that made Jelly-RDF 1.1 faster as well. This page compares Jelly-RDF 1.2 with Jelly-RDF 1.1, and also shows the performance improvements coming from the implementation.
Jelly-RDF 1.2 features a new column layout for stream frames, which improves throughput and reduces serialized size (see the [changes from Jelly-RDF 1.1](../specification/serialization.md#changes-from-jelly-rdf-11)). The first versions to support it are Jelly-JVM 4.0.0 and pyjelly 0.9.0. Both also include other performance improvements in the implementation itself that made Jelly-RDF 1.1 faster as well. This page compares Jelly-RDF 1.2 with Jelly-RDF 1.1, and also shows the performance improvements coming from the implementations.

The charts are interactive – hover over them for details.

## Benchmark setup
**Jump to:**

- **[Jelly-JVM](#jelly-jvm)** – Apache Jena and Eclipse RDF4J
- **[pyjelly](#pyjelly)** – RDFLib and the generic API

## Jelly-JVM

### Benchmark setup

The same datasets, hardware, software, and method were used as in the [Jelly-RDF benchmarks](rdf.md#benchmark-setup), with the "big" preset. Three versions were compared:

Expand All @@ -14,9 +21,9 @@ The same datasets, hardware, software, and method were used as in the [Jelly-RDF

Each of the three versions was run 6 times for every dataset. The result for a dataset is the median of the 6 runs.

## Results
### Results

### Throughput
#### Throughput

--8<-- "snippets/performance/e-versions.html"

Expand All @@ -32,7 +39,7 @@ Throughput on each of the 13 datasets (median of 6 runs). Log scale.

--8<-- "snippets/performance/k-speed-variants.data.html"

### Serialized size
#### Serialized size

--8<-- "snippets/performance/i-size-summary.html"

Expand All @@ -52,7 +59,7 @@ Serialized size on each of the 13 datasets, uncompressed and with zstd. Sorted b

--8<-- "snippets/performance/j-size-variants.data.html"

### Throughput and serialized size
#### Throughput and serialized size

--8<-- "snippets/performance/l-speed-size-variants.html"

Expand All @@ -61,8 +68,64 @@ Size (uncompressed) against throughput on each of the 13 datasets. Each line con

--8<-- "snippets/performance/l-speed-size-variants.data.html"

## pyjelly

### Benchmark setup

The same datasets, hardware, software, and method were used as in the [pyjelly Jelly-RDF benchmarks](rdf.md#pyjelly), with pyjelly's streaming API and the "big" preset. Throughput was measured with RDFLib terms and with the terms of pyjelly's [generic API]({{ python_link('generic-sink') }}), which does not use RDFLib. Three versions were compared:

- **0.8 (Jelly-RDF 1.0)** – pyjelly 0.8.2, Jelly-RDF 1.0, frame size 250 (its defaults)
- **0.9 (Jelly-RDF 1.1)** – pyjelly 0.9.0 writing Jelly-RDF 1.1, frame size 1024
- **0.9 (Jelly-RDF 1.2)** – pyjelly 0.9.0, Jelly-RDF 1.2, frame size 1024

Each of the three versions was run 6 times for every dataset. The result for a dataset is the median of the 6 runs. Serialized sizes are from the RDFLib integration (the generic API writes the same bytes in 0.9.0, and within 3% of them in 0.8.2).

### Results

#### Throughput

--8<-- "snippets/performance/py-e-versions.html"

Step-by-step throughput improvements: the code changes in pyjelly 0.9.0 (with Jelly-RDF 1.1), and the Jelly-RDF 1.2 format. Teal: geometric mean over 13 datasets, gray: each dataset.
{ .jc-caption }

--8<-- "snippets/performance/py-e-versions.data.html"

--8<-- "snippets/performance/py-k-speed-variants.html"

Throughput on each of the 13 datasets (median of 6 runs). Log scale.
{ .jc-caption }

--8<-- "snippets/performance/py-k-speed-variants.data.html"

#### Serialized size

--8<-- "snippets/performance/py-i-size-summary.html"

Serialized size, geometric mean over 13 datasets.
{ .jc-caption }

--8<-- "snippets/performance/py-i-size-summary.data.html"

--8<-- "snippets/performance/py-j-size-variants.html"

Serialized size on each of the 13 datasets, uncompressed and with zstd. Sorted by how much smaller Jelly-RDF 1.2 is. Log scale.
{ .jc-caption }

--8<-- "snippets/performance/py-j-size-variants.data.html"

#### Throughput and serialized size

--8<-- "snippets/performance/py-l-speed-size-variants.html"

Size (uncompressed) against throughput on each of the 13 datasets. Each line connects a dataset's results for 0.8 (Jelly-RDF 1.0) and 0.9 (Jelly-RDF 1.2). Up and to the left is better. Log scales.
{ .jc-caption }

--8<-- "snippets/performance/py-l-speed-size-variants.data.html"

## See also

- [Jelly-RDF benchmarks](rdf.md)
- [Jelly-RDF 1.2 specification](../specification/serialization.md)
- [Benchmark code](https://github.com/Jelly-RDF/jelly-jvm/tree/main/jmh)
- [Jelly-JVM benchmark code](https://github.com/Jelly-RDF/jelly-jvm/tree/main/jmh)
- [pyjelly benchmark code](https://github.com/Jelly-RDF/pyjelly/tree/main/benchmarks)
144 changes: 130 additions & 14 deletions docs/performance/rdf.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
# Jelly-RDF benchmarks

The following results were obtained by benchmarking [Jelly-JVM]({{ jvm_link() }}) 4.0.0 against the RDF serialization formats built into Apache Jena and Eclipse RDF4J. For SPARQL query results, see [Jelly-SPARQL benchmarks](sparql.md).
The following results were obtained by benchmarking [Jelly-JVM]({{ jvm_link() }}) 4.0.0 against the RDF serialization formats built into Apache Jena and Eclipse RDF4J, and [pyjelly]({{ python_link() }}) 0.9.0 against the formats built into RDFLib. For SPARQL query results, see [Jelly-SPARQL benchmarks](sparql.md).

The benchmarks were performed on flat streams of RDF triples or quads.

The charts are interactive – hover over them for details.

## Benchmark setup
**Jump to:**

- **[Jelly-JVM](#jelly-jvm)** – Apache Jena and Eclipse RDF4J
- **[pyjelly](#pyjelly)** – RDFLib

## Jelly-JVM

### Benchmark setup

All benchmarks were performed on the first 100,000 statements of each dataset in [RiverBench](https://w3id.org/riverbench/v/dev) (development version, October 2026), except `yago-annotated-facts`, which uses RDF-star. Out of the 13 used datasets, 3 include RDF quads (`assist-iot-weather-graphs`, `citypulse-traffic-graphs`, `nanopubs`).

Expand All @@ -20,7 +27,7 @@ Hardware: NVIDIA DGX Spark (GB10 Grace Blackwell Superchip with 10 Cortex-X925 a

Software: Linux kernel 6.17, GraalVM CE 25.4 (Java 25.0.4), Apache Jena 6.2.0, Eclipse RDF4J 6.1.0, [Jelly-JVM]({{ jvm_link() }}) 4.0.0.

### Tested methods
#### Tested methods

- W3C N-Triples / N-Quads (Apache Jena, Eclipse RDF4J)
- W3C Turtle / TriG (Apache Jena `TURTLE_BLOCKS` and `TRIG_BLOCKS`, Eclipse RDF4J without pretty-printing)
Expand All @@ -40,9 +47,9 @@ Software: Linux kernel 6.17, GraalVM CE 25.4 (Java 25.0.4), Apache Jena 6.2.0, E

See the [dedicated Jelly-RDF 1.1 vs 1.2 comparison](rdf-1-1-vs-1-2.md) for details on the performance improvements.

## Results
### Results

### Throughput
#### Throughput

--8<-- "snippets/performance/a-overview-rdf.html"

Expand All @@ -51,14 +58,14 @@ Throughput, geometric mean over 13 datasets (10 for RDF/JSON and RDF/XML, which

<div class="jelly-chart-notes" markdown>

- \* buffering writer: drops duplicate statements (write throughput still counts every input statement)
- \* buffering writer: deduplicates statements (write throughput still counts every input statement)
- ° RDF4J's writer is given an OutputStream and writes it one character at a time. With a BufferedWriter it could be 1.28–1.56× faster

</div>

--8<-- "snippets/performance/a-overview-rdf.data.html"

### Throughput and serialized size
#### Throughput and serialized size

--8<-- "snippets/performance/b-speed-size-rdf-triples.html"

Expand All @@ -67,14 +74,14 @@ Serialized size (uncompressed) against throughput, geometric mean over 10 datase

<div class="jelly-chart-notes" markdown>

- \* buffering writer: drops duplicate statements (write throughput still counts every input statement)
- \* buffering writer: deduplicates statements (write throughput still counts every input statement)
- ° RDF4J's writer is given an OutputStream and writes it one character at a time. With a BufferedWriter it could be 1.28–1.56× faster

</div>

--8<-- "snippets/performance/b-speed-size-rdf-triples.data.html"

### Throughput per dataset
#### Throughput per dataset

--8<-- "snippets/performance/c-heatmap-rdf.html"

Expand All @@ -83,7 +90,7 @@ Throughput on each dataset, in statements/s. The last 3 datasets contain quads.

<div class="jelly-chart-notes" markdown>

- \* buffering writer: drops duplicate statements (write throughput still counts every input statement)
- \* buffering writer: deduplicates statements (write throughput still counts every input statement)
- § the 3 sweeps differ by more than 10%
- ° RDF4J's writer is given an OutputStream and writes it one character at a time. With a BufferedWriter it could be 1.28–1.56× faster

Expand All @@ -98,15 +105,15 @@ Throughput on each dataset (dots) and the geometric mean over 10 datasets (bar).

<div class="jelly-chart-notes" markdown>

- \* buffering writer: drops duplicate statements (write throughput still counts every input statement)
- \* buffering writer: deduplicates statements (write throughput still counts every input statement)
- § the 3 sweeps differ by more than 10%
- ° RDF4J's writer is given an OutputStream and writes it one character at a time. With a BufferedWriter it could be 1.28–1.56× faster

</div>

--8<-- "snippets/performance/d-spread-rdf-triples.data.html"

### Serialized size
#### Serialized size

--8<-- "snippets/performance/f-size-rdf-triples.html"

Expand All @@ -115,16 +122,125 @@ Serialized size in bytes/statement, uncompressed and with zstd, geometric mean o

<div class="jelly-chart-notes" markdown>

- \* buffering writer: drops duplicate statements (write throughput still counts every input statement)
- \* buffering writer: deduplicates statements (write throughput still counts every input statement)

</div>

--8<-- "snippets/performance/f-size-rdf-triples.data.html"

## pyjelly

### Benchmark setup

The same 13 datasets (first 100,000 statements of each) and the same hardware were used as for [Jelly-JVM](#benchmark-setup).

- Each method/dataset combination was run in 3 separate Python processes (sweeps), each with 1 × 1 s warmup and 5 × 1 s measurement. The result for a dataset is the median of the 3 sweeps.
- Serialization: the data was preloaded into memory as a list of RDFLib terms and serialized to a null output stream. RDFLib can write Turtle, RDF/XML, and JSON-LD only from a whole `Graph` or `Dataset`, so for these formats the graph was built beforehand, and only `serialize` was measured.
- Deserialization: the data was parsed from bytes in memory. Only the parsed statements were counted in the throughput numbers.
- Writers were given the prefixes of the dataset.
- Averages are geometric means over the datasets.

Hardware: the same NVIDIA DGX Spark, pinned to 5 Cortex-X925 cores. The benchmarks are single-threaded.

Software: Linux kernel 6.17, CPython 3.15.1, RDFLib 7.6.0, protobuf 7.36.2 (upb backend), [pyjelly]({{ python_link() }}) 0.9.0 (compiled with mypyc).

#### Tested methods

- W3C N-Triples / N-Quads (RDFLib's serializer, run line by line over the list of statements)
- W3C Turtle / TriG (RDFLib)
- W3C RDF/XML (RDFLib) – triples only. RDFLib can't parse its own RDF/XML for 4 of the datasets, so there are results for 6 datasets.
- W3C JSON-LD (RDFLib)
- Jelly (pyjelly 0.9.0, Jelly-RDF 1.2, "big" preset), with pyjelly's streaming API (`flat_stream_to_file` and `parse_jelly_flat`)
```protobuf
max_name_table_size = 4000;
max_prefix_table_size = 150;
max_datatype_table_size = 32;
```
- Jelly (Graph API) – the same, through RDFLib's plugin API (`Graph.serialize(format="jelly")` and `Graph.parse(format="jelly")`)

RDFLib changes some literals when parsing them (e.g., `"73"^^xsd:float` is parsed as `"73.0"`), so N-Triples, Turtle, RDF/XML, and JSON-LD don't round-trip exactly on 7 of the 13 datasets. These results are still included. Jelly round-trips exactly on all datasets.

### Results

#### Throughput

--8<-- "snippets/performance/py-a-overview-rdf.html"

Throughput, geometric mean over 13 datasets (6 for RDF/XML). Log scale.
{ .jc-caption }

<div class="jelly-chart-notes" markdown>

- \* writes from an RDFLib `Graph`, which deduplicates statements (throughput counts only the statements written)

</div>

--8<-- "snippets/performance/py-a-overview-rdf.data.html"

#### Throughput and serialized size

--8<-- "snippets/performance/py-b-speed-size-rdf-triples.html"

Serialized size (uncompressed) against throughput, geometric mean over 10 datasets (6 for RDF/XML). Log scale. Top left is better.
{ .jc-caption }

<div class="jelly-chart-notes" markdown>

- \* writes from an RDFLib `Graph`, which deduplicates statements (throughput counts only the statements written)

</div>

--8<-- "snippets/performance/py-b-speed-size-rdf-triples.data.html"

#### Throughput per dataset

--8<-- "snippets/performance/py-c-heatmap-rdf.html"

Throughput on each dataset, in statements/s. The last 3 datasets contain quads.
{ .jc-caption }

<div class="jelly-chart-notes" markdown>

- \* writes from an RDFLib `Graph`, which deduplicates statements (throughput counts only the statements written)

</div>

--8<-- "snippets/performance/py-c-heatmap-rdf.data.html"

--8<-- "snippets/performance/py-d-spread-rdf-triples.html"

Throughput on each dataset (dots) and the geometric mean over 10 datasets (bar; 6 for RDF/XML). Log scale.
{ .jc-caption }

<div class="jelly-chart-notes" markdown>

- \* writes from an RDFLib `Graph`, which deduplicates statements (throughput counts only the statements written)

</div>

--8<-- "snippets/performance/py-d-spread-rdf-triples.data.html"

#### Serialized size

--8<-- "snippets/performance/py-f-size-rdf-triples.html"

Serialized size in bytes/statement, uncompressed and with zstd, geometric mean over 10 datasets (6 for RDF/XML). Log scale.
{ .jc-caption }

<div class="jelly-chart-notes" markdown>

- \* writes from an RDFLib `Graph`, which deduplicates statements (throughput counts only the statements written)

</div>

--8<-- "snippets/performance/py-f-size-rdf-triples.data.html"

## See also

- [Jelly-SPARQL benchmarks](sparql.md)
- [Benchmark code](https://github.com/Jelly-RDF/jelly-jvm/tree/main/jmh)
- [Jelly-JVM benchmark code](https://github.com/Jelly-RDF/jelly-jvm/tree/main/jmh)
- [RiverBench benchmark suite](https://w3id.org/riverbench/)
- [Jelly-JVM]({{ jvm_link() }}) – the Jelly implementation used in the benchmarks
- [User guide](../user-guide.md)
- [pyjelly benchmark code](https://github.com/Jelly-RDF/pyjelly/tree/main/benchmarks)
- [pyjelly]({{ python_link() }}) – the Jelly implementation for Python
Loading
Loading