gossipsub: Topic Streams Extension by MarcoPolo · Pull Request #729 · libp2p/specs

MarcoPolo · 2026-07-08T16:45:20Z

Gossipsub v1.3 uses a single stream per direction for all RPCs. This introduces some problems: unnecessary head of line blocking between messages (especially problematic when a large message in one topic delays small messages in another topic) and topic name overhead on each message.

The Topic Streams Extension addresses these problems. It moves topic scoped application messages to separate long-lived streams.

sukunrt · 2026-07-14T10:11:46Z

+When this extension is negotiated, the original gossipsub stream becomes the
+control stream. Application messages (such as the `Message` or
+`PartialMessagesExtension` messages) MUST NOT be published on the control
+stream.


What do you think about a design where we have per topic control streams too?

So the control stream would share the topic relevant: IHave, IWant, IDontWant, and maybe Graft and Prune

This allows:
Easier implementation. Need to read a control scoped RPC.
Heartbeat can now be per topic and be staggered by having different topics at different times, and also more frequent for some topics.
Less HOL Blocking when sharing IHave, IWant, IDontWant

And mostly the fact that there really isn't any reason that control messages for a topic should interleave with control messages of another topic.

Similarly What do you think about?
1 topic control stream as defined above
1 topic data stream per message.
With this design we get nice cancellation properties. So we can cancel in progress sends easily. And if we add message preambles / ids before sending messages we can also reset on the receive side.

Discussed synchronously. Resolved that theoretically 1 topic control stream would be better, the actual utility doesn't justify its implementation complexity. Decided to leave as is.

sukunrt · 2026-07-14T10:12:26Z

+If there are multiple streams for a single topic, the receiver SHOULD process
+them in the order the streams were opened by the initiator. The receiver
+SHOULD limit the number of concurrent topic streams for the same topic to 3 and
+downscore peers that open more. Initiators SHOULD limit the number of
+concurrent topic streams to 1 per topic. The initiator MUST close the old
+stream before writing on a new stream for a given topic.


it's a bit confusing, why would there be 3 streams?

Is this because of subscribe, unsubscribe, subscribe, unsubscribe kind of pattern?

right, or if the implementation opens a new stream per message. Because there is a potential for packets to be reordered or delayed, the sender should avoid concurrent streams that could appear to the receiver as multiple concurrent streams open at the same time.

Add a Shadow scenario that publishes a 1MiB message then a 1KiB message on different topics, and a log-timestamp check that the small message is not head-of-line blocked. Wire the go binary to the topic-streams pubsub fork (libp2p/specs#729) with WithTopicStreams enabled.

gmelodie · 2026-08-12T19:35:46Z

+If a receiver receives a message from a peer that violates a MUST condition,
+the receiver MUST reset the connection to the peer and send error code
+`0xd52505` when the transport allows it. This code signifies a Topic Streams


the message violates the condition, no? not the peer

Suggested change

If a receiver receives a message from a peer that violates a MUST condition,

the receiver MUST reset the connection to the peer and send error code

`0xd52505` when the transport allows it. This code signifies a Topic Streams

If a peer receives a message that violates a MUST condition,

the receiver MUST reset the connection to the peer and send error code

`0xd52505` when the transport allows it. This code signifies a Topic Streams

gmelodie · 2026-08-12T19:45:12Z

+Protocol Violation error, and is derived from the first 3 bytes of the sha256
+hashsum of the string `gossipsub-topic-streams`. (i.e. `echo -n
+"gossipsub-topic-streams" | sha256sum | head -c 6`)


Is this usual for specs to do? Otherwise, I see no point to explain where it comes from (maybe for curiosity in an appendix?

gmelodie · 2026-08-12T19:45:53Z

+
+## Topic Streams
+
+A peer opens a bidirectional stream for each topic that it wishes to send


Suggested change

A peer opens a bidirectional stream for each topic that it wishes to send

A peer MUST open a bidirectional stream for each topic that it wishes to send

gmelodie · 2026-08-12T19:46:49Z

+The responder of the bidirectional stream MUST NOT write on the stream after
+protocol negotiation completes.


So it is only bidirectional to accomodate the initial handshake? If so this should be stated explicitly

gmelodie · 2026-08-13T12:00:50Z

+
+If there are multiple streams for a single topic, the receiver SHOULD process
+them in the order the streams were opened by the initiator. The receiver
+SHOULD limit the number of concurrent topic streams for the same topic to 3 and


I wonder if it'd be better to use variables here for this, as well as the stream reset timeout or other constants, and then add a table below with recommended default values for these.

gmelodie · 2026-08-13T12:04:05Z

+downscore peers that open more. Initiators SHOULD limit the number of
+concurrent topic streams to 1 per topic. The initiator MUST close the old


Initiators SHOULD limit the number of concurrent topic streams to 1 per topic

Should we include an explanation about what to do when there are multiple concurrent topic streams for a single topic? Because I can't see how that would work exactly. If it makes no sense whatsoever, then maybe we can change that SHOULD to a MUST?

gmelodie · 2026-08-13T12:05:20Z

+If the receiver receives a topic stream for a topic it is not subscribed to and
+has not recently published partial messages to (via fanout), it SHOULD
+downscore the peer. The receiver MUST NOT downscore a peer for opening a topic


it SHOULD downscore the peer

Is this true for other spec violations here as well? Maybe we should include this sentence in other places as well

gmelodie · 2026-08-13T12:06:01Z

+downscore the peer. The receiver MUST NOT downscore a peer for opening a topic
+stream for a topic the receiver recently unsubscribed from, as the peer may not
+have received the unsubscribe message before opening the topic stream.


nit

Suggested change

downscore the peer. The receiver MUST NOT downscore a peer for opening a topic

stream for a topic the receiver recently unsubscribed from, as the peer may not

have received the unsubscribe message before opening the topic stream.

downscore the peer.

**Note:** The receiver MUST NOT downscore a peer for opening a topic

stream for a topic the receiver recently unsubscribed from, as the peer may not

have received the unsubscribe message before opening the topic stream.

gmelodie · 2026-08-13T12:08:16Z

+When a peer wishes to publish a message, it MUST publish a `TopicScopedMessage`
+and it MUST NOT publish a message on the control stream.


This got me thinking... how does a peer know that a message is not a control message? Apologies if I'm missing important gossipsub-related information about this.

gmelodie · 2026-08-13T12:09:48Z

+If the partial message extension has been negotiated with this extension, peers
+MUST send each other Partial Messages on the topic stream, not the control
+stream.


nit

Suggested change

If the partial message extension has been negotiated with this extension, peers

MUST send each other Partial Messages on the topic stream, not the control

stream.

If the partial message extension has been negotiated with this extension, peers

MUST send each other Partial Messages on the topic stream.

Peers MUST NOT send Partial Messages on the control stream.

richard-ramos · 2026-08-13T18:19:37Z

+`TopicRPC` messages MUST NOT be empty. They MUST contain either a partial or
+publish message. The data length of the application message MUST be non zero.


The data length of the application message MUST be non zero

Is this restriction mandatory? because pubsub does allow empty data.
https://github.com/libp2p/specs/blob/master/pubsub/README.md#the-message

richard-ramos · 2026-08-13T18:40:07Z

+not publish messages to a peer before learning of its subscriptions, there is
+no window when a publisher wishes to publish a message, but does not know if
+the peer supports Topic Streams.


What to do in case of races?: Assuming you have to peers (A and B) the following can happen (specially in quic):

A and B write topicStreams=true on their gossip streams

A receives B's extension advertisement so A knows that both sides support the extensions

A opens /gsts/v0beta

But, because the streams are muxed independently, B can receive the topic stream before it read A extension advertisement,

Then in that moment B knows it support topic streams, however it still does not know that A supports it.

Should A open the stream ONLY after receiving the extension advertisement from B? or should B assume A opening a stream for /gsts/v0beta is proof enough that A supports topic streams?

richard-ramos · 2026-08-13T19:17:28Z

+A topic stream is created when a node publishes a topic message to a peer. It is
+closed when either the peer unsubscribes from the topic, or the publisher will
+no longer publish on the topic. Either side may close the stream.


Suggested change

A topic stream is created when a node publishes a topic message to a peer. It is

closed when either the peer unsubscribes from the topic, or the publisher will

no longer publish on the topic. Either side may close the stream.

A topic stream is created when a node publishes a topic message to a peer. It is

closed when the peer unsubscribes or the sender no longer intends to send

messages on that topic. Either side may close the stream.

richard-ramos · 2026-08-13T19:21:51Z

+  // Included for computing signatures, not used on the wire
+  optional string unset_topic_name = 4;


Should this use
https://protobuf.dev/programming-guides/proto3/#reserved ?

Suggested change

// Included for computing signatures, not used on the wire

optional string unset_topic_name = 4;

// Included for computing signatures, not used on the wire

reserved 4;

richard-ramos · 2026-08-13T19:36:33Z

+If there are multiple streams for a single topic, the receiver SHOULD process
+them in the order the streams were opened by the initiator. The receiver
+SHOULD limit the number of concurrent topic streams for the same topic to 3 and
+downscore peers that open more. Initiators SHOULD limit the number of


What happens with streams that are opened after the limit is reached? are they reset?

gossipsub: Topic Streams Extension

ae4a036

MarcoPolo requested a review from sukunrt July 8, 2026 16:45

github-project-automation Bot added this to libp2p Specs Jul 8, 2026

github-project-automation Bot moved this to Triage in libp2p Specs Jul 8, 2026

MarcoPolo requested a review from jxs July 8, 2026 16:55

sukunrt reviewed Jul 14, 2026

View reviewed changes

specify non 0 data length

6b526f0

This was referenced Jul 29, 2026

peercomm: refactor to a single actor per peer libp2p/go-libp2p-pubsub#724

Open

gossipsub: implement topic streams extension libp2p/go-libp2p-pubsub#725

Open

MarcoPolo mentioned this pull request Jul 29, 2026

feat(gossipsub-interop): add topic-streams HoL scenario libp2p/test-plans#860

Open

gmelodie reviewed Aug 13, 2026

View reviewed changes

richard-ramos reviewed Aug 13, 2026

View reviewed changes


		## Topic Streams

		A peer opens a bidirectional stream for each topic that it wishes to send

	A peer opens a bidirectional stream for each topic that it wishes to send
	A peer MUST open a bidirectional stream for each topic that it wishes to send

		The responder of the bidirectional stream MUST NOT write on the stream after
		protocol negotiation completes.

		downscore peers that open more. Initiators SHOULD limit the number of
		concurrent topic streams to 1 per topic. The initiator MUST close the old

		When a peer wishes to publish a message, it MUST publish a `TopicScopedMessage`
		and it MUST NOT publish a message on the control stream.

		`TopicRPC` messages MUST NOT be empty. They MUST contain either a partial or
		publish message. The data length of the application message MUST be non zero.

		// Included for computing signatures, not used on the wire
		optional string unset_topic_name = 4;

Uh oh!

Conversation

MarcoPolo commented Jul 8, 2026

Uh oh!

Uh oh!

sukunrt Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sukunrt Jul 14, 2026 •

edited

Loading