Skip to content

rpk: don't require --partition when using %p in topic produce#30879

Open
telegraphchi wants to merge 1 commit into
redpanda-data:devfrom
telegraphchi:rpk-produce-percent-p
Open

rpk: don't require --partition when using %p in topic produce#30879
telegraphchi wants to merge 1 commit into
redpanda-data:devfrom
telegraphchi:rpk-produce-percent-p

Conversation

@telegraphchi

Copy link
Copy Markdown

rpk topic produce only enabled franz-go's manual partitioner when
--partition was set to a non-negative value, so a %p directive in the
--format string was silently ignored unless -p was also passed. As the
issue notes, the flag's presence — not its value — was the real switch,
which is confusing:

$ rpk topic produce t -f '%p %v\n'        # %p ignored, default partitioning
$ rpk topic produce t -f '%p %v\n' -p 1   # %p honored

This PR enables the manual partitioner whenever the input format references the
partition via %p, so %p works on its own. A small helper detects a real
%p directive while respecting the %%, %{ and %} escapes (so %%p is
literal text, not a directive). The help text and the --partition flag
description are updated to match.

Fixes #17697

Backports Required

  • none - not a bug fix

Release Notes

Improvements

  • rpk topic produce now honors a %p partition directive in --format
    without requiring the --partition flag.

rpk topic produce only enabled the manual partitioner when --partition
was set to a non-negative value, so a %p directive in the --format
string was silently ignored unless the flag was also passed. The flag's
presence, rather than its value, was the real switch, which was
confusing.

Enable the manual partitioner whenever the input format references the
partition via %p, so %p works on its own. Update the help text and the
flag description to match.

Closes redpanda-data#17697

Signed-off-by: Mohamad Reza Chegini <mrchcoin@gmail.com>
@WillemKauf

Copy link
Copy Markdown
Contributor

Docs here seem to indicate this is expected behavior. They will at the very least, need updating, should we alter the behavior like this

@telegraphchi

Copy link
Copy Markdown
Author

Good catch — agreed the docs need updating. That examples page describes today's behavior (where %p requires a non-negative --partition), and this PR does change it, so I've already updated the in-repo --help/long text and I'm happy to follow up with a PR to redpanda-data/docs for that page.

On whether we should change it: the motivation (#17697) is that the current behavior is surprising — when %p is present the --partition value is unused; only the flag's presence decides whether %p is honored, so %p without -p silently no-ops. Since the only reason to put %p in a produce format is to direct records by the parsed partition, enabling the manual partitioner whenever %p appears makes it behave the way the format implies. It doesn't affect any invocation that already passed --partition (manual partitioning was already on there) — it only fixes the %p-without---partition case that previously did nothing.

Happy to gate it differently if you'd prefer (e.g. keep it flag-driven but document clearly), but this seemed like the least-surprising option. Let me know your preference on the docs follow-up too.

@WillemKauf

Copy link
Copy Markdown
Contributor

Good catch — agreed the docs need updating. That examples page describes today's behavior (where %p requires a non-negative --partition), and this PR does change it, so I've already updated the in-repo --help/long text and I'm happy to follow up with a PR to redpanda-data/docs for that page.

I don't mind if your code is AI generated but I wish your responses to me weren't, just FYI.

I think the new behavior is probably better, but @r-vasquez is probably the person to make that call.

@telegraphchi

Copy link
Copy Markdown
Author

Thanks Willem, appreciate you taking a look. Agreed it's @r-vasquez's call — I'm happy either way. Whichever way it lands, I'll update that docs page to match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Do not require -p when using %p in rpk topic produce

2 participants