Summary
On a real 2-speaker conversation, OfflineDiarizerManager with .default config returns the correct speaker count (2) but a wrong partition. Pinning clustering.numSpeakers = 2 returns the same count with a materially different and correct partition.
Same N, different answer. That asymmetry is the core of this report: if auto already estimates 2, I would expect it to then partition as if N=2.
Scored against a pyannote speaker-diarization-community-1 reference on identical bytes (10ms frames, optimal label permutation, overlap-forgiven):
| config |
speakers |
speech correctly attributed |
confused |
.default (auto) |
2 |
69.0% |
25.07s |
clustering.numSpeakers = 2 |
2 |
92.0% |
6.09s |
Detail
Audio: 86.5s, 16kHz mono, two speakers, a balanced 63/37 split by speech time. Speaker B holds 0.0-7.0 and then 62.0-86.9; speaker A holds the middle.
- auto partitions at 82.0s:
A 0.0-82.0, B 82.0-86.5. Speaker B's 25-second closing stretch is absorbed into A.
numSpeakers=2 partitions at 62.0s: 0-62 / 62-86.5, which matches the reference except for the 0-7 opening turn.- The
62.0 boundary is genuinely found by both engines independently, so segmentation is proposing it; the auto path just does not partition on it.
- Deterministic: 3 identical runs, byte-identical output.
Other observations from a 20+ config sweep
Clustering.threshold appears inverted relative to its docstring. It is documented as a "Euclidean distance threshold for unit-normalized embeddings", so lowering it should split more. Observed: 0.45, 0.4, 0.3 all merge to 1 speaker, while 0.5, 0.6 (stock), 0.7, 0.9, 1.1 are byte-identical to each other. Also numSpeakers=2 + threshold=0.5 reverts to the bad 69% partition, i.e. setting threshold to a value that is otherwise a no-op cancels the numSpeakers win. Is the direction intended, and is it meant to interact with numSpeakers this way?minSpeakers=2 alone is a no-op here (byte-identical to stock), which makes sense since auto already returns 2, so the floor never binds. minSpeakers=2, maxSpeakers=2 is byte-identical to numSpeakers=2. Worth a docs note: only an exact count (or min+max together) changes the partition; the floor alone cannot correct an under-partition.numSpeakers behaves as a target, not a constraint. On a separate 15.8s clip, numSpeakers=4 returned 3 and numSpeakers=3 returned 2. That seems reasonable, but it is not obvious from the API surface, and it does mean callers cannot rely on the requested count. A docs note would help.ZeroVoteReembed(enabled: true) at minDurationSeconds 0.4 / 0.25 / 0.1 is byte-identical to stock on this audio (with and without numSpeakers), so it does not address this case. Its doc comment describes a similar-sounding symptom ("silently absorbing whole speaker turns into the surrounding speaker's segment"), which is what led me to try it.Embedding.minSegmentDurationSeconds = 0.25 (vs stock 1.0) is also byte-identical here.
Environment
- FluidAudio 0.15.5 (exact pin), models
FluidInference/speaker-diarization-coreml.
- Reproduced on macOS 26.5 (arm64, Swift 6.3) via a small harness calling
OfflineDiarizerManager directly, and observed in an app on iPadOS (iPad Pro M2).
- One caveat I want to be upfront about: on this clip the macOS harness reports 2 speakers where the iPad ANE reports 1. The device is worse. I have not explained that gap; it may be marginal numerical differences on a borderline cluster. On a 4-speaker control clip, harness and device agree exactly (4 vs 4).
What I can and cannot share
The audio is a private personal recording, so I cannot attach it. I am happy to:
- run any config or patch you suggest against it and report numbers,
- try to reproduce on a public corpus if you can point me at a case with a similar shape (one speaker opening briefly, then returning for a long closing stretch).
I also want to flag, in case it saves someone time: I initially "confirmed" this was unfixable by scoring configs against a truncated reference (I had capped the reference turn list, which hid the closing 25 seconds). That produced a confident and completely wrong conclusion until I uncapped it. Given #752 turned out to be placeholder ground truth in the benchmark, corrupted references seem to be an easy trap in this problem space.
Questions
- Is auto-vs-
numSpeakers=N producing different partitions at the same N expected behaviour, or a bug in the count-estimation / VBx warm-start path?
- Is
threshold's observed direction (lower = fewer clusters) intended?
Summary
On a real 2-speaker conversation,
OfflineDiarizerManagerwith.defaultconfig returns the correct speaker count (2) but a wrong partition. Pinningclustering.numSpeakers = 2returns the same count with a materially different and correct partition.Same N, different answer. That asymmetry is the core of this report: if auto already estimates 2, I would expect it to then partition as if N=2.
Scored against a pyannote
speaker-diarization-community-1reference on identical bytes (10ms frames, optimal label permutation, overlap-forgiven):.default(auto)clustering.numSpeakers = 2Detail
Audio: 86.5s, 16kHz mono, two speakers, a balanced 63/37 split by speech time. Speaker B holds
0.0-7.0and then62.0-86.9; speaker A holds the middle.A 0.0-82.0,B 82.0-86.5. Speaker B's 25-second closing stretch is absorbed into A.numSpeakers=2partitions at 62.0s:0-62/62-86.5, which matches the reference except for the0-7opening turn.62.0boundary is genuinely found by both engines independently, so segmentation is proposing it; the auto path just does not partition on it.Other observations from a 20+ config sweep
Clustering.thresholdappears inverted relative to its docstring. It is documented as a "Euclidean distance threshold for unit-normalized embeddings", so lowering it should split more. Observed:0.45,0.4,0.3all merge to 1 speaker, while0.5,0.6(stock),0.7,0.9,1.1are byte-identical to each other. AlsonumSpeakers=2 + threshold=0.5reverts to the bad 69% partition, i.e. setting threshold to a value that is otherwise a no-op cancels thenumSpeakerswin. Is the direction intended, and is it meant to interact withnumSpeakersthis way?minSpeakers=2alone is a no-op here (byte-identical to stock), which makes sense since auto already returns 2, so the floor never binds.minSpeakers=2, maxSpeakers=2is byte-identical tonumSpeakers=2. Worth a docs note: only an exact count (or min+max together) changes the partition; the floor alone cannot correct an under-partition.numSpeakersbehaves as a target, not a constraint. On a separate 15.8s clip,numSpeakers=4returned 3 andnumSpeakers=3returned 2. That seems reasonable, but it is not obvious from the API surface, and it does mean callers cannot rely on the requested count. A docs note would help.ZeroVoteReembed(enabled: true)atminDurationSeconds0.4 / 0.25 / 0.1 is byte-identical to stock on this audio (with and withoutnumSpeakers), so it does not address this case. Its doc comment describes a similar-sounding symptom ("silently absorbing whole speaker turns into the surrounding speaker's segment"), which is what led me to try it.Embedding.minSegmentDurationSeconds = 0.25(vs stock 1.0) is also byte-identical here.Environment
FluidInference/speaker-diarization-coreml.OfflineDiarizerManagerdirectly, and observed in an app on iPadOS (iPad Pro M2).What I can and cannot share
The audio is a private personal recording, so I cannot attach it. I am happy to:
I also want to flag, in case it saves someone time: I initially "confirmed" this was unfixable by scoring configs against a truncated reference (I had capped the reference turn list, which hid the closing 25 seconds). That produced a confident and completely wrong conclusion until I uncapped it. Given #752 turned out to be placeholder ground truth in the benchmark, corrupted references seem to be an easy trap in this problem space.
Questions
numSpeakers=Nproducing different partitions at the same N expected behaviour, or a bug in the count-estimation / VBx warm-start path?threshold's observed direction (lower = fewer clusters) intended?