Skip to content

Add HuggingFace WavLM backend to speaker similarity metric - #87

Open
NewGamezzz wants to merge 8 commits into
wavlab-speech:mainfrom
NewGamezzz:feat/wavlm_speaker_similarity
Open

Add HuggingFace WavLM backend to speaker similarity metric#87
NewGamezzz wants to merge 8 commits into
wavlab-speech:mainfrom
NewGamezzz:feat/wavlm_speaker_similarity

Conversation

@NewGamezzz

Copy link
Copy Markdown

Why

Most recent TTS and voice-conversion papers report speaker similarity (SIM) using HuggingFace speaker-verification models, most commonly microsoft/wavlm-base-sv (or its variants, e.g., wavlm-base-plus-sv). Since the speaker metric only supports ESPnet-SPK models, VERSA scores cannot be directly compared against the numbers reported in those papers. This PR adds support for HF x-vector models so that users can reproduce the evaluation setup with the same config format as before:

- name: speaker
  model_tag: microsoft/wavlm-base-sv

File Changes:

  • versa/utterance_metrics/speaker.py: added resolve_speaker_backend(), which auto-detects the backend from model_tag. A thin HFSpeakerModel wrapper mirrors Speech2Embedding's call signature so both backends share the existing cosine-similarity code. No new dependencies: transformers was already in the ml extra and the import is guarded.
  • versa/scorer_shared.py: made the shared cache namespace config-aware so HF speaker models cache under huggingface/ instead of espnet_model_zoo/.
  • versa/config_validation.py: dependency validation now only requires the selected backend's package, so a transformers-only install can run WavLM configs without being asked to install espnet2 (and vice versa).
  • egs/separate_metrics/spk_similarity_wavlm.yaml (new) and docs/supported_metrics.md: example config and docs for the new backend.
  • test/test_metrics/test_speaker.py and test/test_pipeline/test_speaker_wavlm.py (new): unit tests for backend resolution, caching, and validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant