Skip to content

Data preparation: no-BOS tokenizer support and configurable train_on_eos#534

Merged
jlamypoirier merged 5 commits into
mainfrom
jlp_no-bos-data-prep
Jun 5, 2026
Merged

Data preparation: no-BOS tokenizer support and configurable train_on_eos#534
jlamypoirier merged 5 commits into
mainfrom
jlp_no-bos-data-prep

Tighten add_bos/add_eos field descriptions

8099634
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar