Add CARBON pLM, integrated onto main

[lhallee]

Integrates the CARBON DNA language models from #54 onto current main, which had moved on to FastPLMs 1.0 and the ESMC SAE runtime since that branch was cut. This is a real merge of add-carbon (930f6ae), so that history is preserved and #54 closes with it.

Conflicts resolved toward main

• pooler.py keeps main's reformatted signatures and its removal of the parti try/except that swallowed a failure into mean pooling.
• embedder.py keeps pools_internally and sparse_storage; the branch's already_pooled no longer exists.
• get_base_models.py keeps the -sae dispatch and the pooling_types parameter, and does not reintroduce vec2vec.
• data_mixin.py keeps _as_row, so a matrix cache is still averaged per sequence rather than reshaped.
• Dockerfile and setup_protify.sh are unchanged. Those pins targeted torch 2.11 + cu128, which main superseded with torch 2.13 + cu130.

Review findings fixed

Blocking:

• parti pooling raised TypeError on every run. Pooler.__call__ hands every pooling function input_ids and eos_token_id, which _pool_parti did not accept. It now takes **kwargs, and test_parti_runs_through_call exercises __call__ rather than calling _pool_parti directly.
• --download_embeddings would have 404'd for all 31 published caches. They predate max_length joining the cache identity, and the bare except swallowed it. _download_embeddings now falls back to legacy_embedding_filename, and only at PUBLISHED_EMBEDDING_MAX_LENGTH, below which the published file holds more of each sequence than the run asked for.

Also:

• resolve_data_max_length replaces the inline min/max, which raised max() arg is an empty sequence for an empty model list.
• wrap_lora takes the model name and adds the Llama projection targets only for CARBON, instead of unioning them for every model and silently changing adapter placement in other Llama-derived checkpoints.
• _check_vocabulary_agreement compares the checkpoint's vocab_size with the locally reconstructed tokenizer at load time.
• AuditedCarbonTokenizer honors pad_to_multiple_of under --padding longest.
• The GUI warns when the model selection outgrows the budget the loaded data was prepared for.

Tokenizer parity

Protify reimplements CARBON's DNA tokenizer rather than executing the model repository's tokenizer.py. That reimplementation was checked against HuggingFaceBio/Carbon-500M's tokenizer.py at the pinned revision: base order A, T, C, G, specials ahead of the 4096 6-mers, 128-alignment padding, the A right-pad for a partial final k-mer, and the per-chunk OOV rule all match, and dna_start_id 151669, dna_vocab_size 4107, and vocab_size 155776 reproduce exactly. Those constants are pinned in test_carbon.py.

Behavior change worth noting

_sanitize_sequence uppercases before filtering instead of dropping lowercase characters. That is required for lowercase DNA, but it also means lowercase soft-masked residues in protein datasets are now embedded rather than silently removed.

Tests

CPU suite: 470 passed, 13 failed, 5 skipped. All 13 failures are the pre-existing environment ones on this machine, unchanged from main: 11 FastPLMs AttentionInterface import errors and 2 parallel-probe expected scalar type Long but found Int.

test_esmc_sae_models.py and test_fastplms_integration_contracts.py still cannot be collected locally for the same transformers reason, so the SAE-plus-eos interaction is reasoned about but not executed. Worth one run in protify-env:latest.

🤖 Generated with Claude Code