Skip to content

Bound the RAM caches in bytes - #69

Open
lkdvos wants to merge 1 commit into
mainfrom
claude/cache-bytes
Open

lkdvos wants to merge 1 commit into
mainfrom
claude/cache-bytes

Conversation

@lkdvos

@lkdvos lkdvos commented Oct 28, 2026

Copy link
Copy Markdown
Member

The RAM caches of the CGCs, F-symbols and R-symbols were bounded at 100 000 entries each. That bound is meaningless for the CGCs, which range from bytes to hundreds of megabytes. With TensorKitSectors' sector testsuite (#44), the pentagon test for SU(3) alone keeps about 28 000 CGCs of large intermediate irreps in RAM, using 34 GiB, and the full run was killed at 143 GB.

This PR bounds the caches in bytes instead (MemoizationKit's measure = "bytes", using Base.summarysize):

cache before now
SUN_CGC 10⁵ entries 4 GiB
SUN_Fsymbol 10⁵ entries 512 MiB
SUN_Rsymbol 10⁵ entries 128 MiB

As before, these are defaults that can be changed through preferences, e.g. MemoizationKit.set_cache_preferences!(SUNRepresentations.SUN_CGC; maxsize = 2^34). CGCs evicted from RAM are reloaded from the disk cache.

The F- and R-symbols are small arrays, so their byte bounds hold many more entries than before. The F-symbol cache was thrashing in the pentagon test: it ran at its 10⁵-entry limit with 2.7 million misses, while those 2.5 million F-symbols take only 180 MB.

Measuring is cheap: Base.summarysize of a CGC with 10⁵ nonzero entries takes about 30 µs, since the Dict of a SparseArray holds only isbits data.

Pentagon test of the sector testsuite for SU(3), smallset of #44:

peak RSS time
count bounds (main) 36.7 GiB 880 s
byte bounds (2 GiB for CGCs) 5.1 GiB 1748 s

The time measured here is with a 2 GiB CGC bound, and that test is too large regardless; #44 will need a smaller smallset. The full test suite passes.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/caching.jl 94.44% <ø> (ø)
src/clebschgordan.jl 97.89% <100.00%> (-0.02%) ⬇️
src/sector.jl 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant