Skip to content

Call init_suitesparse when KLU loads - #57

Merged
ViralBShah merged 1 commit into
mainfrom
init-suitesparse
Oct 31, 2026
Merged

ViralBShah merged 1 commit into
mainfrom
init-suitesparse

Conversation

@ViralBShah

Copy link
Copy Markdown
Member

Calls SparseArrays.LibSuiteSparse.init_suitesparse() in __init__. Its docstring says that any package directly wrapping KLU must do this before using the library, and KLU.jl never did.

The bug

On Julia 1.12 and later, SparseArrays sets SuiteSparse's allocator lazily, the first time one of its solvers runs. All SuiteSparse libraries, including libklu, allocate through that one shared setting. So this sequence mixes allocators:

  1. klu(A) allocates with libc's malloc.
  2. lu, cholesky, qr or \ on any sparse matrix switches the allocator to jl_malloc.
  3. The KLU factors are freed through jl_free, which counts them as freed although the GC never counted them as allocated.

After that, GC can become extremely slow. On Linux with Julia 1.13, a single GC.gc() took about 23s after qr with two large factorizations, against 0.015s without the switch. klu, then lu, then freeing 20 large factorizations (m = 200) stalled the process until it was killed at 45s. Julia 1.12 needed more or larger factorizations to trigger it. The accounting explanation in step 3 fits the source and these results, but I haven't traced it inside the GC.

Julia 1.10 and 1.11 set the allocator when SparseArrays loads, so they don't have this bug, and the call is skipped there.

Results with this PR

Both stalling cases above now finish on 1.12 and 1.13, with at most 3 GCs in the allocation loop.

The GC can now see libklu's memory. 150 rounds of factoring and discarding on Julia 1.13, measured once:

main This PR
Time 3.02s 3.11s
GCs 4 110
Peak memory 675 MB 345 MB

Things to know

  • Windows: this hasn't been run on Windows except through CI, and nothing here shows that it's the cause of Windows-only low-level errors #38.
  • set_libdir!: on SparseArrays main, it throws once a library has loaded, and __init__ now loads them at using KLU, so it would have to be called before that. Julia 1.12 and 1.13 don't have this function.

Testing

A new test runs in a subprocess and checks that a later lu doesn't change the allocator. It fails on main with Julia 1.13.

The full suite passes locally on Linux with Julia 1.10, 1.11, 1.12 and 1.13 at -t 2, and Aqua passes.

🤖 Generated with Claude Code

SparseArrays requires this before any use of libklu. Without it, on Julia 1.12 and later, KLU
allocated with libc's malloc until a SparseArrays solver switched the allocator, and those
factors were then freed through jl_free, after which GC could become extremely slow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 97.30%. Comparing base (a0e78a3) to head (6da75ad).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/KLU.jl 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #57      +/-   ##
==========================================
- Coverage   97.54%   97.30%   -0.25%     
==========================================
  Files           1        1              
  Lines         367      371       +4     
==========================================
+ Hits          358      361       +3     
- Misses          9       10       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ViralBShah
ViralBShah merged commit 149db7f into main Oct 31, 2026
6 of 8 checks passed
@ViralBShah
ViralBShah deleted the init-suitesparse branch October 31, 2026 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant