implement core replay buffer logic

[GHOpenonic]

add replay.py script and test. This does not add replay buffer as a training option, it only implements the core logic. Wiring it into training should be based on the LLC data munging stack #670, #671, #672, #673

[jder]

@GHOpenonic can you point me to where this is used in your code? It's a bit hard to understand how these pieces are put together from this alone.

[GHOpenonic]

@jder most of it is used/further developed in datasets.py, config.py, base.py, stepper.py, and train.py in replay buffer agent attempt.
replay.py is updated and with implementations in datasets.py and train.py in slow replay buffer
I increase replay's dataloading/handling speed to be roughly gpu-compute bound by changing datasets.py and train.py in improve replay buffer train speed --> gpu bound.
Additionally, if you're curious, I make further changes to all these scripts and replay.py in the Jul 19, 2026 commits in llc_cpu_working, mostly shardtensor implementation phase 3: replay buffer, to get shardtensor to work with replay buffer training

[jder]

ok, thanks. I think unless someone needs this sooner (@amogh-gulati maybe?) I vote we stage this after we land the LLC work #670 and rust data loader #800, since building this on top of those probably makes sense.

[amogh-gulati]

ok, thanks. I think unless someone needs this sooner (@amogh-gulati maybe?) I vote we stage this after we land the LLC work #670 and rust data loader #800, since building this on top of those probably makes sense.

I could work on the branch meanwhile