Skip to content

Add gradient-tracking implementation (v0.2.2) for comparison - #193

Open
mbilalazam wants to merge 1 commit into
DeepLearnPhysics:mainfrom
mbilalazam:compare/grad-tracker-v0.2.2
Open

mbilalazam wants to merge 1 commit into
DeepLearnPhysics:mainfrom
mbilalazam:compare/grad-tracker-v0.2.2

Conversation

@mbilalazam

Copy link
Copy Markdown

Gradient tracking implementation from my GrapPA shower study on SPINE v0.2.2, uploaded as the full manager.py exactly as it ran.

It adds an optional grad_tracker dict to initialize_train (a submodule prefix and an optional depth) and a compute_grad_norms() method. After loss.backward() and before optimizer.step(), it takes the L2 norm of each parameter's gradient, strips the submodule prefix so standalone and full-chain column names match, and adds grad_<name> and grad_total to the result, which puts them in the training CSV.

Config:

train:
  grad_tracker:
    submodule: grappa_shower

On a full-chain GrapPA shower run the CSV gained 160 grad_ columns, 159 parameters plus grad_total. Leaving the block out changes nothing.

Gradient tracking implementation from my GrapPA shower study on SPINE v0.2.2, uploaded as the full manager.py exactly as it ran.

It adds an optional grad_tracker dict to initialize_train (a submodule prefix and an optional depth) and a compute_grad_norms() method. After loss.backward() and before optimizer.step(), it takes the L2 norm of each parameter's gradient, strips the submodule prefix so standalone and full-chain column names match, and adds grad_<name> and grad_total to the result, which puts them in the training CSV.

Config:

train:
  grad_tracker:
    submodule: grappa_shower

On a full-chain GrapPA shower run the CSV gained 160 grad_ columns, 159 parameters plus grad_total. Leaving the block out changes nothing.
@mbilalazam
mbilalazam marked this pull request as ready for review November 17, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant