Skip to content

fix: update report merging logic to handle non-unique method names - #89

Open
tlambert03 wants to merge 1 commit into
funkelab:mainfrom
tlambert03:from-directory-collision
Open

tlambert03 wants to merge 1 commit into
funkelab:mainfrom
tlambert03:from-directory-collision

Conversation

@tlambert03

Copy link
Copy Markdown
Member

Report.from_directory collects reports in a dict keyed by the name stored inside each JSON file. run_evaluation.py stores every method's report under the same name (default_report), so with the documented layout

reports/DDeepLift/default_report.json
reports/DIntegratedGradients/default_report.json

each file overwrites the previous one and merge_reports is handed a single report. The result is one method's report, not the per-sample best across methods (with no error). This means the per-image choice between Discriminative
IG and Discriminative DeepLift described in the paper never happens.

Checklist

  • I have added tests to show that my feature works in various situations.
  • I have tested the updated pipeline on an example dataset.
  • I have included docstrings
  • I have updated the How-To in the documentation to reflect the changes
  • I have updated the Changelog to reflect any changes to the default function of the pipeline

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant