Repository navigation
fix: keep cache info of removed files when another file cannot be deleted - #541
Conversation
…info The cleanup started removing files without waiting and then deleted the cache info right away, so the cache info could be gone while the files were still on disk. If the app was killed in between, those files were left with nothing referencing them. The cleanup now waits for the file removals first. The cache info of a file that could not be removed is kept, so the next cleanup retries it.
3d58574 to
519ad8b
Compare
|
Hey @rickdijk , Please could you confirm that the following pull request #542 fixes the issue presented here in this pull request? I believe it does so thank you 🙏 I do have data to back up the orphaned files scenario that we are seeing and before closing this pull request I would like to confirm that this issue is fixed please. Please could you confirm this? Thanks |
rickdijk
left a comment
There was a problem hiding this comment.
Thanks for the detailed write-up, @junsworth. To answer your question first: yes, #542 (released in 3.4.6) already fixes the ordering problem this PR describes. On current main, _cleanupCache awaits Future.wait(futures) before provider.deleteAll(toRemove), so if the app is killed between the two steps you get a row without a file, not a file without a row. Your branch is based on that commit.
Does it fix something? Not the headline bug, since that is already fixed. There is still one real improvement, though. On main, if a single file can't be deleted (for example a file locked on Windows), Future.wait throws and deleteAll is skipped for the whole batch. getOldObjects and getObjectsOverCapacity return at most 100 rows each, so one file that stays undeletable makes every cleanup reselect the same rows and stop there. Keeping only the failed row and deleting the rest fixes that.
Does it add value? Yes, but only the partial-failure part. The title, description and CHANGELOG should describe that change rather than the ordering fix.
Tests: The second test (should keep cache info of a file that cannot be removed) covers the actual change and fails on main. On main it fails because deleteAll is never called, not because the call includes id 2. The first test already passes on main since #542. It's fine to keep as a regression guard, but it doesn't verify this PR.
Breaking change? No. There are no API changes, and the only behavior change is in the failure path.
Contribution guidelines: The template is filled in, there's one package, and the version is bumped with a dated entry. Some of the text is out of date: the description still says both PRs bump to 3.4.6 and that both tests fail on current main, and the CHANGELOG entry credits 3.4.7 with a fix that 3.4.6 already shipped. "All projects build" is left unchecked, so CI will settle that.
I've left inline comments on the details.
Move the partial-failure handling into _removeCachedFile so cleanup, emptyCache and removeCachedFile all delete the cache info of removed files and keep only the failed one. A failing deleteAll is logged instead of replacing the file removal error.
|
Thanks a lot for the thorough review, @rickdijk, and for confirming #542 covers the ordering problem. That makes sense, my branch was just based on a point before it landed. I've reworked this PR around the partial-failure part you described: the title, description and CHANGELOG now say that, and I used your suggested CHANGELOG wording. I've also moved the fix into |
rickdijk
left a comment
There was a problem hiding this comment.
Wait for the CI to finish, then it can be merged
✨ What kind of change does this PR introduce? (Bug fix, feature, docs update...)
Bug fix.
If a cached file can't be deleted (for example a file locked on Windows),
Future.waitthrows anddeleteAllis skipped for the whole batch.getOldObjectsandgetObjectsOverCapacityreturn at most 100 rows each, so one file that stays undeletable makes every cleanup reselect the same rows and stop there.emptyCachehas the same all-or-nothing behavior.🆕 What is the new behavior (if this is a feature change)?
_removeCachedFileonly marks a file for cache-info deletion once the file is gone or confirmed missing. Cleanup,emptyCacheandremoveCachedFilethen delete the cache info of every removed file and keep only the failed file's cache info, so the next cleanup retries it. The file error is still reported, and ifdeleteAllalso fails that is logged instead of replacing it.💥 Does this PR introduce a breaking change?
No.
🐛 Recommendations for testing
New tests in
cache_store_test.dartcover a failed removal during cleanup, a failed removal inemptyCache, and the file error not being masked whendeleteAllalso fails. The ordering test ("should not remove cache info before the cached file is removed") already passes onmainsince #542, I kept it as a regression guard.flutter test,dart format --set-exit-if-changed .andflutter analyzepass locally on Flutter 3.47.4. I haven't built the example apps locally, CI does that.📝 Links to relevant issues/docs
Follows up on #542, which already fixed the ordering of file removal and cache-info deletion. Related: #535.
🤔 Checklist before submitting