Skip to content

fix(kvstoreentry): close directory upload files and rewind before retry - #1966

Open
Tyagiquamar wants to merge 1 commit into
fastly:mainfrom
Tyagiquamar:fix/1942-kvstoreentry-dir-close
Open

Tyagiquamar wants to merge 1 commit into
fastly:mainfrom
Tyagiquamar:fix/1942-kvstoreentry-dir-close

Conversation

@Tyagiquamar

Copy link
Copy Markdown
Contributor

Problem

kv-store-entry create --dir opens each file with os.Open in ProcessDir and never closes it, which can exhaust file descriptors on large directories. On a retry after net/http: cannot rewind body after connection loss, the same reader is reused after the first attempt already consumed the body, so the retry uploads empty or truncated content.

Reproduction

See issue #1942: read from the captured request body after upload (still open, not os.ErrClosed); simulate connection-loss error on first insert and observe empty body on retry.

Root cause

Missing Close() on opened files and no seek back to offset 0 before rebuilding the length reader for the retry path.

Solution

defer f.Close() after each successful os.Open, and on the rewind retry path Seek(0) plus a fresh FileLengthReader before the second insertKey.

Tests

Added create_processdir_test.go with:

  • TestProcessDirClosesUploadedFiles
  • TestProcessDirRetriesAfterConnectionLossWithRewoundBody
  • TestProcessDirRetryStillFailsReportsConnectionLoss

Local go test was not run on this Windows host (go not on agent PATH); prior PR #1965 demonstrated full CI coverage on this fork.

Fixes #1942

Close each file opened in ProcessDir and seek to the start before
retrying insert after a connection-loss rewind error.

Fixes fastly#1942

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown

Thanks for the contribution! This pull request doesn't follow our pull request template:

  • the ### Change summary section is missing
  • the All Submissions: checklist is missing

Please edit the description to use the template. If it is still missing after 3 days, this pull request will be closed automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kv-store-entry create --dir never closes uploaded files and retries with an empty body

1 participant