Repository navigation
[stable35] perf: Partition preview migration job - #65464
Open
backportbot[bot] wants to merge 6 commits into
Open
backportbot[bot] wants to merge 6 commits into
backportbot[bot] wants to merge 6 commits into
Conversation
backportbot
Bot
requested review from
AndyScherzinger,
CarlSchwan and
artonge
October 13, 2026 12:43
Signed-off-by: Carl Schwan <carl@carlschwan.eu> Assisted-by: ClaudeCode:claude-sonnet-5
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Use 16 partitions Assisted-by: ClaudeCode:claude-opus-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Also optimize fetching the previews to migrate by directly querying the filecache with an optimized SQL query. Migrating 100 000 previews now takes 22s instead of 646s on PostgreSQL and 50s instead of 449s on SQLite. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Set the location of migrated previews before inserting them, resolving it once per bucket and batch, instead of updating every preview after its insert. Orphaned previews are deleted from the object store by urn directly, with their filecache rows removed in bulk. Migrating 100 000 previews on an object store primary storage with PostgreSQL now takes 24s instead of 123s. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Move the filecache queries from the job into the migration service, so the job only walks the preview folders of its partition, and share the source file lookup and the preview creation between the batch and the single file migration. Assisted-by: ClaudeCode:claude-opus-5-5 Signed-off-by: Carl Schwan <carl@carlschwan.eu>
CarlSchwan
force-pushed
the
backport/64515/stable35
branch
from
October 14, 2026 07:40
d45a0bf to
874ebfa
Compare
CarlSchwan
marked this pull request as ready for review
October 14, 2026 07:49
CarlSchwan
requested review from
leftybournes,
salmart-dev and
sorbaugh
and removed request for
a team
October 14, 2026 07:49
CarlSchwan
approved these changes
Oct 14, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #64515
Warning, This backport's changes differ from the original and might be incomplete⚠️
Todo
Learn more about backports at https://docs.nextcloud.com/server/stable/go.php?to=developer-backports.