Skip to content

Return redis connections to the worker pool - #59

Merged
Frenzie merged 1 commit into
koreader:masterfrom
pid1:perf/redis-keepalive
Sep 22, 2026
Merged

Frenzie merged 1 commit into
koreader:masterfrom
pid1:perf/redis-keepalive

Conversation

@pid1

@pid1 pid1 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

DbSettings declares pool = 5 for every environment and nothing reads it. Connections are dropped at the end of each request, so every request pays a TCP handshake and a SELECT.

set_keepalive returns the socket to OpenResty's per-worker pool, sized from that setting and held for 60 seconds. config/redis.conf sets timeout 0, so redis never closes one first.

The pool is named host:port/database, so a connection taken from it has already selected the database its name states, and SELECT runs only when get_reused_times() is zero. Development, test and production are databases 3, 2 and 1 on one server and cannot share a pool.

Release happens in the content phase, wrapped around each action rather than at each return: gin turns raise_error into a response through its own pcall, so an action has no single return point, and nginx finalizes cosockets before log_by_lua runs, where set_keepalive fails with "closed".

Measured over 121 requests: SELECT falls to one per nginx worker. Throughput 1,565 to 3,583 requests per second serially and 8,417 to 20,965 at eight connections, p50 630 to 279 microseconds, five interleaved repetitions paired against master.

28 tests pass. The busted harness starts and stops nginx around every request, so no two requests share a worker and it cannot exercise pooling; the command counts above come from a running server instead.

Builds on #58, and is the smaller half of the gain without it.


This change is Reviewable

Comment thread db/redis.lua
})
if ok then
red:select(option.database)
if red:get_reused_times() == 0 then

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't select still supposed to happen? The reuse thing is just about the TCP connection afaik.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/ledgetech/lua-resty-redis-connector/issues/41#issuecomment-863246307 says that too:

You cannot share connections across requests in a safe way.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, the Claude commit message was rather wordy (I'd prefer if you wrote your own really) but I think it's saying it's safe in the very specific way it's used here? But if so it should say so in a (non-Claude verbose style) comment.

@pid1

pid1 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

Yes, that is it. SELECT runs on every new connection; a reused one is the same TCP connection to the same server, so it is already on the database it selected. The pool name carries the database so a connection that selected a different one is never handed over, which matters here because development, test and production are 3, 2 and 1 on one server.

The connector thread is about holding a connection object across requests. set_keepalive releases the object and puts the socket back in the pool for the next connect to take.

I will add the comment and rewrite those commit messages.

The skip is also droppable. Running SELECT unconditionally keeps the connection reuse, which is the bulk of the gain, and removes both the reuse check and the custom pool name. I have not measured the two apart and can if it is worth knowing. Upstream added a db option on 18 September that does what this does, naming the pool <host>:<port>/db=<n>, but it is not in 0.33 and OpenResty 1.29.2.3 bundles 0.32.

@Frenzie

Frenzie commented Sep 22, 2026

Copy link
Copy Markdown
Member

I think the select should be fairly negligible compared to the overhead of starting up a TCP connection, but measuring is always better.

pool = 5 was already set in DbSettings and unused. set_keepalive puts the socket
back for the next request instead of dropping it, and SELECT then only runs on a
new one. One pool per database keeps dev, test and prod apart.

Release happens around the action because gin turns raise_error into a response
through pcall, and nginx closes cosockets before log_by_lua.
@pid1
pid1 force-pushed the perf/redis-keepalive branch from 2357317 to a03e923 Compare September 22, 2026 21:11
@Frenzie
Frenzie merged commit c2fbf6b into koreader:master Sep 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants