Pipeline delayed job counting

[top-sigrid]

Resque.count_all_scheduled_jobs from resque-scheduler is very inefficient - issueing one sequential LLEN Redis call per delayed timestamp. With many delayed jobs this becomes the bottleneck during Prometheus metric collection, causing scrape timeouts.

This PR provides a pipelined version of count_all_scheduled_jobs. I intend to also contribute this to resque-scheduler itself, but for now I think this is also a good place here.

On our production system I measured this 18,6x faster counting all delayed jobs:

	Jobs	Timestamps	Duration
Original (sequential)	~35,950	~20,300	5,641ms
Pipelined	~35,950	~20,300	304ms

[jbockler]

Thanks for the PR 🙏
Even though i would rather see this in the resque-scheduler itself, adding this temporarily should be fine.

[top-sigrid]

Thanks for the PR 🙏 Even though i would rather see this in the resque-scheduler itself, adding this temporarily should be fine.

Thanks for being open to this! I agree and will definitely also contribute this to resque-scheduler itself as indicated in the PR description. I have some PRs waiting there for many months, so I hope it won't take forever. Up until then, we can battle test this version in production :p

[jbockler]

Released as 1.3.0 :)

[top-sigrid]

I also opened a PR for resque-scheduler directly as discussed :) resque/resque-scheduler#822