PROP-239 - Add latent tasks for on-demand FaaS execution - #252
Open
JeffMboya wants to merge 10 commits into
Open
PROP-239 - Add latent tasks for on-demand FaaS execution#252JeffMboya wants to merge 10 commits into
JeffMboya wants to merge 10 commits into
Conversation
…ocation Latent tasks pull and precompile their WASM component on deploy but do not execute it. The compiled component is cached and instantiated on demand when an invoke request arrives; concurrent invocations each get a fresh instance so they run in parallel. - Runtime gains precompile() and invoke() (default: unsupported) - WasmtimeRuntime caches precompiled components and reuses the shared component run paths for both start and invoke - StartRequest gains a latent flag; latent starts precompile and return - InvokeRequest and a control/manager/invoke handler drive invocations, publishing results to control/proplet/invoke_results
Adds a Latent flag on tasks that is forwarded to the proplet in the
start payload, so a task can be deployed without executing. A new
InvokeTask service method and POST /tasks/{taskID}/invoke endpoint
publish invocations to control/manager/invoke; per-invocation results
arrive on control/proplet/invoke_results and are recorded without
marking the deployed task terminal.
- Service.InvokeTask + publishInvoke + updateInvokeResultsHandler
- ActionInvoke plugin authorization and middleware wrappers
- invoke-task HTTP endpoint, decoder and mock
Expose InvokeTask on the SDK and a "tasks invoke <id> [inputs...]" CLI command so latent tasks can be triggered on demand.
JeffMboya
changed the title
Store the precompiled component's wasm bytes once behind an Arc instead of deep-copying the whole StartConfig (including the wasm binary) on every invocation. invoke now clones only an Arc refcount and an empty config; run_component_export takes the bytes as a parameter, removing a second per-call copy on both the start and invoke paths.
Invoke now always targets exactly one proplet: a pinned task uses its deployed proplet, and a broadcast-deployed latent task (present on every active proplet) is load-balanced via the round-robin scheduler instead of fanning the request out to all of them. This distributes request load across proplets and gives each invocation a single responder.
Invoke is now request/response instead of fire-and-forget. The manager
tags each invocation with an id, registers a waiter, and blocks until
the proplet publishes the matching result on control/proplet/invoke_results
(or the request times out). The proplet echoes the invocation id back in
an InvokeResultMessage. InvokeTask, the SDK, the CLI and POST
/tasks/{id}/invoke now surface the function's output to the caller, so an
external HTTP client gets the result directly.
SelectProplet mutates LastProplet without synchronization. The latent invoke path now calls it concurrently (load-balancing broadcast tasks), racing with concurrent task starts. Add a mutex around the counter and a concurrent regression test.
InvokeTask used time.After, which leaks a 30s timer every time the result or context case wins the select — the common path on the FaaS hot path. Use time.NewTimer with defer Stop().
updateInvokeResultsHandler only set the task error on failure, so a successful invocation after a failed one left a stale error on the task. Set it unconditionally so the persisted result and error always reflect the latest invocation.
The synchronous invoke path already returns each result to the caller via the pending-invocation waiter, so writing the last invocation's result back onto the task was pure overhead: a task-repo write per invocation plus a last-write-wins race with concurrent invokes. The invoke-results handler now only routes the result to the waiting caller.
rodneyosodo
reviewed
Jul 6, 2026
rodneyosodo
left a comment
rodneyosodo
left a comment
Contributor
There was a problem hiding this comment.
Metrics observations while reviewing latent tasks
| } | ||
|
|
||
| info!( | ||
| "Latent task {} precompiled and ready for on-demand invocation", |
Contributor
There was a problem hiding this comment.
tasks_running is incremented at L539 for all start commands (including latent), but only decremented on the error path (L691). The success path just returns Ok(()) and leaks the counter.
| let invoke_span = tracing::info_span!("task.invoke.execute", task_id = %task_id); | ||
| tokio::spawn( | ||
| async move { | ||
| metrics.tasks_started.inc(); |
Contributor
There was a problem hiding this comment.
tasks_started is already incremented during precompile (L538). This second increment on invoke means tasks_started = 1 + N while tasks_completed = N — the metric invariant is off by one.
| metrics.tasks_started.inc(); | ||
| metrics.tasks_running.inc(); | ||
|
|
||
| let result = runtime.invoke(task_id.clone(), inputs, env).await; |
Contributor
There was a problem hiding this comment.
If runtime.invoke() panics, tasks_running.dec() at L1129 is skipped. Consider wrapping in catch_unwind or noting the gap.
| } | ||
|
|
||
| return svc.taskPropletRepo.Get(ctx, t.ID) | ||
| } |
Contributor
There was a problem hiding this comment.
publishInvoke uses map[string]any while every other publish method uses a typed struct. Also InvokeRequest.env exists but is never populated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Feature: adds latent FaaS tasks that precompile on deploy and instantiate on demand.
What does this do?
Deploys a task that precompiles its WASM component without running, then instantiates it per request in parallel.
Which issue(s) does this PR fix/relate to?
Resolves #239
Have you included tests for your changes?
Yes. Proplet precompile/invoke and concurrent-invocation tests, plus a manager InvokeTask service test.
Did you document any new/modified features?
No README/API docs changed; usage is the new
tasks invoke <id>CLI command.Notes
Latent flag persists only in memory/badger stores, matching existing Priority and Schedule handling.