Skip to main content

Async Returns + Virtual-Thread Dispatch

@RestOp handler methods may return CompletableFuture<T> (or any CompletionStage<T>); the framework unwraps the value through the standard response-processor chain when the future resolves. A separate, opt-in @Rest(virtualThreads=true) flag dispatches handler invocation onto Java 21+ virtual threads with graceful degradation on Java 17.

The two features compose: a CompletableFuture return on a virtualThreads=true resource gives you the high-throughput "no-thread-blocked" pattern without changing the rest of the request pipeline.

Async returns (on by default — return-type opt-in)​

Returning a CompletableFuture<T> or CompletionStage<T> is recognized by a new AsyncResponseProcessor slotted ahead of SerializedPojoProcessor in the default processor chain. Synchronous handlers — methods returning String, MyDto, byte[], Reader, Throwable, etc. — keep their existing path with zero behavioral change.

@Rest(path="/orders")
public class OrderResource extends BasicRestServlet {

    @Inject OrderService orders;

    @RestGet("/{id}")
    public CompletableFuture<Order> get(@Path long id) {
        return orders.lookupAsync(id);                // resource thread is not blocked
    }

    @RestGet("/{id}/timeline")
    public CompletionStage<Timeline> timeline(@Path long id) {
        return orders.timelineAsync(id);              // CompletionStage works too
    }
}

Resolution mechanics​

  1. The handler returns a CompletionStage<T> (any concrete subclass — CompletableFuture<T> is the common one).
  2. AsyncResponseProcessor calls req.startAsync() to obtain an AsyncContext. If the underlying servlet container does not support async dispatch (e.g. unit-test mock environments), the processor falls back to a bounded CompletableFuture.get(timeout) and re-feeds the unwrapped value through the rest of the chain on the calling thread.
  3. A whenComplete((value, error) -> ...) callback on the future:
    • Success — sets the unwrapped value back onto the RestResponse and re-invokes the response-processor chain (skipping AsyncResponseProcessor itself to avoid recursion). The standard SerializedPojoProcessor then handles serialization just like a synchronous return.
    • Failure — routes the throwable through the existing exception pipeline (ThrowableProcessor / ProblemDetailsProcessor). RFC 7807 / 9457 problem-details rendering works unchanged for failed futures when @Rest(problemDetails="true") is set.
  4. AsyncContext.complete() runs exactly once (state-machined behind an AtomicBoolean).

Bare Future<T> is rejected​

Bare java.util.concurrent.Future<T> (i.e. NOT CompletableFuture or CompletionStage) is explicitly rejected with HTTP 500 and a clear message:

Bare java.util.concurrent.Future is not supported as a return type from @RestOp methods. Return CompletableFuture or CompletionStage instead, or block on the result yourself before returning.

This was a deliberate choice: honoring it via a blocking submit(future::get) was weighed against rejecting it, and rejection won because callers who want async should already be reaching for CompletableFuture, and silently blocking a request thread on a Future.get() is a footgun.

Timeout​

The default async-completion timeout is 30 seconds — chosen to match typical proxy timeouts. On timeout, the response is 504 Gateway Timeout and the AsyncContext is aborted.

Override per-resource or per-op:

@Rest(asyncTimeoutMillis = "60000")           // 60s for every op on this resource
public class LongRunningResource extends BasicRestServlet {

    @RestOp(method = "GET", path = "/quick", asyncTimeoutMillis = "5000")
    public CompletableFuture<Quote> quick() { ... }   // overrides resource-level to 5s

    @RestOp(method = "GET", path = "/poll", asyncTimeoutMillis = "0")
    public CompletableFuture<Job> poll() { ... }      // 0 disables the timeout
}

The value is in milliseconds; 0 means no timeout (rely on the container's default); negative values are treated as "use the default 30s". SVL is supported (asyncTimeoutMillis = "${TIMEOUT_MS:30000}").

Cancellation​

Cancellation is best-effort — when the servlet container reports a connection drop via AsyncListener.onError or onTimeout, the underlying CompletableFuture is cancelled with mayInterruptIfRunning=true. The framework does not attempt to cancel the future on every client disconnect (containers vary in how reliably they surface that signal).

SLF4J MDC propagation (MdcAsyncListener)​

As of 10.0, SLF4J MDC context is propagated automatically across CompletableFuture async hops via MdcAsyncListener. Any diagnostic context set on the request thread (e.g. requestId, userId) is visible to log statements emitted inside the whenComplete callback, even though that callback runs on a different thread.

How it works:

  1. At dispatch time, AsyncResponseProcessor snapshots the request thread's MDC via MDC.getCopyOfContextMap().
  2. The snapshot is carried into the completion callback via MdcAsyncListener.wrap(callback, snapshot).
  3. On the completion thread: the pre-existing MDC (if any) is saved, the request-thread snapshot is installed, the callback body runs, and the completion thread's MDC is restored (or cleared) in a finally block.
  4. If the request thread had no MDC (snapshot is null or empty), or SLF4J is absent from the classpath, the original callback is returned unchanged — zero overhead.

The listener is on by default when SLF4J is on the classpath. To opt out programmatically:

// In a RestContext builder or AbstractRestBuilder subclass:
builder.mdcAsyncPropagation(false);

Note: this class bridges SLF4J MDC only. Users of Log4j2's ThreadContext directly (without the SLF4J facade) are not covered and should propagate context manually.

RequestAttributes, VarResolverSession, Locale, and the resolved JUL logger reference/level snapshot that drives REST debug capture are all request-scoped (carried on RestSession / RestOpSession), not ThreadLocal-backed — they survive the async hop without any extra wiring. Security contexts that rely on ThreadLocal directly (e.g. classic SecurityContextHolder) are not propagated by MdcAsyncListener; for those, use OpenTelemetry's Context.makeCurrent() which is async-friendly by design.

Async completion executor (@Rest(asyncCompletionExecutor=...))​

By default, the whenComplete callback runs on whatever thread completes the CompletableFuture — often an I/O-driver or database-callback thread. When that is undesirable (for example, response serialization is expensive and should not block the driver thread), you can route the callback through a named application-owned Executor:

@Rest(path="/api", asyncCompletionExecutor="myCompletionPool")
public class ApiResource extends BasicRestServlet {

    @Bean(name="myCompletionPool")
    public Executor completionPool() {
        return Executors.newFixedThreadPool(8);
    }

    @RestGet("/orders/{id}")
    public CompletableFuture<Order> getOrder(@Path String id) {
        return orderService.fetchAsync(id);   // callback runs on myCompletionPool
    }
}

The value is the @Bean(name=...) lookup key for an Executor registered in the resource's bean store. An empty string (the default) means no override — completion callbacks run on the future's natural thread.

Per-operation override is available via @RestOp(asyncCompletionExecutor=...) (and the verb annotations); op-level wins over resource-level. A reference to a bean name that does not resolve in the bean store causes a startup failure with a clear error.

MDC propagation still works when this executor is set — MdcAsyncListener wraps the callback before it is routed through the executor, so the MDC snapshot is restored on whichever thread the executor picks.

When to return CompletableFuture vs. block in the handler​

Return CompletableFuture / CompletionStage when the work you're doing already produces one natively — Java 11+ HttpClient.sendAsync(...), Project Reactor's Mono.toFuture(), the reactive Mongo / Cassandra drivers, R2DBC, or CompletableFuture.supplyAsync(...) against an explicit application-owned Executor. The request thread is freed immediately and the response is written on whatever thread completes the future.

Block in the handler when the work is synchronous-only (JDBC, classic Apache HttpClient blocking API, file I/O, JMS) AND virtualThreads=true is NOT enabled. Wrapping a blocking call in CompletableFuture.supplyAsync(...) against the common fork-join pool is an anti-pattern — you've moved the block from one platform-thread pool to another and added scheduling overhead.

The two patterns compose with virtual-thread dispatch: with virtualThreads=true you can return a CompletableFuture AND block inside the handler safely, because the handler is running on a virtual thread and the future just defers the response write. The decision is per-op, not service-wide — mix freely.

@RestGet("/external/{id}")
public CompletableFuture<Quote> external(@Path long id) {
    return http.sendAsync(req(id), ofString())          // already async — return the future
        .thenApply(r -> Quote.parse(r.body()));
}

@RestGet("/db/{id}")
public Order db(@Path long id) throws SQLException {
    return jdbcRepo.lookup(id);                          // JDBC is blocking — just block;
}                                                        // pair with virtualThreads=true if hot

Virtual-thread dispatch (Java 21+, opt-in, off by default)​

Virtual-thread dispatch is opt-in and disabled by default — the same non-negotiable contract used for Jakarta Bean Validation and Observability. A fresh RestContext built with no virtualThreads attribute set never instantiates a virtual-thread executor, even on Java 21+. Opt in per-resource or per-op:

@Rest(path="/orders", virtualThreads = "true")        // every op on this resource
public class OrderResource extends BasicRestServlet { ... }

@Rest(path="/mixed")
public class MixedResource extends BasicRestServlet {
    @RestOp(method = "GET", path = "/heavy", virtualThreads = "true")    // just this op
    public Report heavy() { ... }
}

When enabled (and the runtime is Java 21+), RestOpInvoker dispatches the entire handler call — parameter binding, the user method body, and the standard exception-mapping path — onto a virtual thread submitted to a per-resource Executors.newVirtualThreadPerTaskExecutor(). The platform "carrier" thread that received the request is parked on CompletableFuture.get() until the virtual thread finishes, then resumes to write the response.

Java 17 graceful degradation​

The Juneau runtime floor is Java 17. On Java 17 / 18 / 19 / 20, the virtualThreads=true flag is detected at first use, a one-shot WARNING is logged via Logger.getLogger("org.apache.juneau.rest.server.RestContext.async"), and the framework falls back to caller-thread dispatch — exactly the existing pre-10.0.0 behavior. No runtime error.

The implementation uses runtime reflection for the executor creation:

if (Runtime.version().feature() < 21) {
    log.warning("virtualThreads=true configured but runtime is Java <21 — falling back to caller-thread dispatch.");
    return null;
}
return (Executor) Executors.class.getMethod("newVirtualThreadPerTaskExecutor").invoke(null);

This means the framework compiles cleanly on Java 17 (no <source>21</source> needed) and the virtual-thread path simply does nothing on older JVMs. Tests are guarded with @EnabledForJreRange(min = JAVA_21) for the virtual-thread-active path and @DisabledForJreRange(min = JAVA_21) for the warning-and-fallback path.

Composing with async returns​

The two features are independent but compose naturally — a CompletableFuture return on a virtualThreads=true resource gives you both the structured-concurrency benefits of virtual threads AND a non-blocking response path:

@Rest(path="/users", virtualThreads = "true")
public class UserResource extends BasicRestServlet {

    @Inject UserRepo repo;

    @RestGet("/{id}")
    public CompletableFuture<User> get(@Path long id) {
        return repo.findAsync(id);                            // VT runs the handler;
    }                                                          // future resumes the response on whatever
                                                               // thread completes it.
}

Virtual-thread pinning caveat​

synchronized blocks and JNI calls pin a virtual thread to its underlying carrier thread for their duration, which defeats the point of going virtual in the first place. If you opt in to virtualThreads=true, prefer java.util.concurrent.locks.ReentrantLock over synchronized in handler-side code, especially around shared resources held during blocking I/O.

When to enable virtual-thread dispatch​

The headline win is scaling I/O-bound endpoints without thread-pool exhaustion — anything where each request spends most of its wall-clock time blocked on someone else's I/O.

ScenarioWhy VTs help
JDBC / JPA-backed CRUDJDBC is blocking by spec. A platform-thread Tomcat caps at ~200 in-flight requests; with VTs the same JVM serves thousands of concurrent DB-bound requests because blocked VTs cost ~KB of heap each, not ~1 MB of OS stack.
Fan-out aggregator endpoints (GET /dashboard calling 5 downstream services)Write straight-line synchronous code with a blocking HTTP client and still get massive concurrency. No CompletableFuture.allOf(...) choreography required if you don't want it.
Webhook receivers / spiky workloadsStripe / GitHub / Slack can hammer the service with 10k concurrent POSTs during an event. VTs absorb the spike without queue rejection.
Long-polling / SSE / WebSocket-upgrade handshakesConnections that block for many seconds. With VTs, holding 100k open is feasible on a single JVM.
Slow third-party API integrations (payment / identity / geocoding)When an upstream brownout pushes p99 to 8 seconds, platform threads exhaust the pool and unrelated endpoints start failing too. With VTs the blocked threads are essentially free — degradation stays localized to the slow endpoint.
Mixed workloads on one resourcePer-op @RestOp(virtualThreads="true") opts in only the I/O-heavy ops (e.g., the JDBC-backed search) while keeping cheap ops (admin / health / metrics) on platform threads.

Think of virtualThreads=true as "this endpoint is going to spend most of its wall-clock time blocked on someone else's I/O, and I want each request to be as cheap as a coroutine." That's why the flag is opt-in per-resource and per-op rather than a global default — the framework can't know whether your handler is JDBC-blocked or CPU-bound, but you do.

When NOT to enable virtual-thread dispatch​

  • CPU-bound endpoints — image resizing, crypto, big serialization. You're still bounded by cores; VTs just add scheduler overhead.
  • Already non-blocking code — if the handler returns CompletableFuture and calls HttpClient.sendAsync(...), the platform thread is freed at await time anyway. VTs are redundant; the async-returns half of this feature is enough on its own.
  • Hot paths with heavy synchronized — on Java 21, synchronized blocks pin the carrier thread (see Virtual-thread pinning caveat). If a hot path has contended synchronized around expensive work, throughput can actually drop. Pinning is fully fixed in JDK 24 / JEP 491.
  • Frameworks that hard-bind state to ThreadLocal with assumptions of bounded thread count — legacy MDC / session code can balloon memory under millions of virtual threads.

When in doubt, leave it off. The default is correct for CPU-bound services and for services that already use a non-blocking client end-to-end.

Observability composition​

The MetricsRecorder / TracerHook boundary is preserved across both async returns and virtual-thread dispatch — see Observability:

  • Virtual-thread dispatch sits ABOVE the observability hooks. When virtualThreads=true, the entire invoke(...) body — including tracerHook.startSpan(...) and metricsRecorder.record(...) — runs on the virtual thread. OpenTelemetry's Context.makeCurrent() works as expected because the OTel Scope is opened and closed on the same (virtual) thread.
  • Async returns defer observability completion to future-resolution time. When the handler returns a CompletionStage, the framework attaches a whenComplete callback that closes the OTel Scope and records the Micrometer Timer sample at the moment the future resolves — not the synchronous "future returned" moment. The metric / span describes the actual end-to-end latency including the async wait, and the resolved HTTP status / exception is captured correctly even when the future fails.

Configuration summary​

SettingTypeDefaultEffect
@Rest(virtualThreads)String ("true" / "false" / "")"" (off)Enable virtual-thread dispatch for every op on the resource.
@RestOp(virtualThreads)String ("true" / "false" / "")"" (inherit from @Rest)Per-op override. "false" opts out of a resource-level enablement.
@Rest(asyncTimeoutMillis)String (long, ms)"" (use 30000ms default)Async-completion timeout for every async-returning op on the resource.
@RestOp(asyncTimeoutMillis)String (long, ms)"" (inherit from @Rest)Per-op timeout override. "0" disables the timeout for this op.

All four attributes accept SVL (${...}) so values can flow from system properties, env vars, or Config.

Acceptance verification​

The contract is verified end-to-end in juneau-integration-tests:

BehaviorTest
CompletableFuture<String> happy pathAsyncResponseProcessor_Test#a01_completableFutureString_happyPath
CompletableFuture<MyDto> serializes through the standard processor chainAsyncResponseProcessor_Test#a04_completableFuturePojo_serializesBean
Failed future propagates through ThrowableProcessor (404, 500, etc.)AsyncResponseProcessor_Test#b0*_*_completableFuture
Never-completing future → 504 Gateway TimeoutAsyncResponseProcessor_Test#c01_neverCompletingFuture_504OnTimeout
Per-op asyncTimeoutMillis overrides resource-levelAsyncResponseProcessor_Test#d01_perOpTimeoutOverridesResourceLevel
Bare Future rejected with InternalServerErrorAsyncResponseProcessor_Test#e01_bareFutureRejected
virtualThreads=true on Java 21+ → handler runs on a virtual threadVirtualThreadDispatch_Test#a01_java21Plus_virtualThreadDispatch
virtualThreads=true + CompletableFuture returns combineVirtualThreadDispatch_Test#a02_java21Plus_virtualThreadDispatch_combinedWithCompletableFuture
Java 17 graceful degradation — handler still works, warning emittedVirtualThreadDispatch_Test#b01_degradation_handlerStillWorks + b02_java17_logsWarningOnce
Per-op virtualThreads="false" overrides resource-level "true"VirtualThreadDispatch_Test#c01_perOpOptOutOverridesResourceLevel
Per-op virtualThreads="true" works without resource-level enablementVirtualThreadDispatch_Test#c02_perOpOptInWithoutResourceLevel
Default off-by-default — no annotation, no virtual-thread dispatch even on Java 21+VirtualThreadDispatch_Test#d01_offByDefault

See also​