Optimize blocking calls to avoid app thread pool
This reduces the necessary number of threads in the application executor
and provides a small improvement in latency (~15μs, which is normally in
the noise, but would be a 5% improvement).
Benchmark (direct) (transport) Mode Cnt Score Error Units
Before:
TransportBenchmark.unaryCall1024 true INPROCESS avgt 10 1566.168 ± 13.677 ns/op
TransportBenchmark.unaryCall1024 false INPROCESS avgt 10 35769.532 ± 2358.967 ns/op
After:
TransportBenchmark.unaryCall1024 true INPROCESS avgt 10 1813.778 ± 19.995 ns/op
TransportBenchmark.unaryCall1024 false INPROCESS avgt 10 18568.223 ± 1679.306 ns/op
The benchmark results are exactly what we would expect, assuming that
half of the benefit of direct is on server and half on client:
1566 + (35769 - 1566) / 2 = 18668 ns --vs-- 18568 ns
It is expected that direct=true would get worse, because
SerializingExecutor is now used instead of
SerializeReentrantCallsDirectExecutor plus the additional cost of
ThreadlessExecutor.
In the future we could try to detect the ThreadlessExecutor and ellide
Serializ*Executor completely (as is possible for any single-threaded
executor). We could also optimize the queue used in ThreadlessExecutor
to be single-producer, single-consumer. I don't expect to do those
optimizations soon, however.
diff --git a/compiler/src/test/golden/TestService.java.txt b/compiler/src/test/golden/TestService.java.txt
index 77d5bb2..904963e 100644
--- a/compiler/src/test/golden/TestService.java.txt
+++ b/compiler/src/test/golden/TestService.java.txt
@@ -186,14 +186,14 @@
@java.lang.Override
public io.grpc.testing.integration.Test.SimpleResponse unaryCall(io.grpc.testing.integration.Test.SimpleRequest request) {
return blockingUnaryCall(
- getChannel().newCall(METHOD_UNARY_CALL, getCallOptions()), request);
+ getChannel(), METHOD_UNARY_CALL, getCallOptions(), request);
}
@java.lang.Override
public java.util.Iterator<io.grpc.testing.integration.Test.StreamingOutputCallResponse> streamingOutputCall(
io.grpc.testing.integration.Test.StreamingOutputCallRequest request) {
return blockingServerStreamingCall(
- getChannel().newCall(METHOD_STREAMING_OUTPUT_CALL, getCallOptions()), request);
+ getChannel(), METHOD_STREAMING_OUTPUT_CALL, getCallOptions(), request);
}
}