Merge branch 'httpsclibazel'
diff --git a/doc/command_line_tool.md b/doc/command_line_tool.md
index 77c3d07..5465005 100644
--- a/doc/command_line_tool.md
+++ b/doc/command_line_tool.md
@@ -58,50 +58,140 @@
The main file can be found at
https://github.com/grpc/grpc/blob/master/test/cpp/util/grpc_cli.cc
+## Prerequisites
+
+Most `grpc_cli` commands need the server to support server reflection. See
+guides for
+[Java](https://github.com/grpc/grpc-java/blob/master/documentation/server-reflection-tutorial.md#enable-server-reflection)
+, [C++](https://github.com/grpc/grpc/blob/master/doc/server_reflection_tutorial.md)
+and [Go](https://github.com/grpc/grpc-go/blob/master/Documentation/server-reflection-tutorial.md)
+
## Usage
-### Basic usage
+### List services
-Send a rpc to a helloworld server at `localhost:50051`:
+`grpc_cli ls` command lists services and methods exposed at a given port
-```
-$ bins/opt/grpc_cli call localhost:50051 SayHello "name: 'world'" \
- --enable_ssl=false
-```
+- List all the services exposed at a given port
-On success, the tool will print out
+ ```sh
+ $ grpc_cli ls localhost:50051
+ ```
-```
-Rpc succeeded with OK status
-Response:
- message: "Hello world"
-```
+ output:
-The `localhost:50051` part indicates the server you are connecting to. `SayHello` is (part of) the
-gRPC method string. Then `"name: 'world'"` is the text format of the request proto message. We are
-not using ssl here by `--enable_ssl=false`. For information on more flags, look at the comments of `grpc_cli.cc`.
+ ```none
+ helloworld.Greeter
+ grpc.reflection.v1alpha.ServerReflection
+ ```
-### Use local proto files
+ The `localhost:50051` part indicates the server you are connecting to.
-If the server does not have the server reflection service, you will need to provide local proto
-files containing the service definition. The tool will try to find request/response types from
-them.
+- List one service with details
-```
-$ bins/opt/grpc_cli call localhost:50051 SayHello "name: 'world'" \
- --protofiles=examples/protos/helloworld.proto --enable_ssl=false
-```
+ `grpc_cli ls` command inspects a service given its full name (in the format
+ of \<package\>.\<service\>). It can print information with a long listing
+ format when `-l` flag is set. This flag can be used to get more details
+ about a service.
-If the proto files is not under current directory, you can use `--proto_path` to specify a new
-search root.
+ ```sh
+ $ grpc_cli ls localhost:50051 helloworld.Greeter -l
+ ```
-### Send non-proto rpc
+ `helloworld.Greeter` is full name of the service.
-For using gRPC with protocols other than probobuf, you will need the exact method name string
-and a file containing the raw bytes to be sent on the wire
+ output:
-```
-$ bins/opt/grpc_cli call localhost:50051 /helloworld.Greeter/SayHello --input_binary_file=input.bin \
- --output_binary_file=output.bin
-```
-On success, you will need to read or decode the response from the `output.bin` file.
+ ```proto
+ filename: helloworld.proto
+ package: helloworld;
+ service Greeter {
+ rpc SayHello(helloworld.HelloRequest) returns (helloworld.HelloReply) {}
+ }
+
+ ```
+
+### List methods
+
+- List one method with details
+
+ `grpc_cli ls` command also inspects a method given its full name (in the
+ format of \<package\>.\<service\>.\<method\>).
+
+ ```sh
+ $ grpc_cli ls localhost:50051 helloworld.Greeter.SayHello -l
+ ```
+
+ `helloworld.Greeter.SayHello` is full name of the method.
+
+ output:
+
+ ```proto
+ rpc SayHello(helloworld.HelloRequest) returns (helloworld.HelloReply) {}
+ ```
+
+### Inspect message types
+
+We can use `grpc_cli type` command to inspect request/response types given the
+full name of the type (in the format of \<package\>.\<type\>).
+
+- Get information about the request type
+
+ ```sh
+ $ grpc_cli type localhost:50051 helloworld.HelloRequest
+ ```
+
+ `helloworld.HelloRequest` is the full name of the request type.
+
+ output:
+
+ ```proto
+ message HelloRequest {
+ optional string name = 1;
+ }
+ ```
+
+### Call a remote method
+
+We can send RPCs to a server and get responses using `grpc_cli call` command.
+
+- Call a unary method Send a rpc to a helloworld server at `localhost:50051`:
+
+ ```sh
+ $ grpc_cli call localhost:50051 SayHello "name: 'gRPC CLI'"
+ ```
+
+ output: `sh message: "Hello gRPC CLI"`
+
+ `SayHello` is (part of) the gRPC method string. Then `"name: 'world'"` is
+ the text format of the request proto message. For information on more flags,
+ look at the comments of `grpc_cli.cc`.
+
+- Use local proto files
+
+ If the server does not have the server reflection service, you will need to
+ provide local proto files containing the service definition. The tool will
+ try to find request/response types from them.
+
+ ```sh
+ $ grpc_cli call localhost:50051 SayHello "name: 'world'" \
+ --protofiles=examples/protos/helloworld.proto
+ ```
+
+ If the proto file is not under the current directory, you can use
+ `--proto_path` to specify a new search root.
+
+- Send non-proto rpc
+
+ For using gRPC with protocols other than probobuf, you will need the exact
+ method name string and a file containing the raw bytes to be sent on the
+ wire.
+
+ ```bash
+ $ grpc_cli call localhost:50051 /helloworld.Greeter/SayHello \
+ --input_binary_file=input.bin \
+ --output_binary_file=output.bin
+ ```
+
+ On success, you will need to read or decode the response from the
+ `output.bin` file.
diff --git a/gRPC-C++.podspec b/gRPC-C++.podspec
index 55f33d9..45cee35 100644
--- a/gRPC-C++.podspec
+++ b/gRPC-C++.podspec
@@ -263,8 +263,8 @@
'src/core/lib/security/transport/security_handshaker.h',
'src/core/lib/security/transport/tsi_error.h',
'src/core/lib/security/util/json_util.h',
+ 'src/core/tsi/alts_transport_security.h',
'src/core/tsi/fake_transport_security.h',
- 'src/core/tsi/gts_transport_security.h',
'src/core/tsi/ssl_transport_security.h',
'src/core/tsi/ssl_types.h',
'src/core/tsi/transport_security_grpc.h',
diff --git a/src/python/grpcio/grpc/_channel.py b/src/python/grpcio/grpc/_channel.py
index bfc7208..25a4210 100644
--- a/src/python/grpcio/grpc/_channel.py
+++ b/src/python/grpcio/grpc/_channel.py
@@ -13,10 +13,10 @@
# limitations under the License.
"""Invocation-side implementation of gRPC Python."""
+import logging
import sys
import threading
import time
-import logging
import grpc
from grpc import _common
@@ -882,8 +882,12 @@
def _options(options):
- return list(options) + [(cygrpc.ChannelArgKey.primary_user_agent_string,
- _USER_AGENT)]
+ return list(options) + [
+ (
+ cygrpc.ChannelArgKey.primary_user_agent_string,
+ _USER_AGENT,
+ ),
+ ]
class Channel(grpc.Channel):
@@ -892,14 +896,13 @@
def __init__(self, target, options, credentials):
"""Constructor.
- Args:
- target: The target to which to connect.
- options: Configuration options for the channel.
- credentials: A cygrpc.ChannelCredentials or None.
- """
+ Args:
+ target: The target to which to connect.
+ options: Configuration options for the channel.
+ credentials: A cygrpc.ChannelCredentials or None.
+ """
self._channel = cygrpc.Channel(
- _common.encode(target), _common.channel_args(_options(options)),
- credentials)
+ _common.encode(target), _options(options), credentials)
self._call_state = _ChannelCallState(self._channel)
self._connectivity_state = _ChannelConnectivityState(self._channel)
diff --git a/src/python/grpcio/grpc/_common.py b/src/python/grpcio/grpc/_common.py
index 130fc42..bbb69ad 100644
--- a/src/python/grpcio/grpc/_common.py
+++ b/src/python/grpcio/grpc/_common.py
@@ -79,16 +79,6 @@
return b.decode('latin1')
-def channel_args(options):
- cygrpc_args = []
- for key, value in options:
- if isinstance(value, six.string_types):
- cygrpc_args.append(cygrpc.ChannelArg(encode(key), encode(value)))
- else:
- cygrpc_args.append(cygrpc.ChannelArg(encode(key), value))
- return cygrpc.ChannelArgs(cygrpc_args)
-
-
def _transform(message, transformer, exception_message):
if transformer is None:
return message
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi
new file mode 100644
index 0000000..853bf6f
--- /dev/null
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pxd.pxi
@@ -0,0 +1,40 @@
+# Copyright 2018 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+cdef void* _copy_pointer(void* pointer)
+
+
+cdef void _destroy_pointer(void* pointer)
+
+
+cdef int _compare_pointer(void* first_pointer, void* second_pointer)
+
+
+cdef class _ArgumentProcessor:
+
+ cdef grpc_arg c_argument
+
+ cdef void c(self, argument, grpc_arg_pointer_vtable *vtable, references)
+
+
+cdef class _ArgumentsProcessor:
+
+ cdef readonly tuple _arguments
+ cdef list _argument_processors
+ cdef readonly list _references
+ cdef grpc_channel_args _c_arguments
+
+ cdef grpc_channel_args *c(self, grpc_arg_pointer_vtable *vtable)
+ cdef un_c(self)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi
new file mode 100644
index 0000000..65de308
--- /dev/null
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/arguments.pyx.pxi
@@ -0,0 +1,88 @@
+# Copyright 2018 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+cimport cpython
+
+
+cdef void* _copy_pointer(void* pointer):
+ return pointer
+
+
+cdef void _destroy_pointer(void* pointer):
+ pass
+
+
+cdef int _compare_pointer(void* first_pointer, void* second_pointer):
+ if first_pointer < second_pointer:
+ return -1
+ elif first_pointer > second_pointer:
+ return 1
+ else:
+ return 0
+
+
+cdef class _ArgumentProcessor:
+
+ cdef void c(self, argument, grpc_arg_pointer_vtable *vtable, references):
+ key, value = argument
+ cdef bytes encoded_key = _encode(key)
+ if encoded_key is not key:
+ references.append(encoded_key)
+ self.c_argument.key = encoded_key
+ if isinstance(value, int):
+ self.c_argument.type = GRPC_ARG_INTEGER
+ self.c_argument.value.integer = value
+ elif isinstance(value, (bytes, str, unicode,)):
+ self.c_argument.type = GRPC_ARG_STRING
+ encoded_value = _encode(value)
+ if encoded_value is not value:
+ references.append(encoded_value)
+ self.c_argument.value.string = encoded_value
+ elif hasattr(value, '__int__'):
+ # Pointer objects must override __int__() to return
+ # the underlying C address (Python ints are word size). The
+ # lifecycle of the pointer is fixed to the lifecycle of the
+ # python object wrapping it.
+ self.c_argument.type = GRPC_ARG_POINTER
+ self.c_argument.value.pointer.vtable = vtable
+ self.c_argument.value.pointer.address = <void*>(<intptr_t>int(value))
+ else:
+ raise TypeError(
+ 'Expected int, bytes, or behavior, got {}'.format(type(value)))
+
+
+cdef class _ArgumentsProcessor:
+
+ def __cinit__(self, arguments):
+ self._arguments = () if arguments is None else tuple(arguments)
+ self._argument_processors = []
+ self._references = []
+
+ cdef grpc_channel_args *c(self, grpc_arg_pointer_vtable *vtable):
+ self._c_arguments.arguments_length = len(self._arguments)
+ if self._c_arguments.arguments_length == 0:
+ return NULL
+ else:
+ self._c_arguments.arguments = <grpc_arg *>gpr_malloc(
+ self._c_arguments.arguments_length * sizeof(grpc_arg))
+ for index, argument in enumerate(self._arguments):
+ argument_processor = _ArgumentProcessor()
+ argument_processor.c(argument, vtable, self._references)
+ self._c_arguments.arguments[index] = argument_processor.c_argument
+ self._argument_processors.append(argument_processor)
+ return &self._c_arguments
+
+ cdef un_c(self):
+ if self._arguments:
+ gpr_free(self._c_arguments.arguments)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi
index 4b07e71..1ba76b7 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pxd.pxi
@@ -15,5 +15,7 @@
cdef class Channel:
+ cdef grpc_arg_pointer_vtable _vtable
cdef grpc_channel *c_channel
cdef list references
+ cdef readonly _ArgumentsProcessor _arguments_processor
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi
index efe5f2e..a396649 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi
@@ -17,26 +17,25 @@
cdef class Channel:
- def __cinit__(self, bytes target, ChannelArgs arguments,
+ def __cinit__(self, bytes target, object arguments,
ChannelCredentials channel_credentials=None):
grpc_init()
- cdef grpc_channel_args *c_arguments = NULL
- cdef char *c_target = NULL
- self.c_channel = NULL
+ self._vtable.copy = &_copy_pointer
+ self._vtable.destroy = &_destroy_pointer
+ self._vtable.cmp = &_compare_pointer
+ cdef _ArgumentsProcessor arguments_processor = _ArgumentsProcessor(
+ arguments)
+ cdef grpc_channel_args *c_arguments = arguments_processor.c(&self._vtable)
self.references = []
- if len(arguments) > 0:
- c_arguments = &arguments.c_args
- self.references.append(arguments)
c_target = target
if channel_credentials is None:
- with nogil:
- self.c_channel = grpc_insecure_channel_create(c_target, c_arguments,
- NULL)
+ self.c_channel = grpc_insecure_channel_create(c_target, c_arguments, NULL)
else:
c_channel_credentials = channel_credentials.c()
self.c_channel = grpc_secure_channel_create(
c_channel_credentials, c_target, c_arguments, NULL)
grpc_channel_credentials_release(c_channel_credentials)
+ arguments_processor.un_c()
self.references.append(target)
self.references.append(arguments)
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/records.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/records.pxd.pxi
index 297bbad..35e1bdb 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/records.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/records.pxd.pxi
@@ -29,19 +29,6 @@
cdef readonly object private_key, certificate_chain
-cdef class ChannelArg:
-
- cdef grpc_arg c_arg
- cdef grpc_arg_pointer_vtable ptr_vtable
- cdef readonly object key, value
-
-
-cdef class ChannelArgs:
-
- cdef grpc_channel_args c_args
- cdef list args
-
-
cdef class CompressionOptions:
cdef grpc_compression_options c_options
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi
index f9d4171..1bcea8d 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/records.pyx.pxi
@@ -157,81 +157,6 @@
self.c_pair.certificate_chain = self.certificate_chain
-
-cdef void* copy_ptr(void* ptr):
- return ptr
-
-
-cdef void destroy_ptr(void* ptr):
- pass
-
-
-cdef int compare_ptr(void* ptr1, void* ptr2):
- if ptr1 < ptr2:
- return -1
- elif ptr1 > ptr2:
- return 1
- else:
- return 0
-
-
-cdef class ChannelArg:
-
- def __cinit__(self, bytes key, value):
- self.key = key
- self.value = value
- self.c_arg.key = self.key
- if isinstance(value, int):
- self.c_arg.type = GRPC_ARG_INTEGER
- self.c_arg.value.integer = self.value
- elif isinstance(value, bytes):
- self.c_arg.type = GRPC_ARG_STRING
- self.c_arg.value.string = self.value
- elif hasattr(value, '__int__'):
- # Pointer objects must override __int__() to return
- # the underlying C address (Python ints are word size). The
- # lifecycle of the pointer is fixed to the lifecycle of the
- # python object wrapping it.
- self.ptr_vtable.copy = ©_ptr
- self.ptr_vtable.destroy = &destroy_ptr
- self.ptr_vtable.cmp = &compare_ptr
- self.c_arg.type = GRPC_ARG_POINTER
- self.c_arg.value.pointer.vtable = &self.ptr_vtable
- self.c_arg.value.pointer.address = <void*>(<intptr_t>int(self.value))
- else:
- # TODO Add supported pointer types to this message
- raise TypeError('Expected int or bytes, got {}'.format(type(value)))
-
-
-cdef class ChannelArgs:
-
- def __cinit__(self, args):
- grpc_init()
- self.args = list(args)
- for arg in self.args:
- if not isinstance(arg, ChannelArg):
- raise TypeError("expected list of ChannelArg")
- self.c_args.arguments_length = len(self.args)
- with nogil:
- self.c_args.arguments = <grpc_arg *>gpr_malloc(
- self.c_args.arguments_length*sizeof(grpc_arg))
- for i in range(self.c_args.arguments_length):
- self.c_args.arguments[i] = (<ChannelArg>self.args[i]).c_arg
-
- def __dealloc__(self):
- with nogil:
- gpr_free(self.c_args.arguments)
- grpc_shutdown()
-
- def __len__(self):
- # self.args is never stale; it's only updated from this file
- return len(self.args)
-
- def __getitem__(self, size_t i):
- # self.args is never stale; it's only updated from this file
- return self.args[i]
-
-
cdef class CompressionOptions:
def __cinit__(self):
@@ -254,8 +179,10 @@
return result
def to_channel_arg(self):
- return ChannelArg(GRPC_COMPRESSION_CHANNEL_ENABLED_ALGORITHMS_BITSET,
- self.c_options.enabled_algorithms_bitset)
+ return (
+ GRPC_COMPRESSION_CHANNEL_ENABLED_ALGORITHMS_BITSET,
+ self.c_options.enabled_algorithms_bitset,
+ )
def compression_algorithm_name(grpc_compression_algorithm algorithm):
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi
index df4577e..4588db3 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/server.pxd.pxi
@@ -15,6 +15,8 @@
cdef class Server:
+ cdef grpc_arg_pointer_vtable _vtable
+ cdef readonly _ArgumentsProcessor _arguments_processor
cdef grpc_server *c_server
cdef bint is_started # start has been called
cdef bint is_shutting_down # shutdown has been called
diff --git a/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi b/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi
index e5d28a8..707ec74 100644
--- a/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi
+++ b/src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi
@@ -57,16 +57,19 @@
cdef class Server:
- def __cinit__(self, ChannelArgs arguments):
+ def __cinit__(self, object arguments):
grpc_init()
- cdef grpc_channel_args *c_arguments = NULL
self.references = []
self.registered_completion_queues = []
- if len(arguments) > 0:
- c_arguments = &arguments.c_args
- self.references.append(arguments)
- with nogil:
- self.c_server = grpc_server_create(c_arguments, NULL)
+ self._vtable.copy = &_copy_pointer
+ self._vtable.destroy = &_destroy_pointer
+ self._vtable.cmp = &_compare_pointer
+ cdef _ArgumentsProcessor arguments_processor = _ArgumentsProcessor(
+ arguments)
+ cdef grpc_channel_args *c_arguments = arguments_processor.c(&self._vtable)
+ self.c_server = grpc_server_create(c_arguments, NULL)
+ arguments_processor.un_c()
+ self.references.append(arguments)
self.is_started = False
self.is_shutting_down = False
self.is_shutdown = False
diff --git a/src/python/grpcio/grpc/_cython/cygrpc.pxd b/src/python/grpcio/grpc/_cython/cygrpc.pxd
index 01e2da6..b6a794c 100644
--- a/src/python/grpcio/grpc/_cython/cygrpc.pxd
+++ b/src/python/grpcio/grpc/_cython/cygrpc.pxd
@@ -14,6 +14,7 @@
include "_cygrpc/grpc.pxi"
+include "_cygrpc/arguments.pxd.pxi"
include "_cygrpc/call.pxd.pxi"
include "_cygrpc/channel.pxd.pxi"
include "_cygrpc/credentials.pxd.pxi"
diff --git a/src/python/grpcio/grpc/_cython/cygrpc.pyx b/src/python/grpcio/grpc/_cython/cygrpc.pyx
index d8ac84a..2ee2e6b 100644
--- a/src/python/grpcio/grpc/_cython/cygrpc.pyx
+++ b/src/python/grpcio/grpc/_cython/cygrpc.pyx
@@ -21,6 +21,7 @@
# TODO(atash): figure out why the coverage tool gets confused about the Cython
# coverage plugin when the following files don't have a '.pxi' suffix.
include "_cygrpc/grpc_string.pyx.pxi"
+include "_cygrpc/arguments.pyx.pxi"
include "_cygrpc/call.pyx.pxi"
include "_cygrpc/channel.pyx.pxi"
include "_cygrpc/credentials.pyx.pxi"
diff --git a/src/python/grpcio/grpc/_server.py b/src/python/grpcio/grpc/_server.py
index 56122fe..c988e0c 100644
--- a/src/python/grpcio/grpc/_server.py
+++ b/src/python/grpcio/grpc/_server.py
@@ -797,7 +797,7 @@
def __init__(self, thread_pool, generic_handlers, interceptors, options,
maximum_concurrent_rpcs):
completion_queue = cygrpc.CompletionQueue()
- server = cygrpc.Server(_common.channel_args(options))
+ server = cygrpc.Server(options)
server.register_completion_queue(completion_queue)
self._state = _ServerState(completion_queue, server, generic_handlers,
_interceptor.service_pipeline(interceptors),
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_cancel_many_calls_test.py b/src/python/grpcio_tests/tests/unit/_cython/_cancel_many_calls_test.py
index 2ca1fa8..3765ce4 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_cancel_many_calls_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_cancel_many_calls_test.py
@@ -141,13 +141,16 @@
test_constants.THREAD_CONCURRENCY)
server_completion_queue = cygrpc.CompletionQueue()
- server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ server = cygrpc.Server([
+ (
+ b'grpc.so_reuseport',
+ 0,
+ ),
+ ])
server.register_completion_queue(server_completion_queue)
port = server.add_http2_port(b'[::]:0')
server.start()
- channel = cygrpc.Channel('localhost:{}'.format(port).encode(),
- cygrpc.ChannelArgs([]))
+ channel = cygrpc.Channel('localhost:{}'.format(port).encode(), None)
state = _State()
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py b/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
index c22c77d..7305d0f 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_channel_test.py
@@ -22,7 +22,7 @@
def _channel_and_completion_queue():
- channel = cygrpc.Channel(b'localhost:54321', cygrpc.ChannelArgs(()))
+ channel = cygrpc.Channel(b'localhost:54321', ())
completion_queue = cygrpc.CompletionQueue()
return channel, completion_queue
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_common.py b/src/python/grpcio_tests/tests/unit/_cython/_common.py
index d4b01ca..7fd3d19 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_common.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_common.py
@@ -96,13 +96,11 @@
def setUp(self):
self.server_completion_queue = cygrpc.CompletionQueue()
- self.server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ self.server = cygrpc.Server([(b'grpc.so_reuseport', 0)])
self.server.register_completion_queue(self.server_completion_queue)
port = self.server.add_http2_port(b'[::]:0')
self.server.start()
- self.channel = cygrpc.Channel('localhost:{}'.format(port).encode(),
- cygrpc.ChannelArgs([]))
+ self.channel = cygrpc.Channel('localhost:{}'.format(port).encode(), [])
self._server_shutdown_tag = 'server_shutdown_tag'
self.server_condition = threading.Condition()
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_read_some_but_not_all_responses_test.py b/src/python/grpcio_tests/tests/unit/_cython/_read_some_but_not_all_responses_test.py
index ecd23af..bc63b54 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_read_some_but_not_all_responses_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_read_some_but_not_all_responses_test.py
@@ -111,13 +111,14 @@
def testReadSomeButNotAllResponses(self):
server_completion_queue = cygrpc.CompletionQueue()
- server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ server = cygrpc.Server([(
+ b'grpc.so_reuseport',
+ 0,
+ )])
server.register_completion_queue(server_completion_queue)
port = server.add_http2_port(b'[::]:0')
server.start()
- channel = cygrpc.Channel('localhost:{}'.format(port).encode(),
- cygrpc.ChannelArgs([]))
+ channel = cygrpc.Channel('localhost:{}'.format(port).encode(), set())
server_shutdown_tag = 'server_shutdown_tag'
server_driver = _ServerDriver(server_completion_queue,
diff --git a/src/python/grpcio_tests/tests/unit/_cython/_server_test.py b/src/python/grpcio_tests/tests/unit/_cython/_server_test.py
index 12bf40b..bbd2545 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/_server_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/_server_test.py
@@ -25,7 +25,7 @@
def test_lonely_server(self):
server_call_completion_queue = cygrpc.CompletionQueue()
server_shutdown_completion_queue = cygrpc.CompletionQueue()
- server = cygrpc.Server(cygrpc.ChannelArgs([]))
+ server = cygrpc.Server(None)
server.register_completion_queue(server_call_completion_queue)
server.register_completion_queue(server_shutdown_completion_queue)
port = server.add_http2_port(b'[::]:0')
diff --git a/src/python/grpcio_tests/tests/unit/_cython/cygrpc_test.py b/src/python/grpcio_tests/tests/unit/_cython/cygrpc_test.py
index 561adf7..9045ff5 100644
--- a/src/python/grpcio_tests/tests/unit/_cython/cygrpc_test.py
+++ b/src/python/grpcio_tests/tests/unit/_cython/cygrpc_test.py
@@ -42,12 +42,16 @@
del completion_queue
def testServerUpDown(self):
- server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ server = cygrpc.Server(set([
+ (
+ b'grpc.so_reuseport',
+ 0,
+ ),
+ ]))
del server
def testChannelUpDown(self):
- channel = cygrpc.Channel(b'[::]:0', cygrpc.ChannelArgs([]))
+ channel = cygrpc.Channel(b'[::]:0', None)
del channel
def test_metadata_plugin_call_credentials_up_down(self):
@@ -55,8 +59,12 @@
b'test plugin name!')
def testServerStartNoExplicitShutdown(self):
- server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ server = cygrpc.Server([
+ (
+ b'grpc.so_reuseport',
+ 0,
+ ),
+ ])
completion_queue = cygrpc.CompletionQueue()
server.register_completion_queue(completion_queue)
port = server.add_http2_port(b'[::]:0')
@@ -66,8 +74,12 @@
def testServerStartShutdown(self):
completion_queue = cygrpc.CompletionQueue()
- server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ server = cygrpc.Server([
+ (
+ b'grpc.so_reuseport',
+ 0,
+ ),
+ ])
server.add_http2_port(b'[::]:0')
server.register_completion_queue(completion_queue)
server.start()
@@ -85,8 +97,12 @@
def setUpMixin(self, server_credentials, client_credentials, host_override):
self.server_completion_queue = cygrpc.CompletionQueue()
- self.server = cygrpc.Server(
- cygrpc.ChannelArgs([cygrpc.ChannelArg(b'grpc.so_reuseport', 0)]))
+ self.server = cygrpc.Server([
+ (
+ b'grpc.so_reuseport',
+ 0,
+ ),
+ ])
self.server.register_completion_queue(self.server_completion_queue)
if server_credentials:
self.port = self.server.add_http2_port(b'[::]:0',
@@ -96,16 +112,16 @@
self.server.start()
self.client_completion_queue = cygrpc.CompletionQueue()
if client_credentials:
- client_channel_arguments = cygrpc.ChannelArgs([
- cygrpc.ChannelArg(cygrpc.ChannelArgKey.ssl_target_name_override,
- host_override)
- ])
+ client_channel_arguments = ((
+ cygrpc.ChannelArgKey.ssl_target_name_override,
+ host_override,
+ ),)
self.client_channel = cygrpc.Channel('localhost:{}'.format(
self.port).encode(), client_channel_arguments,
client_credentials)
else:
self.client_channel = cygrpc.Channel('localhost:{}'.format(
- self.port).encode(), cygrpc.ChannelArgs([]))
+ self.port).encode(), set())
if host_override:
self.host_argument = None # default host
self.expected_host = host_override
diff --git a/src/python/grpcio_tests/tests/unit/_metadata_test.py b/src/python/grpcio_tests/tests/unit/_metadata_test.py
index a918066..5908421 100644
--- a/src/python/grpcio_tests/tests/unit/_metadata_test.py
+++ b/src/python/grpcio_tests/tests/unit/_metadata_test.py
@@ -80,7 +80,7 @@
_EXPECTED_TRAILING_METADATA = _TRAILING_METADATA
-def user_agent(metadata):
+def _user_agent(metadata):
for key, val in metadata:
if key == 'user-agent':
return val
@@ -88,16 +88,14 @@
def validate_client_metadata(test, servicer_context):
+ invocation_metadata = servicer_context.invocation_metadata()
test.assertTrue(
- test_common.metadata_transmitted(
- _EXPECTED_INVOCATION_METADATA,
- servicer_context.invocation_metadata()))
+ test_common.metadata_transmitted(_EXPECTED_INVOCATION_METADATA,
+ invocation_metadata))
+ user_agent = _user_agent(invocation_metadata)
test.assertTrue(
- user_agent(servicer_context.invocation_metadata())
- .startswith('primary-agent ' + _channel._USER_AGENT))
- test.assertTrue(
- user_agent(servicer_context.invocation_metadata())
- .endswith('secondary-agent'))
+ user_agent.startswith('primary-agent ' + _channel._USER_AGENT))
+ test.assertTrue(user_agent.endswith('secondary-agent'))
def handle_unary_unary(test, request, servicer_context):
diff --git a/test/core/end2end/h2_ssl_cert_test.cc b/test/core/end2end/h2_ssl_cert_test.cc
index 3383d6d..9adb96e 100644
--- a/test/core/end2end/h2_ssl_cert_test.cc
+++ b/test/core/end2end/h2_ssl_cert_test.cc
@@ -272,6 +272,20 @@
static void shutdown_server(grpc_end2end_test_fixture* f) {
if (!f->server) return;
+ /* Perform a completion queue next, so that any pending operations can be
+ * finished, and resources can be released. This is so that, shutdown does not
+ * hang. For example, the server might be stuck in the handshaking code, which
+ * keeps a ref to a listener. Unless, it is unref'd, shutdown won't be able
+ * to proceed.
+ *
+ * (If shutdown times out, it is probably because 100ms wasn't enough. In that
+ * case, the deadline can be increased. Or, we could simply have another
+ * thread for the server to poll the completion queue while the shutdown
+ * progresses.)
+ */
+ GPR_ASSERT(grpc_completion_queue_next(
+ f->cq, grpc_timeout_milliseconds_to_deadline(100), nullptr)
+ .type == GRPC_QUEUE_TIMEOUT);
grpc_server_shutdown_and_notify(f->server, f->shutdown_cq, tag(1000));
GPR_ASSERT(grpc_completion_queue_pluck(f->shutdown_cq, tag(1000),
grpc_timeout_seconds_to_deadline(5),
@@ -288,8 +302,8 @@
}
static void end_test(grpc_end2end_test_fixture* f) {
- shutdown_server(f);
shutdown_client(f);
+ shutdown_server(f);
grpc_completion_queue_shutdown(f->cq);
drain_cq(f->cq);
diff --git a/test/core/tsi/ssl_transport_security_test.cc b/test/core/tsi/ssl_transport_security_test.cc
index 8939c04..bf54383 100644
--- a/test/core/tsi/ssl_transport_security_test.cc
+++ b/test/core/tsi/ssl_transport_security_test.cc
@@ -667,9 +667,15 @@
ssl_tsi_test_do_handshake_with_server_name_indication_wild_star_domain();
ssl_tsi_test_do_handshake_with_bad_server_cert();
ssl_tsi_test_do_handshake_with_bad_client_cert();
- ssl_tsi_test_do_handshake_alpn_client_no_server();
+ // TODO: BoringSSL and OpenSSL have different behaviors on handling mismatched
+ // ALPN. Re-enable this test if we can detect in the runtime which SSL library
+ // is used.
+ // ssl_tsi_test_do_handshake_alpn_client_no_server();
ssl_tsi_test_do_handshake_alpn_server_no_client();
- ssl_tsi_test_do_handshake_alpn_client_server_mismatch();
+ // TODO: BoringSSL and OpenSSL have different behaviors on handling mismatched
+ // ALPN. Re-enable this test if we can detect in the runtime which SSL library
+ // is used.
+ // ssl_tsi_test_do_handshake_alpn_client_server_mismatch();
ssl_tsi_test_do_handshake_alpn_client_server_ok();
ssl_tsi_test_do_round_trip_for_all_configs();
ssl_tsi_test_do_round_trip_odd_buffer_size();
diff --git a/test/cpp/end2end/end2end_test.cc b/test/cpp/end2end/end2end_test.cc
index 28887f8..2edd108 100644
--- a/test/cpp/end2end/end2end_test.cc
+++ b/test/cpp/end2end/end2end_test.cc
@@ -706,7 +706,6 @@
if (GetParam().inproc) {
return;
}
- gpr_setenv("GRPC_CLIENT_CHANNEL_BACKUP_POLL_INTERVAL_MS", "200");
int poller_slowdown_factor = 1;
// It needs 2 pollset_works to reconnect the channel with polling engine
// "poll"
@@ -1828,6 +1827,7 @@
} // namespace grpc
int main(int argc, char** argv) {
+ gpr_setenv("GRPC_CLIENT_CHANNEL_BACKUP_POLL_INTERVAL_MS", "200");
grpc_test_init(argc, argv);
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
diff --git a/tools/distrib/python/grpcio_tools/README.rst b/tools/distrib/python/grpcio_tools/README.rst
index fb44cfa..32873b2 100644
--- a/tools/distrib/python/grpcio_tools/README.rst
+++ b/tools/distrib/python/grpcio_tools/README.rst
@@ -61,7 +61,7 @@
$ python ../make_grpcio_tools.py
# For the next command do `sudo pip install` if you get permission-denied errors
- $ pip install .
+ $ GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install .
You cannot currently install Python from source on Windows. Things might work
out for you in MSYS2 (follow the Linux instructions), but it isn't officially
diff --git a/tools/run_tests/dockerize/build_and_run_docker.sh b/tools/run_tests/dockerize/build_and_run_docker.sh
index 323c2f7..b8f0a55 100755
--- a/tools/run_tests/dockerize/build_and_run_docker.sh
+++ b/tools/run_tests/dockerize/build_and_run_docker.sh
@@ -18,7 +18,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
git_root=$(pwd)
cd -
@@ -31,36 +31,38 @@
# $@ - Extra args to pass to docker run
# Use image name based on Dockerfile location checksum
-DOCKER_IMAGE_NAME=$(basename $DOCKERFILE_DIR)_$(sha1sum $DOCKERFILE_DIR/Dockerfile | cut -f1 -d\ )
+DOCKER_IMAGE_NAME=$(basename "$DOCKERFILE_DIR")_$(sha1sum "$DOCKERFILE_DIR/Dockerfile" | cut -f1 -d\ )
# Pull the base image to force an update
if [ "$DOCKER_BASE_IMAGE" != "" ]
then
- time docker pull $DOCKER_BASE_IMAGE
+ time docker pull "$DOCKER_BASE_IMAGE"
fi
if [ "$DOCKERHUB_ORGANIZATION" != "" ]
then
DOCKER_IMAGE_NAME=$DOCKERHUB_ORGANIZATION/$DOCKER_IMAGE_NAME
- time docker pull $DOCKER_IMAGE_NAME
+ time docker pull "$DOCKER_IMAGE_NAME"
else
# Make sure docker image has been built. Should be instantaneous if so.
- docker build -t $DOCKER_IMAGE_NAME $DOCKERFILE_DIR
+ docker build -t "$DOCKER_IMAGE_NAME" "$DOCKERFILE_DIR"
fi
# Choose random name for docker container
CONTAINER_NAME="build_and_run_docker_$(uuidgen)"
# Run command inside docker
+# TODO: use a proper array instead of $EXTRA_DOCKER_ARGS
+# shellcheck disable=SC2086
docker run \
"$@" \
-e EXTERNAL_GIT_ROOT="/var/local/jenkins/grpc" \
-e THIS_IS_REALLY_NEEDED='see https://github.com/docker/docker/issues/14203 for why docker is awful' \
-v "$git_root:/var/local/jenkins/grpc:ro" \
-w /var/local/git/grpc \
- --name=$CONTAINER_NAME \
+ --name="$CONTAINER_NAME" \
$EXTRA_DOCKER_ARGS \
- $DOCKER_IMAGE_NAME \
+ "$DOCKER_IMAGE_NAME" \
/bin/bash -l "/var/local/jenkins/grpc/$DOCKER_RUN_SCRIPT" || FAILED="true"
# Copy output artifacts
@@ -70,7 +72,7 @@
fi
# remove the container, possibly killing it first
-docker rm -f $CONTAINER_NAME || true
+docker rm -f "$CONTAINER_NAME" || true
if [ "$FAILED" != "" ]
then
diff --git a/tools/run_tests/dockerize/build_docker_and_run_tests.sh b/tools/run_tests/dockerize/build_docker_and_run_tests.sh
index 32de3fa..8dca05e 100755
--- a/tools/run_tests/dockerize/build_docker_and_run_tests.sh
+++ b/tools/run_tests/dockerize/build_docker_and_run_tests.sh
@@ -18,7 +18,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
git_root=$(pwd)
cd -
@@ -35,15 +35,15 @@
# DOCKERHUB_ORGANIZATION - If set, pull a prebuilt image from given dockerhub org.
# Use image name based on Dockerfile location checksum
-DOCKER_IMAGE_NAME=$(basename $DOCKERFILE_DIR)_$(sha1sum $DOCKERFILE_DIR/Dockerfile | cut -f1 -d\ )
+DOCKER_IMAGE_NAME=$(basename "$DOCKERFILE_DIR")_$(sha1sum "$DOCKERFILE_DIR/Dockerfile" | cut -f1 -d\ )
if [ "$DOCKERHUB_ORGANIZATION" != "" ]
then
DOCKER_IMAGE_NAME=$DOCKERHUB_ORGANIZATION/$DOCKER_IMAGE_NAME
- time docker pull $DOCKER_IMAGE_NAME
+ time docker pull "$DOCKER_IMAGE_NAME"
else
# Make sure docker image has been built. Should be instantaneous if so.
- docker build -t $DOCKER_IMAGE_NAME $DOCKERFILE_DIR
+ docker build -t "$DOCKER_IMAGE_NAME" "$DOCKERFILE_DIR"
fi
# Choose random name for docker container
@@ -54,6 +54,8 @@
# Run tests inside docker
DOCKER_EXIT_CODE=0
+# TODO: silence complaint about $TTY_FLAG expansion in some other way
+# shellcheck disable=SC2086
docker run \
-e "RUN_TESTS_COMMAND=$RUN_TESTS_COMMAND" \
-e "config=$config" \
@@ -61,7 +63,7 @@
-e CCACHE_DIR=/tmp/ccache \
-e XDG_CACHE_HOME=/tmp/xdg-cache-home \
-e THIS_IS_REALLY_NEEDED='see https://github.com/docker/docker/issues/14203 for why docker is awful' \
- -e HOST_GIT_ROOT=$git_root \
+ -e HOST_GIT_ROOT="$git_root" \
-e LOCAL_GIT_ROOT=$docker_instance_git_root \
-e "BUILD_ID=$BUILD_ID" \
-e "BUILD_URL=$BUILD_URL" \
@@ -70,7 +72,8 @@
-e "KOKORO_BUILD_NUMBER=$KOKORO_BUILD_NUMBER" \
-e "KOKORO_BUILD_URL=$KOKORO_BUILD_URL" \
-e "KOKORO_JOB_NAME=$KOKORO_JOB_NAME" \
- -i $TTY_FLAG \
+ -i \
+ $TTY_FLAG \
--sysctl net.ipv6.conf.all.disable_ipv6=0 \
-v ~/.config/gcloud:/root/.config/gcloud \
-v "$git_root:$docker_instance_git_root" \
@@ -78,18 +81,18 @@
-v /tmp/npm-cache:/tmp/npm-cache \
-v /tmp/xdg-cache-home:/tmp/xdg-cache-home \
-w /var/local/git/grpc \
- --name=$CONTAINER_NAME \
- $DOCKER_IMAGE_NAME \
+ --name="$CONTAINER_NAME" \
+ "$DOCKER_IMAGE_NAME" \
bash -l "/var/local/jenkins/grpc/$DOCKER_RUN_SCRIPT" || DOCKER_EXIT_CODE=$?
# use unique name for reports.zip to prevent clash between concurrent
# run_tests.py runs
-TEMP_REPORTS_ZIP=`mktemp`
-docker cp "$CONTAINER_NAME:/var/local/git/grpc/reports.zip" ${TEMP_REPORTS_ZIP} || true
-unzip -o ${TEMP_REPORTS_ZIP} -d $git_root || true
-rm -f ${TEMP_REPORTS_ZIP}
+TEMP_REPORTS_ZIP=$(mktemp)
+docker cp "$CONTAINER_NAME:/var/local/git/grpc/reports.zip" "${TEMP_REPORTS_ZIP}" || true
+unzip -o "${TEMP_REPORTS_ZIP}" -d "$git_root" || true
+rm -f "${TEMP_REPORTS_ZIP}"
# remove the container, possibly killing it first
-docker rm -f $CONTAINER_NAME || true
+docker rm -f "$CONTAINER_NAME" || true
exit $DOCKER_EXIT_CODE
diff --git a/tools/run_tests/dockerize/build_interop_image.sh b/tools/run_tests/dockerize/build_interop_image.sh
index dbc6bde..90605d9 100755
--- a/tools/run_tests/dockerize/build_interop_image.sh
+++ b/tools/run_tests/dockerize/build_interop_image.sh
@@ -28,7 +28,7 @@
# GRPC_GO_ROOT - grpc-go base directory, default to '$GRPC_ROOT/../grpc-go'
# GRPC_JAVA_ROOT - grpc-java base directory, default to '$GRPC_ROOT/../grpc-java'
-cd `dirname $0`/../../..
+cd "$(dirname "$0")/../../.."
echo "GRPC_ROOT: ${GRPC_ROOT:=$(pwd)}"
MOUNT_ARGS="-v $GRPC_ROOT:/var/local/jenkins/grpc:ro"
@@ -61,7 +61,7 @@
# Mount service account dir if available.
# If service_directory does not contain the service account JSON file,
# some of the tests will fail.
-if [ -e $HOME/service_account ]
+if [ -e "$HOME/service_account" ]
then
MOUNT_ARGS+=" -v $HOME/service_account:/var/local/jenkins/service_account:ro"
fi
@@ -70,18 +70,18 @@
# on OSX use md5 instead of sha1sum
if which sha1sum > /dev/null;
then
- BASE_IMAGE=${BASE_NAME}_`sha1sum tools/dockerfile/interoptest/$BASE_NAME/Dockerfile | cut -f1 -d\ `
+ BASE_IMAGE=${BASE_NAME}_$(sha1sum "tools/dockerfile/interoptest/$BASE_NAME/Dockerfile" | cut -f1 -d\ )
else
- BASE_IMAGE=${BASE_NAME}_`md5 -r tools/dockerfile/interoptest/$BASE_NAME/Dockerfile | cut -f1 -d\ `
+ BASE_IMAGE=${BASE_NAME}_$(md5 -r "tools/dockerfile/interoptest/$BASE_NAME/Dockerfile" | cut -f1 -d\ )
fi
if [ "$DOCKERHUB_ORGANIZATION" != "" ]
then
BASE_IMAGE=$DOCKERHUB_ORGANIZATION/$BASE_IMAGE
- time docker pull $BASE_IMAGE
+ time docker pull "$BASE_IMAGE"
else
# Make sure docker image has been built. Should be instantaneous if so.
- docker build -t $BASE_IMAGE --force-rm=true tools/dockerfile/interoptest/$BASE_NAME || exit $?
+ docker build -t "$BASE_IMAGE" --force-rm=true "tools/dockerfile/interoptest/$BASE_NAME" || exit $?
fi
# Create a local branch so the child Docker script won't complain
@@ -90,22 +90,28 @@
CONTAINER_NAME="build_${BASE_NAME}_$(uuidgen)"
# Prepare image for interop tests, commit it on success.
+# TODO: Figure out if is safe to eliminate the suppression. It's currently here
+# because $MOUNT_ARGS and $BUILD_INTEROP_DOCKER_EXTRA_ARGS can have legitimate
+# spaces, but the "correct" way to do this is to utilize proper arrays.
+# Same for $TTY_FLAG
+# shellcheck disable=SC2086
(docker run \
-e CCACHE_DIR=/tmp/ccache \
-e THIS_IS_REALLY_NEEDED='see https://github.com/docker/docker/issues/14203 for why docker is awful' \
-e THIS_IS_REALLY_NEEDED_ONCE_AGAIN='For issue 4835. See https://github.com/docker/docker/issues/14203 for why docker is awful' \
- -i $TTY_FLAG \
+ -i \
+ $TTY_FLAG \
$MOUNT_ARGS \
$BUILD_INTEROP_DOCKER_EXTRA_ARGS \
-v /tmp/ccache:/tmp/ccache \
- --name=$CONTAINER_NAME \
- $BASE_IMAGE \
- bash -l /var/local/jenkins/grpc/tools/dockerfile/interoptest/$BASE_NAME/build_interop.sh \
- && docker commit $CONTAINER_NAME $INTEROP_IMAGE \
+ --name="$CONTAINER_NAME" \
+ "$BASE_IMAGE" \
+ bash -l "/var/local/jenkins/grpc/tools/dockerfile/interoptest/$BASE_NAME/build_interop.sh" \
+ && docker commit "$CONTAINER_NAME" "$INTEROP_IMAGE" \
&& echo "Successfully built image $INTEROP_IMAGE")
EXITCODE=$?
# remove intermediate container, possibly killing it first
-docker rm -f $CONTAINER_NAME
+docker rm -f "$CONTAINER_NAME"
exit $EXITCODE
diff --git a/tools/run_tests/dockerize/docker_run.sh b/tools/run_tests/dockerize/docker_run.sh
index 992e4f2..ac0d09c 100755
--- a/tools/run_tests/dockerize/docker_run.sh
+++ b/tools/run_tests/dockerize/docker_run.sh
@@ -21,9 +21,11 @@
if [ "$RELATIVE_COPY_PATH" == "" ]
then
mkdir -p /var/local/git
- git clone $EXTERNAL_GIT_ROOT /var/local/git/grpc
+ git clone "$EXTERNAL_GIT_ROOT" /var/local/git/grpc
# clone gRPC submodules, use data from locally cloned submodules where possible
- (cd ${EXTERNAL_GIT_ROOT} && git submodule foreach 'cd /var/local/git/grpc \
+ # TODO: figure out a way to eliminate this following shellcheck suppressions
+ # shellcheck disable=SC2016,SC1004
+ (cd "${EXTERNAL_GIT_ROOT}" && git submodule foreach 'cd /var/local/git/grpc \
&& git submodule update --init --reference ${EXTERNAL_GIT_ROOT}/${name} \
${name}')
else
diff --git a/tools/run_tests/dockerize/docker_run_tests.sh b/tools/run_tests/dockerize/docker_run_tests.sh
index 15d7039..df9d3d1 100755
--- a/tools/run_tests/dockerize/docker_run_tests.sh
+++ b/tools/run_tests/dockerize/docker_run_tests.sh
@@ -24,11 +24,13 @@
# Ensure that programs depending on current-user-ownership of cache directories
# are satisfied (it's being mounted from outside the image).
-chown $(whoami) $XDG_CACHE_HOME
+chown "$(whoami)" "$XDG_CACHE_HOME"
mkdir -p /var/local/git
git clone /var/local/jenkins/grpc /var/local/git/grpc
# clone gRPC submodules, use data from locally cloned submodules where possible
+# TODO: figure out a way to eliminate this shellcheck suppression:
+# shellcheck disable=SC2016,SC1004
(cd /var/local/jenkins/grpc/ && git submodule foreach 'cd /var/local/git/grpc \
&& git submodule update --init --reference /var/local/jenkins/grpc/${name} \
${name}')
@@ -52,8 +54,8 @@
cd ..
zip -r reports.zip reports
-find . -name report.xml | xargs -r zip reports.zip
-find . -name sponge_log.xml | xargs -r zip reports.zip
-find . -name 'report_*.xml' | xargs -r zip reports.zip
+find . -name report.xml -print0 | xargs -0 -r zip reports.zip
+find . -name sponge_log.xml -print0 | xargs -0 -r zip reports.zip
+find . -name 'report_*.xml' -print0 | xargs -0 -r zip reports.zip
exit $exit_code
diff --git a/tools/run_tests/interop/android/android_interop_helper.sh b/tools/run_tests/interop/android/android_interop_helper.sh
index 9c0d18f..116549b 100755
--- a/tools/run_tests/interop/android/android_interop_helper.sh
+++ b/tools/run_tests/interop/android/android_interop_helper.sh
@@ -18,7 +18,7 @@
SERVICE_KEY=$1
-gcloud auth activate-service-account --key-file=$SERVICE_KEY || exit 1
+gcloud auth activate-service-account --key-file="$SERVICE_KEY" || exit 1
gcloud config set project grpc-testing || exit 1
rm -rf grpc-java
diff --git a/tools/run_tests/interop/android/run_android_tests_on_firebase.sh b/tools/run_tests/interop/android/run_android_tests_on_firebase.sh
index 0b48113..f6472eb 100755
--- a/tools/run_tests/interop/android/run_android_tests_on_firebase.sh
+++ b/tools/run_tests/interop/android/run_android_tests_on_firebase.sh
@@ -21,7 +21,7 @@
SERVICE_KEY=~/android-interops-service-key.json
HELPER=$(pwd)/tools/run_tests/interop/android/android_interop_helper.sh
-docker build -t $DOCKER_TAG -f $DOCKERFILE .
+docker build -t "$DOCKER_TAG" -f "$DOCKERFILE" .
docker run --interactive --rm \
--volume="$SERVICE_KEY":/service-key.json:ro \
diff --git a/tools/run_tests/interop/with_nvm.sh b/tools/run_tests/interop/with_nvm.sh
index a1c3f3e..887f9f6 100755
--- a/tools/run_tests/interop/with_nvm.sh
+++ b/tools/run_tests/interop/with_nvm.sh
@@ -16,4 +16,4 @@
# Makes sure NVM is loaded before executing the command passed as an argument
source ~/.nvm/nvm.sh
-$@
+"$@"
diff --git a/tools/run_tests/interop/with_rvm.sh b/tools/run_tests/interop/with_rvm.sh
index c3e2882..41e6efc 100755
--- a/tools/run_tests/interop/with_rvm.sh
+++ b/tools/run_tests/interop/with_rvm.sh
@@ -16,4 +16,4 @@
# Makes sure RVM is loaded before executing the command passed as an argument
source /usr/local/rvm/scripts/rvm
-$@
+"$@"
diff --git a/tools/run_tests/performance/build_performance.sh b/tools/run_tests/performance/build_performance.sh
index e7d8db8..22e0ca9 100755
--- a/tools/run_tests/performance/build_performance.sh
+++ b/tools/run_tests/performance/build_performance.sh
@@ -16,7 +16,7 @@
source ~/.rvm/scripts/rvm
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
CONFIG=${CONFIG:-opt}
@@ -28,11 +28,11 @@
# TODO(jtattermusch): not embedding OpenSSL breaks the C# build because
# grpc_csharp_ext needs OpenSSL embedded and some intermediate files from
# this build will be reused.
- make CONFIG=${CONFIG} EMBED_OPENSSL=true EMBED_ZLIB=true qps_worker qps_json_driver -j8
+ make CONFIG="${CONFIG}" EMBED_OPENSSL=true EMBED_ZLIB=true qps_worker qps_json_driver -j8
fi
PHP_ALREADY_BUILT=""
-for language in $@
+for language in "$@"
do
case "$language" in
"c++")
@@ -53,10 +53,10 @@
fi
;;
"csharp")
- python tools/run_tests/run_tests.py -l $language -c $CONFIG --build_only -j 8 --compiler coreclr
+ python tools/run_tests/run_tests.py -l "$language" -c "$CONFIG" --build_only -j 8 --compiler coreclr
;;
*)
- python tools/run_tests/run_tests.py -l $language -c $CONFIG --build_only -j 8
+ python tools/run_tests/run_tests.py -l "$language" -c "$CONFIG" --build_only -j 8
;;
esac
done
diff --git a/tools/run_tests/performance/build_performance_go.sh b/tools/run_tests/performance/build_performance_go.sh
index 5e97fb6..812728d 100755
--- a/tools/run_tests/performance/build_performance_go.sh
+++ b/tools/run_tests/performance/build_performance_go.sh
@@ -15,7 +15,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
export GOPATH=$(pwd)/../gopath
@@ -24,6 +24,6 @@
rm -rf "${GOPATH}/src/google.golang.org/grpc"
# Get the revision of grpc-go we want to test
-git clone --recursive ../grpc-go ${GOPATH}/src/google.golang.org/grpc
+git clone --recursive ../grpc-go "${GOPATH}/src/google.golang.org/grpc"
-(cd ${GOPATH}/src/google.golang.org/grpc/benchmark/worker && go install)
+(cd "${GOPATH}/src/google.golang.org/grpc/benchmark/worker" && go install)
diff --git a/tools/run_tests/performance/build_performance_php7.sh b/tools/run_tests/performance/build_performance_php7.sh
index 141c9fd..37ca9ee 100755
--- a/tools/run_tests/performance/build_performance_php7.sh
+++ b/tools/run_tests/performance/build_performance_php7.sh
@@ -15,9 +15,9 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
CONFIG=${CONFIG:-opt}
-python tools/run_tests/run_tests.py -l php7 -c $CONFIG --build_only -j 8
+python tools/run_tests/run_tests.py -l php7 -c "$CONFIG" --build_only -j 8
# Set up all dependences needed for PHP QPS test
cd src/php/tests/qps
diff --git a/tools/run_tests/performance/kill_workers.sh b/tools/run_tests/performance/kill_workers.sh
index efe7282..95a5bf5 100755
--- a/tools/run_tests/performance/kill_workers.sh
+++ b/tools/run_tests/performance/kill_workers.sh
@@ -15,7 +15,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
# Make sure there are no pre-existing QPS workers around before starting
# the performance test suite
@@ -24,13 +24,17 @@
killall -9 qps_worker || true
# C#
+# shellcheck disable=SC2009
ps -C mono -o pid=,cmd= | grep QpsWorker | awk '{print $1}' | xargs kill -9 || true
+# shellcheck disable=SC2009
ps -C dotnet -o pid=,cmd= | grep QpsWorker | awk '{print $1}' | xargs kill -9 || true
# Ruby
+# shellcheck disable=SC2009
ps -C ruby -o pid=,cmd= | grep 'qps/worker.rb' | awk '{print $1}' | xargs kill -9 || true
# Python
+# shellcheck disable=SC2009
ps -C python -o pid=,cmd= | grep 'qps_worker.py' | awk '{print $1}' | xargs kill -9 || true
# Java
diff --git a/tools/run_tests/performance/process_local_perf_flamegraphs.sh b/tools/run_tests/performance/process_local_perf_flamegraphs.sh
index d0c22f2..ccb5b19 100755
--- a/tools/run_tests/performance/process_local_perf_flamegraphs.sh
+++ b/tools/run_tests/performance/process_local_perf_flamegraphs.sh
@@ -13,13 +13,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-mkdir -p $OUTPUT_DIR
+mkdir -p "$OUTPUT_DIR"
-PERF_DATA_FILE=${PERF_BASE_NAME}-perf.data
-PERF_SCRIPT_OUTPUT=${PERF_BASE_NAME}-out.perf
+PERF_DATA_FILE="${PERF_BASE_NAME}-perf.data"
+PERF_SCRIPT_OUTPUT="${PERF_BASE_NAME}-out.perf"
# Generate Flame graphs
echo "running perf script on $PERF_DATA_FILE"
-perf script -i $PERF_DATA_FILE > $PERF_SCRIPT_OUTPUT
+perf script -i "$PERF_DATA_FILE" > "$PERF_SCRIPT_OUTPUT"
-~/FlameGraph/stackcollapse-perf.pl $PERF_SCRIPT_OUTPUT | ~/FlameGraph/flamegraph.pl > ${OUTPUT_DIR}/${OUTPUT_FILENAME}.svg
+~/FlameGraph/stackcollapse-perf.pl "$PERF_SCRIPT_OUTPUT" | ~/FlameGraph/flamegraph.pl > "${OUTPUT_DIR}/${OUTPUT_FILENAME}.svg"
diff --git a/tools/run_tests/performance/process_remote_perf_flamegraphs.sh b/tools/run_tests/performance/process_remote_perf_flamegraphs.sh
index a0c4f6f..2ea6b4f 100755
--- a/tools/run_tests/performance/process_remote_perf_flamegraphs.sh
+++ b/tools/run_tests/performance/process_remote_perf_flamegraphs.sh
@@ -13,17 +13,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-mkdir -p $OUTPUT_DIR
+mkdir -p "$OUTPUT_DIR"
-PERF_DATA_FILE=${PERF_BASE_NAME}-perf.data
-PERF_SCRIPT_OUTPUT=${PERF_BASE_NAME}-out.perf
+PERF_DATA_FILE="${PERF_BASE_NAME}-perf.data"
+PERF_SCRIPT_OUTPUT="${PERF_BASE_NAME}-out.perf"
# Generate Flame graphs
echo "running perf script on $USER_AT_HOST with perf.data"
-ssh $USER_AT_HOST "cd ~/performance_workspace/grpc && perf script -i $PERF_DATA_FILE | gzip > ${PERF_SCRIPT_OUTPUT}.gz"
+# shellcheck disable=SC2029
+ssh "$USER_AT_HOST" "cd ~/performance_workspace/grpc && perf script -i $PERF_DATA_FILE | gzip > ${PERF_SCRIPT_OUTPUT}.gz"
-scp $USER_AT_HOST:~/performance_workspace/grpc/$PERF_SCRIPT_OUTPUT.gz .
+scp "$USER_AT_HOST:~/performance_workspace/grpc/$PERF_SCRIPT_OUTPUT.gz" .
-gzip -d -f $PERF_SCRIPT_OUTPUT.gz
+gzip -d -f "$PERF_SCRIPT_OUTPUT.gz"
-~/FlameGraph/stackcollapse-perf.pl --kernel $PERF_SCRIPT_OUTPUT | ~/FlameGraph/flamegraph.pl --color=java --hash > ${OUTPUT_DIR}/${OUTPUT_FILENAME}.svg
+~/FlameGraph/stackcollapse-perf.pl --kernel "$PERF_SCRIPT_OUTPUT" | ~/FlameGraph/flamegraph.pl --color=java --hash > "${OUTPUT_DIR}/${OUTPUT_FILENAME}.svg"
diff --git a/tools/run_tests/performance/remote_host_build.sh b/tools/run_tests/performance/remote_host_build.sh
index 75352e9..862bd6c 100755
--- a/tools/run_tests/performance/remote_host_build.sh
+++ b/tools/run_tests/performance/remote_host_build.sh
@@ -15,7 +15,8 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
# execute the build script remotely
+# shellcheck disable=SC2029
ssh "${USER_AT_HOST}" "CONFIG=${CONFIG} ~/performance_workspace/grpc/tools/run_tests/performance/build_performance.sh $*"
diff --git a/tools/run_tests/performance/remote_host_prepare.sh b/tools/run_tests/performance/remote_host_prepare.sh
index bf91acb..d69c85b 100755
--- a/tools/run_tests/performance/remote_host_prepare.sh
+++ b/tools/run_tests/performance/remote_host_prepare.sh
@@ -15,7 +15,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
# TODO(jtattermusch): To be sure there are no running processes that would
# mess with the results, be rough and reboot the slave here
@@ -24,7 +24,7 @@
# On Windows, killall is not supported & we need to kill all pending workers
# before attempting to delete the workspace
-ssh "${USER_AT_HOST}" "ps -e | egrep 'qps_worker|dotnet' | awk '{print $1}' | xargs kill -9 || true"
+ssh "${USER_AT_HOST}" "ps -e | egrep 'qps_worker|dotnet' | awk '{print \$1}' | xargs kill -9 || true"
# cleanup after previous builds
ssh "${USER_AT_HOST}" "rm -rf ~/performance_workspace && mkdir -p ~/performance_workspace"
@@ -36,4 +36,5 @@
ssh "${USER_AT_HOST}" "tar -xf ~/performance_workspace/grpc.tar -C ~/performance_workspace || tar -xf ~/performance_workspace/grpc.tar -C ~/performance_workspace"
# For consistency with local run, invoke the kill_workers script remotely.
+# shellcheck disable=SC2088
ssh "${USER_AT_HOST}" "~/performance_workspace/grpc/tools/run_tests/performance/kill_workers.sh"
diff --git a/tools/run_tests/performance/run_netperf.sh b/tools/run_tests/performance/run_netperf.sh
index b415ede..2a32051 100755
--- a/tools/run_tests/performance/run_netperf.sh
+++ b/tools/run_tests/performance/run_netperf.sh
@@ -15,7 +15,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
netperf >netperf_latency.txt -P 0 -t TCP_RR -H "$NETPERF_SERVER_HOST" -- -r 1,1 -o P50_LATENCY,P90_LATENCY,P99_LATENCY
diff --git a/tools/run_tests/performance/run_qps_driver.sh b/tools/run_tests/performance/run_qps_driver.sh
index 1851632..2d9e310 100755
--- a/tools/run_tests/performance/run_qps_driver.sh
+++ b/tools/run_tests/performance/run_qps_driver.sh
@@ -15,7 +15,7 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
bins/opt/qps_json_driver "$@"
diff --git a/tools/run_tests/performance/run_worker_csharp.sh b/tools/run_tests/performance/run_worker_csharp.sh
index e2e941f..6546d60 100755
--- a/tools/run_tests/performance/run_worker_csharp.sh
+++ b/tools/run_tests/performance/run_worker_csharp.sh
@@ -15,9 +15,9 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
# needed to correctly locate testca
cd src/csharp/Grpc.IntegrationTesting.QpsWorker/bin/Release/netcoreapp1.0
-dotnet exec Grpc.IntegrationTesting.QpsWorker.dll $@
+dotnet exec Grpc.IntegrationTesting.QpsWorker.dll "$@"
diff --git a/tools/run_tests/performance/run_worker_go.sh b/tools/run_tests/performance/run_worker_go.sh
index 6d8abb4..f8e821a 100755
--- a/tools/run_tests/performance/run_worker_go.sh
+++ b/tools/run_tests/performance/run_worker_go.sh
@@ -15,8 +15,8 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
export GOPATH=$(pwd)/../gopath
-${GOPATH}/bin/worker $@
+"${GOPATH}/bin/worker" "$@"
diff --git a/tools/run_tests/performance/run_worker_java.sh b/tools/run_tests/performance/run_worker_java.sh
index 039a961..cff6faf 100755
--- a/tools/run_tests/performance/run_worker_java.sh
+++ b/tools/run_tests/performance/run_worker_java.sh
@@ -16,9 +16,9 @@
set -ex
# Enter repo root
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
# Enter the grpc-java repo root (expected to be next to grpc repo root)
cd ../grpc-java
-benchmarks/build/install/grpc-benchmarks/bin/benchmark_worker $@
+benchmarks/build/install/grpc-benchmarks/bin/benchmark_worker "$@"
diff --git a/tools/run_tests/performance/run_worker_php.sh b/tools/run_tests/performance/run_worker_php.sh
index 8c7ceef..2fe2493 100755
--- a/tools/run_tests/performance/run_worker_php.sh
+++ b/tools/run_tests/performance/run_worker_php.sh
@@ -16,8 +16,8 @@
source ~/.rvm/scripts/rvm
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
# The proxy worker for PHP is implemented in Ruby
-ruby src/ruby/qps/proxy-worker.rb $@
+ruby src/ruby/qps/proxy-worker.rb "$@"
diff --git a/tools/run_tests/performance/run_worker_python.sh b/tools/run_tests/performance/run_worker_python.sh
index cd7d0eb..01241c8 100755
--- a/tools/run_tests/performance/run_worker_python.sh
+++ b/tools/run_tests/performance/run_worker_python.sh
@@ -15,6 +15,6 @@
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
-PYTHONPATH=src/python/grpcio_tests:src/python/gens py27/bin/python src/python/grpcio_tests/tests/qps/qps_worker.py $@
+PYTHONPATH=src/python/grpcio_tests:src/python/gens py27/bin/python src/python/grpcio_tests/tests/qps/qps_worker.py "$@"
diff --git a/tools/run_tests/performance/run_worker_ruby.sh b/tools/run_tests/performance/run_worker_ruby.sh
index db8a7d8..729c5ce 100755
--- a/tools/run_tests/performance/run_worker_ruby.sh
+++ b/tools/run_tests/performance/run_worker_ruby.sh
@@ -16,6 +16,6 @@
source ~/.rvm/scripts/rvm
set -ex
-cd $(dirname $0)/../../..
+cd "$(dirname "$0")/../../.."
-ruby src/ruby/qps/worker.rb $@
+ruby src/ruby/qps/worker.rb "$@"
diff --git a/tools/run_tests/sanity/check_shellcheck.sh b/tools/run_tests/sanity/check_shellcheck.sh
index b30feba..64f59cb 100755
--- a/tools/run_tests/sanity/check_shellcheck.sh
+++ b/tools/run_tests/sanity/check_shellcheck.sh
@@ -19,9 +19,7 @@
ROOT="$(dirname "$0")/../../.."
DIRS=(
- 'tools/run_tests/artifacts'
- 'tools/run_tests/helper_scripts'
- 'tools/run_tests/sanity'
+ 'tools/run_tests'
)
for dir in "${DIRS[@]}"; do