Add a new flavor of command executer.
Why -
The usefulness of this new cmd executor is that it allows programmer
to examine the execution output/err stream while the binary is being
executed.
What we can do with our current weapons in the command_executor
repertoire is to kick-off the execution, capture all the command
output/err in memory, wait for its finish and examine the output in
memory.
A concrete example -
Let me give a concrete example for this - while I was reading one of
the nightly scripts, I came across this -
commands = ("{0}/utils/buildbot_json.py builds "
"http://chromegw/p/tryserver.chromiumos/"
.format(file_dir))
_, buildinfo, _ = ce.RunCommand(commands, return_output=True,
print_to_console=False)
This kicks-off a long-run command line, captures all of its output in a string, and then parse to pick out some information.
There is some issues with this -
a) - memory consumption - we capture all output in a python string
b) - no way to control the execution, what if what we are interested
in is at the very beginning or in the middle of the command output,
and once we get that we are done, and we don't want it to run any
longer.
c) - we only have hinder sight about the execution, what if the
execution spit out errors like 'unable to connect, wait for 30
minutes till timeout', which we may know of and could act earlier
(for example, kill it prematurely and report back to user).
A better approach
So here is how the proposed cmd executor works to address the above limitations -
... ...
def my_line_consumer(line, output):
if output == 'output':
# parsing this line
if output == 'err':
# ignore this line
return true; // keep execution, false to kill the current execution.
commands = ("....")
rv = ce.RunCommand(commands, my_line_consumer) # no output string is returned.
... ...
The only difference is that we pass a 'line_consumer' to the cmd
executor, this consumer will be fed with output/stderr (in unit of
lines), and depending on line content, it decides whether to kill the
execution prematurely.
Current status
I have this implemented (not many lines of code, < 100) in my
bisecting scripts, and I plan to port it to toolchain_utils/utils.
Change-Id: I2318dda796b3dcfa6ebe604091b41f9d68525d95
Reviewed-on: https://chrome-internal-review.googlesource.com/208619
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
1 file changed