rbtools.utils.process¶

Utilities for running processes.

Functions

Classes

Exceptions

class rbtools.utils.process.RunProcessResult(*, command: str, exit_code: int = 0, ignored_error: bool = False, stdout: bytes = b'', stderr: bytes = b'', encoding: str = 'utf-8')[source]¶

Bases: object

The result of running a process.

This provides information on the command that was run, the return code, flags indicating if an error was met or ignored, and access to the raw or decoded standard output and error streams.

This should only be constructed by run_process() or in unit tests when spying.

New in version 4.0.

__init__(*, command: str, exit_code: int = 0, ignored_error: bool = False, stdout: bytes = b'', stderr: bytes = b'', encoding: str = 'utf-8') → None[source]¶

Initialize the process result.

Parameters:

• command (str) – The string form of the command that was run.

• exit_code (int, optional) – The exit code of the process.

• ignored_error (bool, optional) – Whether a non-0 exit code was ignored.

• stdout (bytes, optional) – The standard output from the process.

• stderr (bytes, optional) – The standard error output from the process.

• encoding (str, optional) – The expected encoding for the output streams.

command: str¶

A string representation of the command that was run.

Type:

str

exit_code: int¶

The exit code from the process.

Type:

int

ignored_error: bool¶

Whether this returned an exit code that was ignored.

Type:

bool

encoding: str¶

The encoding expected for any standard output or errors.

This is used for decoding the streams when accessing stdout or stderr.

Type:

str

stdout_bytes: BytesIO¶

The raw standard output from the process.

This is a standard bytes I/O stream, which can be used to read some or all of the data from the process, either as raw bytes or lines of bytes.

This is pre-populated with the entire contents of the process’s standard output.

Type:

io.BytesIO

stderr_bytes: BytesIO¶

The raw standard error output from the process.

This is a standard bytes I/O stream, which can be used to read some or all of the data from the process, either as raw bytes or lines of bytes.

This is pre-populated with the entire contents of the process’s standard error output.

Type:

io.BytesIO

property stdout: TextIOWrapper[source]¶

The standard output as a decoded Unicode stream.

This will construct a text I/O wrapper on first access, wrapping stdout_bytes and decoding it using encoding.

Type:

io.TextIOWrapper

property stderr: TextIOWrapper[source]¶

The standard error output as a decoded Unicode stream.

This will construct a text I/O wrapper on first access, wrapping stderr_bytes and decoding it using encoding.

Type:

io.TextIOWrapper

exception rbtools.utils.process.RunProcessError(result: RunProcessResult)[source]¶

Bases: Exception

An error running a process.

The error code and standard output/error streams are available through the result attribute. This can be used to further evaluate the cause of the error.

New in version 4.0.

__init__(result: RunProcessResult) → None[source]¶

Initialize the error.

Parameters:

result (RunProcessResult) – The result of running the process.

result: RunProcessResult¶

The result of running the process.

Type:

RunProcessResult

rbtools.utils.process.run_process(command: Union[AnyStr, List], *, cwd: Optional[str] = None, env: Optional[Dict[str, str]] = None, encoding: str = 'utf-8', needs_stdout: bool = True, needs_stderr: bool = True, redirect_stderr: bool = False, ignore_errors: Union[bool, Tuple[int, ...]] = False, log_debug_output_on_error: bool = True) → RunProcessResult[source]¶

Run a command and return the results.

This will run the provided command and its arguments, optionally with the provided environment and working directory, returning a result that can be processed by the caller.

Callers have access to the raw byte streams of the process’s standard output and error streams, and can use those to decode or further process any results.

This is the successor to execute(), which will be removed in a future release.

Note that unit tests should not spy on this function. Instead, spy on run_process_exec().

New in version 4.0.

Parameters:

• command (list of str) –

  The command to execute.

  This should always be passed as a list of strings. It does accept passing a single string, or passing bytes instead of Unicode strings, but this is not recommended and is mainly for backwards-compatibility with execute().

• cwd (str, optional) – An optional working directory in which to run the command.

• env (dict, optional) –

  Environment variables to pass to the called executable.

  These will be combined with the current environment and used for the process.

  LC_ALL and LANGUAGE will added to the final environment, set to en_US.UTF-8.

• encoding (str, optional) – The encoding used to convert any output to Unicode strings. This can usually be left as the default of utf-8.

• needs_stdout (bool, optional) –

  Whether the caller needs standard output captured.

  If True (the default), RunProcessResult.stdout_bytes and RunProcessResult.stdout will contain any standard output from the process.

  If False, standard output will be not be captured, and those will contain empty strings.

• needs_stderr (bool, optional) –

  Whether the caller needs standard error output captured.

  If True (the default), RunProcessResult.stderr_bytes and RunProcessResult.stderr will contain any standard error output from the process.

  If False, standard error output will be not be captured, and those will contain empty strings.

  Note that redirect_stderr takes precedence over this.

• redirect_stderr (bool, optional) –

  Whether to redirect stderr output to stdout, combining the results into one.

  If set, RunProcessResult.stderr_bytes and RunProcessResult.stderr will be empty. Instead, any standard error output (if any) will be set in RunProcessResult.stdout (note that this also requires needs_stdout=True to stay set, which is the default).

• ignore_errors (bool or tuple, optional) –

  Whether to ignore errors, or specific exit codes to ignore.

  If False (the default), non-0 exit codes will raise a RunProcessError.

  If True, exit codes will never cause the exception to be raised.

  If set to a tuple of exit codes, then those codes (including 0) will be ignored, and any other non-0 code will raise the exception.

  This is a convenience over catching RunProcessError and accessing RunProcessError.result.

• log_debug_output_on_error (bool, optional) –

  Whether to log the full output and errors of a command if it returns a non-0 exit code.

  Non-0 error codes will always log a debug message about the result. However, if this is True, the output and errors will also be logged.

  The default is True.

Returns:

The result of running the process, if no errors in execution were encountered.

Return type:

RunProcessResult

Raises:

• Exception – Any unexpected exceptions from running the command.

• FileNotFoundError – The provided program could not be found.

• PermissionError – The user didn’t have permissions to run the provided program, or the program wasn’t executable.

• RunProcessError – The command returned a non-0 exit code, and that code wasn’t ignored. Details of the command and its results will be available as part of the exception.

• TypeError – The value for command was not a string, bytes, or list of either.

rbtools.utils.process.run_process_exec(command: Union[AnyStr, List], cwd: Optional[str], env: Dict[str, str], needs_stdout: bool, needs_stderr: bool, redirect_stderr: bool) → Tuple[int, Optional[bytes], Optional[bytes]][source]¶

Executes a command for run_process, returning results.

This normally wraps subprocess.run(), returning results for use in run_process().

Unit tests should override this method to return results, rather than spying on run_process() itself. This will ensure the most accurate test results. run_process() will sanity-check the results to ensure they match the input parameters.

New in version 4.0.

Parameters:

• command (str) – The command to run.

• cwd (str, optional) – An optional working directory in which to run the command.

• env (dict, optional) – Environment variables to pass to the called executable.

• needs_stdout (bool, optional) – Whether the caller needs standard output captured.

• needs_stderr (bool, optional) – Whether the caller needs standard error output captured.

• redirect_stderr (bool, optional) – Whether to redirect stderr output to stdout, combining the results into one.

Returns:

A 3-tuple containing:

Tuple:

• 0 (int) – The exit code.

• 1 (bytes) – The standard output, or None.

• 2 (bytes) – The standard error output, or None.

Return type:

tuple

Raises:

Exception – All exceptions will be bubbled up to run_process().

rbtools.utils.process.execute(command: Union[AnyStr, List], env: Optional[Dict[str, str]] = None, cwd: Optional[str] = None, split_lines: bool = False, ignore_errors: bool = False, extra_ignore_errors: Tuple[int, ...] = (), with_errors: bool = True, none_on_ignored_error: bool = False, return_error_code: bool = False, log_output_on_error: bool = True, results_unicode: bool = True, return_errors: bool = False) → Any[source]¶

Execute a command and return the output.

Deprecated since version 4.0: This has been replaced with run_process(), which is more future-proof and has better type safety.

Parameters:

• command (unicode or list of unicode) – The command to execute.

• env (dict, optional) – Environment variables to pass to the called executable. These will be added to the current environment.

• cwd (unicode, optional) – An optional working directory to change to before executing the process.

• split_lines (bool, optional) – Whether to return the output as a list of lines or a single string.

• ignore_errors (bool, optional) – Whether to ignore errors. If False, this will raise an exception.

• extra_ignore_errors (tuple, optional) – A set of errors to ignore even when ignore_errors is False. This is used because some commands (such as diff) use non-zero return codes even when the command was successful.

• with_errors (bool, optional) – Whether to combine the output and error streams of the command together into a single return value. This argument is mutually exclusive with the return_errors argument.

• none_on_ignored_error (bool, optional) – Whether to return None in the case that an error was ignored (instead of the output of the command).

• return_error_code (bool, optional) – Whether to include the exit status of the executed command in addition to the output

• log_output_on_error (bool, optional) – If True, the output from the command will be logged in the case that the command returned a non-zero exit code.

• results_unicode (bool, optional) – If True, the output will be treated as text and returned as unicode strings instead of bytes.

• return_errors (bool, optional) – Whether to return the content of the stderr stream. This argument is mutually exclusive with the with_errors argument.

Returns:

This returns a single value, 2-tuple, or 3-tuple depending on the arguments.

If return_error_code is True, the error code of the process will be returned as the first element of the tuple.

If return_errors is True, the process’ standard error stream will be returned as the last element of the tuple.

If both of return_error_code and return_errors are False, then the process’ output will be returned. If either or both of them are True, then this is the other element of the returned tuple.