docs: docs update (#911)
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/google-api-python-client/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
Fixes #<issue_number_goes_here> 🦕
diff --git a/docs/dyn/testing_v1.projects.testMatrices.html b/docs/dyn/testing_v1.projects.testMatrices.html
index c5281db..f9f6fb0 100644
--- a/docs/dyn/testing_v1.projects.testMatrices.html
+++ b/docs/dyn/testing_v1.projects.testMatrices.html
@@ -108,7 +108,7 @@
An object of the form:
{ # Response containing the current state of the specified test matrix.
- "testState": "A String", # The current rolled-up state of the test matrix.
+ "testState": "A String", # The current rolled-up state of the test matrix.
# If this state is already final, then the cancelation request will
# have no effect.
}</pre>
@@ -134,487 +134,137 @@
{ # TestMatrix captures all details about a test. It contains the environment
# configuration, test specification, test executions and overall state and
# outcome.
- "clientInfo": { # Information about the client which invoked the test. # Information about the client which invoked the test.
- "clientInfoDetails": [ # The list of detailed information about client.
- { # Key-value pair of detailed information about the client which invoked the
- # test. Examples: {'Version', '1.0'}, {'Release Track', 'BETA'}.
- "key": "A String", # Required. The key of detailed client information.
- "value": "A String", # Required. The value of detailed client information.
- },
- ],
- "name": "A String", # Required. Client name, such as gcloud.
- },
- "projectId": "A String", # The cloud project that owns the test matrix.
- "resultStorage": { # Locations where the results of running the test are stored. # Required. Where the results for the matrix are written.
- "toolResultsHistory": { # Represents a tool results history resource. # The tool results history that contains the tool results execution that
- # results are written to.
- #
- # If not provided, the service will choose an appropriate value.
- "projectId": "A String", # Required. The cloud project that owns the tool results history.
- "historyId": "A String", # Required. A tool results history ID.
- },
- "googleCloudStorage": { # A storage location within Google cloud storage (GCS). # Required.
- "gcsPath": "A String", # Required. The path to a directory in GCS that will
- # eventually contain the results for this test.
- # The requesting user must have write access on the bucket in the supplied
- # path.
- },
- "toolResultsExecution": { # Represents a tool results execution resource. # Output only. The tool results execution that results are written to.
- #
- # This has the results of a TestMatrix.
- "projectId": "A String", # Output only. The cloud project that owns the tool results execution.
- "executionId": "A String", # Output only. A tool results execution ID.
- "historyId": "A String", # Output only. A tool results history ID.
- },
- "resultsUrl": "A String", # Output only. URL to the results in the Firebase Web Console.
- },
- "flakyTestAttempts": 42, # The number of times a TestExecution should be re-attempted if one or more
- # of its test cases fail for any reason.
- # The maximum number of reruns allowed is 10.
- #
- # Default is 0, which implies no reruns.
- "state": "A String", # Output only. Indicates the current progress of the test matrix.
- "testExecutions": [ # Output only. The list of test executions that the service creates for
- # this matrix.
- { # A single test executed in a single environment.
- "timestamp": "A String", # Output only. The time this test execution was initially created.
- "matrixId": "A String", # Output only. Id of the containing TestMatrix.
- "shard": { # Output only. Details about the shard. # Output only. Details about the shard.
- "numShards": 42, # Output only. The total number of shards.
- "testTargetsForShard": { # Test targets for a shard. # Output only. Test targets for each shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
- # The targets need to be specified in AndroidJUnitRunner argument format. For
- # example, “package com.my.packages” “class com.my.package.MyClass”.
- #
- # The number of shard_test_targets must be greater than 0.
- "A String",
- ],
- },
- "shardIndex": 42, # Output only. The index of the shard among all the shards.
- },
- "id": "A String", # Output only. Unique id set by the service.
- "environment": { # The environment in which the test is run. # Output only. How the host machine(s) are configured.
- "iosDevice": { # A single iOS device. # An iOS device which must be used with an iOS test.
- "locale": "A String", # Required. The locale the test device used for testing.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "iosModelId": "A String", # Required. The id of the iOS device to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- },
- "androidDevice": { # A single Android device. # An Android device which must be used with an Android test.
- "locale": "A String", # Required. The locale the test device used for testing.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "androidModelId": "A String", # Required. The id of the Android device to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- },
- },
- "state": "A String", # Output only. Indicates the current progress of the test execution
- # (e.g., FINISHED).
- "testSpecification": { # A description of how to run the test. # Output only. How to run the test.
- "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- },
- "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
- # Xcode supports the option to "build for testing", which generates an
- # .xctestrun file that contains a test specification (arguments, test methods,
- # etc). This test type accepts a zip file containing the .xctestrun file and
- # the corresponding contents of the Build/Products directory that contains all
- # the binaries needed to run the tests.
- "xcodeVersion": "A String", # The Xcode version that should be used for the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- # Defaults to the latest Xcode version Firebase Test Lab supports.
- "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
- # tests zip. Because the .xctestrun file contains environment variables along
- # with test methods to run and/or ignore, this can be useful for sharding
- # tests. Default is taken from the tests zip.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
- # DerivedData/Build/Products directory.
- # The .xctestrun file in this zip is ignored if the xctestrun field is
- # specified.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- },
- "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
- # This test type accepts an archived application (.ipa file) and a list of
- # integer scenarios that will be executed on the app sequentially.
- "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
- # single scenario 0 if unspecified.
- 42,
- ],
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testTimeout": "A String", # Max time a test execution is allowed to run before it is
- # automatically cancelled.
- # The default value is 5 min.
- "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
- # scripts.
- "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
- "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
- # If set, the service automatically generates a Google test account and adds
- # it to the device, before executing the test. Note that test accounts might be
- # reused.
- # Many applications show their full set of functionalities when an account is
- # present on the device. Logging into the device with these generated accounts
- # allows testing more functionalities.
- },
- },
- "systrace": { # Systrace configuration for the run.
- # If set a systrace will be taken, starting on test start and lasting for the
- # configured duration. The systrace file thus obtained is put in the results
- # bucket together with the other artifacts from the run.
- "durationSeconds": 42, # Systrace duration in seconds.
- # Should be between 1 and 30 seconds. 0 disables systrace.
- },
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
- # they must be absolute paths under /sdcard or /data/local/tmp.
- # Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
- #
- # Note: The paths /sdcard and /data will be made available and treated as
- # implicit path substitutions. E.g. if /sdcard on a particular device does
- # not map to external storage, the system will replace it with the external
- # storage path prefix for that device.
- "A String",
- ],
- "environmentVariables": [ # Environment variables to set for the test (only applicable for
- # instrumentation tests).
- { # A key-value pair passed as an environment variable to the test.
- "key": "A String", # Key for the environment variable.
- "value": "A String", # Value for the environment variable.
- },
- ],
- "filesToPush": [ # List of files to push to the device before starting the test.
- { # A single device file description.
- "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
- "content": { # A reference to a file, used for user inputs. # Required. The source file.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
- # whitelisted path. If the file exists, it will be replaced.
- # The following device-side directories and any of their subdirectories are
- # whitelisted:
- # <p>${EXTERNAL_STORAGE}, or /sdcard</p>
- # <p>${ANDROID_DATA}/local/tmp, or /data/local/tmp</p>
- # <p>Specifying a path outside of these directory trees is invalid.
- #
- # <p> The paths /sdcard and /data will be made available and treated as
- # implicit path substitutions. E.g. if /sdcard on a particular device does
- # not map to external storage, the system will replace it with the external
- # storage path prefix for that device and copy the file there.
- #
- # <p> It is strongly advised to use the <a href=
- # "http://developer.android.com/reference/android/os/Environment.html">
- # Environment API</a> in app and test code to access files on the device in a
- # portable way.
- },
- "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
- "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
- # Android
- # e.g. [main|patch].0300110.com.example.android.obb
- # which will be installed into
- # \<shared-storage\>/Android/obb/\<package-name\>/
- # on the device.
- "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- },
- ],
- "additionalApks": [ # APKs to install in addition to those being directly tested.
- # Currently capped at 100.
- { # An Android package file to install.
- "packageName": "A String", # The java package for the APK to be installed.
- # Value is determined by examining the application's manifest.
- "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- ],
- },
- "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
- "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
- # The intent \<intent-name\> will be implicitly added, since Games is the only
- # user of this api, for the time being.
- "scenarios": [ # The list of scenarios that should be run during the test.
- # The default is all test loops, derived from the application's
- # manifest.
- 42,
- ],
- "appPackageId": "A String", # The java package for the application under test.
- # The default is determined by examining the application's manifest.
- "scenarioLabels": [ # The list of scenario labels that should be run during the test.
- # The scenario labels should map to labels defined in the application's
- # manifest. For example, player_experience and
- # com.google.test.loops.player_experience add all of the loops labeled in the
- # manifest with the com.google.test.loops.player_experience name to the
- # execution.
- # Scenarios can also be specified in the scenarios field.
- "A String",
- ],
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
- "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
- # or physical Android Device, finding culprits and crashes as it goes.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "roboDirectives": [ # A set of directives Robo should apply during the crawl.
- # This allows users to customize the crawl. For example, the username and
- # password for a test account can be provided.
- { # Directs Robo to interact with a specific UI element if it is encountered
- # during the crawl. Currently, Robo can perform text entry or element click.
- "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
- # treated as a CLICK on the element matching the resource_name.
- "resourceName": "A String", # Required. The android resource name of the target UI element.
- # For example,
- # in Java: R.string.foo
- # in xml: @string/foo
- # Only the "foo" part is needed.
- # Reference doc:
- # https://developer.android.com/guide/topics/resources/accessing-resources.html
- "actionType": "A String", # Required. The type of action that Robo should perform on the specified
- # element.
- },
- ],
- "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
- # for the crawl.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
- # 2 to make Robo explore the app beyond the first activity.
- # Default is 50.
- "startingIntents": [ # The intents used to launch the app for the crawl.
- # If none are provided, then the main launcher activity is launched.
- # If some are provided, then only those provided are launched (the main
- # launcher activity must be provided explicitly).
- { # Message for specifying the start activities to crawl.
- "timeout": "A String", # Timeout in seconds for each intent.
- "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
- "action": "A String", # Action name.
- # Required for START_ACTIVITY.
- "uri": "A String", # URI for the action.
- "categories": [ # Intent categories to set on the intent.
- "A String",
- ],
- },
- "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
- },
- },
- ],
- "maxSteps": 42, # The max number of steps Robo can execute.
- # Default is no limit.
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "appInitialActivity": "A String", # The initial activity that should be used to start the app.
- },
- "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
- # independently of its normal lifecycle.
- # Android instrumentation tests run an application APK and test APK inside the
- # same process on a virtual or physical AndroidDevice. They also specify
- # a test runner class, such as com.google.GoogleTestRunner, which can vary
- # on the specific instrumentation framework chosen.
- #
- # See <http://developer.android.com/tools/testing/testing_android.html> for
- # more information on types of Android tests.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testRunnerClass": "A String", # The InstrumentationTestRunner class.
- # The default value is determined by examining the application's manifest.
- "testPackageId": "A String", # The java package for the test to be executed.
- # The default value is determined by examining the application's manifest.
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
- # instrumentation with Android Test Orchestrator or not.
- # ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
- # higher! **
- # Orchestrator offers the following benefits:
- # - No shared state
- # - Crashes are isolated
- # - Logs are scoped per test
- #
- # See
- # <https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator>
- # for more information about Android Test Orchestrator.
- #
- # If not set, the test will be run without the orchestrator.
- "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
- "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
- # methods.
- # methods.
- #
- # With manual sharding enabled, specifying test targets via
- # environment_variables or in InstrumentationTest is invalid.
- "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
- # each shard. The number of shard_test_targets must be >= 1 and <= 50.
- { # Test targets for a shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
- # The targets need to be specified in AndroidJUnitRunner argument format. For
- # example, “package com.my.packages” “class com.my.package.MyClass”.
- #
- # The number of shard_test_targets must be greater than 0.
- "A String",
- ],
- },
- ],
- },
- "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
- #
- # For Instrumentation test, it will be translated to “-e numShard” “-e
- # shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
- # specifying these sharding arguments via environment_variables is invalid.
- "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
- },
- },
- "testTargets": [ # Each target must be fully qualified with the package name or class name,
- # in one of these formats:
- # - "package package_name"
- # - "class package_name.class_name"
- # - "class package_name.class_name#method_name"
- #
- # If empty, all targets in the module will be run.
- "A String",
- ],
- },
- },
- "projectId": "A String", # Output only. The cloud project that owns the test execution.
- "toolResultsStep": { # Represents a tool results step resource. # Output only. Where the results for this execution are written.
- #
- # This has the results of a TestExecution.
- "projectId": "A String", # Output only. The cloud project that owns the tool results step.
- "executionId": "A String", # Output only. A tool results execution ID.
- "stepId": "A String", # Output only. A tool results step ID.
- "historyId": "A String", # Output only. A tool results history ID.
- },
- "testDetails": { # Additional details about the progress of the running test. # Output only. Additional details about the running test.
- "progressMessages": [ # Output only. Human-readable, detailed descriptions of the test's progress.
- # For example: "Provisioning a device", "Starting Test".
- #
- # During the course of execution new data may be appended
- # to the end of progress_messages.
- "A String",
- ],
- "errorMessage": "A String", # Output only. If the TestState is ERROR, then this string will contain
- # human-readable details about the error.
- },
- },
- ],
- "testSpecification": { # A description of how to run the test. # Required. How to run the test.
- "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- },
- "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
- # Xcode supports the option to "build for testing", which generates an
+ "invalidMatrixDetails": "A String", # Output only. Describes why the matrix is considered invalid.
+ # Only useful for matrices in the INVALID state.
+ "outcomeSummary": "A String", # Output Only. The overall outcome of the test.
+ # Only set when the test matrix state is FINISHED.
+ "testSpecification": { # A description of how to run the test. # Required. How to run the test.
+ "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
+ # Xcode supports the option to "build for testing", which generates an
# .xctestrun file that contains a test specification (arguments, test methods,
# etc). This test type accepts a zip file containing the .xctestrun file and
# the corresponding contents of the Build/Products directory that contains all
# the binaries needed to run the tests.
- "xcodeVersion": "A String", # The Xcode version that should be used for the test.
+ "xcodeVersion": "A String", # The Xcode version that should be used for the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
# Defaults to the latest Xcode version Firebase Test Lab supports.
- "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
# tests zip. Because the .xctestrun file contains environment variables along
# with test methods to run and/or ignore, this can be useful for sharding
# tests. Default is taken from the tests zip.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
+ "testSpecialEntitlements": True or False, # The option to test special app entitlements. Setting this would re-sign the
+ # app having special entitlements with an explicit application-identifier.
+ # Currently supports testing aps-environment entitlement.
+ "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
# DerivedData/Build/Products directory.
# The .xctestrun file in this zip is ignored if the xctestrun field is
# specified.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
},
- "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
+ "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
# This test type accepts an archived application (.ipa file) and a list of
# integer scenarios that will be executed on the app sequentially.
- "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
# single scenario 0 if unspecified.
42,
],
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
},
- "testTimeout": "A String", # Max time a test execution is allowed to run before it is
+ "testTimeout": "A String", # Max time a test execution is allowed to run before it is
# automatically cancelled.
# The default value is 5 min.
- "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
+ "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ },
+ "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
+ # or physical Android Device, finding culprits and crashes as it goes.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "startingIntents": [ # The intents used to launch the app for the crawl.
+ # If none are provided, then the main launcher activity is launched.
+ # If some are provided, then only those provided are launched (the main
+ # launcher activity must be provided explicitly).
+ { # Message for specifying the start activities to crawl.
+ "timeout": "A String", # Timeout in seconds for each intent.
+ "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
+ "action": "A String", # Action name.
+ # Required for START_ACTIVITY.
+ "uri": "A String", # URI for the action.
+ "categories": [ # Intent categories to set on the intent.
+ "A String",
+ ],
+ },
+ "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
+ },
+ },
+ ],
+ "appInitialActivity": "A String", # The initial activity that should be used to start the app.
+ "maxSteps": 42, # The max number of steps Robo can execute.
+ # Default is no limit.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "roboDirectives": [ # A set of directives Robo should apply during the crawl.
+ # This allows users to customize the crawl. For example, the username and
+ # password for a test account can be provided.
+ { # Directs Robo to interact with a specific UI element if it is encountered
+ # during the crawl. Currently, Robo can perform text entry or element click.
+ "actionType": "A String", # Required. The type of action that Robo should perform on the specified
+ # element.
+ "resourceName": "A String", # Required. The android resource name of the target UI element.
+ # For example,
+ # in Java: R.string.foo
+ # in xml: @string/foo
+ # Only the "foo" part is needed.
+ # Reference doc:
+ # https://developer.android.com/guide/topics/resources/accessing-resources.html
+ "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
+ # treated as a CLICK on the element matching the resource_name.
+ },
+ ],
+ "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
+ # for the crawl.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
+ # 2 to make Robo explore the app beyond the first activity.
+ # Default is 50.
+ },
+ "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
# scripts.
- "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
- "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
+ "systrace": { # Systrace configuration for the run.
+ # If set a systrace will be taken, starting on test start and lasting for the
+ # configured duration. The systrace file thus obtained is put in the results
+ # bucket together with the other artifacts from the run.
+ "durationSeconds": 42, # Systrace duration in seconds.
+ # Should be between 1 and 30 seconds. 0 disables systrace.
+ },
+ "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
+ "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
# If set, the service automatically generates a Google test account and adds
# it to the device, before executing the test. Note that test accounts might be
# reused.
@@ -623,18 +273,7 @@
# allows testing more functionalities.
},
},
- "systrace": { # Systrace configuration for the run.
- # If set a systrace will be taken, starting on test start and lasting for the
- # configured duration. The systrace file thus obtained is put in the results
- # bucket together with the other artifacts from the run.
- "durationSeconds": 42, # Systrace duration in seconds.
- # Should be between 1 and 30 seconds. 0 disables systrace.
- },
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
+ "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
# they must be absolute paths under /sdcard or /data/local/tmp.
# Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
#
@@ -642,23 +281,46 @@
# implicit path substitutions. E.g. if /sdcard on a particular device does
# not map to external storage, the system will replace it with the external
# storage path prefix for that device.
- "A String",
+ "A String",
],
- "environmentVariables": [ # Environment variables to set for the test (only applicable for
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ "environmentVariables": [ # Environment variables to set for the test (only applicable for
# instrumentation tests).
{ # A key-value pair passed as an environment variable to the test.
- "key": "A String", # Key for the environment variable.
- "value": "A String", # Value for the environment variable.
+ "value": "A String", # Value for the environment variable.
+ "key": "A String", # Key for the environment variable.
},
],
- "filesToPush": [ # List of files to push to the device before starting the test.
+ "additionalApks": [ # APKs to install in addition to those being directly tested.
+ # Currently capped at 100.
+ { # An Android package file to install.
+ "packageName": "A String", # The java package for the APK to be installed.
+ # Value is determined by examining the application's manifest.
+ "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ ],
+ "filesToPush": [ # List of files to push to the device before starting the test.
{ # A single device file description.
- "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
- "content": { # A reference to a file, used for user inputs. # Required. The source file.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
+ "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
+ "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
+ # Android
+ # e.g. [main|patch].0300110.com.example.android.obb
+ # which will be installed into
+ # \<shared-storage\>/Android/obb/\<package-name\>/
+ # on the device.
+ },
+ "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
+ "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
# whitelisted path. If the file exists, it will be replaced.
# The following device-side directories and any of their subdirectories are
# whitelisted:
@@ -672,137 +334,51 @@
# storage path prefix for that device and copy the file there.
#
# <p> It is strongly advised to use the <a href=
- # "http://developer.android.com/reference/android/os/Environment.html">
+ # "http://developer.android.com/reference/android/os/Environment.html">
# Environment API</a> in app and test code to access files on the device in a
# portable way.
- },
- "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
- "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
- # Android
- # e.g. [main|patch].0300110.com.example.android.obb
- # which will be installed into
- # \<shared-storage\>/Android/obb/\<package-name\>/
- # on the device.
- "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "content": { # A reference to a file, used for user inputs. # Required. The source file.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
},
},
],
- "additionalApks": [ # APKs to install in addition to those being directly tested.
- # Currently capped at 100.
- { # An Android package file to install.
- "packageName": "A String", # The java package for the APK to be installed.
- # Value is determined by examining the application's manifest.
- "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- ],
},
- "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
- "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
+ "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
# The intent \<intent-name\> will be implicitly added, since Games is the only
# user of this api, for the time being.
- "scenarios": [ # The list of scenarios that should be run during the test.
- # The default is all test loops, derived from the application's
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default is determined by examining the application's manifest.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test.
+ # The default is all test loops, derived from the application's
# manifest.
42,
],
- "appPackageId": "A String", # The java package for the application under test.
- # The default is determined by examining the application's manifest.
- "scenarioLabels": [ # The list of scenario labels that should be run during the test.
- # The scenario labels should map to labels defined in the application's
+ "scenarioLabels": [ # The list of scenario labels that should be run during the test.
+ # The scenario labels should map to labels defined in the application's
# manifest. For example, player_experience and
# com.google.test.loops.player_experience add all of the loops labeled in the
# manifest with the com.google.test.loops.player_experience name to the
# execution.
# Scenarios can also be specified in the scenarios field.
- "A String",
+ "A String",
],
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
},
- "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
- "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
- # or physical Android Device, finding culprits and crashes as it goes.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "roboDirectives": [ # A set of directives Robo should apply during the crawl.
- # This allows users to customize the crawl. For example, the username and
- # password for a test account can be provided.
- { # Directs Robo to interact with a specific UI element if it is encountered
- # during the crawl. Currently, Robo can perform text entry or element click.
- "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
- # treated as a CLICK on the element matching the resource_name.
- "resourceName": "A String", # Required. The android resource name of the target UI element.
- # For example,
- # in Java: R.string.foo
- # in xml: @string/foo
- # Only the "foo" part is needed.
- # Reference doc:
- # https://developer.android.com/guide/topics/resources/accessing-resources.html
- "actionType": "A String", # Required. The type of action that Robo should perform on the specified
- # element.
- },
- ],
- "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
- # for the crawl.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
- # 2 to make Robo explore the app beyond the first activity.
- # Default is 50.
- "startingIntents": [ # The intents used to launch the app for the crawl.
- # If none are provided, then the main launcher activity is launched.
- # If some are provided, then only those provided are launched (the main
- # launcher activity must be provided explicitly).
- { # Message for specifying the start activities to crawl.
- "timeout": "A String", # Timeout in seconds for each intent.
- "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
- "action": "A String", # Action name.
- # Required for START_ACTIVITY.
- "uri": "A String", # URI for the action.
- "categories": [ # Intent categories to set on the intent.
- "A String",
- ],
- },
- "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
- },
- },
- ],
- "maxSteps": 42, # The max number of steps Robo can execute.
- # Default is no limit.
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "appInitialActivity": "A String", # The initial activity that should be used to start the app.
- },
- "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
+ "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
# independently of its normal lifecycle.
# Android instrumentation tests run an application APK and test APK inside the
# same process on a virtual or physical AndroidDevice. They also specify
@@ -811,30 +387,7 @@
#
# See <http://developer.android.com/tools/testing/testing_android.html> for
# more information on types of Android tests.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testRunnerClass": "A String", # The InstrumentationTestRunner class.
- # The default value is determined by examining the application's manifest.
- "testPackageId": "A String", # The java package for the test to be executed.
- # The default value is determined by examining the application's manifest.
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
+ "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
# instrumentation with Android Test Orchestrator or not.
# ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
# higher! **
@@ -848,105 +401,558 @@
# for more information about Android Test Orchestrator.
#
# If not set, the test will be run without the orchestrator.
- "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
- "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "testTargets": [ # Each target must be fully qualified with the package name or class name,
+ # in one of these formats:
+ # - "package package_name"
+ # - "class package_name.class_name"
+ # - "class package_name.class_name#method_name"
+ #
+ # If empty, all targets in the module will be run.
+ "A String",
+ ],
+ "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testRunnerClass": "A String", # The InstrumentationTestRunner class.
+ # The default value is determined by examining the application's manifest.
+ "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
+ "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
# methods.
# methods.
#
# With manual sharding enabled, specifying test targets via
# environment_variables or in InstrumentationTest is invalid.
- "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
+ "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
# each shard. The number of shard_test_targets must be >= 1 and <= 50.
{ # Test targets for a shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
# The targets need to be specified in AndroidJUnitRunner argument format. For
# example, “package com.my.packages” “class com.my.package.MyClass”.
#
# The number of shard_test_targets must be greater than 0.
- "A String",
+ "A String",
],
},
],
},
- "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
+ "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
#
# For Instrumentation test, it will be translated to “-e numShard” “-e
# shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
# specifying these sharding arguments via environment_variables is invalid.
- "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
+ "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
},
},
- "testTargets": [ # Each target must be fully qualified with the package name or class name,
- # in one of these formats:
- # - "package package_name"
- # - "class package_name.class_name"
- # - "class package_name.class_name#method_name"
- #
- # If empty, all targets in the module will be run.
- "A String",
- ],
+ "testPackageId": "A String", # The java package for the test to be executed.
+ # The default value is determined by examining the application's manifest.
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
},
+ "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
+ "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
},
- "testMatrixId": "A String", # Output only. Unique id set by the service.
- "timestamp": "A String", # Output only. The time this test matrix was initially created.
- "invalidMatrixDetails": "A String", # Output only. Describes why the matrix is considered invalid.
- # Only useful for matrices in the INVALID state.
- "environmentMatrix": { # The matrix of environments in which the test is to be executed. # Required. The devices the tests are being executed on.
- "androidDeviceList": { # A list of Android device configurations in which the test is to be executed. # A list of Android devices; the test will be run only on the specified
+ "projectId": "A String", # The cloud project that owns the test matrix.
+ "timestamp": "A String", # Output only. The time this test matrix was initially created.
+ "flakyTestAttempts": 42, # The number of times a TestExecution should be re-attempted if one or more
+ # of its test cases fail for any reason.
+ # The maximum number of reruns allowed is 10.
+ #
+ # Default is 0, which implies no reruns.
+ "state": "A String", # Output only. Indicates the current progress of the test matrix.
+ "clientInfo": { # Information about the client which invoked the test. # Information about the client which invoked the test.
+ "clientInfoDetails": [ # The list of detailed information about client.
+ { # Key-value pair of detailed information about the client which invoked the
+ # test. Examples: {'Version', '1.0'}, {'Release Track', 'BETA'}.
+ "value": "A String", # Required. The value of detailed client information.
+ "key": "A String", # Required. The key of detailed client information.
+ },
+ ],
+ "name": "A String", # Required. Client name, such as gcloud.
+ },
+ "testExecutions": [ # Output only. The list of test executions that the service creates for
+ # this matrix.
+ { # A single test executed in a single environment.
+ "timestamp": "A String", # Output only. The time this test execution was initially created.
+ "shard": { # Output only. Details about the shard. # Output only. Details about the shard.
+ "shardIndex": 42, # Output only. The index of the shard among all the shards.
+ "testTargetsForShard": { # Test targets for a shard. # Output only. Test targets for each shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ # The targets need to be specified in AndroidJUnitRunner argument format. For
+ # example, “package com.my.packages” “class com.my.package.MyClass”.
+ #
+ # The number of shard_test_targets must be greater than 0.
+ "A String",
+ ],
+ },
+ "numShards": 42, # Output only. The total number of shards.
+ },
+ "matrixId": "A String", # Output only. Id of the containing TestMatrix.
+ "testDetails": { # Additional details about the progress of the running test. # Output only. Additional details about the running test.
+ "progressMessages": [ # Output only. Human-readable, detailed descriptions of the test's progress.
+ # For example: "Provisioning a device", "Starting Test".
+ #
+ # During the course of execution new data may be appended
+ # to the end of progress_messages.
+ "A String",
+ ],
+ "errorMessage": "A String", # Output only. If the TestState is ERROR, then this string will contain
+ # human-readable details about the error.
+ },
+ "environment": { # The environment in which the test is run. # Output only. How the host machine(s) are configured.
+ "iosDevice": { # A single iOS device. # An iOS device which must be used with an iOS test.
+ "orientation": "A String", # Required. How the device is oriented during the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "locale": "A String", # Required. The locale the test device used for testing.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "iosModelId": "A String", # Required. The id of the iOS device to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ },
+ "androidDevice": { # A single Android device. # An Android device which must be used with an Android test.
+ "androidModelId": "A String", # Required. The id of the Android device to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "orientation": "A String", # Required. How the device is oriented during the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "locale": "A String", # Required. The locale the test device used for testing.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ },
+ },
+ "state": "A String", # Output only. Indicates the current progress of the test execution
+ # (e.g., FINISHED).
+ "toolResultsStep": { # Represents a tool results step resource. # Output only. Where the results for this execution are written.
+ #
+ # This has the results of a TestExecution.
+ "executionId": "A String", # Output only. A tool results execution ID.
+ "stepId": "A String", # Output only. A tool results step ID.
+ "projectId": "A String", # Output only. The cloud project that owns the tool results step.
+ "historyId": "A String", # Output only. A tool results history ID.
+ },
+ "projectId": "A String", # Output only. The cloud project that owns the test execution.
+ "testSpecification": { # A description of how to run the test. # Output only. How to run the test.
+ "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
+ # Xcode supports the option to "build for testing", which generates an
+ # .xctestrun file that contains a test specification (arguments, test methods,
+ # etc). This test type accepts a zip file containing the .xctestrun file and
+ # the corresponding contents of the Build/Products directory that contains all
+ # the binaries needed to run the tests.
+ "xcodeVersion": "A String", # The Xcode version that should be used for the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ # Defaults to the latest Xcode version Firebase Test Lab supports.
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
+ # tests zip. Because the .xctestrun file contains environment variables along
+ # with test methods to run and/or ignore, this can be useful for sharding
+ # tests. Default is taken from the tests zip.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testSpecialEntitlements": True or False, # The option to test special app entitlements. Setting this would re-sign the
+ # app having special entitlements with an explicit application-identifier.
+ # Currently supports testing aps-environment entitlement.
+ "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
+ # DerivedData/Build/Products directory.
+ # The .xctestrun file in this zip is ignored if the xctestrun field is
+ # specified.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
+ # This test type accepts an archived application (.ipa file) and a list of
+ # integer scenarios that will be executed on the app sequentially.
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
+ # single scenario 0 if unspecified.
+ 42,
+ ],
+ },
+ "testTimeout": "A String", # Max time a test execution is allowed to run before it is
+ # automatically cancelled.
+ # The default value is 5 min.
+ "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ },
+ "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
+ # or physical Android Device, finding culprits and crashes as it goes.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "startingIntents": [ # The intents used to launch the app for the crawl.
+ # If none are provided, then the main launcher activity is launched.
+ # If some are provided, then only those provided are launched (the main
+ # launcher activity must be provided explicitly).
+ { # Message for specifying the start activities to crawl.
+ "timeout": "A String", # Timeout in seconds for each intent.
+ "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
+ "action": "A String", # Action name.
+ # Required for START_ACTIVITY.
+ "uri": "A String", # URI for the action.
+ "categories": [ # Intent categories to set on the intent.
+ "A String",
+ ],
+ },
+ "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
+ },
+ },
+ ],
+ "appInitialActivity": "A String", # The initial activity that should be used to start the app.
+ "maxSteps": 42, # The max number of steps Robo can execute.
+ # Default is no limit.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "roboDirectives": [ # A set of directives Robo should apply during the crawl.
+ # This allows users to customize the crawl. For example, the username and
+ # password for a test account can be provided.
+ { # Directs Robo to interact with a specific UI element if it is encountered
+ # during the crawl. Currently, Robo can perform text entry or element click.
+ "actionType": "A String", # Required. The type of action that Robo should perform on the specified
+ # element.
+ "resourceName": "A String", # Required. The android resource name of the target UI element.
+ # For example,
+ # in Java: R.string.foo
+ # in xml: @string/foo
+ # Only the "foo" part is needed.
+ # Reference doc:
+ # https://developer.android.com/guide/topics/resources/accessing-resources.html
+ "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
+ # treated as a CLICK on the element matching the resource_name.
+ },
+ ],
+ "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
+ # for the crawl.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
+ # 2 to make Robo explore the app beyond the first activity.
+ # Default is 50.
+ },
+ "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
+ # scripts.
+ "systrace": { # Systrace configuration for the run.
+ # If set a systrace will be taken, starting on test start and lasting for the
+ # configured duration. The systrace file thus obtained is put in the results
+ # bucket together with the other artifacts from the run.
+ "durationSeconds": 42, # Systrace duration in seconds.
+ # Should be between 1 and 30 seconds. 0 disables systrace.
+ },
+ "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
+ "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
+ # If set, the service automatically generates a Google test account and adds
+ # it to the device, before executing the test. Note that test accounts might be
+ # reused.
+ # Many applications show their full set of functionalities when an account is
+ # present on the device. Logging into the device with these generated accounts
+ # allows testing more functionalities.
+ },
+ },
+ "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
+ # they must be absolute paths under /sdcard or /data/local/tmp.
+ # Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
+ #
+ # Note: The paths /sdcard and /data will be made available and treated as
+ # implicit path substitutions. E.g. if /sdcard on a particular device does
+ # not map to external storage, the system will replace it with the external
+ # storage path prefix for that device.
+ "A String",
+ ],
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ "environmentVariables": [ # Environment variables to set for the test (only applicable for
+ # instrumentation tests).
+ { # A key-value pair passed as an environment variable to the test.
+ "value": "A String", # Value for the environment variable.
+ "key": "A String", # Key for the environment variable.
+ },
+ ],
+ "additionalApks": [ # APKs to install in addition to those being directly tested.
+ # Currently capped at 100.
+ { # An Android package file to install.
+ "packageName": "A String", # The java package for the APK to be installed.
+ # Value is determined by examining the application's manifest.
+ "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ ],
+ "filesToPush": [ # List of files to push to the device before starting the test.
+ { # A single device file description.
+ "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
+ "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
+ # Android
+ # e.g. [main|patch].0300110.com.example.android.obb
+ # which will be installed into
+ # \<shared-storage\>/Android/obb/\<package-name\>/
+ # on the device.
+ },
+ "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
+ "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
+ # whitelisted path. If the file exists, it will be replaced.
+ # The following device-side directories and any of their subdirectories are
+ # whitelisted:
+ # <p>${EXTERNAL_STORAGE}, or /sdcard</p>
+ # <p>${ANDROID_DATA}/local/tmp, or /data/local/tmp</p>
+ # <p>Specifying a path outside of these directory trees is invalid.
+ #
+ # <p> The paths /sdcard and /data will be made available and treated as
+ # implicit path substitutions. E.g. if /sdcard on a particular device does
+ # not map to external storage, the system will replace it with the external
+ # storage path prefix for that device and copy the file there.
+ #
+ # <p> It is strongly advised to use the <a href=
+ # "http://developer.android.com/reference/android/os/Environment.html">
+ # Environment API</a> in app and test code to access files on the device in a
+ # portable way.
+ "content": { # A reference to a file, used for user inputs. # Required. The source file.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ },
+ ],
+ },
+ "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
+ # The intent \<intent-name\> will be implicitly added, since Games is the only
+ # user of this api, for the time being.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default is determined by examining the application's manifest.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test.
+ # The default is all test loops, derived from the application's
+ # manifest.
+ 42,
+ ],
+ "scenarioLabels": [ # The list of scenario labels that should be run during the test.
+ # The scenario labels should map to labels defined in the application's
+ # manifest. For example, player_experience and
+ # com.google.test.loops.player_experience add all of the loops labeled in the
+ # manifest with the com.google.test.loops.player_experience name to the
+ # execution.
+ # Scenarios can also be specified in the scenarios field.
+ "A String",
+ ],
+ },
+ "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
+ # independently of its normal lifecycle.
+ # Android instrumentation tests run an application APK and test APK inside the
+ # same process on a virtual or physical AndroidDevice. They also specify
+ # a test runner class, such as com.google.GoogleTestRunner, which can vary
+ # on the specific instrumentation framework chosen.
+ #
+ # See <http://developer.android.com/tools/testing/testing_android.html> for
+ # more information on types of Android tests.
+ "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
+ # instrumentation with Android Test Orchestrator or not.
+ # ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
+ # higher! **
+ # Orchestrator offers the following benefits:
+ # - No shared state
+ # - Crashes are isolated
+ # - Logs are scoped per test
+ #
+ # See
+ # <https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator>
+ # for more information about Android Test Orchestrator.
+ #
+ # If not set, the test will be run without the orchestrator.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "testTargets": [ # Each target must be fully qualified with the package name or class name,
+ # in one of these formats:
+ # - "package package_name"
+ # - "class package_name.class_name"
+ # - "class package_name.class_name#method_name"
+ #
+ # If empty, all targets in the module will be run.
+ "A String",
+ ],
+ "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testRunnerClass": "A String", # The InstrumentationTestRunner class.
+ # The default value is determined by examining the application's manifest.
+ "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
+ "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
+ # methods.
+ # methods.
+ #
+ # With manual sharding enabled, specifying test targets via
+ # environment_variables or in InstrumentationTest is invalid.
+ "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
+ # each shard. The number of shard_test_targets must be >= 1 and <= 50.
+ { # Test targets for a shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ # The targets need to be specified in AndroidJUnitRunner argument format. For
+ # example, “package com.my.packages” “class com.my.package.MyClass”.
+ #
+ # The number of shard_test_targets must be greater than 0.
+ "A String",
+ ],
+ },
+ ],
+ },
+ "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
+ #
+ # For Instrumentation test, it will be translated to “-e numShard” “-e
+ # shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
+ # specifying these sharding arguments via environment_variables is invalid.
+ "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
+ },
+ },
+ "testPackageId": "A String", # The java package for the test to be executed.
+ # The default value is determined by examining the application's manifest.
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
+ "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
+ },
+ "id": "A String", # Output only. Unique id set by the service.
+ },
+ ],
+ "environmentMatrix": { # The matrix of environments in which the test is to be executed. # Required. The devices the tests are being executed on.
+ "androidDeviceList": { # A list of Android device configurations in which the test is to be executed. # A list of Android devices; the test will be run only on the specified
# devices.
- "androidDevices": [ # Required. A list of Android devices.
+ "androidDevices": [ # Required. A list of Android devices.
{ # A single Android device.
- "locale": "A String", # Required. The locale the test device used for testing.
+ "androidModelId": "A String", # Required. The id of the Android device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
+ "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "androidModelId": "A String", # Required. The id of the Android device to be used.
+ "orientation": "A String", # Required. How the device is oriented during the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
+ "locale": "A String", # Required. The locale the test device used for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
},
],
},
- "androidMatrix": { # A set of Android device configuration permutations is defined by the # A matrix of Android devices.
+ "androidMatrix": { # A set of Android device configuration permutations is defined by the # A matrix of Android devices.
# the cross-product of the given axes. Internally, the given AndroidMatrix
# will be expanded into a set of AndroidDevices.
#
# Only supported permutations will be instantiated. Invalid permutations
# (e.g., incompatible models/versions) are ignored.
- "orientations": [ # Required. The set of orientations to test with.
+ "androidModelIds": [ # Required. The ids of the set of Android device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "locales": [ # Required. The set of locales the test device will enable for testing.
+ "orientations": [ # Required. The set of orientations to test with.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "androidVersionIds": [ # Required. The ids of the set of Android OS version to be used.
+ "androidVersionIds": [ # Required. The ids of the set of Android OS version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "androidModelIds": [ # Required. The ids of the set of Android device to be used.
+ "locales": [ # Required. The set of locales the test device will enable for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
},
- "iosDeviceList": { # A list of iOS device configurations in which the test is to be executed. # A list of iOS devices.
- "iosDevices": [ # Required. A list of iOS devices.
+ "iosDeviceList": { # A list of iOS device configurations in which the test is to be executed. # A list of iOS devices.
+ "iosDevices": [ # Required. A list of iOS devices.
{ # A single iOS device.
- "locale": "A String", # Required. The locale the test device used for testing.
+ "orientation": "A String", # Required. How the device is oriented during the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
+ "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
+ "locale": "A String", # Required. The locale the test device used for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "iosModelId": "A String", # Required. The id of the iOS device to be used.
+ "iosModelId": "A String", # Required. The id of the iOS device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
},
],
},
},
- "outcomeSummary": "A String", # Output Only. The overall outcome of the test.
- # Only set when the test matrix state is FINISHED.
+ "resultStorage": { # Locations where the results of running the test are stored. # Required. Where the results for the matrix are written.
+ "resultsUrl": "A String", # Output only. URL to the results in the Firebase Web Console.
+ "toolResultsHistory": { # Represents a tool results history resource. # The tool results history that contains the tool results execution that
+ # results are written to.
+ #
+ # If not provided, the service will choose an appropriate value.
+ "projectId": "A String", # Required. The cloud project that owns the tool results history.
+ "historyId": "A String", # Required. A tool results history ID.
+ },
+ "googleCloudStorage": { # A storage location within Google cloud storage (GCS). # Required.
+ "gcsPath": "A String", # Required. The path to a directory in GCS that will
+ # eventually contain the results for this test.
+ # The requesting user must have write access on the bucket in the supplied
+ # path.
+ },
+ "toolResultsExecution": { # Represents a tool results execution resource. # Output only. The tool results execution that results are written to.
+ #
+ # This has the results of a TestMatrix.
+ "projectId": "A String", # Output only. The cloud project that owns the tool results execution.
+ "historyId": "A String", # Output only. A tool results history ID.
+ "executionId": "A String", # Output only. A tool results execution ID.
+ },
+ },
+ "testMatrixId": "A String", # Output only. Unique id set by the service.
}
requestId: string, A string id used to detect duplicated requests.
@@ -966,487 +972,137 @@
{ # TestMatrix captures all details about a test. It contains the environment
# configuration, test specification, test executions and overall state and
# outcome.
- "clientInfo": { # Information about the client which invoked the test. # Information about the client which invoked the test.
- "clientInfoDetails": [ # The list of detailed information about client.
- { # Key-value pair of detailed information about the client which invoked the
- # test. Examples: {'Version', '1.0'}, {'Release Track', 'BETA'}.
- "key": "A String", # Required. The key of detailed client information.
- "value": "A String", # Required. The value of detailed client information.
- },
- ],
- "name": "A String", # Required. Client name, such as gcloud.
- },
- "projectId": "A String", # The cloud project that owns the test matrix.
- "resultStorage": { # Locations where the results of running the test are stored. # Required. Where the results for the matrix are written.
- "toolResultsHistory": { # Represents a tool results history resource. # The tool results history that contains the tool results execution that
- # results are written to.
- #
- # If not provided, the service will choose an appropriate value.
- "projectId": "A String", # Required. The cloud project that owns the tool results history.
- "historyId": "A String", # Required. A tool results history ID.
- },
- "googleCloudStorage": { # A storage location within Google cloud storage (GCS). # Required.
- "gcsPath": "A String", # Required. The path to a directory in GCS that will
- # eventually contain the results for this test.
- # The requesting user must have write access on the bucket in the supplied
- # path.
- },
- "toolResultsExecution": { # Represents a tool results execution resource. # Output only. The tool results execution that results are written to.
- #
- # This has the results of a TestMatrix.
- "projectId": "A String", # Output only. The cloud project that owns the tool results execution.
- "executionId": "A String", # Output only. A tool results execution ID.
- "historyId": "A String", # Output only. A tool results history ID.
- },
- "resultsUrl": "A String", # Output only. URL to the results in the Firebase Web Console.
- },
- "flakyTestAttempts": 42, # The number of times a TestExecution should be re-attempted if one or more
- # of its test cases fail for any reason.
- # The maximum number of reruns allowed is 10.
- #
- # Default is 0, which implies no reruns.
- "state": "A String", # Output only. Indicates the current progress of the test matrix.
- "testExecutions": [ # Output only. The list of test executions that the service creates for
- # this matrix.
- { # A single test executed in a single environment.
- "timestamp": "A String", # Output only. The time this test execution was initially created.
- "matrixId": "A String", # Output only. Id of the containing TestMatrix.
- "shard": { # Output only. Details about the shard. # Output only. Details about the shard.
- "numShards": 42, # Output only. The total number of shards.
- "testTargetsForShard": { # Test targets for a shard. # Output only. Test targets for each shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
- # The targets need to be specified in AndroidJUnitRunner argument format. For
- # example, “package com.my.packages” “class com.my.package.MyClass”.
- #
- # The number of shard_test_targets must be greater than 0.
- "A String",
- ],
- },
- "shardIndex": 42, # Output only. The index of the shard among all the shards.
- },
- "id": "A String", # Output only. Unique id set by the service.
- "environment": { # The environment in which the test is run. # Output only. How the host machine(s) are configured.
- "iosDevice": { # A single iOS device. # An iOS device which must be used with an iOS test.
- "locale": "A String", # Required. The locale the test device used for testing.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "iosModelId": "A String", # Required. The id of the iOS device to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- },
- "androidDevice": { # A single Android device. # An Android device which must be used with an Android test.
- "locale": "A String", # Required. The locale the test device used for testing.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "androidModelId": "A String", # Required. The id of the Android device to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- },
- },
- "state": "A String", # Output only. Indicates the current progress of the test execution
- # (e.g., FINISHED).
- "testSpecification": { # A description of how to run the test. # Output only. How to run the test.
- "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- },
- "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
- # Xcode supports the option to "build for testing", which generates an
- # .xctestrun file that contains a test specification (arguments, test methods,
- # etc). This test type accepts a zip file containing the .xctestrun file and
- # the corresponding contents of the Build/Products directory that contains all
- # the binaries needed to run the tests.
- "xcodeVersion": "A String", # The Xcode version that should be used for the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- # Defaults to the latest Xcode version Firebase Test Lab supports.
- "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
- # tests zip. Because the .xctestrun file contains environment variables along
- # with test methods to run and/or ignore, this can be useful for sharding
- # tests. Default is taken from the tests zip.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
- # DerivedData/Build/Products directory.
- # The .xctestrun file in this zip is ignored if the xctestrun field is
- # specified.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- },
- "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
- # This test type accepts an archived application (.ipa file) and a list of
- # integer scenarios that will be executed on the app sequentially.
- "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
- # single scenario 0 if unspecified.
- 42,
- ],
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testTimeout": "A String", # Max time a test execution is allowed to run before it is
- # automatically cancelled.
- # The default value is 5 min.
- "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
- # scripts.
- "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
- "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
- # If set, the service automatically generates a Google test account and adds
- # it to the device, before executing the test. Note that test accounts might be
- # reused.
- # Many applications show their full set of functionalities when an account is
- # present on the device. Logging into the device with these generated accounts
- # allows testing more functionalities.
- },
- },
- "systrace": { # Systrace configuration for the run.
- # If set a systrace will be taken, starting on test start and lasting for the
- # configured duration. The systrace file thus obtained is put in the results
- # bucket together with the other artifacts from the run.
- "durationSeconds": 42, # Systrace duration in seconds.
- # Should be between 1 and 30 seconds. 0 disables systrace.
- },
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
- # they must be absolute paths under /sdcard or /data/local/tmp.
- # Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
- #
- # Note: The paths /sdcard and /data will be made available and treated as
- # implicit path substitutions. E.g. if /sdcard on a particular device does
- # not map to external storage, the system will replace it with the external
- # storage path prefix for that device.
- "A String",
- ],
- "environmentVariables": [ # Environment variables to set for the test (only applicable for
- # instrumentation tests).
- { # A key-value pair passed as an environment variable to the test.
- "key": "A String", # Key for the environment variable.
- "value": "A String", # Value for the environment variable.
- },
- ],
- "filesToPush": [ # List of files to push to the device before starting the test.
- { # A single device file description.
- "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
- "content": { # A reference to a file, used for user inputs. # Required. The source file.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
- # whitelisted path. If the file exists, it will be replaced.
- # The following device-side directories and any of their subdirectories are
- # whitelisted:
- # <p>${EXTERNAL_STORAGE}, or /sdcard</p>
- # <p>${ANDROID_DATA}/local/tmp, or /data/local/tmp</p>
- # <p>Specifying a path outside of these directory trees is invalid.
- #
- # <p> The paths /sdcard and /data will be made available and treated as
- # implicit path substitutions. E.g. if /sdcard on a particular device does
- # not map to external storage, the system will replace it with the external
- # storage path prefix for that device and copy the file there.
- #
- # <p> It is strongly advised to use the <a href=
- # "http://developer.android.com/reference/android/os/Environment.html">
- # Environment API</a> in app and test code to access files on the device in a
- # portable way.
- },
- "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
- "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
- # Android
- # e.g. [main|patch].0300110.com.example.android.obb
- # which will be installed into
- # \<shared-storage\>/Android/obb/\<package-name\>/
- # on the device.
- "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- },
- ],
- "additionalApks": [ # APKs to install in addition to those being directly tested.
- # Currently capped at 100.
- { # An Android package file to install.
- "packageName": "A String", # The java package for the APK to be installed.
- # Value is determined by examining the application's manifest.
- "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- ],
- },
- "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
- "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
- # The intent \<intent-name\> will be implicitly added, since Games is the only
- # user of this api, for the time being.
- "scenarios": [ # The list of scenarios that should be run during the test.
- # The default is all test loops, derived from the application's
- # manifest.
- 42,
- ],
- "appPackageId": "A String", # The java package for the application under test.
- # The default is determined by examining the application's manifest.
- "scenarioLabels": [ # The list of scenario labels that should be run during the test.
- # The scenario labels should map to labels defined in the application's
- # manifest. For example, player_experience and
- # com.google.test.loops.player_experience add all of the loops labeled in the
- # manifest with the com.google.test.loops.player_experience name to the
- # execution.
- # Scenarios can also be specified in the scenarios field.
- "A String",
- ],
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
- "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
- # or physical Android Device, finding culprits and crashes as it goes.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "roboDirectives": [ # A set of directives Robo should apply during the crawl.
- # This allows users to customize the crawl. For example, the username and
- # password for a test account can be provided.
- { # Directs Robo to interact with a specific UI element if it is encountered
- # during the crawl. Currently, Robo can perform text entry or element click.
- "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
- # treated as a CLICK on the element matching the resource_name.
- "resourceName": "A String", # Required. The android resource name of the target UI element.
- # For example,
- # in Java: R.string.foo
- # in xml: @string/foo
- # Only the "foo" part is needed.
- # Reference doc:
- # https://developer.android.com/guide/topics/resources/accessing-resources.html
- "actionType": "A String", # Required. The type of action that Robo should perform on the specified
- # element.
- },
- ],
- "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
- # for the crawl.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
- # 2 to make Robo explore the app beyond the first activity.
- # Default is 50.
- "startingIntents": [ # The intents used to launch the app for the crawl.
- # If none are provided, then the main launcher activity is launched.
- # If some are provided, then only those provided are launched (the main
- # launcher activity must be provided explicitly).
- { # Message for specifying the start activities to crawl.
- "timeout": "A String", # Timeout in seconds for each intent.
- "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
- "action": "A String", # Action name.
- # Required for START_ACTIVITY.
- "uri": "A String", # URI for the action.
- "categories": [ # Intent categories to set on the intent.
- "A String",
- ],
- },
- "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
- },
- },
- ],
- "maxSteps": 42, # The max number of steps Robo can execute.
- # Default is no limit.
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "appInitialActivity": "A String", # The initial activity that should be used to start the app.
- },
- "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
- # independently of its normal lifecycle.
- # Android instrumentation tests run an application APK and test APK inside the
- # same process on a virtual or physical AndroidDevice. They also specify
- # a test runner class, such as com.google.GoogleTestRunner, which can vary
- # on the specific instrumentation framework chosen.
- #
- # See <http://developer.android.com/tools/testing/testing_android.html> for
- # more information on types of Android tests.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testRunnerClass": "A String", # The InstrumentationTestRunner class.
- # The default value is determined by examining the application's manifest.
- "testPackageId": "A String", # The java package for the test to be executed.
- # The default value is determined by examining the application's manifest.
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
- # instrumentation with Android Test Orchestrator or not.
- # ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
- # higher! **
- # Orchestrator offers the following benefits:
- # - No shared state
- # - Crashes are isolated
- # - Logs are scoped per test
- #
- # See
- # <https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator>
- # for more information about Android Test Orchestrator.
- #
- # If not set, the test will be run without the orchestrator.
- "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
- "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
- # methods.
- # methods.
- #
- # With manual sharding enabled, specifying test targets via
- # environment_variables or in InstrumentationTest is invalid.
- "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
- # each shard. The number of shard_test_targets must be >= 1 and <= 50.
- { # Test targets for a shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
- # The targets need to be specified in AndroidJUnitRunner argument format. For
- # example, “package com.my.packages” “class com.my.package.MyClass”.
- #
- # The number of shard_test_targets must be greater than 0.
- "A String",
- ],
- },
- ],
- },
- "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
- #
- # For Instrumentation test, it will be translated to “-e numShard” “-e
- # shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
- # specifying these sharding arguments via environment_variables is invalid.
- "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
- },
- },
- "testTargets": [ # Each target must be fully qualified with the package name or class name,
- # in one of these formats:
- # - "package package_name"
- # - "class package_name.class_name"
- # - "class package_name.class_name#method_name"
- #
- # If empty, all targets in the module will be run.
- "A String",
- ],
- },
- },
- "projectId": "A String", # Output only. The cloud project that owns the test execution.
- "toolResultsStep": { # Represents a tool results step resource. # Output only. Where the results for this execution are written.
- #
- # This has the results of a TestExecution.
- "projectId": "A String", # Output only. The cloud project that owns the tool results step.
- "executionId": "A String", # Output only. A tool results execution ID.
- "stepId": "A String", # Output only. A tool results step ID.
- "historyId": "A String", # Output only. A tool results history ID.
- },
- "testDetails": { # Additional details about the progress of the running test. # Output only. Additional details about the running test.
- "progressMessages": [ # Output only. Human-readable, detailed descriptions of the test's progress.
- # For example: "Provisioning a device", "Starting Test".
- #
- # During the course of execution new data may be appended
- # to the end of progress_messages.
- "A String",
- ],
- "errorMessage": "A String", # Output only. If the TestState is ERROR, then this string will contain
- # human-readable details about the error.
- },
- },
- ],
- "testSpecification": { # A description of how to run the test. # Required. How to run the test.
- "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- },
- "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
- # Xcode supports the option to "build for testing", which generates an
+ "invalidMatrixDetails": "A String", # Output only. Describes why the matrix is considered invalid.
+ # Only useful for matrices in the INVALID state.
+ "outcomeSummary": "A String", # Output Only. The overall outcome of the test.
+ # Only set when the test matrix state is FINISHED.
+ "testSpecification": { # A description of how to run the test. # Required. How to run the test.
+ "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
+ # Xcode supports the option to "build for testing", which generates an
# .xctestrun file that contains a test specification (arguments, test methods,
# etc). This test type accepts a zip file containing the .xctestrun file and
# the corresponding contents of the Build/Products directory that contains all
# the binaries needed to run the tests.
- "xcodeVersion": "A String", # The Xcode version that should be used for the test.
+ "xcodeVersion": "A String", # The Xcode version that should be used for the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
# Defaults to the latest Xcode version Firebase Test Lab supports.
- "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
# tests zip. Because the .xctestrun file contains environment variables along
# with test methods to run and/or ignore, this can be useful for sharding
# tests. Default is taken from the tests zip.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
+ "testSpecialEntitlements": True or False, # The option to test special app entitlements. Setting this would re-sign the
+ # app having special entitlements with an explicit application-identifier.
+ # Currently supports testing aps-environment entitlement.
+ "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
# DerivedData/Build/Products directory.
# The .xctestrun file in this zip is ignored if the xctestrun field is
# specified.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
},
- "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
+ "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
# This test type accepts an archived application (.ipa file) and a list of
# integer scenarios that will be executed on the app sequentially.
- "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
# single scenario 0 if unspecified.
42,
],
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
},
- "testTimeout": "A String", # Max time a test execution is allowed to run before it is
+ "testTimeout": "A String", # Max time a test execution is allowed to run before it is
# automatically cancelled.
# The default value is 5 min.
- "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
+ "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ },
+ "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
+ # or physical Android Device, finding culprits and crashes as it goes.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "startingIntents": [ # The intents used to launch the app for the crawl.
+ # If none are provided, then the main launcher activity is launched.
+ # If some are provided, then only those provided are launched (the main
+ # launcher activity must be provided explicitly).
+ { # Message for specifying the start activities to crawl.
+ "timeout": "A String", # Timeout in seconds for each intent.
+ "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
+ "action": "A String", # Action name.
+ # Required for START_ACTIVITY.
+ "uri": "A String", # URI for the action.
+ "categories": [ # Intent categories to set on the intent.
+ "A String",
+ ],
+ },
+ "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
+ },
+ },
+ ],
+ "appInitialActivity": "A String", # The initial activity that should be used to start the app.
+ "maxSteps": 42, # The max number of steps Robo can execute.
+ # Default is no limit.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "roboDirectives": [ # A set of directives Robo should apply during the crawl.
+ # This allows users to customize the crawl. For example, the username and
+ # password for a test account can be provided.
+ { # Directs Robo to interact with a specific UI element if it is encountered
+ # during the crawl. Currently, Robo can perform text entry or element click.
+ "actionType": "A String", # Required. The type of action that Robo should perform on the specified
+ # element.
+ "resourceName": "A String", # Required. The android resource name of the target UI element.
+ # For example,
+ # in Java: R.string.foo
+ # in xml: @string/foo
+ # Only the "foo" part is needed.
+ # Reference doc:
+ # https://developer.android.com/guide/topics/resources/accessing-resources.html
+ "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
+ # treated as a CLICK on the element matching the resource_name.
+ },
+ ],
+ "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
+ # for the crawl.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
+ # 2 to make Robo explore the app beyond the first activity.
+ # Default is 50.
+ },
+ "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
# scripts.
- "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
- "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
+ "systrace": { # Systrace configuration for the run.
+ # If set a systrace will be taken, starting on test start and lasting for the
+ # configured duration. The systrace file thus obtained is put in the results
+ # bucket together with the other artifacts from the run.
+ "durationSeconds": 42, # Systrace duration in seconds.
+ # Should be between 1 and 30 seconds. 0 disables systrace.
+ },
+ "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
+ "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
# If set, the service automatically generates a Google test account and adds
# it to the device, before executing the test. Note that test accounts might be
# reused.
@@ -1455,18 +1111,7 @@
# allows testing more functionalities.
},
},
- "systrace": { # Systrace configuration for the run.
- # If set a systrace will be taken, starting on test start and lasting for the
- # configured duration. The systrace file thus obtained is put in the results
- # bucket together with the other artifacts from the run.
- "durationSeconds": 42, # Systrace duration in seconds.
- # Should be between 1 and 30 seconds. 0 disables systrace.
- },
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
+ "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
# they must be absolute paths under /sdcard or /data/local/tmp.
# Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
#
@@ -1474,23 +1119,46 @@
# implicit path substitutions. E.g. if /sdcard on a particular device does
# not map to external storage, the system will replace it with the external
# storage path prefix for that device.
- "A String",
+ "A String",
],
- "environmentVariables": [ # Environment variables to set for the test (only applicable for
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ "environmentVariables": [ # Environment variables to set for the test (only applicable for
# instrumentation tests).
{ # A key-value pair passed as an environment variable to the test.
- "key": "A String", # Key for the environment variable.
- "value": "A String", # Value for the environment variable.
+ "value": "A String", # Value for the environment variable.
+ "key": "A String", # Key for the environment variable.
},
],
- "filesToPush": [ # List of files to push to the device before starting the test.
+ "additionalApks": [ # APKs to install in addition to those being directly tested.
+ # Currently capped at 100.
+ { # An Android package file to install.
+ "packageName": "A String", # The java package for the APK to be installed.
+ # Value is determined by examining the application's manifest.
+ "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ ],
+ "filesToPush": [ # List of files to push to the device before starting the test.
{ # A single device file description.
- "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
- "content": { # A reference to a file, used for user inputs. # Required. The source file.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
+ "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
+ "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
+ # Android
+ # e.g. [main|patch].0300110.com.example.android.obb
+ # which will be installed into
+ # \<shared-storage\>/Android/obb/\<package-name\>/
+ # on the device.
+ },
+ "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
+ "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
# whitelisted path. If the file exists, it will be replaced.
# The following device-side directories and any of their subdirectories are
# whitelisted:
@@ -1504,137 +1172,51 @@
# storage path prefix for that device and copy the file there.
#
# <p> It is strongly advised to use the <a href=
- # "http://developer.android.com/reference/android/os/Environment.html">
+ # "http://developer.android.com/reference/android/os/Environment.html">
# Environment API</a> in app and test code to access files on the device in a
# portable way.
- },
- "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
- "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
- # Android
- # e.g. [main|patch].0300110.com.example.android.obb
- # which will be installed into
- # \<shared-storage\>/Android/obb/\<package-name\>/
- # on the device.
- "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "content": { # A reference to a file, used for user inputs. # Required. The source file.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
},
},
],
- "additionalApks": [ # APKs to install in addition to those being directly tested.
- # Currently capped at 100.
- { # An Android package file to install.
- "packageName": "A String", # The java package for the APK to be installed.
- # Value is determined by examining the application's manifest.
- "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- ],
},
- "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
- "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
+ "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
# The intent \<intent-name\> will be implicitly added, since Games is the only
# user of this api, for the time being.
- "scenarios": [ # The list of scenarios that should be run during the test.
- # The default is all test loops, derived from the application's
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default is determined by examining the application's manifest.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test.
+ # The default is all test loops, derived from the application's
# manifest.
42,
],
- "appPackageId": "A String", # The java package for the application under test.
- # The default is determined by examining the application's manifest.
- "scenarioLabels": [ # The list of scenario labels that should be run during the test.
- # The scenario labels should map to labels defined in the application's
+ "scenarioLabels": [ # The list of scenario labels that should be run during the test.
+ # The scenario labels should map to labels defined in the application's
# manifest. For example, player_experience and
# com.google.test.loops.player_experience add all of the loops labeled in the
# manifest with the com.google.test.loops.player_experience name to the
# execution.
# Scenarios can also be specified in the scenarios field.
- "A String",
+ "A String",
],
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
},
- "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
- "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
- # or physical Android Device, finding culprits and crashes as it goes.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "roboDirectives": [ # A set of directives Robo should apply during the crawl.
- # This allows users to customize the crawl. For example, the username and
- # password for a test account can be provided.
- { # Directs Robo to interact with a specific UI element if it is encountered
- # during the crawl. Currently, Robo can perform text entry or element click.
- "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
- # treated as a CLICK on the element matching the resource_name.
- "resourceName": "A String", # Required. The android resource name of the target UI element.
- # For example,
- # in Java: R.string.foo
- # in xml: @string/foo
- # Only the "foo" part is needed.
- # Reference doc:
- # https://developer.android.com/guide/topics/resources/accessing-resources.html
- "actionType": "A String", # Required. The type of action that Robo should perform on the specified
- # element.
- },
- ],
- "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
- # for the crawl.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
- # 2 to make Robo explore the app beyond the first activity.
- # Default is 50.
- "startingIntents": [ # The intents used to launch the app for the crawl.
- # If none are provided, then the main launcher activity is launched.
- # If some are provided, then only those provided are launched (the main
- # launcher activity must be provided explicitly).
- { # Message for specifying the start activities to crawl.
- "timeout": "A String", # Timeout in seconds for each intent.
- "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
- "action": "A String", # Action name.
- # Required for START_ACTIVITY.
- "uri": "A String", # URI for the action.
- "categories": [ # Intent categories to set on the intent.
- "A String",
- ],
- },
- "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
- },
- },
- ],
- "maxSteps": 42, # The max number of steps Robo can execute.
- # Default is no limit.
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "appInitialActivity": "A String", # The initial activity that should be used to start the app.
- },
- "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
+ "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
# independently of its normal lifecycle.
# Android instrumentation tests run an application APK and test APK inside the
# same process on a virtual or physical AndroidDevice. They also specify
@@ -1643,30 +1225,7 @@
#
# See <http://developer.android.com/tools/testing/testing_android.html> for
# more information on types of Android tests.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testRunnerClass": "A String", # The InstrumentationTestRunner class.
- # The default value is determined by examining the application's manifest.
- "testPackageId": "A String", # The java package for the test to be executed.
- # The default value is determined by examining the application's manifest.
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
+ "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
# instrumentation with Android Test Orchestrator or not.
# ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
# higher! **
@@ -1680,105 +1239,558 @@
# for more information about Android Test Orchestrator.
#
# If not set, the test will be run without the orchestrator.
- "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
- "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "testTargets": [ # Each target must be fully qualified with the package name or class name,
+ # in one of these formats:
+ # - "package package_name"
+ # - "class package_name.class_name"
+ # - "class package_name.class_name#method_name"
+ #
+ # If empty, all targets in the module will be run.
+ "A String",
+ ],
+ "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testRunnerClass": "A String", # The InstrumentationTestRunner class.
+ # The default value is determined by examining the application's manifest.
+ "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
+ "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
# methods.
# methods.
#
# With manual sharding enabled, specifying test targets via
# environment_variables or in InstrumentationTest is invalid.
- "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
+ "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
# each shard. The number of shard_test_targets must be >= 1 and <= 50.
{ # Test targets for a shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
# The targets need to be specified in AndroidJUnitRunner argument format. For
# example, “package com.my.packages” “class com.my.package.MyClass”.
#
# The number of shard_test_targets must be greater than 0.
- "A String",
+ "A String",
],
},
],
},
- "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
+ "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
#
# For Instrumentation test, it will be translated to “-e numShard” “-e
# shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
# specifying these sharding arguments via environment_variables is invalid.
- "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
+ "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
},
},
- "testTargets": [ # Each target must be fully qualified with the package name or class name,
- # in one of these formats:
- # - "package package_name"
- # - "class package_name.class_name"
- # - "class package_name.class_name#method_name"
- #
- # If empty, all targets in the module will be run.
- "A String",
- ],
+ "testPackageId": "A String", # The java package for the test to be executed.
+ # The default value is determined by examining the application's manifest.
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
},
+ "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
+ "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
},
- "testMatrixId": "A String", # Output only. Unique id set by the service.
- "timestamp": "A String", # Output only. The time this test matrix was initially created.
- "invalidMatrixDetails": "A String", # Output only. Describes why the matrix is considered invalid.
- # Only useful for matrices in the INVALID state.
- "environmentMatrix": { # The matrix of environments in which the test is to be executed. # Required. The devices the tests are being executed on.
- "androidDeviceList": { # A list of Android device configurations in which the test is to be executed. # A list of Android devices; the test will be run only on the specified
+ "projectId": "A String", # The cloud project that owns the test matrix.
+ "timestamp": "A String", # Output only. The time this test matrix was initially created.
+ "flakyTestAttempts": 42, # The number of times a TestExecution should be re-attempted if one or more
+ # of its test cases fail for any reason.
+ # The maximum number of reruns allowed is 10.
+ #
+ # Default is 0, which implies no reruns.
+ "state": "A String", # Output only. Indicates the current progress of the test matrix.
+ "clientInfo": { # Information about the client which invoked the test. # Information about the client which invoked the test.
+ "clientInfoDetails": [ # The list of detailed information about client.
+ { # Key-value pair of detailed information about the client which invoked the
+ # test. Examples: {'Version', '1.0'}, {'Release Track', 'BETA'}.
+ "value": "A String", # Required. The value of detailed client information.
+ "key": "A String", # Required. The key of detailed client information.
+ },
+ ],
+ "name": "A String", # Required. Client name, such as gcloud.
+ },
+ "testExecutions": [ # Output only. The list of test executions that the service creates for
+ # this matrix.
+ { # A single test executed in a single environment.
+ "timestamp": "A String", # Output only. The time this test execution was initially created.
+ "shard": { # Output only. Details about the shard. # Output only. Details about the shard.
+ "shardIndex": 42, # Output only. The index of the shard among all the shards.
+ "testTargetsForShard": { # Test targets for a shard. # Output only. Test targets for each shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ # The targets need to be specified in AndroidJUnitRunner argument format. For
+ # example, “package com.my.packages” “class com.my.package.MyClass”.
+ #
+ # The number of shard_test_targets must be greater than 0.
+ "A String",
+ ],
+ },
+ "numShards": 42, # Output only. The total number of shards.
+ },
+ "matrixId": "A String", # Output only. Id of the containing TestMatrix.
+ "testDetails": { # Additional details about the progress of the running test. # Output only. Additional details about the running test.
+ "progressMessages": [ # Output only. Human-readable, detailed descriptions of the test's progress.
+ # For example: "Provisioning a device", "Starting Test".
+ #
+ # During the course of execution new data may be appended
+ # to the end of progress_messages.
+ "A String",
+ ],
+ "errorMessage": "A String", # Output only. If the TestState is ERROR, then this string will contain
+ # human-readable details about the error.
+ },
+ "environment": { # The environment in which the test is run. # Output only. How the host machine(s) are configured.
+ "iosDevice": { # A single iOS device. # An iOS device which must be used with an iOS test.
+ "orientation": "A String", # Required. How the device is oriented during the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "locale": "A String", # Required. The locale the test device used for testing.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "iosModelId": "A String", # Required. The id of the iOS device to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ },
+ "androidDevice": { # A single Android device. # An Android device which must be used with an Android test.
+ "androidModelId": "A String", # Required. The id of the Android device to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "orientation": "A String", # Required. How the device is oriented during the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "locale": "A String", # Required. The locale the test device used for testing.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ },
+ },
+ "state": "A String", # Output only. Indicates the current progress of the test execution
+ # (e.g., FINISHED).
+ "toolResultsStep": { # Represents a tool results step resource. # Output only. Where the results for this execution are written.
+ #
+ # This has the results of a TestExecution.
+ "executionId": "A String", # Output only. A tool results execution ID.
+ "stepId": "A String", # Output only. A tool results step ID.
+ "projectId": "A String", # Output only. The cloud project that owns the tool results step.
+ "historyId": "A String", # Output only. A tool results history ID.
+ },
+ "projectId": "A String", # Output only. The cloud project that owns the test execution.
+ "testSpecification": { # A description of how to run the test. # Output only. How to run the test.
+ "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
+ # Xcode supports the option to "build for testing", which generates an
+ # .xctestrun file that contains a test specification (arguments, test methods,
+ # etc). This test type accepts a zip file containing the .xctestrun file and
+ # the corresponding contents of the Build/Products directory that contains all
+ # the binaries needed to run the tests.
+ "xcodeVersion": "A String", # The Xcode version that should be used for the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ # Defaults to the latest Xcode version Firebase Test Lab supports.
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
+ # tests zip. Because the .xctestrun file contains environment variables along
+ # with test methods to run and/or ignore, this can be useful for sharding
+ # tests. Default is taken from the tests zip.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testSpecialEntitlements": True or False, # The option to test special app entitlements. Setting this would re-sign the
+ # app having special entitlements with an explicit application-identifier.
+ # Currently supports testing aps-environment entitlement.
+ "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
+ # DerivedData/Build/Products directory.
+ # The .xctestrun file in this zip is ignored if the xctestrun field is
+ # specified.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
+ # This test type accepts an archived application (.ipa file) and a list of
+ # integer scenarios that will be executed on the app sequentially.
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
+ # single scenario 0 if unspecified.
+ 42,
+ ],
+ },
+ "testTimeout": "A String", # Max time a test execution is allowed to run before it is
+ # automatically cancelled.
+ # The default value is 5 min.
+ "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ },
+ "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
+ # or physical Android Device, finding culprits and crashes as it goes.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "startingIntents": [ # The intents used to launch the app for the crawl.
+ # If none are provided, then the main launcher activity is launched.
+ # If some are provided, then only those provided are launched (the main
+ # launcher activity must be provided explicitly).
+ { # Message for specifying the start activities to crawl.
+ "timeout": "A String", # Timeout in seconds for each intent.
+ "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
+ "action": "A String", # Action name.
+ # Required for START_ACTIVITY.
+ "uri": "A String", # URI for the action.
+ "categories": [ # Intent categories to set on the intent.
+ "A String",
+ ],
+ },
+ "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
+ },
+ },
+ ],
+ "appInitialActivity": "A String", # The initial activity that should be used to start the app.
+ "maxSteps": 42, # The max number of steps Robo can execute.
+ # Default is no limit.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "roboDirectives": [ # A set of directives Robo should apply during the crawl.
+ # This allows users to customize the crawl. For example, the username and
+ # password for a test account can be provided.
+ { # Directs Robo to interact with a specific UI element if it is encountered
+ # during the crawl. Currently, Robo can perform text entry or element click.
+ "actionType": "A String", # Required. The type of action that Robo should perform on the specified
+ # element.
+ "resourceName": "A String", # Required. The android resource name of the target UI element.
+ # For example,
+ # in Java: R.string.foo
+ # in xml: @string/foo
+ # Only the "foo" part is needed.
+ # Reference doc:
+ # https://developer.android.com/guide/topics/resources/accessing-resources.html
+ "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
+ # treated as a CLICK on the element matching the resource_name.
+ },
+ ],
+ "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
+ # for the crawl.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
+ # 2 to make Robo explore the app beyond the first activity.
+ # Default is 50.
+ },
+ "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
+ # scripts.
+ "systrace": { # Systrace configuration for the run.
+ # If set a systrace will be taken, starting on test start and lasting for the
+ # configured duration. The systrace file thus obtained is put in the results
+ # bucket together with the other artifacts from the run.
+ "durationSeconds": 42, # Systrace duration in seconds.
+ # Should be between 1 and 30 seconds. 0 disables systrace.
+ },
+ "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
+ "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
+ # If set, the service automatically generates a Google test account and adds
+ # it to the device, before executing the test. Note that test accounts might be
+ # reused.
+ # Many applications show their full set of functionalities when an account is
+ # present on the device. Logging into the device with these generated accounts
+ # allows testing more functionalities.
+ },
+ },
+ "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
+ # they must be absolute paths under /sdcard or /data/local/tmp.
+ # Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
+ #
+ # Note: The paths /sdcard and /data will be made available and treated as
+ # implicit path substitutions. E.g. if /sdcard on a particular device does
+ # not map to external storage, the system will replace it with the external
+ # storage path prefix for that device.
+ "A String",
+ ],
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ "environmentVariables": [ # Environment variables to set for the test (only applicable for
+ # instrumentation tests).
+ { # A key-value pair passed as an environment variable to the test.
+ "value": "A String", # Value for the environment variable.
+ "key": "A String", # Key for the environment variable.
+ },
+ ],
+ "additionalApks": [ # APKs to install in addition to those being directly tested.
+ # Currently capped at 100.
+ { # An Android package file to install.
+ "packageName": "A String", # The java package for the APK to be installed.
+ # Value is determined by examining the application's manifest.
+ "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ ],
+ "filesToPush": [ # List of files to push to the device before starting the test.
+ { # A single device file description.
+ "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
+ "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
+ # Android
+ # e.g. [main|patch].0300110.com.example.android.obb
+ # which will be installed into
+ # \<shared-storage\>/Android/obb/\<package-name\>/
+ # on the device.
+ },
+ "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
+ "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
+ # whitelisted path. If the file exists, it will be replaced.
+ # The following device-side directories and any of their subdirectories are
+ # whitelisted:
+ # <p>${EXTERNAL_STORAGE}, or /sdcard</p>
+ # <p>${ANDROID_DATA}/local/tmp, or /data/local/tmp</p>
+ # <p>Specifying a path outside of these directory trees is invalid.
+ #
+ # <p> The paths /sdcard and /data will be made available and treated as
+ # implicit path substitutions. E.g. if /sdcard on a particular device does
+ # not map to external storage, the system will replace it with the external
+ # storage path prefix for that device and copy the file there.
+ #
+ # <p> It is strongly advised to use the <a href=
+ # "http://developer.android.com/reference/android/os/Environment.html">
+ # Environment API</a> in app and test code to access files on the device in a
+ # portable way.
+ "content": { # A reference to a file, used for user inputs. # Required. The source file.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ },
+ ],
+ },
+ "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
+ # The intent \<intent-name\> will be implicitly added, since Games is the only
+ # user of this api, for the time being.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default is determined by examining the application's manifest.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test.
+ # The default is all test loops, derived from the application's
+ # manifest.
+ 42,
+ ],
+ "scenarioLabels": [ # The list of scenario labels that should be run during the test.
+ # The scenario labels should map to labels defined in the application's
+ # manifest. For example, player_experience and
+ # com.google.test.loops.player_experience add all of the loops labeled in the
+ # manifest with the com.google.test.loops.player_experience name to the
+ # execution.
+ # Scenarios can also be specified in the scenarios field.
+ "A String",
+ ],
+ },
+ "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
+ # independently of its normal lifecycle.
+ # Android instrumentation tests run an application APK and test APK inside the
+ # same process on a virtual or physical AndroidDevice. They also specify
+ # a test runner class, such as com.google.GoogleTestRunner, which can vary
+ # on the specific instrumentation framework chosen.
+ #
+ # See <http://developer.android.com/tools/testing/testing_android.html> for
+ # more information on types of Android tests.
+ "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
+ # instrumentation with Android Test Orchestrator or not.
+ # ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
+ # higher! **
+ # Orchestrator offers the following benefits:
+ # - No shared state
+ # - Crashes are isolated
+ # - Logs are scoped per test
+ #
+ # See
+ # <https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator>
+ # for more information about Android Test Orchestrator.
+ #
+ # If not set, the test will be run without the orchestrator.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "testTargets": [ # Each target must be fully qualified with the package name or class name,
+ # in one of these formats:
+ # - "package package_name"
+ # - "class package_name.class_name"
+ # - "class package_name.class_name#method_name"
+ #
+ # If empty, all targets in the module will be run.
+ "A String",
+ ],
+ "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testRunnerClass": "A String", # The InstrumentationTestRunner class.
+ # The default value is determined by examining the application's manifest.
+ "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
+ "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
+ # methods.
+ # methods.
+ #
+ # With manual sharding enabled, specifying test targets via
+ # environment_variables or in InstrumentationTest is invalid.
+ "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
+ # each shard. The number of shard_test_targets must be >= 1 and <= 50.
+ { # Test targets for a shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ # The targets need to be specified in AndroidJUnitRunner argument format. For
+ # example, “package com.my.packages” “class com.my.package.MyClass”.
+ #
+ # The number of shard_test_targets must be greater than 0.
+ "A String",
+ ],
+ },
+ ],
+ },
+ "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
+ #
+ # For Instrumentation test, it will be translated to “-e numShard” “-e
+ # shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
+ # specifying these sharding arguments via environment_variables is invalid.
+ "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
+ },
+ },
+ "testPackageId": "A String", # The java package for the test to be executed.
+ # The default value is determined by examining the application's manifest.
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
+ "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
+ },
+ "id": "A String", # Output only. Unique id set by the service.
+ },
+ ],
+ "environmentMatrix": { # The matrix of environments in which the test is to be executed. # Required. The devices the tests are being executed on.
+ "androidDeviceList": { # A list of Android device configurations in which the test is to be executed. # A list of Android devices; the test will be run only on the specified
# devices.
- "androidDevices": [ # Required. A list of Android devices.
+ "androidDevices": [ # Required. A list of Android devices.
{ # A single Android device.
- "locale": "A String", # Required. The locale the test device used for testing.
+ "androidModelId": "A String", # Required. The id of the Android device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
+ "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "androidModelId": "A String", # Required. The id of the Android device to be used.
+ "orientation": "A String", # Required. How the device is oriented during the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
+ "locale": "A String", # Required. The locale the test device used for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
},
],
},
- "androidMatrix": { # A set of Android device configuration permutations is defined by the # A matrix of Android devices.
+ "androidMatrix": { # A set of Android device configuration permutations is defined by the # A matrix of Android devices.
# the cross-product of the given axes. Internally, the given AndroidMatrix
# will be expanded into a set of AndroidDevices.
#
# Only supported permutations will be instantiated. Invalid permutations
# (e.g., incompatible models/versions) are ignored.
- "orientations": [ # Required. The set of orientations to test with.
+ "androidModelIds": [ # Required. The ids of the set of Android device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "locales": [ # Required. The set of locales the test device will enable for testing.
+ "orientations": [ # Required. The set of orientations to test with.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "androidVersionIds": [ # Required. The ids of the set of Android OS version to be used.
+ "androidVersionIds": [ # Required. The ids of the set of Android OS version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "androidModelIds": [ # Required. The ids of the set of Android device to be used.
+ "locales": [ # Required. The set of locales the test device will enable for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
},
- "iosDeviceList": { # A list of iOS device configurations in which the test is to be executed. # A list of iOS devices.
- "iosDevices": [ # Required. A list of iOS devices.
+ "iosDeviceList": { # A list of iOS device configurations in which the test is to be executed. # A list of iOS devices.
+ "iosDevices": [ # Required. A list of iOS devices.
{ # A single iOS device.
- "locale": "A String", # Required. The locale the test device used for testing.
+ "orientation": "A String", # Required. How the device is oriented during the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
+ "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
+ "locale": "A String", # Required. The locale the test device used for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "iosModelId": "A String", # Required. The id of the iOS device to be used.
+ "iosModelId": "A String", # Required. The id of the iOS device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
},
],
},
},
- "outcomeSummary": "A String", # Output Only. The overall outcome of the test.
- # Only set when the test matrix state is FINISHED.
+ "resultStorage": { # Locations where the results of running the test are stored. # Required. Where the results for the matrix are written.
+ "resultsUrl": "A String", # Output only. URL to the results in the Firebase Web Console.
+ "toolResultsHistory": { # Represents a tool results history resource. # The tool results history that contains the tool results execution that
+ # results are written to.
+ #
+ # If not provided, the service will choose an appropriate value.
+ "projectId": "A String", # Required. The cloud project that owns the tool results history.
+ "historyId": "A String", # Required. A tool results history ID.
+ },
+ "googleCloudStorage": { # A storage location within Google cloud storage (GCS). # Required.
+ "gcsPath": "A String", # Required. The path to a directory in GCS that will
+ # eventually contain the results for this test.
+ # The requesting user must have write access on the bucket in the supplied
+ # path.
+ },
+ "toolResultsExecution": { # Represents a tool results execution resource. # Output only. The tool results execution that results are written to.
+ #
+ # This has the results of a TestMatrix.
+ "projectId": "A String", # Output only. The cloud project that owns the tool results execution.
+ "historyId": "A String", # Output only. A tool results history ID.
+ "executionId": "A String", # Output only. A tool results execution ID.
+ },
+ },
+ "testMatrixId": "A String", # Output only. Unique id set by the service.
}</pre>
</div>
@@ -1806,487 +1818,137 @@
{ # TestMatrix captures all details about a test. It contains the environment
# configuration, test specification, test executions and overall state and
# outcome.
- "clientInfo": { # Information about the client which invoked the test. # Information about the client which invoked the test.
- "clientInfoDetails": [ # The list of detailed information about client.
- { # Key-value pair of detailed information about the client which invoked the
- # test. Examples: {'Version', '1.0'}, {'Release Track', 'BETA'}.
- "key": "A String", # Required. The key of detailed client information.
- "value": "A String", # Required. The value of detailed client information.
- },
- ],
- "name": "A String", # Required. Client name, such as gcloud.
- },
- "projectId": "A String", # The cloud project that owns the test matrix.
- "resultStorage": { # Locations where the results of running the test are stored. # Required. Where the results for the matrix are written.
- "toolResultsHistory": { # Represents a tool results history resource. # The tool results history that contains the tool results execution that
- # results are written to.
- #
- # If not provided, the service will choose an appropriate value.
- "projectId": "A String", # Required. The cloud project that owns the tool results history.
- "historyId": "A String", # Required. A tool results history ID.
- },
- "googleCloudStorage": { # A storage location within Google cloud storage (GCS). # Required.
- "gcsPath": "A String", # Required. The path to a directory in GCS that will
- # eventually contain the results for this test.
- # The requesting user must have write access on the bucket in the supplied
- # path.
- },
- "toolResultsExecution": { # Represents a tool results execution resource. # Output only. The tool results execution that results are written to.
- #
- # This has the results of a TestMatrix.
- "projectId": "A String", # Output only. The cloud project that owns the tool results execution.
- "executionId": "A String", # Output only. A tool results execution ID.
- "historyId": "A String", # Output only. A tool results history ID.
- },
- "resultsUrl": "A String", # Output only. URL to the results in the Firebase Web Console.
- },
- "flakyTestAttempts": 42, # The number of times a TestExecution should be re-attempted if one or more
- # of its test cases fail for any reason.
- # The maximum number of reruns allowed is 10.
- #
- # Default is 0, which implies no reruns.
- "state": "A String", # Output only. Indicates the current progress of the test matrix.
- "testExecutions": [ # Output only. The list of test executions that the service creates for
- # this matrix.
- { # A single test executed in a single environment.
- "timestamp": "A String", # Output only. The time this test execution was initially created.
- "matrixId": "A String", # Output only. Id of the containing TestMatrix.
- "shard": { # Output only. Details about the shard. # Output only. Details about the shard.
- "numShards": 42, # Output only. The total number of shards.
- "testTargetsForShard": { # Test targets for a shard. # Output only. Test targets for each shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
- # The targets need to be specified in AndroidJUnitRunner argument format. For
- # example, “package com.my.packages” “class com.my.package.MyClass”.
- #
- # The number of shard_test_targets must be greater than 0.
- "A String",
- ],
- },
- "shardIndex": 42, # Output only. The index of the shard among all the shards.
- },
- "id": "A String", # Output only. Unique id set by the service.
- "environment": { # The environment in which the test is run. # Output only. How the host machine(s) are configured.
- "iosDevice": { # A single iOS device. # An iOS device which must be used with an iOS test.
- "locale": "A String", # Required. The locale the test device used for testing.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "iosModelId": "A String", # Required. The id of the iOS device to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- },
- "androidDevice": { # A single Android device. # An Android device which must be used with an Android test.
- "locale": "A String", # Required. The locale the test device used for testing.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "androidModelId": "A String", # Required. The id of the Android device to be used.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- },
- },
- "state": "A String", # Output only. Indicates the current progress of the test execution
- # (e.g., FINISHED).
- "testSpecification": { # A description of how to run the test. # Output only. How to run the test.
- "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- },
- "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
- # Xcode supports the option to "build for testing", which generates an
- # .xctestrun file that contains a test specification (arguments, test methods,
- # etc). This test type accepts a zip file containing the .xctestrun file and
- # the corresponding contents of the Build/Products directory that contains all
- # the binaries needed to run the tests.
- "xcodeVersion": "A String", # The Xcode version that should be used for the test.
- # Use the TestEnvironmentDiscoveryService to get supported options.
- # Defaults to the latest Xcode version Firebase Test Lab supports.
- "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
- # tests zip. Because the .xctestrun file contains environment variables along
- # with test methods to run and/or ignore, this can be useful for sharding
- # tests. Default is taken from the tests zip.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
- # DerivedData/Build/Products directory.
- # The .xctestrun file in this zip is ignored if the xctestrun field is
- # specified.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- },
- "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
- # This test type accepts an archived application (.ipa file) and a list of
- # integer scenarios that will be executed on the app sequentially.
- "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
- # single scenario 0 if unspecified.
- 42,
- ],
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testTimeout": "A String", # Max time a test execution is allowed to run before it is
- # automatically cancelled.
- # The default value is 5 min.
- "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
- # scripts.
- "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
- "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
- # If set, the service automatically generates a Google test account and adds
- # it to the device, before executing the test. Note that test accounts might be
- # reused.
- # Many applications show their full set of functionalities when an account is
- # present on the device. Logging into the device with these generated accounts
- # allows testing more functionalities.
- },
- },
- "systrace": { # Systrace configuration for the run.
- # If set a systrace will be taken, starting on test start and lasting for the
- # configured duration. The systrace file thus obtained is put in the results
- # bucket together with the other artifacts from the run.
- "durationSeconds": 42, # Systrace duration in seconds.
- # Should be between 1 and 30 seconds. 0 disables systrace.
- },
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
- # they must be absolute paths under /sdcard or /data/local/tmp.
- # Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
- #
- # Note: The paths /sdcard and /data will be made available and treated as
- # implicit path substitutions. E.g. if /sdcard on a particular device does
- # not map to external storage, the system will replace it with the external
- # storage path prefix for that device.
- "A String",
- ],
- "environmentVariables": [ # Environment variables to set for the test (only applicable for
- # instrumentation tests).
- { # A key-value pair passed as an environment variable to the test.
- "key": "A String", # Key for the environment variable.
- "value": "A String", # Value for the environment variable.
- },
- ],
- "filesToPush": [ # List of files to push to the device before starting the test.
- { # A single device file description.
- "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
- "content": { # A reference to a file, used for user inputs. # Required. The source file.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
- # whitelisted path. If the file exists, it will be replaced.
- # The following device-side directories and any of their subdirectories are
- # whitelisted:
- # <p>${EXTERNAL_STORAGE}, or /sdcard</p>
- # <p>${ANDROID_DATA}/local/tmp, or /data/local/tmp</p>
- # <p>Specifying a path outside of these directory trees is invalid.
- #
- # <p> The paths /sdcard and /data will be made available and treated as
- # implicit path substitutions. E.g. if /sdcard on a particular device does
- # not map to external storage, the system will replace it with the external
- # storage path prefix for that device and copy the file there.
- #
- # <p> It is strongly advised to use the <a href=
- # "http://developer.android.com/reference/android/os/Environment.html">
- # Environment API</a> in app and test code to access files on the device in a
- # portable way.
- },
- "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
- "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
- # Android
- # e.g. [main|patch].0300110.com.example.android.obb
- # which will be installed into
- # \<shared-storage\>/Android/obb/\<package-name\>/
- # on the device.
- "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- },
- ],
- "additionalApks": [ # APKs to install in addition to those being directly tested.
- # Currently capped at 100.
- { # An Android package file to install.
- "packageName": "A String", # The java package for the APK to be installed.
- # Value is determined by examining the application's manifest.
- "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- ],
- },
- "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
- "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
- # The intent \<intent-name\> will be implicitly added, since Games is the only
- # user of this api, for the time being.
- "scenarios": [ # The list of scenarios that should be run during the test.
- # The default is all test loops, derived from the application's
- # manifest.
- 42,
- ],
- "appPackageId": "A String", # The java package for the application under test.
- # The default is determined by examining the application's manifest.
- "scenarioLabels": [ # The list of scenario labels that should be run during the test.
- # The scenario labels should map to labels defined in the application's
- # manifest. For example, player_experience and
- # com.google.test.loops.player_experience add all of the loops labeled in the
- # manifest with the com.google.test.loops.player_experience name to the
- # execution.
- # Scenarios can also be specified in the scenarios field.
- "A String",
- ],
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
- "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
- # or physical Android Device, finding culprits and crashes as it goes.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "roboDirectives": [ # A set of directives Robo should apply during the crawl.
- # This allows users to customize the crawl. For example, the username and
- # password for a test account can be provided.
- { # Directs Robo to interact with a specific UI element if it is encountered
- # during the crawl. Currently, Robo can perform text entry or element click.
- "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
- # treated as a CLICK on the element matching the resource_name.
- "resourceName": "A String", # Required. The android resource name of the target UI element.
- # For example,
- # in Java: R.string.foo
- # in xml: @string/foo
- # Only the "foo" part is needed.
- # Reference doc:
- # https://developer.android.com/guide/topics/resources/accessing-resources.html
- "actionType": "A String", # Required. The type of action that Robo should perform on the specified
- # element.
- },
- ],
- "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
- # for the crawl.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
- # 2 to make Robo explore the app beyond the first activity.
- # Default is 50.
- "startingIntents": [ # The intents used to launch the app for the crawl.
- # If none are provided, then the main launcher activity is launched.
- # If some are provided, then only those provided are launched (the main
- # launcher activity must be provided explicitly).
- { # Message for specifying the start activities to crawl.
- "timeout": "A String", # Timeout in seconds for each intent.
- "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
- "action": "A String", # Action name.
- # Required for START_ACTIVITY.
- "uri": "A String", # URI for the action.
- "categories": [ # Intent categories to set on the intent.
- "A String",
- ],
- },
- "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
- },
- },
- ],
- "maxSteps": 42, # The max number of steps Robo can execute.
- # Default is no limit.
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "appInitialActivity": "A String", # The initial activity that should be used to start the app.
- },
- "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
- # independently of its normal lifecycle.
- # Android instrumentation tests run an application APK and test APK inside the
- # same process on a virtual or physical AndroidDevice. They also specify
- # a test runner class, such as com.google.GoogleTestRunner, which can vary
- # on the specific instrumentation framework chosen.
- #
- # See <http://developer.android.com/tools/testing/testing_android.html> for
- # more information on types of Android tests.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testRunnerClass": "A String", # The InstrumentationTestRunner class.
- # The default value is determined by examining the application's manifest.
- "testPackageId": "A String", # The java package for the test to be executed.
- # The default value is determined by examining the application's manifest.
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
- # instrumentation with Android Test Orchestrator or not.
- # ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
- # higher! **
- # Orchestrator offers the following benefits:
- # - No shared state
- # - Crashes are isolated
- # - Logs are scoped per test
- #
- # See
- # <https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator>
- # for more information about Android Test Orchestrator.
- #
- # If not set, the test will be run without the orchestrator.
- "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
- "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
- # methods.
- # methods.
- #
- # With manual sharding enabled, specifying test targets via
- # environment_variables or in InstrumentationTest is invalid.
- "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
- # each shard. The number of shard_test_targets must be >= 1 and <= 50.
- { # Test targets for a shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
- # The targets need to be specified in AndroidJUnitRunner argument format. For
- # example, “package com.my.packages” “class com.my.package.MyClass”.
- #
- # The number of shard_test_targets must be greater than 0.
- "A String",
- ],
- },
- ],
- },
- "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
- #
- # For Instrumentation test, it will be translated to “-e numShard” “-e
- # shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
- # specifying these sharding arguments via environment_variables is invalid.
- "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
- },
- },
- "testTargets": [ # Each target must be fully qualified with the package name or class name,
- # in one of these formats:
- # - "package package_name"
- # - "class package_name.class_name"
- # - "class package_name.class_name#method_name"
- #
- # If empty, all targets in the module will be run.
- "A String",
- ],
- },
- },
- "projectId": "A String", # Output only. The cloud project that owns the test execution.
- "toolResultsStep": { # Represents a tool results step resource. # Output only. Where the results for this execution are written.
- #
- # This has the results of a TestExecution.
- "projectId": "A String", # Output only. The cloud project that owns the tool results step.
- "executionId": "A String", # Output only. A tool results execution ID.
- "stepId": "A String", # Output only. A tool results step ID.
- "historyId": "A String", # Output only. A tool results history ID.
- },
- "testDetails": { # Additional details about the progress of the running test. # Output only. Additional details about the running test.
- "progressMessages": [ # Output only. Human-readable, detailed descriptions of the test's progress.
- # For example: "Provisioning a device", "Starting Test".
- #
- # During the course of execution new data may be appended
- # to the end of progress_messages.
- "A String",
- ],
- "errorMessage": "A String", # Output only. If the TestState is ERROR, then this string will contain
- # human-readable details about the error.
- },
- },
- ],
- "testSpecification": { # A description of how to run the test. # Required. How to run the test.
- "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- },
- "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
- # Xcode supports the option to "build for testing", which generates an
+ "invalidMatrixDetails": "A String", # Output only. Describes why the matrix is considered invalid.
+ # Only useful for matrices in the INVALID state.
+ "outcomeSummary": "A String", # Output Only. The overall outcome of the test.
+ # Only set when the test matrix state is FINISHED.
+ "testSpecification": { # A description of how to run the test. # Required. How to run the test.
+ "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
+ # Xcode supports the option to "build for testing", which generates an
# .xctestrun file that contains a test specification (arguments, test methods,
# etc). This test type accepts a zip file containing the .xctestrun file and
# the corresponding contents of the Build/Products directory that contains all
# the binaries needed to run the tests.
- "xcodeVersion": "A String", # The Xcode version that should be used for the test.
+ "xcodeVersion": "A String", # The Xcode version that should be used for the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
# Defaults to the latest Xcode version Firebase Test Lab supports.
- "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
# tests zip. Because the .xctestrun file contains environment variables along
# with test methods to run and/or ignore, this can be useful for sharding
# tests. Default is taken from the tests zip.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
+ "testSpecialEntitlements": True or False, # The option to test special app entitlements. Setting this would re-sign the
+ # app having special entitlements with an explicit application-identifier.
+ # Currently supports testing aps-environment entitlement.
+ "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
# DerivedData/Build/Products directory.
# The .xctestrun file in this zip is ignored if the xctestrun field is
# specified.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
},
- "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
+ "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
# This test type accepts an archived application (.ipa file) and a list of
# integer scenarios that will be executed on the app sequentially.
- "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
# single scenario 0 if unspecified.
42,
],
- "appBundleId": "A String", # Output only. The bundle id for the application under test.
- "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
},
- "testTimeout": "A String", # Max time a test execution is allowed to run before it is
+ "testTimeout": "A String", # Max time a test execution is allowed to run before it is
# automatically cancelled.
# The default value is 5 min.
- "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
+ "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ },
+ "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
+ # or physical Android Device, finding culprits and crashes as it goes.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "startingIntents": [ # The intents used to launch the app for the crawl.
+ # If none are provided, then the main launcher activity is launched.
+ # If some are provided, then only those provided are launched (the main
+ # launcher activity must be provided explicitly).
+ { # Message for specifying the start activities to crawl.
+ "timeout": "A String", # Timeout in seconds for each intent.
+ "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
+ "action": "A String", # Action name.
+ # Required for START_ACTIVITY.
+ "uri": "A String", # URI for the action.
+ "categories": [ # Intent categories to set on the intent.
+ "A String",
+ ],
+ },
+ "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
+ },
+ },
+ ],
+ "appInitialActivity": "A String", # The initial activity that should be used to start the app.
+ "maxSteps": 42, # The max number of steps Robo can execute.
+ # Default is no limit.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "roboDirectives": [ # A set of directives Robo should apply during the crawl.
+ # This allows users to customize the crawl. For example, the username and
+ # password for a test account can be provided.
+ { # Directs Robo to interact with a specific UI element if it is encountered
+ # during the crawl. Currently, Robo can perform text entry or element click.
+ "actionType": "A String", # Required. The type of action that Robo should perform on the specified
+ # element.
+ "resourceName": "A String", # Required. The android resource name of the target UI element.
+ # For example,
+ # in Java: R.string.foo
+ # in xml: @string/foo
+ # Only the "foo" part is needed.
+ # Reference doc:
+ # https://developer.android.com/guide/topics/resources/accessing-resources.html
+ "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
+ # treated as a CLICK on the element matching the resource_name.
+ },
+ ],
+ "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
+ # for the crawl.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
+ # 2 to make Robo explore the app beyond the first activity.
+ # Default is 50.
+ },
+ "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
# scripts.
- "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
- "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
+ "systrace": { # Systrace configuration for the run.
+ # If set a systrace will be taken, starting on test start and lasting for the
+ # configured duration. The systrace file thus obtained is put in the results
+ # bucket together with the other artifacts from the run.
+ "durationSeconds": 42, # Systrace duration in seconds.
+ # Should be between 1 and 30 seconds. 0 disables systrace.
+ },
+ "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
+ "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
# If set, the service automatically generates a Google test account and adds
# it to the device, before executing the test. Note that test accounts might be
# reused.
@@ -2295,18 +1957,7 @@
# allows testing more functionalities.
},
},
- "systrace": { # Systrace configuration for the run.
- # If set a systrace will be taken, starting on test start and lasting for the
- # configured duration. The systrace file thus obtained is put in the results
- # bucket together with the other artifacts from the run.
- "durationSeconds": 42, # Systrace duration in seconds.
- # Should be between 1 and 30 seconds. 0 disables systrace.
- },
- "networkProfile": "A String", # The network traffic profile used for running the test.
- # Available network profiles can be queried by using the
- # NETWORK_CONFIGURATION environment type when calling
- # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
- "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
+ "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
# they must be absolute paths under /sdcard or /data/local/tmp.
# Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
#
@@ -2314,23 +1965,46 @@
# implicit path substitutions. E.g. if /sdcard on a particular device does
# not map to external storage, the system will replace it with the external
# storage path prefix for that device.
- "A String",
+ "A String",
],
- "environmentVariables": [ # Environment variables to set for the test (only applicable for
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ "environmentVariables": [ # Environment variables to set for the test (only applicable for
# instrumentation tests).
{ # A key-value pair passed as an environment variable to the test.
- "key": "A String", # Key for the environment variable.
- "value": "A String", # Value for the environment variable.
+ "value": "A String", # Value for the environment variable.
+ "key": "A String", # Key for the environment variable.
},
],
- "filesToPush": [ # List of files to push to the device before starting the test.
+ "additionalApks": [ # APKs to install in addition to those being directly tested.
+ # Currently capped at 100.
+ { # An Android package file to install.
+ "packageName": "A String", # The java package for the APK to be installed.
+ # Value is determined by examining the application's manifest.
+ "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ ],
+ "filesToPush": [ # List of files to push to the device before starting the test.
{ # A single device file description.
- "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
- "content": { # A reference to a file, used for user inputs. # Required. The source file.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
+ "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
- "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
+ "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
+ # Android
+ # e.g. [main|patch].0300110.com.example.android.obb
+ # which will be installed into
+ # \<shared-storage\>/Android/obb/\<package-name\>/
+ # on the device.
+ },
+ "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
+ "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
# whitelisted path. If the file exists, it will be replaced.
# The following device-side directories and any of their subdirectories are
# whitelisted:
@@ -2344,137 +2018,51 @@
# storage path prefix for that device and copy the file there.
#
# <p> It is strongly advised to use the <a href=
- # "http://developer.android.com/reference/android/os/Environment.html">
+ # "http://developer.android.com/reference/android/os/Environment.html">
# Environment API</a> in app and test code to access files on the device in a
# portable way.
- },
- "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
- "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
- # Android
- # e.g. [main|patch].0300110.com.example.android.obb
- # which will be installed into
- # \<shared-storage\>/Android/obb/\<package-name\>/
- # on the device.
- "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ "content": { # A reference to a file, used for user inputs. # Required. The source file.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
# Example: gs://build-app-1414623860166/app-debug-unaligned.apk
},
},
},
],
- "additionalApks": [ # APKs to install in addition to those being directly tested.
- # Currently capped at 100.
- { # An Android package file to install.
- "packageName": "A String", # The java package for the APK to be installed.
- # Value is determined by examining the application's manifest.
- "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- ],
},
- "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
- "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
+ "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
# The intent \<intent-name\> will be implicitly added, since Games is the only
# user of this api, for the time being.
- "scenarios": [ # The list of scenarios that should be run during the test.
- # The default is all test loops, derived from the application's
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default is determined by examining the application's manifest.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test.
+ # The default is all test loops, derived from the application's
# manifest.
42,
],
- "appPackageId": "A String", # The java package for the application under test.
- # The default is determined by examining the application's manifest.
- "scenarioLabels": [ # The list of scenario labels that should be run during the test.
- # The scenario labels should map to labels defined in the application's
+ "scenarioLabels": [ # The list of scenario labels that should be run during the test.
+ # The scenario labels should map to labels defined in the application's
# manifest. For example, player_experience and
# com.google.test.loops.player_experience add all of the loops labeled in the
# manifest with the com.google.test.loops.player_experience name to the
# execution.
# Scenarios can also be specified in the scenarios field.
- "A String",
+ "A String",
],
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
},
- "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
- "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
- # or physical Android Device, finding culprits and crashes as it goes.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "roboDirectives": [ # A set of directives Robo should apply during the crawl.
- # This allows users to customize the crawl. For example, the username and
- # password for a test account can be provided.
- { # Directs Robo to interact with a specific UI element if it is encountered
- # during the crawl. Currently, Robo can perform text entry or element click.
- "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
- # treated as a CLICK on the element matching the resource_name.
- "resourceName": "A String", # Required. The android resource name of the target UI element.
- # For example,
- # in Java: R.string.foo
- # in xml: @string/foo
- # Only the "foo" part is needed.
- # Reference doc:
- # https://developer.android.com/guide/topics/resources/accessing-resources.html
- "actionType": "A String", # Required. The type of action that Robo should perform on the specified
- # element.
- },
- ],
- "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
- # for the crawl.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
- # 2 to make Robo explore the app beyond the first activity.
- # Default is 50.
- "startingIntents": [ # The intents used to launch the app for the crawl.
- # If none are provided, then the main launcher activity is launched.
- # If some are provided, then only those provided are launched (the main
- # launcher activity must be provided explicitly).
- { # Message for specifying the start activities to crawl.
- "timeout": "A String", # Timeout in seconds for each intent.
- "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
- "action": "A String", # Action name.
- # Required for START_ACTIVITY.
- "uri": "A String", # URI for the action.
- "categories": [ # Intent categories to set on the intent.
- "A String",
- ],
- },
- "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
- },
- },
- ],
- "maxSteps": 42, # The max number of steps Robo can execute.
- # Default is no limit.
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "appInitialActivity": "A String", # The initial activity that should be used to start the app.
- },
- "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
+ "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
# independently of its normal lifecycle.
# Android instrumentation tests run an application APK and test APK inside the
# same process on a virtual or physical AndroidDevice. They also specify
@@ -2483,30 +2071,7 @@
#
# See <http://developer.android.com/tools/testing/testing_android.html> for
# more information on types of Android tests.
- "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
- # a base module directory, zero or more dynamic feature module directories.
- # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
- # building App Bundles.
- "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- },
- "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "testRunnerClass": "A String", # The InstrumentationTestRunner class.
- # The default value is determined by examining the application's manifest.
- "testPackageId": "A String", # The java package for the test to be executed.
- # The default value is determined by examining the application's manifest.
- "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
- "gcsPath": "A String", # A path to a file in Google Cloud Storage.
- # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
- },
- "appPackageId": "A String", # The java package for the application under test.
- # The default value is determined by examining the application's manifest.
- "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
+ "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
# instrumentation with Android Test Orchestrator or not.
# ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
# higher! **
@@ -2520,105 +2085,558 @@
# for more information about Android Test Orchestrator.
#
# If not set, the test will be run without the orchestrator.
- "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
- "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "testTargets": [ # Each target must be fully qualified with the package name or class name,
+ # in one of these formats:
+ # - "package package_name"
+ # - "class package_name.class_name"
+ # - "class package_name.class_name#method_name"
+ #
+ # If empty, all targets in the module will be run.
+ "A String",
+ ],
+ "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testRunnerClass": "A String", # The InstrumentationTestRunner class.
+ # The default value is determined by examining the application's manifest.
+ "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
+ "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
# methods.
# methods.
#
# With manual sharding enabled, specifying test targets via
# environment_variables or in InstrumentationTest is invalid.
- "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
+ "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
# each shard. The number of shard_test_targets must be >= 1 and <= 50.
{ # Test targets for a shard.
- "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
# The targets need to be specified in AndroidJUnitRunner argument format. For
# example, “package com.my.packages” “class com.my.package.MyClass”.
#
# The number of shard_test_targets must be greater than 0.
- "A String",
+ "A String",
],
},
],
},
- "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
+ "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
#
# For Instrumentation test, it will be translated to “-e numShard” “-e
# shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
# specifying these sharding arguments via environment_variables is invalid.
- "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
+ "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
},
},
- "testTargets": [ # Each target must be fully qualified with the package name or class name,
- # in one of these formats:
- # - "package package_name"
- # - "class package_name.class_name"
- # - "class package_name.class_name#method_name"
- #
- # If empty, all targets in the module will be run.
- "A String",
- ],
+ "testPackageId": "A String", # The java package for the test to be executed.
+ # The default value is determined by examining the application's manifest.
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
},
+ "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
+ "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
},
- "testMatrixId": "A String", # Output only. Unique id set by the service.
- "timestamp": "A String", # Output only. The time this test matrix was initially created.
- "invalidMatrixDetails": "A String", # Output only. Describes why the matrix is considered invalid.
- # Only useful for matrices in the INVALID state.
- "environmentMatrix": { # The matrix of environments in which the test is to be executed. # Required. The devices the tests are being executed on.
- "androidDeviceList": { # A list of Android device configurations in which the test is to be executed. # A list of Android devices; the test will be run only on the specified
+ "projectId": "A String", # The cloud project that owns the test matrix.
+ "timestamp": "A String", # Output only. The time this test matrix was initially created.
+ "flakyTestAttempts": 42, # The number of times a TestExecution should be re-attempted if one or more
+ # of its test cases fail for any reason.
+ # The maximum number of reruns allowed is 10.
+ #
+ # Default is 0, which implies no reruns.
+ "state": "A String", # Output only. Indicates the current progress of the test matrix.
+ "clientInfo": { # Information about the client which invoked the test. # Information about the client which invoked the test.
+ "clientInfoDetails": [ # The list of detailed information about client.
+ { # Key-value pair of detailed information about the client which invoked the
+ # test. Examples: {'Version', '1.0'}, {'Release Track', 'BETA'}.
+ "value": "A String", # Required. The value of detailed client information.
+ "key": "A String", # Required. The key of detailed client information.
+ },
+ ],
+ "name": "A String", # Required. Client name, such as gcloud.
+ },
+ "testExecutions": [ # Output only. The list of test executions that the service creates for
+ # this matrix.
+ { # A single test executed in a single environment.
+ "timestamp": "A String", # Output only. The time this test execution was initially created.
+ "shard": { # Output only. Details about the shard. # Output only. Details about the shard.
+ "shardIndex": 42, # Output only. The index of the shard among all the shards.
+ "testTargetsForShard": { # Test targets for a shard. # Output only. Test targets for each shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ # The targets need to be specified in AndroidJUnitRunner argument format. For
+ # example, “package com.my.packages” “class com.my.package.MyClass”.
+ #
+ # The number of shard_test_targets must be greater than 0.
+ "A String",
+ ],
+ },
+ "numShards": 42, # Output only. The total number of shards.
+ },
+ "matrixId": "A String", # Output only. Id of the containing TestMatrix.
+ "testDetails": { # Additional details about the progress of the running test. # Output only. Additional details about the running test.
+ "progressMessages": [ # Output only. Human-readable, detailed descriptions of the test's progress.
+ # For example: "Provisioning a device", "Starting Test".
+ #
+ # During the course of execution new data may be appended
+ # to the end of progress_messages.
+ "A String",
+ ],
+ "errorMessage": "A String", # Output only. If the TestState is ERROR, then this string will contain
+ # human-readable details about the error.
+ },
+ "environment": { # The environment in which the test is run. # Output only. How the host machine(s) are configured.
+ "iosDevice": { # A single iOS device. # An iOS device which must be used with an iOS test.
+ "orientation": "A String", # Required. How the device is oriented during the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "locale": "A String", # Required. The locale the test device used for testing.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "iosModelId": "A String", # Required. The id of the iOS device to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ },
+ "androidDevice": { # A single Android device. # An Android device which must be used with an Android test.
+ "androidModelId": "A String", # Required. The id of the Android device to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "orientation": "A String", # Required. How the device is oriented during the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ "locale": "A String", # Required. The locale the test device used for testing.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ },
+ },
+ "state": "A String", # Output only. Indicates the current progress of the test execution
+ # (e.g., FINISHED).
+ "toolResultsStep": { # Represents a tool results step resource. # Output only. Where the results for this execution are written.
+ #
+ # This has the results of a TestExecution.
+ "executionId": "A String", # Output only. A tool results execution ID.
+ "stepId": "A String", # Output only. A tool results step ID.
+ "projectId": "A String", # Output only. The cloud project that owns the tool results step.
+ "historyId": "A String", # Output only. A tool results history ID.
+ },
+ "projectId": "A String", # Output only. The cloud project that owns the test execution.
+ "testSpecification": { # A description of how to run the test. # Output only. How to run the test.
+ "iosXcTest": { # A test of an iOS application that uses the XCTest framework. # An iOS XCTest, via an .xctestrun file.
+ # Xcode supports the option to "build for testing", which generates an
+ # .xctestrun file that contains a test specification (arguments, test methods,
+ # etc). This test type accepts a zip file containing the .xctestrun file and
+ # the corresponding contents of the Build/Products directory that contains all
+ # the binaries needed to run the tests.
+ "xcodeVersion": "A String", # The Xcode version that should be used for the test.
+ # Use the TestEnvironmentDiscoveryService to get supported options.
+ # Defaults to the latest Xcode version Firebase Test Lab supports.
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "xctestrun": { # A reference to a file, used for user inputs. # An .xctestrun file that will override the .xctestrun file in the
+ # tests zip. Because the .xctestrun file contains environment variables along
+ # with test methods to run and/or ignore, this can be useful for sharding
+ # tests. Default is taken from the tests zip.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testSpecialEntitlements": True or False, # The option to test special app entitlements. Setting this would re-sign the
+ # app having special entitlements with an explicit application-identifier.
+ # Currently supports testing aps-environment entitlement.
+ "testsZip": { # A reference to a file, used for user inputs. # Required. The .zip containing the .xctestrun file and the contents of the
+ # DerivedData/Build/Products directory.
+ # The .xctestrun file in this zip is ignored if the xctestrun field is
+ # specified.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "iosTestLoop": { # A test of an iOS application that implements one or more game loop scenarios. # An iOS application with a test loop.
+ # This test type accepts an archived application (.ipa file) and a list of
+ # integer scenarios that will be executed on the app sequentially.
+ "appBundleId": "A String", # Output only. The bundle id for the application under test.
+ "appIpa": { # A reference to a file, used for user inputs. # Required. The .ipa of the application to test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test. Defaults to the
+ # single scenario 0 if unspecified.
+ 42,
+ ],
+ },
+ "testTimeout": "A String", # Max time a test execution is allowed to run before it is
+ # automatically cancelled.
+ # The default value is 5 min.
+ "iosTestSetup": { # A description of how to set up an iOS device prior to running the test. # Test setup requirements for iOS.
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ },
+ "androidRoboTest": { # A test of an android application that explores the application on a virtual # An Android robo test.
+ # or physical Android Device, finding culprits and crashes as it goes.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "startingIntents": [ # The intents used to launch the app for the crawl.
+ # If none are provided, then the main launcher activity is launched.
+ # If some are provided, then only those provided are launched (the main
+ # launcher activity must be provided explicitly).
+ { # Message for specifying the start activities to crawl.
+ "timeout": "A String", # Timeout in seconds for each intent.
+ "startActivity": { # A starting intent specified by an action, uri, and categories. # An intent that starts an activity with specific details.
+ "action": "A String", # Action name.
+ # Required for START_ACTIVITY.
+ "uri": "A String", # URI for the action.
+ "categories": [ # Intent categories to set on the intent.
+ "A String",
+ ],
+ },
+ "launcherActivity": { # Specifies an intent that starts the main launcher activity. # An intent that starts the main launcher activity.
+ },
+ },
+ ],
+ "appInitialActivity": "A String", # The initial activity that should be used to start the app.
+ "maxSteps": 42, # The max number of steps Robo can execute.
+ # Default is no limit.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "roboDirectives": [ # A set of directives Robo should apply during the crawl.
+ # This allows users to customize the crawl. For example, the username and
+ # password for a test account can be provided.
+ { # Directs Robo to interact with a specific UI element if it is encountered
+ # during the crawl. Currently, Robo can perform text entry or element click.
+ "actionType": "A String", # Required. The type of action that Robo should perform on the specified
+ # element.
+ "resourceName": "A String", # Required. The android resource name of the target UI element.
+ # For example,
+ # in Java: R.string.foo
+ # in xml: @string/foo
+ # Only the "foo" part is needed.
+ # Reference doc:
+ # https://developer.android.com/guide/topics/resources/accessing-resources.html
+ "inputText": "A String", # The text that Robo is directed to set. If left empty, the directive will be
+ # treated as a CLICK on the element matching the resource_name.
+ },
+ ],
+ "roboScript": { # A reference to a file, used for user inputs. # A JSON file with a sequence of actions Robo should perform as a prologue
+ # for the crawl.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "maxDepth": 42, # The max depth of the traversal stack Robo can explore. Needs to be at least
+ # 2 to make Robo explore the app beyond the first activity.
+ # Default is 50.
+ },
+ "testSetup": { # A description of how to set up the Android device prior to running the test. # Test setup requirements for Android e.g. files to install, bootstrap
+ # scripts.
+ "systrace": { # Systrace configuration for the run.
+ # If set a systrace will be taken, starting on test start and lasting for the
+ # configured duration. The systrace file thus obtained is put in the results
+ # bucket together with the other artifacts from the run.
+ "durationSeconds": 42, # Systrace duration in seconds.
+ # Should be between 1 and 30 seconds. 0 disables systrace.
+ },
+ "account": { # Identifies an account and how to log into it. # The device will be logged in on this account for the duration of the test.
+ "googleAuto": { # Enables automatic Google account login. # An automatic google login account.
+ # If set, the service automatically generates a Google test account and adds
+ # it to the device, before executing the test. Note that test accounts might be
+ # reused.
+ # Many applications show their full set of functionalities when an account is
+ # present on the device. Logging into the device with these generated accounts
+ # allows testing more functionalities.
+ },
+ },
+ "directoriesToPull": [ # List of directories on the device to upload to GCS at the end of the test;
+ # they must be absolute paths under /sdcard or /data/local/tmp.
+ # Path names are restricted to characters a-z A-Z 0-9 _ - . + and /
+ #
+ # Note: The paths /sdcard and /data will be made available and treated as
+ # implicit path substitutions. E.g. if /sdcard on a particular device does
+ # not map to external storage, the system will replace it with the external
+ # storage path prefix for that device.
+ "A String",
+ ],
+ "networkProfile": "A String", # The network traffic profile used for running the test.
+ # Available network profiles can be queried by using the
+ # NETWORK_CONFIGURATION environment type when calling
+ # TestEnvironmentDiscoveryService.GetTestEnvironmentCatalog.
+ "environmentVariables": [ # Environment variables to set for the test (only applicable for
+ # instrumentation tests).
+ { # A key-value pair passed as an environment variable to the test.
+ "value": "A String", # Value for the environment variable.
+ "key": "A String", # Key for the environment variable.
+ },
+ ],
+ "additionalApks": [ # APKs to install in addition to those being directly tested.
+ # Currently capped at 100.
+ { # An Android package file to install.
+ "packageName": "A String", # The java package for the APK to be installed.
+ # Value is determined by examining the application's manifest.
+ "location": { # A reference to a file, used for user inputs. # The path to an APK to be installed on the device before the test begins.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ ],
+ "filesToPush": [ # List of files to push to the device before starting the test.
+ { # A single device file description.
+ "obbFile": { # An opaque binary blob file to install on the device before the test starts. # A reference to an opaque binary blob file.
+ "obb": { # A reference to a file, used for user inputs. # Required. Opaque Binary Blob (OBB) file(s) to install on the device.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "obbFileName": "A String", # Required. OBB file name which must conform to the format as specified by
+ # Android
+ # e.g. [main|patch].0300110.com.example.android.obb
+ # which will be installed into
+ # \<shared-storage\>/Android/obb/\<package-name\>/
+ # on the device.
+ },
+ "regularFile": { # A file or directory to install on the device before the test starts. # A reference to a regular file.
+ "devicePath": "A String", # Required. Where to put the content on the device. Must be an absolute,
+ # whitelisted path. If the file exists, it will be replaced.
+ # The following device-side directories and any of their subdirectories are
+ # whitelisted:
+ # <p>${EXTERNAL_STORAGE}, or /sdcard</p>
+ # <p>${ANDROID_DATA}/local/tmp, or /data/local/tmp</p>
+ # <p>Specifying a path outside of these directory trees is invalid.
+ #
+ # <p> The paths /sdcard and /data will be made available and treated as
+ # implicit path substitutions. E.g. if /sdcard on a particular device does
+ # not map to external storage, the system will replace it with the external
+ # storage path prefix for that device and copy the file there.
+ #
+ # <p> It is strongly advised to use the <a href=
+ # "http://developer.android.com/reference/android/os/Environment.html">
+ # Environment API</a> in app and test code to access files on the device in a
+ # portable way.
+ "content": { # A reference to a file, used for user inputs. # Required. The source file.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ },
+ ],
+ },
+ "androidTestLoop": { # A test of an Android Application with a Test Loop. # An Android Application with a Test Loop.
+ # The intent \<intent-name\> will be implicitly added, since Games is the only
+ # user of this api, for the time being.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default is determined by examining the application's manifest.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "scenarios": [ # The list of scenarios that should be run during the test.
+ # The default is all test loops, derived from the application's
+ # manifest.
+ 42,
+ ],
+ "scenarioLabels": [ # The list of scenario labels that should be run during the test.
+ # The scenario labels should map to labels defined in the application's
+ # manifest. For example, player_experience and
+ # com.google.test.loops.player_experience add all of the loops labeled in the
+ # manifest with the com.google.test.loops.player_experience name to the
+ # execution.
+ # Scenarios can also be specified in the scenarios field.
+ "A String",
+ ],
+ },
+ "androidInstrumentationTest": { # A test of an Android application that can control an Android component # An Android instrumentation test.
+ # independently of its normal lifecycle.
+ # Android instrumentation tests run an application APK and test APK inside the
+ # same process on a virtual or physical AndroidDevice. They also specify
+ # a test runner class, such as com.google.GoogleTestRunner, which can vary
+ # on the specific instrumentation framework chosen.
+ #
+ # See <http://developer.android.com/tools/testing/testing_android.html> for
+ # more information on types of Android tests.
+ "orchestratorOption": "A String", # The option of whether running each test within its own invocation of
+ # instrumentation with Android Test Orchestrator or not.
+ # ** Orchestrator is only compatible with AndroidJUnitRunner version 1.0 or
+ # higher! **
+ # Orchestrator offers the following benefits:
+ # - No shared state
+ # - Crashes are isolated
+ # - Logs are scoped per test
+ #
+ # See
+ # <https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator>
+ # for more information about Android Test Orchestrator.
+ #
+ # If not set, the test will be run without the orchestrator.
+ "appBundle": { # An Android App Bundle file format, containing a BundleConfig.pb file, # A multi-apk app bundle for the application under test.
+ # a base module directory, zero or more dynamic feature module directories.
+ # <p>See https://developer.android.com/guide/app-bundle/build for guidance on
+ # building App Bundles.
+ "bundleLocation": { # A reference to a file, used for user inputs. # .aab file representing the app bundle under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "testTargets": [ # Each target must be fully qualified with the package name or class name,
+ # in one of these formats:
+ # - "package package_name"
+ # - "class package_name.class_name"
+ # - "class package_name.class_name#method_name"
+ #
+ # If empty, all targets in the module will be run.
+ "A String",
+ ],
+ "testApk": { # A reference to a file, used for user inputs. # Required. The APK containing the test code to be executed.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ "testRunnerClass": "A String", # The InstrumentationTestRunner class.
+ # The default value is determined by examining the application's manifest.
+ "shardingOption": { # Options for enabling sharding. # The option to run tests in multiple shards in parallel.
+ "manualSharding": { # Shards test cases into the specified groups of packages, classes, and/or # Shards test cases into the specified groups of packages, classes, and/or
+ # methods.
+ # methods.
+ #
+ # With manual sharding enabled, specifying test targets via
+ # environment_variables or in InstrumentationTest is invalid.
+ "testTargetsForShard": [ # Required. Group of packages, classes, and/or test methods to be run for
+ # each shard. The number of shard_test_targets must be >= 1 and <= 50.
+ { # Test targets for a shard.
+ "testTargets": [ # Group of packages, classes, and/or test methods to be run for each shard.
+ # The targets need to be specified in AndroidJUnitRunner argument format. For
+ # example, “package com.my.packages” “class com.my.package.MyClass”.
+ #
+ # The number of shard_test_targets must be greater than 0.
+ "A String",
+ ],
+ },
+ ],
+ },
+ "uniformSharding": { # Uniformly shards test cases given a total number of shards. # Uniformly shards test cases given a total number of shards.
+ #
+ # For Instrumentation test, it will be translated to “-e numShard” “-e
+ # shardIndex” AndroidJUnitRunner arguments. With uniform sharding enabled,
+ # specifying these sharding arguments via environment_variables is invalid.
+ "numShards": 42, # Required. Total number of shards. The number must be >= 1 and <= 50.
+ },
+ },
+ "testPackageId": "A String", # The java package for the test to be executed.
+ # The default value is determined by examining the application's manifest.
+ "appPackageId": "A String", # The java package for the application under test.
+ # The default value is determined by examining the application's manifest.
+ "appApk": { # A reference to a file, used for user inputs. # The APK for the application under test.
+ "gcsPath": "A String", # A path to a file in Google Cloud Storage.
+ # Example: gs://build-app-1414623860166/app-debug-unaligned.apk
+ },
+ },
+ "disablePerformanceMetrics": True or False, # Disables performance metrics recording. May reduce test latency.
+ "disableVideoRecording": True or False, # Disables video recording. May reduce test latency.
+ },
+ "id": "A String", # Output only. Unique id set by the service.
+ },
+ ],
+ "environmentMatrix": { # The matrix of environments in which the test is to be executed. # Required. The devices the tests are being executed on.
+ "androidDeviceList": { # A list of Android device configurations in which the test is to be executed. # A list of Android devices; the test will be run only on the specified
# devices.
- "androidDevices": [ # Required. A list of Android devices.
+ "androidDevices": [ # Required. A list of Android devices.
{ # A single Android device.
- "locale": "A String", # Required. The locale the test device used for testing.
+ "androidModelId": "A String", # Required. The id of the Android device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
+ "androidVersionId": "A String", # Required. The id of the Android OS version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "androidModelId": "A String", # Required. The id of the Android device to be used.
+ "orientation": "A String", # Required. How the device is oriented during the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
+ "locale": "A String", # Required. The locale the test device used for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
},
],
},
- "androidMatrix": { # A set of Android device configuration permutations is defined by the # A matrix of Android devices.
+ "androidMatrix": { # A set of Android device configuration permutations is defined by the # A matrix of Android devices.
# the cross-product of the given axes. Internally, the given AndroidMatrix
# will be expanded into a set of AndroidDevices.
#
# Only supported permutations will be instantiated. Invalid permutations
# (e.g., incompatible models/versions) are ignored.
- "orientations": [ # Required. The set of orientations to test with.
+ "androidModelIds": [ # Required. The ids of the set of Android device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "locales": [ # Required. The set of locales the test device will enable for testing.
+ "orientations": [ # Required. The set of orientations to test with.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "androidVersionIds": [ # Required. The ids of the set of Android OS version to be used.
+ "androidVersionIds": [ # Required. The ids of the set of Android OS version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
- "androidModelIds": [ # Required. The ids of the set of Android device to be used.
+ "locales": [ # Required. The set of locales the test device will enable for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "A String",
+ "A String",
],
},
- "iosDeviceList": { # A list of iOS device configurations in which the test is to be executed. # A list of iOS devices.
- "iosDevices": [ # Required. A list of iOS devices.
+ "iosDeviceList": { # A list of iOS device configurations in which the test is to be executed. # A list of iOS devices.
+ "iosDevices": [ # Required. A list of iOS devices.
{ # A single iOS device.
- "locale": "A String", # Required. The locale the test device used for testing.
+ "orientation": "A String", # Required. How the device is oriented during the test.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
+ "iosVersionId": "A String", # Required. The id of the iOS major software version to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "orientation": "A String", # Required. How the device is oriented during the test.
+ "locale": "A String", # Required. The locale the test device used for testing.
# Use the TestEnvironmentDiscoveryService to get supported options.
- "iosModelId": "A String", # Required. The id of the iOS device to be used.
+ "iosModelId": "A String", # Required. The id of the iOS device to be used.
# Use the TestEnvironmentDiscoveryService to get supported options.
},
],
},
},
- "outcomeSummary": "A String", # Output Only. The overall outcome of the test.
- # Only set when the test matrix state is FINISHED.
+ "resultStorage": { # Locations where the results of running the test are stored. # Required. Where the results for the matrix are written.
+ "resultsUrl": "A String", # Output only. URL to the results in the Firebase Web Console.
+ "toolResultsHistory": { # Represents a tool results history resource. # The tool results history that contains the tool results execution that
+ # results are written to.
+ #
+ # If not provided, the service will choose an appropriate value.
+ "projectId": "A String", # Required. The cloud project that owns the tool results history.
+ "historyId": "A String", # Required. A tool results history ID.
+ },
+ "googleCloudStorage": { # A storage location within Google cloud storage (GCS). # Required.
+ "gcsPath": "A String", # Required. The path to a directory in GCS that will
+ # eventually contain the results for this test.
+ # The requesting user must have write access on the bucket in the supplied
+ # path.
+ },
+ "toolResultsExecution": { # Represents a tool results execution resource. # Output only. The tool results execution that results are written to.
+ #
+ # This has the results of a TestMatrix.
+ "projectId": "A String", # Output only. The cloud project that owns the tool results execution.
+ "historyId": "A String", # Output only. A tool results history ID.
+ "executionId": "A String", # Output only. A tool results execution ID.
+ },
+ },
+ "testMatrixId": "A String", # Output only. Unique id set by the service.
}</pre>
</div>