| page.title=Testing Concepts |
| parent.title=Testing |
| parent.link=index.html |
| @jd:body |
| |
| <div id="qv-wrapper"> |
| <div id="qv"> |
| <h2>In this document</h2> |
| <ol> |
| <li><a href="#TestStructure">Test Structure</a></li> |
| <li><a href="#TestAPI">Testing APIs</a> |
| <ol> |
| <li> |
| <a href="#JUnit">JUnit</a> |
| </li> |
| <li> |
| <a href="#Instrumentation">Instrumentation</a> |
| </li> |
| <li> |
| <a href="#TestingSupportLibraryClasses">Android Testing Support Library APIs</a> |
| </li> |
| <li> |
| <a href="#AssertionClasses">Assertion classes</a> |
| </li> |
| </ol> |
| </li> |
| <li> |
| <a href="#Monkeys">Monkey and Monkeyrunner</a> |
| </li> |
| </ol> |
| <h2>See also</h2> |
| <ol> |
| <li><a href="{@docRoot}training/testing/start/index.html">Getting Started with Testing</a></li> |
| </ol> |
| </div> |
| </div> |
| |
| <p> |
| This document describes key concepts related to Android app testing. It assumes you have a basic |
| knowledge of the <a href="http://junit.org/" class="external-link">JUnit</a> testing framework.</p> |
| |
| <h2 id="TestStructure">Test Structure</h2> |
| <p>Android testing is based on JUnit. In general, a JUnit test is a method whose statements test a |
| part of the application. You organize test methods into classes called |
| <em>test cases</em>. You can further organize these classes into <em>test suites</em>.</p> |
| <p>In JUnit, you build one or more test classes and use a test runner to |
| execute them. In Android, you use <a href="{@docRoot}tools/studio/index.html"> |
| Android Studio </a>(or the <a href="{@docRoot}tools/building/plugin-for-gradle.html"> |
| Android Plugin for Gradle</a>) to build one or more test source files into an |
| <em>Android test app</em>.</p> |
| |
| <p>From your testing environment, you can run your test in one of the following ways:</p> |
| <ul> |
| <li><strong>On your local machine:</strong> Compile the test classes and |
| execute them locally on the Java Virtual Machine (JVM) using the JUnit test runner.</li> |
| <li><strong>On a device or emulator</strong>: Install the test app and the app |
| under test to a physical device or emulator, and then execute your tests using an Android-specific |
| test runner (such as <a href="{@docRoot}tools/testing-support-library/index.html#AndroidJUnitRunner"> |
| {@code AndroidJUnitRunner}</a>).</li> |
| </ul> |
| |
| <p>The structure of your test code and the way you build and run the tests in Android Studio depend |
| on the type of testing you are performing. The following table summarizes the common testing types |
| for Android:</p> |
| |
| <table> |
| <tr><th>Type</th> |
| <th>Subtype</th> |
| <th>Description</th></tr> |
| |
| <tr><td rowspan="3">Unit tests</td> |
| <tr><td>Local Unit Tests</td> |
| <td>Unit tests that run on your local machine only. These tests are compiled to run locally |
| on the JVM to minimize execution time. Use this approach to run unit tests |
| that have no dependencies on the Android framework or have dependencies that mock objects can |
| satisfy.</td></tr> |
| <tr><td>Instrumented unit tests</td> |
| <td>Unit tests that run on an Android device or emulator. These tests have access to |
| {@link android.app.Instrumentation} information, such as the {@link android.content.Context} of the |
| app under test. Use this approach to run unit tests that have Android dependencies which mock |
| objects cannot easily satisfy.</td></tr> |
| |
| <tr><td rowspan="3">Integration Tests</td> |
| <tr><td>Components within your app only</td> |
| <td>This type of test verifies that the target app behaves as expected when a user performs |
| a specific action or enters a specific input in its activities. For example, it allows you to check |
| that the target app returns the correct UI output in response to user interactions in the app’s |
| activities. UI testing frameworks like |
| <a href="{@docRoot}tools/testing-support-library/index.html#Espresso">Espresso</a> allow you to |
| programmatically simulate user actions and test complex intra-app user interactions.</td></tr> |
| <tr><td>Cross-app Components</td> |
| <td>This type of test verifies the correct behavior of interactions between different user |
| apps or between user apps and system apps. For example, you might want to test that your app behaves |
| correctly when the user performs an action in the Android Settings menu. UI testing frameworks |
| that support cross-app interactions, such as UI Automator, allow you to create tests for such |
| scenarios.</td></tr> |
| </table> |
| |
| <p>Based on the type of test you want to create, you need to configure the test code source |
| location and the project dependencies in Android Studio as described in |
| <a href="{@docRoot}training/testing/start/index.html">Getting Started with Testing</a>.</p> |
| |
| <h2 id="TestAPI">Testing APIs</h2> |
| <p>The following list summarizes the common APIs related to app testing for Android.</p> |
| |
| <h3 id="JUnit">JUnit</h3> |
| |
| <p>You should write your unit or integration test class as a JUnit 4 test class. |
| <a href="http://junit.org/" class="external-link">JUnit</a> is the most popular |
| and widely-used unit testing framework for Java. The framework offers a convenient way to perform |
| common setup, teardown, and assertion operations in your test.</p> |
| |
| <p>JUnit 4 allows you to write tests in a cleaner and more |
| flexible way than its predecessor versions. Unlike the previous approach to Android unit testing |
| based on JUnit 3, with JUnit 4, you do not need to extend the {@code junit.framework.TestCase} |
| class. You also do not need to prepend the {@code test} keyword to your test method name, or |
| use any classes in the {@code junit.framework} or {@code junit.extensions} package.</p> |
| |
| <p>A basic JUnit 4 test class is a Java class that contains one or more test methods. |
| A test method begins with the {@code @Test} annotation and contains the code to exercise |
| and verify a single functionality (that is, a logical <em>unit</em>) in the component that you want |
| to test.</p> |
| <p>The following snippet shows an example JUnit 4 integration test that uses the Espresso |
| APIs to perform a click action on a UI element, then checks to see if an expected string is |
| displayed.</p> |
| <pre> |
| @RunWith(AndroidJUnit4.class) |
| @LargeTest |
| public class MainActivityInstrumentationTest { |
| |
| @Rule |
| public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<>( |
| MainActivity.class); |
| |
| @Test |
| public void sayHello(){ |
| onView(withText("Say hello!")).perform(click()); |
| |
| onView(withId(R.id.textView)).check(matches(withText("Hello, World!"))); |
| } |
| } |
| </pre> |
| <p>In your JUnit 4 test class, you can call out sections in your test code for |
| special processing by using the following annotations:</p> |
| <ul> |
| <li> |
| {@code @Before}: Use this annotation to specify a block of code that contains test setup |
| operations. The test class invokes this code block before each test. You can have multiple |
| {@code @Before} methods but the order in which the test class calls these methods |
| is not guaranteed. |
| </li> |
| <li> |
| {@code @After}: This annotation specifies a block of code that contains test |
| tear-down operations. The test class calls this code block after every test method. You can define |
| multiple {@code @After} operations in your test code. Use this annotation to release any |
| resources from memory. |
| </li> |
| <li> |
| {@code @Test}: Use this annotation to mark a test method. A single test class can contain |
| multiple test methods, each prefixed with this annotation. |
| </li> |
| <li> |
| {@code @Rule}: Rules allow you to flexibly add or redefine the behavior of each test |
| method in a reusable way. In Android testing, use this annotation together with |
| one of the test rule classes that the Android Testing Support Library provides, such as |
| <a href="{@docRoot}reference/android/support/test/rule/ActivityTestRule.html"> |
| {@code ActivityTestRule}</a> or |
| <a href="{@docRoot}reference/android/support/test/rule/ServiceTestRule.html"> |
| {@code ServiceTestRule}</a>. |
| </li> |
| <li> |
| {@code @BeforeClass}: Use this annotation to specify static methods for each test class to |
| invoke only once. This testing step is useful for expensive operations such as connecting to a |
| database. |
| </li> |
| <li> |
| {@code @AfterClass}: Use this annotation to specify static methods for the test class to invoke |
| only after all tests in the class have run. This testing step is useful for releasing any |
| resources allocated in the {@code @BeforeClass} block. |
| </li> |
| <li> |
| {@code @Test(timeout=<milliseconds>)}: Some annotations support the ability to pass in |
| elements for which you can set values. For example, you can specify a timeout period for the test. |
| If the test starts but does not complete within the given timeout period, it automatically fails. |
| You must specify the timeout period in milliseconds, for example: {@code @Test(timeout=5000)}. |
| </li> |
| </ul> |
| <p>For more annotations, see the documentation for |
| <a href="//junit.sourceforge.net/javadoc/org/junit/package-summary.html" class="external-link"> |
| JUnit annotations</a> and the |
| <a href="{@docRoot}android/support/test/annotation/package-summary.html"> |
| Android-specific annotations</a>. |
| <p> |
| You use the JUnit {@link junit.framework.Assert} class to verify the correctness of an object's |
| state. The assert methods compare values you expect from a test to the actual results and |
| throw an exception if the comparison fails. <a href="#AssertionClasses"> |
| Assertion classes</a> describes these methods in more detail. |
| </p> |
| |
| </p> |
| <h3 id="Instrumentation">Instrumentation</h3> |
| <p> |
| Android instrumentation is a set of control methods or hooks in the Android system. These |
| hooks control an Android component independently of its normal lifecycle. They also control how |
| Android loads applications. |
| </p> |
| <p> |
| The following diagram summarizes the testing framework: |
| </p> |
| <div style="width: 70%; margin-left:auto; margin-right:auto;"> |
| <a href="{@docRoot}images/testing/test_framework.png"> |
| <img src="{@docRoot}images/testing/test_framework.png" |
| alt="The Android testing framework"/> |
| </a> |
| </div> |
| <p> |
| Normally, an Android component runs in a lifecycle that the system determines. For example, an |
| {@link android.app.Activity} object's lifecycle starts when an {@link android.content.Intent} |
| activates the {@link android.app.Activity}. The system calls the object's <code>onCreate()</code> |
| method, on then the <code>onResume()</code> method. When the user starts another application, the |
| system calls the <code>onPause()</code> method. If the {@link android.app.Activity} code calls |
| the <code>finish()</code> method, the system calls the <code>onDestroy()</code> method. The Android |
| framework API does not provide a way for your code to invoke these callback methods directly, but |
| you can do so using instrumentation. |
| </p> |
| <p> |
| The system runs all the components of an application in the same process. You can allow some |
| components, such as content providers, to run in a separate process, |
| but you can't force an application to run in the same process as another application that is |
| already running. |
| </p> |
| <p> |
| Instrumentation can load both a test package and the app under test into the |
| same process. Since the application components and their tests are in the same process, your |
| tests can invoke methods in the components, and modify and examine fields in the components. |
| </p> |
| <h3 id="TestingSupportLibraryClasses">Android Testing Support Library APIs</h3> |
| <p> |
| The <a href="{@docRoot}tools/testing-support-library/index.html">Android Testing Support Library</a> |
| provides a set of APIs that allow you to quickly build and run test code for your apps, including |
| JUnit 4 and functional user interface (UI) tests. The library includes the following |
| instrumentation-based APIs that are useful when you want to automate your tests:</p> |
| <ul> |
| <li><a href="{@docRoot}tools/testing-support-library/index.html#AndroidJUnitRunner"> |
| {@code AndroidJUnitRunner}</a>: |
| JUnit 4-compatible test runner for Android |
| </li> |
| |
| <li><a href="{@docRoot}tools/testing-support-library/index.html#Espresso">Espresso</a>: |
| UI testing framework; suitable for functional UI testing within an app |
| </li> |
| |
| <li><a href="{@docRoot}tools/testing-support-library/index.html#UIAutomator">UI Automator</a>: |
| UI testing framework; suitable for cross-app functional UI testing across system and installed |
| apps</li> |
| </ul> |
| |
| <h3 id="AssertionClasses">Assertion classes</h3> |
| <p>Because Android Testing Support Library APIs extend JUnit, you can use assertion methods to |
| display the results of tests. An assertion method compares an actual value returned by a test to an |
| expected value, and throws an AssertionException if the comparison test fails. Using assertions |
| is more convenient than logging, and provides better test performance. |
| </p> |
| <p>To simplify your test development, we recommend that you use the |
| <a href="https://github.com/hamcrest" |
| class="external-link">Hamcrest</a> library, which lets you create more flexible tests using the |
| Hamcrest matcher APIs.</p> |
| |
| <h2 id="Monkeys">Monkey and Monkeyrunner</h2> |
| <p> |
| The SDK provides two tools for functional-level application testing: |
| </p> |
| <ul> |
| <li> |
| The <a href="{@docRoot}tools/help/monkey.html">UI/Application Exerciser Monkey</a>, |
| usually called "monkey", is a command-line tool that sends pseudo-random streams of |
| keystrokes, touches, and gestures to a device. You run it with the |
| <a href="{@docRoot}tools/help/adb.html">Android Debug Bridge</a> (adb) tool. |
| You use it to stress-test your application and report back errors that are encountered. |
| You can repeat a stream of events by running the tool each time with the same random |
| number seed. |
| </li> |
| <li> |
| The <a href="{@docRoot}tools/help/monkeyrunner_concepts.html">monkeyrunner</a> tool |
| is an API and execution environment for test programs written in Python. The API |
| includes functions for connecting to a device, installing and uninstalling packages, |
| taking screenshots, comparing two images, and running a test package against an |
| application. Using the API, you can write a wide range of large, powerful, and complex |
| tests. You run programs that use the API with the <code>monkeyrunner</code> command-line |
| tool. |
| </li> |
| </ul> |