Added accessibility APIs for introspecting interactive windows.
1. The old introspection model was allowing querying only the active window
which is the one the user is touching or the focused one if no window is
touched. This was limiting as auto completion drop downs were not inspectable,
there was not way to know when the IME toggles, non-focusable windows were
not inspectable if the user taps them as until a screen-reader starts
introspecting the users finger is up, accessibility focus was limited to
only one window and the user couldn't use gestures to visit the whole UI,
and other things I can't remember right now.
The new APIs allow getting all interactive windows, i.e. ones that a
sighted user can interact with. This prevents an accessibility service
from interacting with content a sighter user cannot. The list of windows
can be obtained from an accessibility service or the host window from an
accessibility node info. Introspecting windows obey the same rules for
introspecting node, i.e. the service has to declare this capability
in its manifest.
When some windows change accessibility services receive a new type
of event. Initially the types of windows is very limited. We provide
the bounds in screen, layer, and some other properties which are
enough for a client to determined the spacial and hierarchical
relationship of the windows.
2. Update the documentation in AccessibilityService for newer event types.
3. LongArray was not removing elements properly.
4. Composite accessibility node ids were not properly constructed as they
are composed of two ints, each taking 32 bits. However, the values for
undefined were -1 so composing a 64 long from -1, -1 prevents from getting
back these values when unpacking.
5. Some apps were generating inconsistent AccessibilityNodeInfo tree. Added
a check that enforces such trees to be well formed on dev builds.
6. Removed an necessary code for piping the touch exploration state to
the policy as it should just use the AccessibilityManager from context.
7. When view's visibility changed it was not firing an event to notify
clients it disappeared/appeared. Also ViewGroup was sending accessibility
events for changes if the view is included for accessibility but this is
wrong as there may be a service that want all nodes, hence events from them.
The accessibility manager service takes care of delivering events from
not important for accessibility nodes only to services that want such.
8. Several places were asking for prefetching of sibling but not predecessor
nodes which resulted in prefetching of unconnected subtrees.
9. The local AccessibilityManager implementation was relying on the backing
service being ready when it is created but it can be fetched from a context
before that. If that happens the local manager was in a broken state forever.
Now it is more robust and starts working properly once the backing service
is up. Several places were lacking locking.
bug:13331285
Change-Id: Ie51166d4875d5f3def8d29d77973da4b9251f5c8
diff --git a/core/java/android/accessibilityservice/AccessibilityServiceInfo.java b/core/java/android/accessibilityservice/AccessibilityServiceInfo.java
index bdc4fdde..4f9ba59 100644
--- a/core/java/android/accessibilityservice/AccessibilityServiceInfo.java
+++ b/core/java/android/accessibilityservice/AccessibilityServiceInfo.java
@@ -284,6 +284,27 @@
public static final int FLAG_REQUEST_FILTER_KEY_EVENTS = 0x00000020;
/**
+ * This flag indicates to the system that the accessibility service wants
+ * to access content of all interactive windows. An interactive window is a
+ * window that can be touched by a sighted user when explore by touch is not
+ * enabled. If this flag is not set your service will not receive
+ * {@link android.view.accessibility.AccessibilityEvent#TYPE_WINDOWS_CHANGED}
+ * events, calling AccessibilityService{@link AccessibilityService#getWindows()
+ * AccessibilityService.getWindows()} will return an empty list, and {@link
+ * AccessibilityNodeInfo#getWindow() AccessibilityNodeInfo.getWindow()} will
+ * return null.
+ * <p>
+ * Services that want to set this flag have to declare the capability
+ * to retrieve window content in their meta-data by setting the attribute
+ * {@link android.R.attr#canRetrieveWindowContent canRetrieveWindowContent} to
+ * true, otherwise this flag will be ignored. For how to declare the meta-data
+ * of a service refer to {@value AccessibilityService#SERVICE_META_DATA}.
+ * </p>
+ * @see android.R.styleable#AccessibilityService_canRetrieveWindowContent
+ */
+ public static final int FLAG_RETRIEVE_INTERACTIVE_WINDOWS = 0x00000040;
+
+ /**
* The event types an {@link AccessibilityService} is interested in.
* <p>
* <strong>Can be dynamically set at runtime.</strong>
@@ -302,6 +323,15 @@
* @see android.view.accessibility.AccessibilityEvent#TYPE_VIEW_SCROLLED
* @see android.view.accessibility.AccessibilityEvent#TYPE_VIEW_TEXT_SELECTION_CHANGED
* @see android.view.accessibility.AccessibilityEvent#TYPE_WINDOW_CONTENT_CHANGED
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_TOUCH_INTERACTION_START
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_TOUCH_INTERACTION_END
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_ANNOUNCEMENT
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_GESTURE_DETECTION_START
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_GESTURE_DETECTION_END
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_VIEW_ACCESSIBILITY_FOCUSED
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_VIEW_ACCESSIBILITY_FOCUS_CLEARED
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_VIEW_TEXT_TRAVERSED_AT_MOVEMENT_GRANULARITY
+ * @see android.view.accessibility.AccessibilityEvent#TYPE_WINDOWS_CHANGED
*/
public int eventTypes;
@@ -354,6 +384,7 @@
* @see #FLAG_REQUEST_ENHANCED_WEB_ACCESSIBILITY
* @see #FLAG_REQUEST_FILTER_KEY_EVENTS
* @see #FLAG_REPORT_VIEW_IDS
+ * @see #FLAG_RETRIEVE_INTERACTIVE_WINDOWS
*/
public int flags;
@@ -449,7 +480,7 @@
com.android.internal.R.styleable.AccessibilityService_accessibilityFeedbackType,
0);
notificationTimeout = asAttributes.getInt(
- com.android.internal.R.styleable.AccessibilityService_notificationTimeout,
+ com.android.internal.R.styleable.AccessibilityService_notificationTimeout,
0);
flags = asAttributes.getInt(
com.android.internal.R.styleable.AccessibilityService_accessibilityFlags, 0);
@@ -861,6 +892,8 @@
return "FLAG_REPORT_VIEW_IDS";
case FLAG_REQUEST_FILTER_KEY_EVENTS:
return "FLAG_REQUEST_FILTER_KEY_EVENTS";
+ case FLAG_RETRIEVE_INTERACTIVE_WINDOWS:
+ return "FLAG_RETRIEVE_INTERACTIVE_WINDOWS";
default:
return null;
}