Screen magnification - feature - framework.
This change is the initial check in of the screen magnification
feature. This feature enables magnification of the screen via
global gestures (assuming it has been enabled from settings)
to allow a low vision user to efficiently use an Android device.
Interaction model:
1. Triple tap toggles permanent screen magnification which is magnifying
the area around the location of the triple tap. One can think of the
location of the triple tap as the center of the magnified viewport.
For example, a triple tap when not magnified would magnify the screen
and leave it in a magnified state. A triple tapping when magnified would
clear magnification and leave the screen in a not magnified state.
2. Triple tap and hold would magnify the screen if not magnified and enable
viewport dragging mode until the finger goes up. One can think of this
mode as a way to move the magnified viewport since the area around the
moving finger will be magnified to fit the screen. For example, if the
screen was not magnified and the user triple taps and holds the screen
would magnify and the viewport will follow the user's finger. When the
finger goes up the screen will clear zoom out. If the same user interaction
is performed when the screen is magnified, the viewport movement will
be the same but when the finger goes up the screen will stay magnified.
In other words, the initial magnified state is sticky.
3. Pinching with any number of additional fingers when viewport dragging
is enabled, i.e. the user triple tapped and holds, would adjust the
magnification scale which will become the current default magnification
scale. The next time the user magnifies the same magnification scale
would be used.
4. When in a permanent magnified state the user can use two or more fingers
to pan the viewport. Note that in this mode the content is panned as
opposed to the viewport dragging mode in which the viewport is moved.
5. When in a permanent magnified state the user can use three or more
fingers to change the magnification scale which will become the current
default magnification scale. The next time the user magnifies the same
magnification scale would be used.
6. The magnification scale will be persisted in settings and in the cloud.
Note: Since two fingers are used to pan the content in a permanently magnified
state no other two finger gestures in touch exploration or applications
will work unless the uses zooms out to normal state where all gestures
works as expected. This is an intentional tradeoff to allow efficient
panning since in a permanently magnified state this would be the dominant
action to be performed.
Design:
1. The window manager exposes APIs for setting accessibility transformation
which is a scale and offsets for X and Y axis. The window manager queries
the window policy for which windows will not be magnified. For example,
the IME windows and the navigation bar are not magnified including windows
that are attached to them.
2. The accessibility features such a screen magnification and touch
exploration are now impemented as a sequence of transformations on the
event stream. The accessibility manager service may request each
of these features or both. The behavior of the features is not changed
based on the fact that another one is enabled.
3. The screen magnifier keeps a viewport of the content that is magnified
which is surrounded by a glow in a magnified state. Interactions outside
of the viewport are delegated directly to the application without
interpretation. For example, a triple tap on the letter 'a' of the IME
would type three letters instead of toggling magnified state. The viewport
is updated on screen rotation and on window transitions. For example,
when the IME pops up the viewport shrinks.
4. The glow around the viewport is implemented as a special type of window
that does not take input focus, cannot be touched, is laid out in the
screen coordiates with width and height matching these of the screen.
When the magnified region changes the root view of the window draws the
hightlight but the size of the window does not change - unless a rotation
happens. All changes in the viewport size or showing or hiding it are
animated.
5. The viewport is encapsulated in a class that knows how to show,
hide, and resize the viewport - potentially animating that.
This class uses the new animation framework for animations.
6. The magnification is handled by a magnification controller that
keeps track of the current trnasformation to be applied to the screen
content and the desired such. If these two are not the same it is
responsibility of the magnification controller to reconcile them by
potentially animating the transition from one to the other.
7. A dipslay content observer wathces for winodw transitions, screen
rotations, and when a rectange on the screen has been reqeusted. This
class is responsible for handling interesting state changes such
as changing the viewport bounds on IME pop up or screen rotation,
panning the content to make a requested rectangle visible on the
screen, etc.
8. To implement viewport updates the window manger was updated with APIs
to watch for window transitions and when a rectangle has been requested
on the screen. These APIs are protected by a signature level permission.
Also a parcelable and poolable window info class has been added with
APIs for getting the window info given the window token. This enables
getting some useful information about a window. There APIs are also
signature protected.
bug:6795382
Change-Id: Iec93da8bf6376beebbd4f5167ab7723dc7d9bd00
diff --git a/core/java/android/os/RemoteCallbackList.java b/core/java/android/os/RemoteCallbackList.java
index b74af16..d02a320 100644
--- a/core/java/android/os/RemoteCallbackList.java
+++ b/core/java/android/os/RemoteCallbackList.java
@@ -304,4 +304,25 @@
mBroadcastCount = -1;
}
+
+ /**
+ * Returns the number of registered callbacks. Note that the number of registered
+ * callbacks may differ from the value returned by {@link #beginBroadcast()} since
+ * the former returns the number of callbacks registered at the time of the call
+ * and the second the number of callback to which the broadcast will be delivered.
+ * <p>
+ * This function is useful to decide whether to schedule a broadcast if this
+ * requires doing some work which otherwise would not be performed.
+ * </p>
+ *
+ * @return The size.
+ */
+ public int getRegisteredCallbackCount() {
+ synchronized (mCallbacks) {
+ if (mKilled) {
+ return 0;
+ }
+ return mCallbacks.size();
+ }
+ }
}
diff --git a/core/java/android/provider/Settings.java b/core/java/android/provider/Settings.java
index 6b4bc76..d7ae441 100644
--- a/core/java/android/provider/Settings.java
+++ b/core/java/android/provider/Settings.java
@@ -3400,6 +3400,46 @@
"accessibility_web_content_key_bindings";
/**
+ * Setting that specifies whether the display magnification is enabled.
+ * Display magnifications allows the user to zoom in the display content
+ * and is targeted to low vision users. The current magnification scale
+ * is controlled by {@link #ACCESSIBILITY_DISPLAY_MAGNIFICATION_SCALE}.
+ *
+ * @hide
+ */
+ public static final String ACCESSIBILITY_DISPLAY_MAGNIFICATION_ENABLED =
+ "accessibility_display_magnification_enabled";
+
+ /**
+ * Setting that specifies what the display magnification scale is.
+ * Display magnifications allows the user to zoom in the display
+ * content and is targeted to low vision users. Whether a display
+ * magnification is performed is controlled by
+ * {@link #ACCESSIBILITY_DISPLAY_MAGNIFICATION_ENABLED}
+ *
+ * @hide
+ */
+ public static final String ACCESSIBILITY_DISPLAY_MAGNIFICATION_SCALE =
+ "accessibility_display_magnification_scale";
+
+ /**
+ * Setting that specifies whether the display magnification should be
+ * automatically updated. If this fearture is enabled the system will
+ * exit magnification mode or pan the viewport when a context change
+ * occurs. For example, on staring a new activity or rotating the screen,
+ * the system may zoom out so the user can see the new context he is in.
+ * Another example is on showing a window that is not visible in the
+ * magnified viewport the system may pan the viewport to make the window
+ * the has popped up so the user knows that the context has changed.
+ * Whether a screen magnification is performed is controlled by
+ * {@link #ACCESSIBILITY_DISPLAY_MAGNIFICATION_ENABLED}
+ *
+ * @hide
+ */
+ public static final String ACCESSIBILITY_DISPLAY_MAGNIFICATION_AUTO_UPDATE =
+ "accessibility_display_magnification_auto_update";
+
+ /**
* The timout for considering a press to be a long press in milliseconds.
* @hide
*/
@@ -4806,6 +4846,9 @@
PARENTAL_CONTROL_ENABLED,
PARENTAL_CONTROL_REDIRECT_URL,
USB_MASS_STORAGE_ENABLED,
+ ACCESSIBILITY_DISPLAY_MAGNIFICATION_ENABLED,
+ ACCESSIBILITY_DISPLAY_MAGNIFICATION_SCALE,
+ ACCESSIBILITY_DISPLAY_MAGNIFICATION_AUTO_UPDATE,
ACCESSIBILITY_SCRIPT_INJECTION,
BACKUP_AUTO_RESTORE,
ENABLED_ACCESSIBILITY_SERVICES,
diff --git a/core/java/android/view/IDisplayContentChangeListener.aidl b/core/java/android/view/IDisplayContentChangeListener.aidl
new file mode 100644
index 0000000..8f23ff6
--- /dev/null
+++ b/core/java/android/view/IDisplayContentChangeListener.aidl
@@ -0,0 +1,32 @@
+/*
+** Copyright 2012, The Android Open Source Project
+**
+** Licensed under the Apache License, Version 2.0 (the "License")
+** you may not use this file except in compliance with the License.
+** You may obtain a copy of the License at
+**
+** http://www.apache.org/licenses/LICENSE-2.0
+**
+** Unless required by applicable law or agreed to in writing, software
+** distributed under the License is distributed on an "AS IS" BASIS,
+** WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+** See the License for the specific language governing permissions and
+** limitations under the License.
+*/
+
+package android.view;
+
+import android.os.IBinder;
+import android.view.WindowInfo;
+import android.graphics.Rect;
+
+/**
+ * Interface for observing content changes on a display.
+ *
+ * {@hide}
+ */
+oneway interface IDisplayContentChangeListener {
+ void onWindowTransition(int displayId, int transition, in WindowInfo info);
+ void onRectangleOnScreenRequested(int displayId, in Rect rectangle, boolean immediate);
+ void onRotationChanged(int rotation);
+}
diff --git a/core/java/android/view/IWindowManager.aidl b/core/java/android/view/IWindowManager.aidl
index 7f2de50..b9f3942 100644
--- a/core/java/android/view/IWindowManager.aidl
+++ b/core/java/android/view/IWindowManager.aidl
@@ -26,6 +26,7 @@
import android.graphics.Rect;
import android.os.IRemoteCallback;
import android.view.IApplicationToken;
+import android.view.IDisplayContentChangeListener;
import android.view.IOnKeyguardExitResult;
import android.view.IRotationWatcher;
import android.view.IWindowSession;
@@ -34,7 +35,8 @@
import android.view.MotionEvent;
import android.view.InputChannel;
import android.view.InputDevice;
-import android.view.IInputFilter;
+import android.view.IInputFilter;
+import android.view.WindowInfo;
/**
* System private interface to the window manager.
@@ -214,11 +216,6 @@
IBinder getFocusedWindowToken();
/**
- * Gets the frame on the screen of the window given its token.
- */
- boolean getWindowFrame(IBinder token, out Rect outBounds);
-
- /**
* Gets the compatibility scale of e window given its token.
*/
float getWindowCompatibilityScale(IBinder windowToken);
@@ -227,4 +224,29 @@
* Sets an input filter for manipulating the input event stream.
*/
void setInputFilter(in IInputFilter filter);
+
+ /**
+ * Sets the scale and offset for implementing accessibility magnification.
+ */
+ void magnifyDisplay(int dipslayId, float scale, float offsetX, float offsetY);
+
+ /**
+ * Adds a listener for display content changes.
+ */
+ void addDisplayContentChangeListener(int displayId, IDisplayContentChangeListener listener);
+
+ /**
+ * Removes a listener for display content changes.
+ */
+ void removeDisplayContentChangeListener(int displayId, IDisplayContentChangeListener listener);
+
+ /**
+ * Gets the info for a window given its token.
+ */
+ WindowInfo getWindowInfo(IBinder token);
+
+ /**
+ * Gets the infos for all visible windows.
+ */
+ void getVisibleWindowsForDisplay(int displayId, out List<WindowInfo> outInfos);
}
diff --git a/core/java/android/view/IWindowSession.aidl b/core/java/android/view/IWindowSession.aidl
index c5d9255..ff9dcce 100644
--- a/core/java/android/view/IWindowSession.aidl
+++ b/core/java/android/view/IWindowSession.aidl
@@ -180,4 +180,9 @@
void setUniverseTransform(IBinder window, float alpha, float offx, float offy,
float dsdx, float dtdx, float dsdy, float dtdy);
+
+ /**
+ * Notifies that a rectangle on the screen has been requested.
+ */
+ void onRectangleOnScreenRequested(IBinder token, in Rect rectangle, boolean immediate);
}
diff --git a/core/java/android/view/View.java b/core/java/android/view/View.java
index fc5629e..e5d72ab 100644
--- a/core/java/android/view/View.java
+++ b/core/java/android/view/View.java
@@ -6259,6 +6259,11 @@
if (viewRootImpl != null) {
viewRootImpl.setAccessibilityFocus(this, null);
}
+ if (mAttachInfo != null) {
+ Rect rectangle = mAttachInfo.mTmpInvalRect;
+ getDrawingRect(rectangle);
+ requestRectangleOnScreen(rectangle);
+ }
invalidate();
sendAccessibilityEvent(AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUSED);
notifyAccessibilityStateChanged();
diff --git a/core/java/android/view/ViewRootImpl.java b/core/java/android/view/ViewRootImpl.java
index ffd495e..a4c0235 100644
--- a/core/java/android/view/ViewRootImpl.java
+++ b/core/java/android/view/ViewRootImpl.java
@@ -4649,9 +4649,19 @@
// ViewAncestor never intercepts touch event, so this can be a no-op
}
- public boolean requestChildRectangleOnScreen(View child, Rect rectangle,
- boolean immediate) {
- return scrollToRectOrFocus(rectangle, immediate);
+ public boolean requestChildRectangleOnScreen(View child, Rect rectangle, boolean immediate) {
+ final boolean scrolled = scrollToRectOrFocus(rectangle, immediate);
+ if (rectangle != null) {
+ mTempRect.set(rectangle);
+ mTempRect.offset(0, -mCurScrollY);
+ mTempRect.offset(mAttachInfo.mWindowLeft, mAttachInfo.mWindowTop);
+ try {
+ mWindowSession.onRectangleOnScreenRequested(mWindow, mTempRect, immediate);
+ } catch (RemoteException re) {
+ /* ignore */
+ }
+ }
+ return scrolled;
}
public void childHasTransientStateChanged(View child, boolean hasTransientState) {
diff --git a/core/java/android/view/WindowInfo.aidl b/core/java/android/view/WindowInfo.aidl
new file mode 100644
index 0000000..23e927a
--- /dev/null
+++ b/core/java/android/view/WindowInfo.aidl
@@ -0,0 +1,20 @@
+/*
+**
+** Copyright 2012, The Android Open Source Project
+**
+** Licensed under the Apache License, Version 2.0 (the "License")
+** you may not use this file except in compliance with the License.
+** You may obtain a copy of the License at
+**
+** http://www.apache.org/licenses/LICENSE-2.0
+**
+** Unless required by applicable law or agreed to in writing, software
+** distributed under the License is distributed on an "AS IS" BASIS,
+** WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+** See the License for the specific language governing permissions and
+** limitations under the License.
+*/
+
+package android.view;
+
+parcelable WindowInfo;
diff --git a/core/java/android/view/WindowInfo.java b/core/java/android/view/WindowInfo.java
new file mode 100644
index 0000000..00875ae
--- /dev/null
+++ b/core/java/android/view/WindowInfo.java
@@ -0,0 +1,153 @@
+/*
+ * Copyright (C) 2012 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package android.view;
+
+import android.graphics.Rect;
+import android.os.IBinder;
+import android.os.Parcel;
+import android.os.Parcelable;
+
+/**
+ * Information the state of a window.
+ *
+ * @hide
+ */
+public class WindowInfo implements Parcelable {
+
+ private static final int MAX_POOL_SIZE = 20;
+
+ private static int UNDEFINED = -1;
+
+ private static Object sPoolLock = new Object();
+ private static WindowInfo sPool;
+ private static int sPoolSize;
+
+ private WindowInfo mNext;
+ private boolean mInPool;
+
+ public IBinder token;
+
+ public final Rect frame = new Rect();
+
+ public final Rect touchableRegion = new Rect();
+
+ public int type;
+
+ public float compatibilityScale;
+
+ public boolean visible;
+
+ public int displayId;
+
+ private WindowInfo() {
+ /* do nothing - reduce visibility */
+ }
+
+ @Override
+ public int describeContents() {
+ return 0;
+ }
+
+ @Override
+ public void writeToParcel(Parcel parcel, int flags) {
+ parcel.writeStrongBinder(token);
+ parcel.writeParcelable(frame, 0);
+ parcel.writeParcelable(touchableRegion, 0);
+ parcel.writeInt(type);
+ parcel.writeFloat(compatibilityScale);
+ parcel.writeInt(visible ? 1 : 0);
+ parcel.writeInt(displayId);
+ recycle();
+ }
+
+ private void initFromParcel(Parcel parcel) {
+ token = parcel.readStrongBinder();
+ frame.set((Rect) parcel.readParcelable(null));
+ touchableRegion.set((Rect) parcel.readParcelable(null));
+ type = parcel.readInt();
+ compatibilityScale = parcel.readFloat();
+ visible = (parcel.readInt() == 1);
+ displayId = parcel.readInt();
+ }
+
+ public static WindowInfo obtain(WindowInfo other) {
+ WindowInfo info = obtain();
+ info.token = other.token;
+ info.frame.set(other.frame);
+ info.touchableRegion.set(other.touchableRegion);
+ info.type = other.type;
+ info.displayId = other.displayId;
+ info.compatibilityScale = other.compatibilityScale;
+ info.visible = other.visible;
+ return info;
+ }
+
+ public static WindowInfo obtain() {
+ synchronized (sPoolLock) {
+ if (sPoolSize > 0) {
+ WindowInfo info = sPool;
+ sPool = info.mNext;
+ info.mNext = null;
+ info.mInPool = false;
+ sPoolSize--;
+ return info;
+ } else {
+ return new WindowInfo();
+ }
+ }
+ }
+
+ public void recycle() {
+ if (mInPool) {
+ throw new IllegalStateException("Already recycled.");
+ }
+ clear();
+ synchronized (sPoolLock) {
+ if (sPoolSize < MAX_POOL_SIZE) {
+ mNext = sPool;
+ sPool = this;
+ mInPool = true;
+ sPoolSize++;
+ }
+ }
+ }
+
+ private void clear() {
+ token = null;
+ frame.setEmpty();
+ touchableRegion.setEmpty();
+ type = UNDEFINED;
+ displayId = UNDEFINED;
+ visible = false;
+ }
+
+ /**
+ * @see Parcelable.Creator
+ */
+ public static final Parcelable.Creator<WindowInfo> CREATOR =
+ new Parcelable.Creator<WindowInfo>() {
+ public WindowInfo createFromParcel(Parcel parcel) {
+ WindowInfo info = WindowInfo.obtain();
+ info.initFromParcel(parcel);
+ return info;
+ }
+
+ public WindowInfo[] newArray(int size) {
+ return new WindowInfo[size];
+ }
+ };
+}
diff --git a/core/java/android/view/WindowManager.java b/core/java/android/view/WindowManager.java
index 7412f39..fa2d4e8 100644
--- a/core/java/android/view/WindowManager.java
+++ b/core/java/android/view/WindowManager.java
@@ -205,7 +205,8 @@
@ViewDebug.IntToString(from = TYPE_HIDDEN_NAV_CONSUMER, to = "TYPE_HIDDEN_NAV_CONSUMER"),
@ViewDebug.IntToString(from = TYPE_DREAM, to = "TYPE_DREAM"),
@ViewDebug.IntToString(from = TYPE_NAVIGATION_BAR_PANEL, to = "TYPE_NAVIGATION_BAR_PANEL"),
- @ViewDebug.IntToString(from = TYPE_DISPLAY_OVERLAY, to = "TYPE_DISPLAY_OVERLAY")
+ @ViewDebug.IntToString(from = TYPE_DISPLAY_OVERLAY, to = "TYPE_DISPLAY_OVERLAY"),
+ @ViewDebug.IntToString(from = TYPE_MAGNIFICATION_OVERLAY, to = "TYPE_MAGNIFICATION_OVERLAY")
})
public int type;
@@ -467,6 +468,13 @@
public static final int TYPE_DISPLAY_OVERLAY = FIRST_SYSTEM_WINDOW+26;
/**
+ * Window type: Magnification overlay window. Used to highlight the magnified
+ * portion of a display when accessibility magnification is enabled.
+ * @hide
+ */
+ public static final int TYPE_MAGNIFICATION_OVERLAY = FIRST_SYSTEM_WINDOW+27;
+
+ /**
* End of types of system windows.
*/
public static final int LAST_SYSTEM_WINDOW = 2999;
diff --git a/core/java/android/view/WindowManagerPolicy.java b/core/java/android/view/WindowManagerPolicy.java
index b059afc..75554da 100644
--- a/core/java/android/view/WindowManagerPolicy.java
+++ b/core/java/android/view/WindowManagerPolicy.java
@@ -1106,6 +1106,14 @@
public void setLastInputMethodWindowLw(WindowState ime, WindowState target);
/**
+ * Returns whether magnification can be applied to the given window type.
+ *
+ * @param attrs The window's LayoutParams.
+ * @return Whether magnification can be applied.
+ */
+ public boolean canMagnifyWindow(WindowManager.LayoutParams attrs);
+
+ /**
* Print the WindowManagerPolicy's state into the given stream.
*
* @param prefix Text to print at the front of each line.