Data Usage Docs in Android 4.0 for Systems Integrators.
Change-Id: I65606843ebe5396037c43cf633a0b8c536bfaa1d
diff --git a/src/tech/datausage/excluding-network-types.md b/src/tech/datausage/excluding-network-types.md
new file mode 100644
index 0000000..dcaf64f
--- /dev/null
+++ b/src/tech/datausage/excluding-network-types.md
@@ -0,0 +1,29 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+A mobile operator may wish to exclude specific network types from the
+total data usage calculated by a device. For example, network traffic
+over an MMS APN may be “zero-rated” by a mobile operator. To support
+this, the set of network types used to calculate total data usage can
+be configured through the `config_data_usage_network_types` resource
+at build time.
+
+Some mobile radio implementations may have unique Linux network
+interfaces for each active APN, while other radios may force multiple
+APNs to coexist on a single interface. Android can collect network
+statistics from both designs, but `config_data_usage_network_types` is
+not be effective at excluding APNs forced to coexist on a single
+interface.
diff --git a/src/tech/datausage/iface-overview.md b/src/tech/datausage/iface-overview.md
new file mode 100644
index 0000000..fd1e3cc
--- /dev/null
+++ b/src/tech/datausage/iface-overview.md
@@ -0,0 +1,49 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+
+In Android 4.0, statistics reported by Linux network interfaces are
+recorded over time, and are used to enforce network quota limits,
+render user-visible charts, and more.
+
+Each network device driver (Wi-Fi included) must follow the standard
+kernel device lifecycle, and return correct statistics through
+`dev_get_stats()`. In particular, statistics returned must remain
+strictly monotonic while the interface is active. Drivers may reset
+statistics only after successfully completing an `unregister_netdev()`
+or the equivalent that generates a `NETDEV_UNREGISTER` event for
+callbacks registered with `register_netdevice_notifier()` /
+`register_inetaddr_notifier()` / `register_inet6addr_notifier()`.
+
+Mobile operators typically measure data usage at the Internet layer
+(IP). To match this approach in Android 4.0, we rely on the fact that
+for the kernel devices we care about the `rx_bytes` and `tx_bytes`
+values returned by `dev_get_stats()` return exactly the Internet layer
+(`IP`) bytes transferred. But we understand that for other devices it
+might not be the case. For now, the feature relies on this
+peculiarity. New drivers should have that property also, and the
+`dev_get_stats()` values must not include any encapsulation overhead
+of lower network layers (such as Ethernet headers), and should
+preferably not include other traffic (such as ARP) unless it is
+negligible.
+
+The Android framework only collects statistics from network interfaces
+associated with a `NetworkStateTracker` in `ConnectivityService`. This
+enables the framework to concretely identify each network interface,
+including its type (such as `TYPE_MOBILE` or `TYPE_WIFI`) and
+subscriber identity (such as IMSI). All network interfaces used to
+route data should be represented by a `NetworkStateTracker` so that
+statistics can be accounted correctly.
diff --git a/src/tech/datausage/index.md b/src/tech/datausage/index.md
new file mode 100644
index 0000000..279f851
--- /dev/null
+++ b/src/tech/datausage/index.md
@@ -0,0 +1,37 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Data Usage Technical Information #
+
+Android 4.0 (Ice Cream Sandwich) introduces new features that help
+users understand and control how their device uses network data. It
+monitors overall data usage, and supports warning or limit thresholds
+which will trigger notifications or disable mobile data when usage
+exceeds a specific quota.
+
+Data usage is also tracked on a per-application basis, enabling users
+to visually explore historical usage in the Settings app. Users can
+also restrict how specific applications are allowed to use data when
+running in the background.
+
+The documentation in this section is intended for systems integrators
+and mobile operators, to help explain technical details they should be
+aware of when porting Android to specific devices. These details are
+summarized below, and the
+[android-porting](mailto:android-porting+subscribe@googlegroups.com)
+mailing list is a good place for further discussion.
+
+
diff --git a/src/tech/datausage/kernel-changes.md b/src/tech/datausage/kernel-changes.md
new file mode 100644
index 0000000..f89847f
--- /dev/null
+++ b/src/tech/datausage/kernel-changes.md
@@ -0,0 +1,29 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+This is a summary of the main changes in the kernel that diverge from mainline.
+
+* added net/netfilter/xt_qtaguid*
+* imported then modified net/netfilter/xt_quota2.c from xtables-addons project
+* fixes in net/netfilter/ip6_tables.c
+* modified ip*t_REJECT.c
+* modified net/netfilter/xt_socket.c
+
+A few comments on the kernel configuration:
+
+* xt_qtaguid masquerades as xt_owner and relies on xt_socket and itself relies on the connection tracker.
+* The connection tracker can't handle large SIP packets, it must be disabled.
+* The modified xt_quota2 uses the NFLOG support to notify userspace.
diff --git a/src/tech/datausage/kernel-overview.md b/src/tech/datausage/kernel-overview.md
new file mode 100644
index 0000000..d21e0a3
--- /dev/null
+++ b/src/tech/datausage/kernel-overview.md
@@ -0,0 +1,55 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Overview #
+
+The per-application/delegated data usage monitoring and tracking
+functionality relies on the xt_qtaguid module in the android-3.0 Linux
+kernel (`kernel/net/netfilter/xt_qtaguid`). The socket tagging
+functionality in the framework (`system/core/libcutils/qtaguid.c`)
+relies mainly on the existence of `/proc/net/xt_qtaguid/ctrl`
+interface exported by the `xt_qtaguid` kernel module.
+
+The `quota2` netfilter module (originally part of `xtables-addons`)
+allows the functionality to set named quota limits and was extended to
+support notifying userspace when certain limits are reached. Once the
+quota limit is reached, the `quota2` module discards all subsequent
+network traffic. The framework can also specify additional rules to
+restrict background data traffic for an application (refer to
+`com.android.server.NetworkManagementSocketTagger.setKernelCounterSet`
+and
+`android.net.NetworkPolicyManager.POLICY_REJECT_METERED_BACKGROUND`).
+
+# How does it work? #
+
+The `qtaguid` netfilter module tracks the network traffic on a
+per-socket basis for every application using the unique UID of the
+owning application. There are two tag components associated with any
+socket in the system. The first is the UID which uniquely identifies
+the application which is responsible for the data transfer (Linux
+allows the ability to ascribe the ownership of each network socket to
+the UID of the calling application). The second tag component is used
+to support additional characterization of the traffic into application
+developer specified categories. Using these application level tags, an
+application can profile the traffic into several sub-categories.
+
+In the case of applications that provide network data transfer as a
+service, such as the download manager, media streaming service, etc,
+it is possible to attribute the ownership of the network data transfer
+to the UID of the requesting application using the
+`TrafficStats.setThreadStatsUid()` function call. The caller must hold
+the “`android.permission.MODIFY_NETWORK_ACCOUNTING`” permission to
+re-assign the ownership of the network traffic.
diff --git a/src/tech/datausage/sidebar2.md b/src/tech/datausage/sidebar2.md
new file mode 100644
index 0000000..348c684
--- /dev/null
+++ b/src/tech/datausage/sidebar2.md
@@ -0,0 +1,11 @@
+# Network interface statistics #
+- [Overview](/tech/datausage/iface-overview.html)
+- [Excluding network types from data usage](/tech/datausage/excluding-network-types.html)
+- [Tethering data](/tech/datausage/tethering-data.html)
+- [Usage cycle reset dates](/tech/datausage/usage-cycle-resets-dates.html)
+
+# The xt_qtaguid netfilter kernel module #
+- [Overview](/tech/datausage/kernel-overview.html)
+- [Data usage tags explained](/tech/datausage/tags-explained.html)
+- [Kernel changes](/tech/datausage/kernel-changes.html)
+
diff --git a/src/tech/datausage/tags-explained.md b/src/tech/datausage/tags-explained.md
new file mode 100644
index 0000000..90b6a29
--- /dev/null
+++ b/src/tech/datausage/tags-explained.md
@@ -0,0 +1,48 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+Tags represent one of the metrics the data usage counters will be
+tracked against. By default, and implicitly, a tag is just based on
+the UID. The UID is used as the base for policing, and cannot be
+ignored. So a tag will always at least represent a UID (uid_tag). A
+tag can be explicitly augmented with an "accounting tag" which is
+associated with a UID. User space can use
+`TrafficStats.setThreadStatsTag()` to set the acct_tag portion of the
+tag which is then used with sockets: all data belonging to that
+socket will be counted against the tag. The policing is then based on
+the tag's uid_tag portion, and stats are collected for the acct_tag
+portion separately.
+
+Without explicit tagging, the qtaguid module will assume the
+`default_tag: {acct_tag=0, uid_tag=10003}`
+
+ a: {acct_tag=1, uid_tag=10003}
+ b: {acct_tag=2, uid_tag=10003}
+ c: {acct_tag=3, uid_tag=10003}
+
+`a, b, c…` represent explicit tags associated with specific sockets.
+
+`default_tag (acct_tag=0)` is the default accounting tag that contains
+the total traffic for that uid, including all untagged
+traffic, and is typically used to enforce policing/quota rules.
+
+These tags can be used to profile the network traffic of an
+application into separate logical categories (at a network socket
+level). Such tags can be removed, reapplied, or modified during
+runtime.
+
+The qtaguid module has been implemented on [kernel/common branch of
+android-3.0](https://android-review.googlesource.com/#/q/project:kernel/common+branch:android-3.0,n,z)
diff --git a/src/tech/datausage/tethering-data.md b/src/tech/datausage/tethering-data.md
new file mode 100644
index 0000000..0bc8f89
--- /dev/null
+++ b/src/tech/datausage/tethering-data.md
@@ -0,0 +1,20 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+Tethering involves forwarding of traffic from one network interface to
+another using `iptables` forwarding rules. The framework periodically
+records tethering statistics between any interface pairs returned by
+`ConnectivityService.getTetheredIfacePairs()`.
diff --git a/src/tech/datausage/usage-cycle-resets-dates.md b/src/tech/datausage/usage-cycle-resets-dates.md
new file mode 100644
index 0000000..9bddad6
--- /dev/null
+++ b/src/tech/datausage/usage-cycle-resets-dates.md
@@ -0,0 +1,22 @@
+<!--
+ Copyright 2012 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+Users can specify a day of month upon which their data usage
+resets. Internally, cycle boundaries are defined to end at midnight
+`(00:00) UTC` on the requested day. When a month is shorter than the
+requested day, the cycle resets on the first day of the subsequent
+month. For example, a cycle reset day of the 30th would cause a reset
+on January 30 at `00:00 UTC` and March 1 at `00:00 UTC`.
diff --git a/src/tech/index.md b/src/tech/index.md
index 5653bda..d871a9a 100644
--- a/src/tech/index.md
+++ b/src/tech/index.md
@@ -58,3 +58,13 @@
keyboard, joysticks, mice and other devices.
[» Input Information](/tech/input/index.html)
+
+## Data Usage Technical Information ##
+Android's data usage features allow users to understand and control how their
+device uses network data. This document is designed for systems integrators
+and mobile operators, to help explain technical details they should be aware
+of when porting Android to specific devices.
+
+[» Data Usage Information](/tech/datausage/index.html)
+
+
diff --git a/src/tech/sidebar.md b/src/tech/sidebar.md
index 3f81124..c3701fd 100644
--- a/src/tech/sidebar.md
+++ b/src/tech/sidebar.md
@@ -4,3 +4,4 @@
- [Encryption](/tech/encryption/index.html)
- [Security](/tech/security/index.html)
- [Input](/tech/input/index.html)
+- [Data Usage](/tech/datausage/index.html)