Merge tag 'char-misc-3.6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc
Pull char/misc driver merge from Greg Kroah-Hartman:
"Here is the "big" char/misc driver tree update for the 3.7-rc1 merge
window.
Nothing major, just a number of driver updates and fixes, all of which
have been in the linux-next releases for a while now either in my
tree, or in Andrew's (the lis3l driver changes came from his tree last
week).
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>"
* tag 'char-misc-3.6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc: (52 commits)
drivers/misc/lis3lv02d/lis3lv02d_i2c.c: add lis3lv02d device tree init
drivers/misc/lis3lv02d/lis3lv02d_spi.c: add lis3lv02d device tree init
drivers/misc/lis3lv02d: remove lis3lv02d driver DT init
drivers/misc/lis3lv02d/lis3lv02d_spi.c: add DT matching table passthru code
drivers/misc/lis3lv02d: add generic DT matching code
lis3lv02d: fix some comments specific to lis331dlh driver
MISC: hpilo, remove pci_disable_device
pcmcia: synclink_cs: fix potential tty NULL dereference
drivers/char/mmtimer.c: Remove useless kfree
drivers/char: removes unnecessary semicolon
char/misc: remove CONFIG_EXPERIMENTAL dependencies
mei: don't print buffer as a string
mei: struct mei_message_data doesn't have to be packed
mei: add error messages for open count errors
misc: use module_spi_driver
tifm: use module_pci_driver
misc/at25, dt: Improve at25 SPI eeprom device tree bindings.
mei: add lynx point pci device ids
mei: fix max number of open handles
mei: rename struct pci_dev *mei_device to mei_pdev
...
diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
index 34f5110..dff1f48 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci
+++ b/Documentation/ABI/testing/sysfs-bus-pci
@@ -210,3 +210,15 @@
firmware assigned instance number of the PCI
device that can help in understanding the firmware
intended order of the PCI device.
+
+What: /sys/bus/pci/devices/.../d3cold_allowed
+Date: July 2012
+Contact: Huang Ying <ying.huang@intel.com>
+Description:
+ d3cold_allowed is bit to control whether the corresponding PCI
+ device can be put into D3Cold state. If it is cleared, the
+ device will never be put into D3Cold state. If it is set, the
+ device may be put into D3Cold state if other requirements are
+ satisfied too. Reading this attribute will show the current
+ value of d3cold_allowed bit. Writing this attribute will set
+ the value of d3cold_allowed bit.
diff --git a/Documentation/ABI/testing/sysfs-class-regulator b/Documentation/ABI/testing/sysfs-class-regulator
index e091fa8..bc578bc 100644
--- a/Documentation/ABI/testing/sysfs-class-regulator
+++ b/Documentation/ABI/testing/sysfs-class-regulator
@@ -349,3 +349,24 @@
This will be one of the same strings reported by
the "state" attribute.
+
+What: /sys/class/regulator/.../bypass
+Date: September 2012
+KernelVersion: 3.7
+Contact: Mark Brown <broonie@opensource.wolfsonmicro.com>
+Description:
+ Some regulator directories will contain a field called
+ bypass. This indicates if the device is in bypass mode.
+
+ This will be one of the following strings:
+
+ 'enabled'
+ 'disabled'
+ 'unknown'
+
+ 'enabled' means the regulator is in bypass mode.
+
+ 'disabled' means that the regulator is regulating.
+
+ 'unknown' means software cannot determine the state, or
+ the reported state is invalid.
diff --git a/Documentation/ABI/testing/sysfs-driver-wacom b/Documentation/ABI/testing/sysfs-driver-wacom
index 8d55a83..7fc7810 100644
--- a/Documentation/ABI/testing/sysfs-driver-wacom
+++ b/Documentation/ABI/testing/sysfs-driver-wacom
@@ -1,3 +1,16 @@
+WWhat: /sys/class/hidraw/hidraw*/device/oled*_img
+Date: June 2012
+Contact: linux-bluetooth@vger.kernel.org
+Description:
+ The /sys/class/hidraw/hidraw*/device/oled*_img files control
+ OLED mocro displays on Intuos4 Wireless tablet. Accepted image
+ has to contain 256 bytes (64x32 px 1 bit colour). The format
+ is the same as PBM image 62x32px without header (64 bits per
+ horizontal line, 32 lines). An example of setting OLED No. 0:
+ dd bs=256 count=1 if=img_file of=[path to oled0_img]/oled0_img
+ The attribute is read only and no local copy of the image is
+ stored.
+
What: /sys/class/hidraw/hidraw*/device/speed
Date: April 2010
Kernel Version: 2.6.35
diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
index fc103d7..cdb20d4 100644
--- a/Documentation/RCU/checklist.txt
+++ b/Documentation/RCU/checklist.txt
@@ -310,6 +310,12 @@
code under the influence of preempt_disable(), you instead
need to use synchronize_irq() or synchronize_sched().
+ This same limitation also applies to synchronize_rcu_bh()
+ and synchronize_srcu(), as well as to the asynchronous and
+ expedited forms of the three primitives, namely call_rcu(),
+ call_rcu_bh(), call_srcu(), synchronize_rcu_expedited(),
+ synchronize_rcu_bh_expedited(), and synchronize_srcu_expedited().
+
12. Any lock acquired by an RCU callback must be acquired elsewhere
with softirq disabled, e.g., via spin_lock_irqsave(),
spin_lock_bh(), etc. Failing to disable irq on a given
diff --git a/Documentation/RCU/stallwarn.txt b/Documentation/RCU/stallwarn.txt
index 523364e..1927151 100644
--- a/Documentation/RCU/stallwarn.txt
+++ b/Documentation/RCU/stallwarn.txt
@@ -99,7 +99,7 @@
printed:
INFO: rcu_preempt detected stall on CPU
- 0: (64628 ticks this GP) idle=dd5/3fffffffffffffff/0 drain=0 . timer=-1
+ 0: (64628 ticks this GP) idle=dd5/3fffffffffffffff/0 drain=0 . timer not pending
(t=65000 jiffies)
The "(64628 ticks this GP)" indicates that this CPU has taken more
@@ -116,13 +116,13 @@
be a small positive number if in the idle loop and a very large positive
number (as shown above) otherwise.
-For CONFIG_RCU_FAST_NO_HZ kernels, the "drain=0" indicates that the
-CPU is not in the process of trying to force itself into dyntick-idle
-state, the "." indicates that the CPU has not given up forcing RCU
-into dyntick-idle mode (it would be "H" otherwise), and the "timer=-1"
-indicates that the CPU has not recented forced RCU into dyntick-idle
-mode (it would otherwise indicate the number of microseconds remaining
-in this forced state).
+For CONFIG_RCU_FAST_NO_HZ kernels, the "drain=0" indicates that the CPU is
+not in the process of trying to force itself into dyntick-idle state, the
+"." indicates that the CPU has not given up forcing RCU into dyntick-idle
+mode (it would be "H" otherwise), and the "timer not pending" indicates
+that the CPU has not recently forced RCU into dyntick-idle mode (it
+would otherwise indicate the number of microseconds remaining in this
+forced state).
Multiple Warnings From One Stall
diff --git a/Documentation/RCU/trace.txt b/Documentation/RCU/trace.txt
index f6f15ce..672d190 100644
--- a/Documentation/RCU/trace.txt
+++ b/Documentation/RCU/trace.txt
@@ -333,23 +333,23 @@
The output of "cat rcu/rcu_pending" looks as follows:
rcu_sched:
- 0 np=255892 qsp=53936 rpq=85 cbr=0 cng=14417 gpc=10033 gps=24320 nf=6445 nn=146741
- 1 np=261224 qsp=54638 rpq=33 cbr=0 cng=25723 gpc=16310 gps=2849 nf=5912 nn=155792
- 2 np=237496 qsp=49664 rpq=23 cbr=0 cng=2762 gpc=45478 gps=1762 nf=1201 nn=136629
- 3 np=236249 qsp=48766 rpq=98 cbr=0 cng=286 gpc=48049 gps=1218 nf=207 nn=137723
- 4 np=221310 qsp=46850 rpq=7 cbr=0 cng=26 gpc=43161 gps=4634 nf=3529 nn=123110
- 5 np=237332 qsp=48449 rpq=9 cbr=0 cng=54 gpc=47920 gps=3252 nf=201 nn=137456
- 6 np=219995 qsp=46718 rpq=12 cbr=0 cng=50 gpc=42098 gps=6093 nf=4202 nn=120834
- 7 np=249893 qsp=49390 rpq=42 cbr=0 cng=72 gpc=38400 gps=17102 nf=41 nn=144888
+ 0 np=255892 qsp=53936 rpq=85 cbr=0 cng=14417 gpc=10033 gps=24320 nn=146741
+ 1 np=261224 qsp=54638 rpq=33 cbr=0 cng=25723 gpc=16310 gps=2849 nn=155792
+ 2 np=237496 qsp=49664 rpq=23 cbr=0 cng=2762 gpc=45478 gps=1762 nn=136629
+ 3 np=236249 qsp=48766 rpq=98 cbr=0 cng=286 gpc=48049 gps=1218 nn=137723
+ 4 np=221310 qsp=46850 rpq=7 cbr=0 cng=26 gpc=43161 gps=4634 nn=123110
+ 5 np=237332 qsp=48449 rpq=9 cbr=0 cng=54 gpc=47920 gps=3252 nn=137456
+ 6 np=219995 qsp=46718 rpq=12 cbr=0 cng=50 gpc=42098 gps=6093 nn=120834
+ 7 np=249893 qsp=49390 rpq=42 cbr=0 cng=72 gpc=38400 gps=17102 nn=144888
rcu_bh:
- 0 np=146741 qsp=1419 rpq=6 cbr=0 cng=6 gpc=0 gps=0 nf=2 nn=145314
- 1 np=155792 qsp=12597 rpq=3 cbr=0 cng=0 gpc=4 gps=8 nf=3 nn=143180
- 2 np=136629 qsp=18680 rpq=1 cbr=0 cng=0 gpc=7 gps=6 nf=0 nn=117936
- 3 np=137723 qsp=2843 rpq=0 cbr=0 cng=0 gpc=10 gps=7 nf=0 nn=134863
- 4 np=123110 qsp=12433 rpq=0 cbr=0 cng=0 gpc=4 gps=2 nf=0 nn=110671
- 5 np=137456 qsp=4210 rpq=1 cbr=0 cng=0 gpc=6 gps=5 nf=0 nn=133235
- 6 np=120834 qsp=9902 rpq=2 cbr=0 cng=0 gpc=6 gps=3 nf=2 nn=110921
- 7 np=144888 qsp=26336 rpq=0 cbr=0 cng=0 gpc=8 gps=2 nf=0 nn=118542
+ 0 np=146741 qsp=1419 rpq=6 cbr=0 cng=6 gpc=0 gps=0 nn=145314
+ 1 np=155792 qsp=12597 rpq=3 cbr=0 cng=0 gpc=4 gps=8 nn=143180
+ 2 np=136629 qsp=18680 rpq=1 cbr=0 cng=0 gpc=7 gps=6 nn=117936
+ 3 np=137723 qsp=2843 rpq=0 cbr=0 cng=0 gpc=10 gps=7 nn=134863
+ 4 np=123110 qsp=12433 rpq=0 cbr=0 cng=0 gpc=4 gps=2 nn=110671
+ 5 np=137456 qsp=4210 rpq=1 cbr=0 cng=0 gpc=6 gps=5 nn=133235
+ 6 np=120834 qsp=9902 rpq=2 cbr=0 cng=0 gpc=6 gps=3 nn=110921
+ 7 np=144888 qsp=26336 rpq=0 cbr=0 cng=0 gpc=8 gps=2 nn=118542
As always, this is once again split into "rcu_sched" and "rcu_bh"
portions, with CONFIG_TREE_PREEMPT_RCU kernels having an additional
@@ -377,17 +377,6 @@
o "gps" is the number of times that a new grace period had started,
but this CPU was not yet aware of it.
-o "nf" is the number of times that this CPU suspected that the
- current grace period had run for too long, and thus needed to
- be forced.
-
- Please note that "forcing" consists of sending resched IPIs
- to holdout CPUs. If that CPU really still is in an old RCU
- read-side critical section, then we really do have to wait for it.
- The assumption behing "forcing" is that the CPU is not still in
- an old RCU read-side critical section, but has not yet responded
- for some other reason.
-
o "nn" is the number of times that this CPU needed nothing. Alert
readers will note that the rcu "nn" number for a given CPU very
closely matches the rcu_bh "np" number for that same CPU. This
diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt
index 69ee188..bf0f6de 100644
--- a/Documentation/RCU/whatisRCU.txt
+++ b/Documentation/RCU/whatisRCU.txt
@@ -873,7 +873,7 @@
and code segments with preemption disabled (whether
via preempt_disable(), local_irq_save(), local_bh_disable(),
or some other mechanism) as if they were explicit RCU readers?
- If so, you need RCU-sched.
+ If so, RCU-sched is the only choice that will work for you.
e. Do you need RCU grace periods to complete even in the face
of softirq monopolization of one or more of the CPUs? For
@@ -884,7 +884,12 @@
RCU, but inappropriate for other synchronization mechanisms?
If so, consider SLAB_DESTROY_BY_RCU. But please be careful!
-g. Otherwise, use RCU.
+g. Do you need read-side critical sections that are respected
+ even though they are in the middle of the idle loop, during
+ user-mode execution, or on an offlined CPU? If so, SRCU is the
+ only choice that will work for you.
+
+h. Otherwise, use RCU.
Of course, this all assumes that you have determined that RCU is in fact
the right tool for your job.
diff --git a/Documentation/accounting/getdelays.c b/Documentation/accounting/getdelays.c
index f6318f6..6f706ac 100644
--- a/Documentation/accounting/getdelays.c
+++ b/Documentation/accounting/getdelays.c
@@ -98,10 +98,9 @@
if (rcvbufsz)
if (setsockopt(fd, SOL_SOCKET, SO_RCVBUF,
&rcvbufsz, sizeof(rcvbufsz)) < 0) {
- fprintf(stderr, "Unable to set socket rcv buf size "
- "to %d\n",
+ fprintf(stderr, "Unable to set socket rcv buf size to %d\n",
rcvbufsz);
- return -1;
+ goto error;
}
memset(&local, 0, sizeof(local));
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
new file mode 100644
index 0000000..9c4d388
--- /dev/null
+++ b/Documentation/arm64/booting.txt
@@ -0,0 +1,152 @@
+ Booting AArch64 Linux
+ =====================
+
+Author: Will Deacon <will.deacon@arm.com>
+Date : 07 September 2012
+
+This document is based on the ARM booting document by Russell King and
+is relevant to all public releases of the AArch64 Linux kernel.
+
+The AArch64 exception model is made up of a number of exception levels
+(EL0 - EL3), with EL0 and EL1 having a secure and a non-secure
+counterpart. EL2 is the hypervisor level and exists only in non-secure
+mode. EL3 is the highest priority level and exists only in secure mode.
+
+For the purposes of this document, we will use the term `boot loader'
+simply to define all software that executes on the CPU(s) before control
+is passed to the Linux kernel. This may include secure monitor and
+hypervisor code, or it may just be a handful of instructions for
+preparing a minimal boot environment.
+
+Essentially, the boot loader should provide (as a minimum) the
+following:
+
+1. Setup and initialise the RAM
+2. Setup the device tree
+3. Decompress the kernel image
+4. Call the kernel image
+
+
+1. Setup and initialise RAM
+---------------------------
+
+Requirement: MANDATORY
+
+The boot loader is expected to find and initialise all RAM that the
+kernel will use for volatile data storage in the system. It performs
+this in a machine dependent manner. (It may use internal algorithms
+to automatically locate and size all RAM, or it may use knowledge of
+the RAM in the machine, or any other method the boot loader designer
+sees fit.)
+
+
+2. Setup the device tree
+-------------------------
+
+Requirement: MANDATORY
+
+The device tree blob (dtb) must be no bigger than 2 megabytes in size
+and placed at a 2-megabyte boundary within the first 512 megabytes from
+the start of the kernel image. This is to allow the kernel to map the
+blob using a single section mapping in the initial page tables.
+
+
+3. Decompress the kernel image
+------------------------------
+
+Requirement: OPTIONAL
+
+The AArch64 kernel does not currently provide a decompressor and
+therefore requires decompression (gzip etc.) to be performed by the boot
+loader if a compressed Image target (e.g. Image.gz) is used. For
+bootloaders that do not implement this requirement, the uncompressed
+Image target is available instead.
+
+
+4. Call the kernel image
+------------------------
+
+Requirement: MANDATORY
+
+The decompressed kernel image contains a 32-byte header as follows:
+
+ u32 magic = 0x14000008; /* branch to stext, little-endian */
+ u32 res0 = 0; /* reserved */
+ u64 text_offset; /* Image load offset */
+ u64 res1 = 0; /* reserved */
+ u64 res2 = 0; /* reserved */
+
+The image must be placed at the specified offset (currently 0x80000)
+from the start of the system RAM and called there. The start of the
+system RAM must be aligned to 2MB.
+
+Before jumping into the kernel, the following conditions must be met:
+
+- Quiesce all DMA capable devices so that memory does not get
+ corrupted by bogus network packets or disk data. This will save
+ you many hours of debug.
+
+- Primary CPU general-purpose register settings
+ x0 = physical address of device tree blob (dtb) in system RAM.
+ x1 = 0 (reserved for future use)
+ x2 = 0 (reserved for future use)
+ x3 = 0 (reserved for future use)
+
+- CPU mode
+ All forms of interrupts must be masked in PSTATE.DAIF (Debug, SError,
+ IRQ and FIQ).
+ The CPU must be in either EL2 (RECOMMENDED in order to have access to
+ the virtualisation extensions) or non-secure EL1.
+
+- Caches, MMUs
+ The MMU must be off.
+ Instruction cache may be on or off.
+ Data cache must be off and invalidated.
+ External caches (if present) must be configured and disabled.
+
+- Architected timers
+ CNTFRQ must be programmed with the timer frequency.
+ If entering the kernel at EL1, CNTHCTL_EL2 must have EL1PCTEN (bit 0)
+ set where available.
+
+- Coherency
+ All CPUs to be booted by the kernel must be part of the same coherency
+ domain on entry to the kernel. This may require IMPLEMENTATION DEFINED
+ initialisation to enable the receiving of maintenance operations on
+ each CPU.
+
+- System registers
+ All writable architected system registers at the exception level where
+ the kernel image will be entered must be initialised by software at a
+ higher exception level to prevent execution in an UNKNOWN state.
+
+The boot loader is expected to enter the kernel on each CPU in the
+following manner:
+
+- The primary CPU must jump directly to the first instruction of the
+ kernel image. The device tree blob passed by this CPU must contain
+ for each CPU node:
+
+ 1. An 'enable-method' property. Currently, the only supported value
+ for this field is the string "spin-table".
+
+ 2. A 'cpu-release-addr' property identifying a 64-bit,
+ zero-initialised memory location.
+
+ It is expected that the bootloader will generate these device tree
+ properties and insert them into the blob prior to kernel entry.
+
+- Any secondary CPUs must spin outside of the kernel in a reserved area
+ of memory (communicated to the kernel by a /memreserve/ region in the
+ device tree) polling their cpu-release-addr location, which must be
+ contained in the reserved region. A wfe instruction may be inserted
+ to reduce the overhead of the busy-loop and a sev will be issued by
+ the primary CPU. When a read of the location pointed to by the
+ cpu-release-addr returns a non-zero value, the CPU must jump directly
+ to this value.
+
+- Secondary CPU general-purpose register settings
+ x0 = 0 (reserved for future use)
+ x1 = 0 (reserved for future use)
+ x2 = 0 (reserved for future use)
+ x3 = 0 (reserved for future use)
diff --git a/Documentation/arm64/memory.txt b/Documentation/arm64/memory.txt
new file mode 100644
index 0000000..dbbdcbb
--- /dev/null
+++ b/Documentation/arm64/memory.txt
@@ -0,0 +1,73 @@
+ Memory Layout on AArch64 Linux
+ ==============================
+
+Author: Catalin Marinas <catalin.marinas@arm.com>
+Date : 20 February 2012
+
+This document describes the virtual memory layout used by the AArch64
+Linux kernel. The architecture allows up to 4 levels of translation
+tables with a 4KB page size and up to 3 levels with a 64KB page size.
+
+AArch64 Linux uses 3 levels of translation tables with the 4KB page
+configuration, allowing 39-bit (512GB) virtual addresses for both user
+and kernel. With 64KB pages, only 2 levels of translation tables are
+used but the memory layout is the same.
+
+User addresses have bits 63:39 set to 0 while the kernel addresses have
+the same bits set to 1. TTBRx selection is given by bit 63 of the
+virtual address. The swapper_pg_dir contains only kernel (global)
+mappings while the user pgd contains only user (non-global) mappings.
+The swapper_pgd_dir address is written to TTBR1 and never written to
+TTBR0.
+
+
+AArch64 Linux memory layout:
+
+Start End Size Use
+-----------------------------------------------------------------------
+0000000000000000 0000007fffffffff 512GB user
+
+ffffff8000000000 ffffffbbfffcffff ~240GB vmalloc
+
+ffffffbbfffd0000 ffffffbcfffdffff 64KB [guard page]
+
+ffffffbbfffe0000 ffffffbcfffeffff 64KB PCI I/O space
+
+ffffffbbffff0000 ffffffbcffffffff 64KB [guard page]
+
+ffffffbc00000000 ffffffbdffffffff 8GB vmemmap
+
+ffffffbe00000000 ffffffbffbffffff ~8GB [guard, future vmmemap]
+
+ffffffbffc000000 ffffffbfffffffff 64MB modules
+
+ffffffc000000000 ffffffffffffffff 256GB memory
+
+
+Translation table lookup with 4KB pages:
+
++--------+--------+--------+--------+--------+--------+--------+--------+
+|63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
++--------+--------+--------+--------+--------+--------+--------+--------+
+ | | | | | |
+ | | | | | v
+ | | | | | [11:0] in-page offset
+ | | | | +-> [20:12] L3 index
+ | | | +-----------> [29:21] L2 index
+ | | +---------------------> [38:30] L1 index
+ | +-------------------------------> [47:39] L0 index (not used)
+ +-------------------------------------------------> [63] TTBR0/1
+
+
+Translation table lookup with 64KB pages:
+
++--------+--------+--------+--------+--------+--------+--------+--------+
+|63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
++--------+--------+--------+--------+--------+--------+--------+--------+
+ | | | | |
+ | | | | v
+ | | | | [15:0] in-page offset
+ | | | +----------> [28:16] L3 index
+ | | +--------------------------> [41:29] L2 index (only 38:29 used)
+ | +-------------------------------> [47:42] L1 index (not used)
+ +-------------------------------------------------> [63] TTBR0/1
diff --git a/Documentation/block/00-INDEX b/Documentation/block/00-INDEX
index d111e3b..d18ecd8 100644
--- a/Documentation/block/00-INDEX
+++ b/Documentation/block/00-INDEX
@@ -3,15 +3,21 @@
biodoc.txt
- Notes on the Generic Block Layer Rewrite in Linux 2.5
capability.txt
- - Generic Block Device Capability (/sys/block/<disk>/capability)
+ - Generic Block Device Capability (/sys/block/<device>/capability)
+cfq-iosched.txt
+ - CFQ IO scheduler tunables
+data-integrity.txt
+ - Block data integrity
deadline-iosched.txt
- Deadline IO scheduler tunables
ioprio.txt
- Block io priorities (in CFQ scheduler)
+queue-sysfs.txt
+ - Queue's sysfs entries
request.txt
- The members of struct request (in include/linux/blkdev.h)
stat.txt
- - Block layer statistics in /sys/block/<dev>/stat
+ - Block layer statistics in /sys/block/<device>/stat
switching-sched.txt
- Switching I/O schedulers at runtime
writeback_cache_control.txt
diff --git a/Documentation/block/cfq-iosched.txt b/Documentation/block/cfq-iosched.txt
index 6d670f5..d89b4fe 100644
--- a/Documentation/block/cfq-iosched.txt
+++ b/Documentation/block/cfq-iosched.txt
@@ -1,3 +1,14 @@
+CFQ (Complete Fairness Queueing)
+===============================
+
+The main aim of CFQ scheduler is to provide a fair allocation of the disk
+I/O bandwidth for all the processes which requests an I/O operation.
+
+CFQ maintains the per process queue for the processes which request I/O
+operation(syncronous requests). In case of asynchronous requests, all the
+requests from all the processes are batched together according to their
+process's I/O priority.
+
CFQ ioscheduler tunables
========================
@@ -25,6 +36,72 @@
controller or for storage arrays), setting slice_idle=0 might end up in better
throughput and acceptable latencies.
+back_seek_max
+-------------
+This specifies, given in Kbytes, the maximum "distance" for backward seeking.
+The distance is the amount of space from the current head location to the
+sectors that are backward in terms of distance.
+
+This parameter allows the scheduler to anticipate requests in the "backward"
+direction and consider them as being the "next" if they are within this
+distance from the current head location.
+
+back_seek_penalty
+-----------------
+This parameter is used to compute the cost of backward seeking. If the
+backward distance of request is just 1/back_seek_penalty from a "front"
+request, then the seeking cost of two requests is considered equivalent.
+
+So scheduler will not bias toward one or the other request (otherwise scheduler
+will bias toward front request). Default value of back_seek_penalty is 2.
+
+fifo_expire_async
+-----------------
+This parameter is used to set the timeout of asynchronous requests. Default
+value of this is 248ms.
+
+fifo_expire_sync
+----------------
+This parameter is used to set the timeout of synchronous requests. Default
+value of this is 124ms. In case to favor synchronous requests over asynchronous
+one, this value should be decreased relative to fifo_expire_async.
+
+slice_async
+-----------
+This parameter is same as of slice_sync but for asynchronous queue. The
+default value is 40ms.
+
+slice_async_rq
+--------------
+This parameter is used to limit the dispatching of asynchronous request to
+device request queue in queue's slice time. The maximum number of request that
+are allowed to be dispatched also depends upon the io priority. Default value
+for this is 2.
+
+slice_sync
+----------
+When a queue is selected for execution, the queues IO requests are only
+executed for a certain amount of time(time_slice) before switching to another
+queue. This parameter is used to calculate the time slice of synchronous
+queue.
+
+time_slice is computed using the below equation:-
+time_slice = slice_sync + (slice_sync/5 * (4 - prio)). To increase the
+time_slice of synchronous queue, increase the value of slice_sync. Default
+value is 100ms.
+
+quantum
+-------
+This specifies the number of request dispatched to the device queue. In a
+queue's time slice, a request will not be dispatched if the number of request
+in the device exceeds this parameter. This parameter is used for synchronous
+request.
+
+In case of storage with several disk, this setting can limit the parallel
+processing of request. Therefore, increasing the value can imporve the
+performace although this can cause the latency of some I/O to increase due
+to more number of requests.
+
CFQ IOPS Mode for group scheduling
===================================
Basic CFQ design is to provide priority based time slices. Higher priority
diff --git a/Documentation/block/queue-sysfs.txt b/Documentation/block/queue-sysfs.txt
index 6518a55..e54ac1d 100644
--- a/Documentation/block/queue-sysfs.txt
+++ b/Documentation/block/queue-sysfs.txt
@@ -9,20 +9,71 @@
Files denoted with a RO postfix are readonly and the RW postfix means
read-write.
+add_random (RW)
+----------------
+This file allows to trun off the disk entropy contribution. Default
+value of this file is '1'(on).
+
+discard_granularity (RO)
+-----------------------
+This shows the size of internal allocation of the device in bytes, if
+reported by the device. A value of '0' means device does not support
+the discard functionality.
+
+discard_max_bytes (RO)
+----------------------
+Devices that support discard functionality may have internal limits on
+the number of bytes that can be trimmed or unmapped in a single operation.
+The discard_max_bytes parameter is set by the device driver to the maximum
+number of bytes that can be discarded in a single operation. Discard
+requests issued to the device must not exceed this limit. A discard_max_bytes
+value of 0 means that the device does not support discard functionality.
+
+discard_zeroes_data (RO)
+------------------------
+When read, this file will show if the discarded block are zeroed by the
+device or not. If its value is '1' the blocks are zeroed otherwise not.
+
hw_sector_size (RO)
-------------------
This is the hardware sector size of the device, in bytes.
+iostats (RW)
+-------------
+This file is used to control (on/off) the iostats accounting of the
+disk.
+
+logical_block_size (RO)
+-----------------------
+This is the logcal block size of the device, in bytes.
+
max_hw_sectors_kb (RO)
----------------------
This is the maximum number of kilobytes supported in a single data transfer.
+max_integrity_segments (RO)
+---------------------------
+When read, this file shows the max limit of integrity segments as
+set by block layer which a hardware controller can handle.
+
max_sectors_kb (RW)
-------------------
This is the maximum number of kilobytes that the block layer will allow
for a filesystem request. Must be smaller than or equal to the maximum
size allowed by the hardware.
+max_segments (RO)
+-----------------
+Maximum number of segments of the device.
+
+max_segment_size (RO)
+---------------------
+Maximum segment size of the device.
+
+minimum_io_size (RO)
+--------------------
+This is the smallest preferred io size reported by the device.
+
nomerges (RW)
-------------
This enables the user to disable the lookup logic involved with IO
@@ -45,11 +96,24 @@
each request queue may have upto N request pools, each independently
regulated by nr_requests.
+optimal_io_size (RO)
+--------------------
+This is the optimal io size reported by the device.
+
+physical_block_size (RO)
+------------------------
+This is the physical block size of device, in bytes.
+
read_ahead_kb (RW)
------------------
Maximum number of kilobytes to read-ahead for filesystems on this block
device.
+rotational (RW)
+---------------
+This file is used to stat if the device is of rotational type or
+non-rotational type.
+
rq_affinity (RW)
----------------
If this option is '1', the block layer will migrate request completions to the
diff --git a/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt b/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt
index 70cd49b..1dd6225 100644
--- a/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt
+++ b/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt
@@ -10,8 +10,8 @@
- compatible : Should be "fsl,<chip>-esdhc"
Optional properties:
-- fsl,cd-internal : Indicate to use controller internal card detection
-- fsl,wp-internal : Indicate to use controller internal write protection
+- fsl,cd-controller : Indicate to use controller internal card detection
+- fsl,wp-controller : Indicate to use controller internal write protection
Examples:
@@ -19,8 +19,8 @@
compatible = "fsl,imx51-esdhc";
reg = <0x70004000 0x4000>;
interrupts = <1>;
- fsl,cd-internal;
- fsl,wp-internal;
+ fsl,cd-controller;
+ fsl,wp-controller;
};
esdhc@70008000 {
diff --git a/Documentation/devicetree/bindings/regulator/regulator.txt b/Documentation/devicetree/bindings/regulator/regulator.txt
index 66ece3f..ecfc6cc 100644
--- a/Documentation/devicetree/bindings/regulator/regulator.txt
+++ b/Documentation/devicetree/bindings/regulator/regulator.txt
@@ -11,10 +11,13 @@
- regulator-boot-on: bootloader/firmware enabled regulator
- <name>-supply: phandle to the parent supply/regulator node
- regulator-ramp-delay: ramp delay for regulator(in uV/uS)
+
+Deprecated properties:
- regulator-compatible: If a regulator chip contains multiple
regulators, and if the chip's binding contains a child node that
describes each regulator, then this property indicates which regulator
- this child node is intended to configure.
+ this child node is intended to configure. If this property is missing,
+ the node's name will be used instead.
Example:
diff --git a/Documentation/devicetree/bindings/regulator/tps65217.txt b/Documentation/devicetree/bindings/regulator/tps65217.txt
index 0487e96..d316fb8 100644
--- a/Documentation/devicetree/bindings/regulator/tps65217.txt
+++ b/Documentation/devicetree/bindings/regulator/tps65217.txt
@@ -22,66 +22,49 @@
compatible = "ti,tps65217";
regulators {
- #address-cells = <1>;
- #size-cells = <0>;
-
- dcdc1_reg: regulator@0 {
- reg = <0>;
- regulator-compatible = "dcdc1";
+ dcdc1_reg: dcdc1 {
regulator-min-microvolt = <900000>;
regulator-max-microvolt = <1800000>;
regulator-boot-on;
regulator-always-on;
};
- dcdc2_reg: regulator@1 {
- reg = <1>;
- regulator-compatible = "dcdc2";
+ dcdc2_reg: dcdc2 {
regulator-min-microvolt = <900000>;
regulator-max-microvolt = <3300000>;
regulator-boot-on;
regulator-always-on;
};
- dcdc3_reg: regulator@2 {
- reg = <2>;
- regulator-compatible = "dcdc3";
+ dcdc3_reg: dcc3 {
regulator-min-microvolt = <900000>;
regulator-max-microvolt = <1500000>;
regulator-boot-on;
regulator-always-on;
};
- ldo1_reg: regulator@3 {
- reg = <3>;
- regulator-compatible = "ldo1";
+ ldo1_reg: ldo1 {
regulator-min-microvolt = <1000000>;
regulator-max-microvolt = <3300000>;
regulator-boot-on;
regulator-always-on;
};
- ldo2_reg: regulator@4 {
- reg = <4>;
- regulator-compatible = "ldo2";
+ ldo2_reg: ldo2 {
regulator-min-microvolt = <900000>;
regulator-max-microvolt = <3300000>;
regulator-boot-on;
regulator-always-on;
};
- ldo3_reg: regulator@5 {
- reg = <5>;
- regulator-compatible = "ldo3";
+ ldo3_reg: ldo3 {
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <3300000>;
regulator-boot-on;
regulator-always-on;
};
- ldo4_reg: regulator@6 {
- reg = <6>;
- regulator-compatible = "ldo4";
+ ldo4_reg: ldo4 {
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <3300000>;
regulator-boot-on;
diff --git a/Documentation/devicetree/bindings/regulator/tps6586x.txt b/Documentation/devicetree/bindings/regulator/tps6586x.txt
index da80c2a..07b9ef6 100644
--- a/Documentation/devicetree/bindings/regulator/tps6586x.txt
+++ b/Documentation/devicetree/bindings/regulator/tps6586x.txt
@@ -6,9 +6,13 @@
- interrupts: the interrupt outputs of the controller
- #gpio-cells: number of cells to describe a GPIO
- gpio-controller: mark the device as a GPIO controller
-- regulators: list of regulators provided by this controller, must have
- property "regulator-compatible" to match their hardware counterparts:
- sm[0-2], ldo[0-9] and ldo_rtc
+- regulators: A node that houses a sub-node for each regulator within the
+ device. Each sub-node is identified using the node's name (or the deprecated
+ regulator-compatible property if present), with valid values listed below.
+ The content of each sub-node is defined by the standard binding for
+ regulators; see regulator.txt.
+ sys, sm[0-2], ldo[0-9] and ldo_rtc
+- sys-supply: The input supply for SYS.
- vin-sm0-supply: The input supply for the SM0.
- vin-sm1-supply: The input supply for the SM1.
- vin-sm2-supply: The input supply for the SM2.
@@ -20,6 +24,9 @@
Each regulator is defined using the standard binding for regulators.
+Note: LDO5 and LDO_RTC is supplied by SYS regulator internally and driver
+ take care of making proper parent child relationship.
+
Example:
pmu: tps6586x@34 {
@@ -30,6 +37,7 @@
#gpio-cells = <2>;
gpio-controller;
+ sys-supply = <&some_reg>;
vin-sm0-supply = <&some_reg>;
vin-sm1-supply = <&some_reg>;
vin-sm2-supply = <&some_reg>;
@@ -40,103 +48,80 @@
vinldo9-supply = <...>;
regulators {
- #address-cells = <1>;
- #size-cells = <0>;
+ sys_reg: sys {
+ regulator-name = "vdd_sys";
+ regulator-boot-on;
+ regulator-always-on;
+ };
- sm0_reg: regulator@0 {
- reg = <0>;
- regulator-compatible = "sm0";
+ sm0_reg: sm0 {
regulator-min-microvolt = < 725000>;
regulator-max-microvolt = <1500000>;
regulator-boot-on;
regulator-always-on;
};
- sm1_reg: regulator@1 {
- reg = <1>;
- regulator-compatible = "sm1";
+ sm1_reg: sm1 {
regulator-min-microvolt = < 725000>;
regulator-max-microvolt = <1500000>;
regulator-boot-on;
regulator-always-on;
};
- sm2_reg: regulator@2 {
- reg = <2>;
- regulator-compatible = "sm2";
+ sm2_reg: sm2 {
regulator-min-microvolt = <3000000>;
regulator-max-microvolt = <4550000>;
regulator-boot-on;
regulator-always-on;
};
- ldo0_reg: regulator@3 {
- reg = <3>;
- regulator-compatible = "ldo0";
+ ldo0_reg: ldo0 {
regulator-name = "PCIE CLK";
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
};
- ldo1_reg: regulator@4 {
- reg = <4>;
- regulator-compatible = "ldo1";
+ ldo1_reg: ldo1 {
regulator-min-microvolt = < 725000>;
regulator-max-microvolt = <1500000>;
};
- ldo2_reg: regulator@5 {
- reg = <5>;
- regulator-compatible = "ldo2";
+ ldo2_reg: ldo2 {
regulator-min-microvolt = < 725000>;
regulator-max-microvolt = <1500000>;
};
- ldo3_reg: regulator@6 {
- reg = <6>;
- regulator-compatible = "ldo3";
+ ldo3_reg: ldo3 {
regulator-min-microvolt = <1250000>;
regulator-max-microvolt = <3300000>;
};
- ldo4_reg: regulator@7 {
- reg = <7>;
- regulator-compatible = "ldo4";
+ ldo4_reg: ldo4 {
regulator-min-microvolt = <1700000>;
regulator-max-microvolt = <2475000>;
};
- ldo5_reg: regulator@8 {
- reg = <8>;
- regulator-compatible = "ldo5";
+ ldo5_reg: ldo5 {
regulator-min-microvolt = <1250000>;
regulator-max-microvolt = <3300000>;
};
- ldo6_reg: regulator@9 {
- reg = <9>;
- regulator-compatible = "ldo6";
+ ldo6_reg: ldo6 {
regulator-min-microvolt = <1250000>;
regulator-max-microvolt = <3300000>;
};
- ldo7_reg: regulator@10 {
- reg = <10>;
- regulator-compatible = "ldo7";
+ ldo7_reg: ldo7 {
regulator-min-microvolt = <1250000>;
regulator-max-microvolt = <3300000>;
};
- ldo8_reg: regulator@11 {
- reg = <11>;
- regulator-compatible = "ldo8";
+ ldo8_reg: ldo8 {
regulator-min-microvolt = <1250000>;
regulator-max-microvolt = <3300000>;
};
- ldo9_reg: regulator@12 {
- reg = <12>;
- regulator-compatible = "ldo9";
+ ldo9_reg: ldo9 {
regulator-min-microvolt = <1250000>;
regulator-max-microvolt = <3300000>;
};
diff --git a/Documentation/dontdiff b/Documentation/dontdiff
index 39462cf..74c25c8 100644
--- a/Documentation/dontdiff
+++ b/Documentation/dontdiff
@@ -162,7 +162,6 @@
machtypes.h
map
map_hugetlb
-maui_boot.h
media
mconf
miboot*
diff --git a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt
index afaff31..bc55d38 100644
--- a/Documentation/feature-removal-schedule.txt
+++ b/Documentation/feature-removal-schedule.txt
@@ -253,38 +253,6 @@
-----------------------------
-What: fakephp and associated sysfs files in /sys/bus/pci/slots/
-When: 2011
-Why: In 2.6.27, the semantics of /sys/bus/pci/slots was redefined to
- represent a machine's physical PCI slots. The change in semantics
- had userspace implications, as the hotplug core no longer allowed
- drivers to create multiple sysfs files per physical slot (required
- for multi-function devices, e.g.). fakephp was seen as a developer's
- tool only, and its interface changed. Too late, we learned that
- there were some users of the fakephp interface.
-
- In 2.6.30, the original fakephp interface was restored. At the same
- time, the PCI core gained the ability that fakephp provided, namely
- function-level hot-remove and hot-add.
-
- Since the PCI core now provides the same functionality, exposed in:
-
- /sys/bus/pci/rescan
- /sys/bus/pci/devices/.../remove
- /sys/bus/pci/devices/.../rescan
-
- there is no functional reason to maintain fakephp as well.
-
- We will keep the existing module so that 'modprobe fakephp' will
- present the old /sys/bus/pci/slots/... interface for compatibility,
- but users are urged to migrate their applications to the API above.
-
- After a reasonable transition period, we will remove the legacy
- fakephp interface.
-Who: Alex Chiang <achiang@hp.com>
-
----------------------------
-
What: CONFIG_RFKILL_INPUT
When: 2.6.33
Why: Should be implemented in userspace, policy daemon.
@@ -579,7 +547,7 @@
----------------------------
What: at91-mci driver ("CONFIG_MMC_AT91")
-When: 3.7
+When: 3.8
Why: There are two mci drivers: at91-mci and atmel-mci. The PDC support
was added to atmel-mci as a first step to support more chips.
Then at91-mci was kept only for old IP versions (on at91rm9200 and
diff --git a/Documentation/i2c/busses/i2c-i801 b/Documentation/i2c/busses/i2c-i801
index 615142d..157416e7 100644
--- a/Documentation/i2c/busses/i2c-i801
+++ b/Documentation/i2c/busses/i2c-i801
@@ -21,6 +21,7 @@
* Intel DH89xxCC (PCH)
* Intel Panther Point (PCH)
* Intel Lynx Point (PCH)
+ * Intel Lynx Point-LP (PCH)
Datasheets: Publicly available at the Intel website
On Intel Patsburg and later chipsets, both the normal host SMBus controller
diff --git a/Documentation/ia64/aliasing-test.c b/Documentation/ia64/aliasing-test.c
index 5caa2af..62a190d 100644
--- a/Documentation/ia64/aliasing-test.c
+++ b/Documentation/ia64/aliasing-test.c
@@ -132,6 +132,7 @@
rc = write(fd, "1", 2);
if (rc <= 0) {
+ close(fd);
perror("write");
return -1;
}
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index ad7e2e5..df551df 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1833,6 +1833,12 @@
and restore using xsave. The kernel will fallback to
enabling legacy floating-point and sse state.
+ eagerfpu= [X86]
+ on enable eager fpu restore
+ off disable eager fpu restore
+ auto selects the default scheme, which automatically
+ enables eagerfpu restore for xsaveopt.
+
nohlt [BUGS=ARM,SH] Tells the kernel that the sleep(SH) or
wfi(ARM) instruction doesn't work correctly and not to
use it. This is also useful when using JTAG debugger.
@@ -2385,6 +2391,17 @@
rcutree.rcu_cpu_stall_timeout= [KNL,BOOT]
Set timeout for RCU CPU stall warning messages.
+ rcutree.jiffies_till_first_fqs= [KNL,BOOT]
+ Set delay from grace-period initialization to
+ first attempt to force quiescent states.
+ Units are jiffies, minimum value is zero,
+ and maximum value is HZ.
+
+ rcutree.jiffies_till_next_fqs= [KNL,BOOT]
+ Set delay between subsequent attempts to force
+ quiescent states. Units are jiffies, minimum
+ value is one, and maximum value is HZ.
+
rcutorture.fqs_duration= [KNL,BOOT]
Set duration of force_quiescent_state bursts.
@@ -2638,9 +2655,6 @@
smart2= [HW]
Format: <io1>[,<io2>[,...,<io8>]]
- smp-alt-once [X86-32,SMP] On a hotplug CPU system, only
- attempt to substitute SMP alternatives once at boot.
-
smsc-ircc2.nopnp [HW] Don't use PNP to discover SMC devices
smsc-ircc2.ircc_cfg= [HW] Device configuration I/O port
smsc-ircc2.ircc_sir= [HW] SIR base I/O port
diff --git a/Documentation/power/swsusp.txt b/Documentation/power/swsusp.txt
index 92341b8..0b4b63e 100644
--- a/Documentation/power/swsusp.txt
+++ b/Documentation/power/swsusp.txt
@@ -53,7 +53,7 @@
Article about goals and implementation of Software Suspend for Linux
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Author: G‚ábor Kuti
+Author: Gábor Kuti
Last revised: 2003-10-20 by Pavel Machek
Idea and goals to achieve
diff --git a/Documentation/scheduler/sched-arch.txt b/Documentation/scheduler/sched-arch.txt
index 28aa107..b1b8587 100644
--- a/Documentation/scheduler/sched-arch.txt
+++ b/Documentation/scheduler/sched-arch.txt
@@ -17,16 +17,6 @@
Unlocked context switches introduce only a very minor performance
penalty to the core scheduler implementation in the CONFIG_SMP case.
-2. Interrupt status
-By default, the switch_to arch function is called with interrupts
-disabled. Interrupts may be enabled over the call if it is likely to
-introduce a significant interrupt latency by adding the line
-`#define __ARCH_WANT_INTERRUPTS_ON_CTXSW` in the same place as for
-unlocked context switches. This define also implies
-`__ARCH_WANT_UNLOCKED_CTXSW`. See arch/arm/include/asm/system.h for an
-example.
-
-
CPU idle
========
Your cpu_idle routines need to obey the following rules:
diff --git a/Documentation/trace/kprobetrace.txt b/Documentation/trace/kprobetrace.txt
index d0d0bb9..d68ea5f 100644
--- a/Documentation/trace/kprobetrace.txt
+++ b/Documentation/trace/kprobetrace.txt
@@ -12,7 +12,7 @@
functions). Unlike the Tracepoint based event, this can be added and removed
dynamically, on the fly.
-To enable this feature, build your kernel with CONFIG_KPROBE_TRACING=y.
+To enable this feature, build your kernel with CONFIG_KPROBE_EVENT=y.
Similar to the events tracer, this doesn't need to be activated via
current_tracer. Instead of that, add probe points via
diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt
index 0cb6685..8eda363 100644
--- a/Documentation/vfio.txt
+++ b/Documentation/vfio.txt
@@ -133,7 +133,7 @@
$ lspci -n -s 0000:06:0d.0
06:0d.0 0401: 1102:0002 (rev 08)
# echo 0000:06:0d.0 > /sys/bus/pci/devices/0000:06:0d.0/driver/unbind
-# echo 1102 0002 > /sys/bus/pci/drivers/vfio/new_id
+# echo 1102 0002 > /sys/bus/pci/drivers/vfio-pci/new_id
Now we need to look at what other devices are in the group to free
it for use by VFIO:
diff --git a/Documentation/watchdog/src/watchdog-test.c b/Documentation/watchdog/src/watchdog-test.c
index 73ff5cc..3da8229 100644
--- a/Documentation/watchdog/src/watchdog-test.c
+++ b/Documentation/watchdog/src/watchdog-test.c
@@ -31,7 +31,7 @@
* or "-e" to enable the card.
*/
-void term(int sig)
+static void term(int sig)
{
close(fd);
fprintf(stderr, "Stopping watchdog ticks...\n");
diff --git a/Documentation/x86/x86_64/boot-options.txt b/Documentation/x86/x86_64/boot-options.txt
index c54b4f5..de38429 100644
--- a/Documentation/x86/x86_64/boot-options.txt
+++ b/Documentation/x86/x86_64/boot-options.txt
@@ -50,6 +50,13 @@
monarchtimeout:
Sets the time in us to wait for other CPUs on machine checks. 0
to disable.
+ mce=bios_cmci_threshold
+ Don't overwrite the bios-set CMCI threshold. This boot option
+ prevents Linux from overwriting the CMCI threshold set by the
+ bios. Without this option, Linux always sets the CMCI
+ threshold to 1. Enabling this may make memory predictive failure
+ analysis less effective if the bios sets thresholds for memory
+ errors since we will not see details for all errors.
nomce (for compatibility with i386): same as mce=off
diff --git a/MAINTAINERS b/MAINTAINERS
index fdc0119..c98b1a1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1209,6 +1209,12 @@
F: arch/arm/mach-pxa/z2.c
F: arch/arm/mach-pxa/include/mach/z2.h
+ARM64 PORT (AARCH64 ARCHITECTURE)
+M: Catalin Marinas <catalin.marinas@arm.com>
+L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+F: arch/arm64/
+
ASC7621 HARDWARE MONITOR DRIVER
M: George Joseph <george.joseph@fairview5.com>
L: lm-sensors@lm-sensors.org
@@ -3388,7 +3394,7 @@
L: linux-i2c@vger.kernel.org
W: http://i2c.wiki.kernel.org/
T: quilt kernel.org/pub/linux/kernel/people/jdelvare/linux-2.6/jdelvare-i2c/
-T: git git://git.fluff.org/bjdooks/linux.git
+T: git git://git.pengutronix.de/git/wsa/linux.git
S: Maintained
F: Documentation/i2c/
F: drivers/i2c/
@@ -3552,11 +3558,12 @@
INTEL C600 SERIES SAS CONTROLLER DRIVER
M: Intel SCU Linux support <intel-linux-scu@intel.com>
+M: Lukasz Dorau <lukasz.dorau@intel.com>
+M: Maciej Patelczyk <maciej.patelczyk@intel.com>
M: Dave Jiang <dave.jiang@intel.com>
-M: Ed Nadolski <edmund.nadolski@intel.com>
L: linux-scsi@vger.kernel.org
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/djbw/isci.git
-S: Maintained
+T: git git://git.code.sf.net/p/intel-sas/isci
+S: Supported
F: drivers/scsi/isci/
F: firmware/isci/
@@ -3666,11 +3673,12 @@
F: drivers/net/wireless/ipw2x00/
INTEL(R) TRUSTED EXECUTION TECHNOLOGY (TXT)
-M: Joseph Cihula <joseph.cihula@intel.com>
+M: Richard L Maliszewski <richard.l.maliszewski@intel.com>
+M: Gang Wei <gang.wei@intel.com>
M: Shane Wang <shane.wang@intel.com>
L: tboot-devel@lists.sourceforge.net
W: http://tboot.sourceforge.net
-T: Mercurial http://www.bughost.org/repos.hg/tboot.hg
+T: hg http://tboot.hg.sourceforge.net:8000/hgroot/tboot/tboot
S: Supported
F: Documentation/intel_txt.txt
F: include/linux/tboot.h
@@ -5320,6 +5328,12 @@
S: Maintained
F: drivers/mtd/devices/phram.c
+PICOLCD HID DRIVER
+M: Bruno Prémont <bonbons@linux-vserver.org>
+L: linux-input@vger.kernel.org
+S: Maintained
+F: drivers/hid/hid-picolcd*
+
PICOXCELL SUPPORT
M: Jamie Iles <jamie@jamieiles.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
@@ -5543,6 +5557,8 @@
F: include/linux/pwm.h
F: include/linux/of_pwm.h
F: drivers/pwm/
+F: drivers/video/backlight/pwm_bl.c
+F: include/linux/pwm_backlight.h
PXA2xx/PXA3xx SUPPORT
M: Eric Miao <eric.y.miao@gmail.com>
diff --git a/Makefile b/Makefile
index 3540268..846dd76 100644
--- a/Makefile
+++ b/Makefile
@@ -1,8 +1,8 @@
VERSION = 3
PATCHLEVEL = 6
SUBLEVEL = 0
-EXTRAVERSION = -rc3
-NAME = Saber-toothed Squirrel
+EXTRAVERSION =
+NAME = Terrified Chipmunk
# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
@@ -609,7 +609,11 @@
endif
ifdef CONFIG_FUNCTION_TRACER
-KBUILD_CFLAGS += -pg
+ifdef CONFIG_HAVE_FENTRY
+CC_USING_FENTRY := $(call cc-option, -mfentry -DCC_USING_FENTRY)
+endif
+KBUILD_CFLAGS += -pg $(CC_USING_FENTRY)
+KBUILD_AFLAGS += $(CC_USING_FENTRY)
ifdef CONFIG_DYNAMIC_FTRACE
ifdef CONFIG_HAVE_C_RECORDMCOUNT
BUILD_C_RECORDMCOUNT := y
diff --git a/arch/Kconfig b/arch/Kconfig
index 72f2fa1..a62965d 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -222,6 +222,19 @@
subsystem. Also has support for calculating CPU cycle events
to determine how many clock cycles in a given period.
+config HAVE_PERF_REGS
+ bool
+ help
+ Support selective register dumps for perf events. This includes
+ bit-mapping of each registers and a unique architecture id.
+
+config HAVE_PERF_USER_STACK_DUMP
+ bool
+ help
+ Support user stack dumps for perf event samples. This needs
+ access to the user stack pointer which is not unified across
+ architectures.
+
config HAVE_ARCH_JUMP_LABEL
bool
@@ -281,4 +294,23 @@
See Documentation/prctl/seccomp_filter.txt for details.
+config HAVE_RCU_USER_QS
+ bool
+ help
+ Provide kernel entry/exit hooks necessary for userspace
+ RCU extended quiescent state. Syscalls need to be wrapped inside
+ rcu_user_exit()-rcu_user_enter() through the slow path using
+ TIF_NOHZ flag. Exceptions handlers must be wrapped as well. Irqs
+ are already protected inside rcu_irq_enter/rcu_irq_exit() but
+ preemption or signal handling on irq exit still need to be protected.
+
+config HAVE_VIRT_CPU_ACCOUNTING
+ bool
+
+config HAVE_IRQ_TIME_ACCOUNTING
+ bool
+ help
+ Archs need to ensure they use a high enough resolution clock to
+ support irq time accounting and then call enable_sched_clock_irqtime().
+
source "kernel/gcov/Kconfig"
diff --git a/arch/alpha/kernel/pci.c b/arch/alpha/kernel/pci.c
index 9816d5a..ef75714 100644
--- a/arch/alpha/kernel/pci.c
+++ b/arch/alpha/kernel/pci.c
@@ -256,12 +256,6 @@
}
}
-void __init
-pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
int
pcibios_enable_device(struct pci_dev *dev, int mask)
{
diff --git a/arch/alpha/kernel/process.c b/arch/alpha/kernel/process.c
index d6fde98..83638aa 100644
--- a/arch/alpha/kernel/process.c
+++ b/arch/alpha/kernel/process.c
@@ -28,6 +28,7 @@
#include <linux/tty.h>
#include <linux/console.h>
#include <linux/slab.h>
+#include <linux/rcupdate.h>
#include <asm/reg.h>
#include <asm/uaccess.h>
@@ -54,9 +55,12 @@
/* FIXME -- EV6 and LCA45 know how to power down
the CPU. */
+ rcu_idle_enter();
while (!need_resched())
cpu_relax();
- schedule();
+
+ rcu_idle_exit();
+ schedule_preempt_disabled();
}
}
diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c
index 35ddc02..a41ad90 100644
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -166,6 +166,7 @@
DBGS(("smp_callin: commencing CPU %d current %p active_mm %p\n",
cpuid, current, current->active_mm));
+ preempt_disable();
/* Do nothing. */
cpu_idle();
}
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 6d6e18f..de325f4 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -6,7 +6,7 @@
select HAVE_DMA_API_DEBUG
select HAVE_IDE if PCI || ISA || PCMCIA
select HAVE_DMA_ATTRS
- select HAVE_DMA_CONTIGUOUS if (CPU_V6 || CPU_V6K || CPU_V7)
+ select HAVE_DMA_CONTIGUOUS if MMU
select HAVE_MEMBLOCK
select RTC_LIB
select SYS_SUPPORTS_APM_EMULATION
@@ -273,7 +273,7 @@
select ARM_AMBA
select ARCH_HAS_CPUFREQ
select COMMON_CLK
- select CLK_VERSATILE
+ select COMMON_CLK_VERSATILE
select HAVE_TCM
select ICST
select GENERIC_CLOCKEVENTS
@@ -289,13 +289,12 @@
config ARCH_REALVIEW
bool "ARM Ltd. RealView family"
select ARM_AMBA
- select CLKDEV_LOOKUP
- select HAVE_MACH_CLKDEV
+ select COMMON_CLK
+ select COMMON_CLK_VERSATILE
select ICST
select GENERIC_CLOCKEVENTS
select ARCH_WANT_OPTIONAL_GPIOLIB
select PLAT_VERSATILE
- select PLAT_VERSATILE_CLOCK
select PLAT_VERSATILE_CLCD
select ARM_TIMER_SP804
select GPIO_PL061 if GPIOLIB
@@ -413,7 +412,7 @@
select NO_IOPORT
select ARCH_REQUIRE_GPIOLIB
select GENERIC_CLOCKEVENTS
- select CLKDEV_LOOKUP
+ select COMMON_CLK
select GENERIC_IRQ_CHIP
select MIGHT_HAVE_CACHE_L2X0
select PINCTRL
@@ -2144,6 +2143,7 @@
config CPU_FREQ_IMX
tristate "CPUfreq driver for i.MX CPUs"
depends on ARCH_MXC && CPU_FREQ
+ select CPU_FREQ_TABLE
help
This enables the CPUfreq driver for i.MX CPUs.
diff --git a/arch/arm/Kconfig.debug b/arch/arm/Kconfig.debug
index f15f82b..e968a52 100644
--- a/arch/arm/Kconfig.debug
+++ b/arch/arm/Kconfig.debug
@@ -356,15 +356,15 @@
is nothing connected to read from the DCC.
config DEBUG_SEMIHOSTING
- bool "Kernel low-level debug output via semihosting I"
+ bool "Kernel low-level debug output via semihosting I/O"
help
Semihosting enables code running on an ARM target to use
the I/O facilities on a host debugger/emulator through a
- simple SVC calls. The host debugger or emulator must have
+ simple SVC call. The host debugger or emulator must have
semihosting enabled for the special svc call to be trapped
otherwise the kernel will crash.
- This is known to work with OpenOCD, as wellas
+ This is known to work with OpenOCD, as well as
ARM's Fast Models, or any other controlling environment
that implements semihosting.
diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 30eae87..a051dfb 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -284,10 +284,10 @@
zinstall uinstall install: vmlinux
$(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $@
-%.dtb:
+%.dtb: scripts
$(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $(boot)/$@
-dtbs:
+dtbs: scripts
$(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $(boot)/$@
# We use MRPROPER_FILES and CLEAN_FILES now
diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index b8c64b8..bc67cbf 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -653,16 +653,21 @@
mcrne p15, 0, r0, c8, c7, 0 @ flush I,D TLBs
#endif
mrc p15, 0, r0, c1, c0, 0 @ read control reg
+ bic r0, r0, #1 << 28 @ clear SCTLR.TRE
orr r0, r0, #0x5000 @ I-cache enable, RR cache replacement
orr r0, r0, #0x003c @ write buffer
#ifdef CONFIG_MMU
#ifdef CONFIG_CPU_ENDIAN_BE8
orr r0, r0, #1 << 25 @ big-endian page tables
#endif
+ mrcne p15, 0, r6, c2, c0, 2 @ read ttb control reg
orrne r0, r0, #1 @ MMU enabled
movne r1, #0xfffffffd @ domain 0 = client
+ bic r6, r6, #1 << 31 @ 32-bit translation system
+ bic r6, r6, #3 << 0 @ use only ttbr0
mcrne p15, 0, r3, c2, c0, 0 @ load page table pointer
mcrne p15, 0, r1, c3, c0, 0 @ load domain access control
+ mcrne p15, 0, r6, c2, c0, 2 @ load ttb control
#endif
mcr p15, 0, r0, c7, c5, 4 @ ISB
mcr p15, 0, r0, c1, c0, 0 @ load control register
diff --git a/arch/arm/boot/dts/am33xx.dtsi b/arch/arm/boot/dts/am33xx.dtsi
index 59509c4..bd0cff3 100644
--- a/arch/arm/boot/dts/am33xx.dtsi
+++ b/arch/arm/boot/dts/am33xx.dtsi
@@ -154,5 +154,10 @@
#size-cells = <0>;
ti,hwmods = "i2c3";
};
+
+ wdt2: wdt@44e35000 {
+ compatible = "ti,omap3-wdt";
+ ti,hwmods = "wd_timer2";
+ };
};
};
diff --git a/arch/arm/boot/dts/at91sam9260.dtsi b/arch/arm/boot/dts/at91sam9260.dtsi
index 66389c1..7c95f76 100644
--- a/arch/arm/boot/dts/at91sam9260.dtsi
+++ b/arch/arm/boot/dts/at91sam9260.dtsi
@@ -104,6 +104,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioB: gpio@fffff600 {
@@ -113,6 +114,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioC: gpio@fffff800 {
@@ -122,6 +124,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
dbgu: serial@fffff200 {
diff --git a/arch/arm/boot/dts/at91sam9263.dtsi b/arch/arm/boot/dts/at91sam9263.dtsi
index b460d6c..195019b 100644
--- a/arch/arm/boot/dts/at91sam9263.dtsi
+++ b/arch/arm/boot/dts/at91sam9263.dtsi
@@ -95,6 +95,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioB: gpio@fffff400 {
@@ -104,6 +105,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioC: gpio@fffff600 {
@@ -113,6 +115,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioD: gpio@fffff800 {
@@ -122,6 +125,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioE: gpio@fffffa00 {
@@ -131,6 +135,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
dbgu: serial@ffffee00 {
diff --git a/arch/arm/boot/dts/at91sam9g25ek.dts b/arch/arm/boot/dts/at91sam9g25ek.dts
index 7829a4d..96514c1 100644
--- a/arch/arm/boot/dts/at91sam9g25ek.dts
+++ b/arch/arm/boot/dts/at91sam9g25ek.dts
@@ -15,7 +15,7 @@
compatible = "atmel,at91sam9g25ek", "atmel,at91sam9x5ek", "atmel,at91sam9x5", "atmel,at91sam9";
chosen {
- bootargs = "128M console=ttyS0,115200 root=/dev/mtdblock1 rw rootfstype=ubifs ubi.mtd=1 root=ubi0:rootfs";
+ bootargs = "console=ttyS0,115200 root=/dev/mtdblock1 rw rootfstype=ubifs ubi.mtd=1 root=ubi0:rootfs";
};
ahb {
diff --git a/arch/arm/boot/dts/at91sam9g45.dtsi b/arch/arm/boot/dts/at91sam9g45.dtsi
index bafa880..63751b1 100644
--- a/arch/arm/boot/dts/at91sam9g45.dtsi
+++ b/arch/arm/boot/dts/at91sam9g45.dtsi
@@ -113,6 +113,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioB: gpio@fffff400 {
@@ -122,6 +123,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioC: gpio@fffff600 {
@@ -131,6 +133,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioD: gpio@fffff800 {
@@ -140,6 +143,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioE: gpio@fffffa00 {
@@ -149,6 +153,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
dbgu: serial@ffffee00 {
diff --git a/arch/arm/boot/dts/at91sam9n12.dtsi b/arch/arm/boot/dts/at91sam9n12.dtsi
index bfac0df..ef9336a 100644
--- a/arch/arm/boot/dts/at91sam9n12.dtsi
+++ b/arch/arm/boot/dts/at91sam9n12.dtsi
@@ -107,6 +107,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioB: gpio@fffff600 {
@@ -116,6 +117,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioC: gpio@fffff800 {
@@ -125,6 +127,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioD: gpio@fffffa00 {
@@ -134,6 +137,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
dbgu: serial@fffff200 {
diff --git a/arch/arm/boot/dts/at91sam9x5.dtsi b/arch/arm/boot/dts/at91sam9x5.dtsi
index 4a18c39..8a387a8 100644
--- a/arch/arm/boot/dts/at91sam9x5.dtsi
+++ b/arch/arm/boot/dts/at91sam9x5.dtsi
@@ -115,6 +115,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioB: gpio@fffff600 {
@@ -124,6 +125,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioC: gpio@fffff800 {
@@ -133,6 +135,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
pioD: gpio@fffffa00 {
@@ -142,6 +145,7 @@
#gpio-cells = <2>;
gpio-controller;
interrupt-controller;
+ #interrupt-cells = <2>;
};
dbgu: serial@fffff200 {
diff --git a/arch/arm/boot/dts/imx51-babbage.dts b/arch/arm/boot/dts/imx51-babbage.dts
index cd86177..59d9789 100644
--- a/arch/arm/boot/dts/imx51-babbage.dts
+++ b/arch/arm/boot/dts/imx51-babbage.dts
@@ -25,8 +25,8 @@
aips@70000000 { /* aips-1 */
spba@70000000 {
esdhc@70004000 { /* ESDHC1 */
- fsl,cd-internal;
- fsl,wp-internal;
+ fsl,cd-controller;
+ fsl,wp-controller;
status = "okay";
};
diff --git a/arch/arm/boot/dts/kirkwood-iconnect.dts b/arch/arm/boot/dts/kirkwood-iconnect.dts
index 52d9470..f8ca6fa 100644
--- a/arch/arm/boot/dts/kirkwood-iconnect.dts
+++ b/arch/arm/boot/dts/kirkwood-iconnect.dts
@@ -41,9 +41,13 @@
};
power-blue {
label = "power:blue";
- gpios = <&gpio1 11 0>;
+ gpios = <&gpio1 10 0>;
linux,default-trigger = "timer";
};
+ power-red {
+ label = "power:red";
+ gpios = <&gpio1 11 0>;
+ };
usb1 {
label = "usb1:blue";
gpios = <&gpio1 12 0>;
diff --git a/arch/arm/boot/dts/twl6030.dtsi b/arch/arm/boot/dts/twl6030.dtsi
index 3b2f351..d351b27 100644
--- a/arch/arm/boot/dts/twl6030.dtsi
+++ b/arch/arm/boot/dts/twl6030.dtsi
@@ -66,6 +66,7 @@
vcxio: regulator@8 {
compatible = "ti,twl6030-vcxio";
+ regulator-always-on;
};
vusb: regulator@9 {
@@ -74,10 +75,12 @@
v1v8: regulator@10 {
compatible = "ti,twl6030-v1v8";
+ regulator-always-on;
};
v2v1: regulator@11 {
compatible = "ti,twl6030-v2v1";
+ regulator-always-on;
};
clk32kg: regulator@12 {
diff --git a/arch/arm/configs/armadillo800eva_defconfig b/arch/arm/configs/armadillo800eva_defconfig
index 7d87184..90610c7 100644
--- a/arch/arm/configs/armadillo800eva_defconfig
+++ b/arch/arm/configs/armadillo800eva_defconfig
@@ -33,7 +33,7 @@
CONFIG_FORCE_MAX_ZONEORDER=13
CONFIG_ZBOOT_ROM_TEXT=0x0
CONFIG_ZBOOT_ROM_BSS=0x0
-CONFIG_CMDLINE="console=tty0 console=ttySC1,115200 earlyprintk=sh-sci.1,115200 ignore_loglevel root=/dev/nfs ip=dhcp nfsroot=,rsize=4096,wsize=4096"
+CONFIG_CMDLINE="console=tty0 console=ttySC1,115200 earlyprintk=sh-sci.1,115200 ignore_loglevel root=/dev/nfs ip=dhcp nfsroot=,rsize=4096,wsize=4096 rw"
CONFIG_CMDLINE_FORCE=y
CONFIG_KEXEC=y
CONFIG_VFP=y
diff --git a/arch/arm/configs/u8500_defconfig b/arch/arm/configs/u8500_defconfig
index 2d4f661..da68454 100644
--- a/arch/arm/configs/u8500_defconfig
+++ b/arch/arm/configs/u8500_defconfig
@@ -86,6 +86,7 @@
CONFIG_LEDS_CLASS=y
CONFIG_LEDS_LM3530=y
CONFIG_LEDS_LP5521=y
+CONFIG_LEDS_GPIO=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_DRV_AB8500=y
CONFIG_RTC_DRV_PL031=y
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 03fb936..5c8b3bf4 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -320,4 +320,12 @@
.size \name , . - \name
.endm
+ .macro check_uaccess, addr:req, size:req, limit:req, tmp:req, bad:req
+#ifndef CONFIG_CPU_USE_DOMAINS
+ adds \tmp, \addr, #\size - 1
+ sbcccs \tmp, \tmp, \limit
+ bcs \bad
+#endif
+ .endm
+
#endif /* __ASM_ASSEMBLER_H__ */
diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index 2ae842d..5c44dcb 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -203,6 +203,13 @@
}
/*
+ * This can be called during early boot to increase the size of the atomic
+ * coherent DMA pool above the default value of 256KiB. It must be called
+ * before postcore_initcall.
+ */
+extern void __init init_dma_coherent_pool_size(unsigned long size);
+
+/*
* This can be called during boot to increase the size of the consistent
* DMA region above it's default value of 2MB. It must be called before the
* memory allocator is initialised, i.e. before any core_initcall.
diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
index e965f1b..5f6ddcc 100644
--- a/arch/arm/include/asm/memory.h
+++ b/arch/arm/include/asm/memory.h
@@ -187,6 +187,7 @@
#define __phys_to_virt(x) ((x) - PHYS_OFFSET + PAGE_OFFSET)
#endif
#endif
+#endif /* __ASSEMBLY__ */
#ifndef PHYS_OFFSET
#ifdef PLAT_PHYS_OFFSET
@@ -196,6 +197,8 @@
#endif
#endif
+#ifndef __ASSEMBLY__
+
/*
* PFNs are used to describe any physical page; this means
* PFN 0 == physical address 0.
diff --git a/arch/arm/include/asm/tlb.h b/arch/arm/include/asm/tlb.h
index 314d466..99a1951 100644
--- a/arch/arm/include/asm/tlb.h
+++ b/arch/arm/include/asm/tlb.h
@@ -199,6 +199,9 @@
{
pgtable_page_dtor(pte);
+#ifdef CONFIG_ARM_LPAE
+ tlb_add_flush(tlb, addr);
+#else
/*
* With the classic ARM MMU, a pte page has two corresponding pmd
* entries, each covering 1MB.
@@ -206,6 +209,7 @@
addr &= PMD_MASK;
tlb_add_flush(tlb, addr + SZ_1M - PAGE_SIZE);
tlb_add_flush(tlb, addr + SZ_1M);
+#endif
tlb_remove_page(tlb, pte);
}
diff --git a/arch/arm/include/asm/uaccess.h b/arch/arm/include/asm/uaccess.h
index 479a635..77bd79f 100644
--- a/arch/arm/include/asm/uaccess.h
+++ b/arch/arm/include/asm/uaccess.h
@@ -101,28 +101,39 @@
extern int __get_user_2(void *);
extern int __get_user_4(void *);
-#define __get_user_x(__r2,__p,__e,__s,__i...) \
+#define __GUP_CLOBBER_1 "lr", "cc"
+#ifdef CONFIG_CPU_USE_DOMAINS
+#define __GUP_CLOBBER_2 "ip", "lr", "cc"
+#else
+#define __GUP_CLOBBER_2 "lr", "cc"
+#endif
+#define __GUP_CLOBBER_4 "lr", "cc"
+
+#define __get_user_x(__r2,__p,__e,__l,__s) \
__asm__ __volatile__ ( \
__asmeq("%0", "r0") __asmeq("%1", "r2") \
+ __asmeq("%3", "r1") \
"bl __get_user_" #__s \
: "=&r" (__e), "=r" (__r2) \
- : "0" (__p) \
- : __i, "cc")
+ : "0" (__p), "r" (__l) \
+ : __GUP_CLOBBER_##__s)
-#define get_user(x,p) \
+#define __get_user_check(x,p) \
({ \
+ unsigned long __limit = current_thread_info()->addr_limit - 1; \
register const typeof(*(p)) __user *__p asm("r0") = (p);\
register unsigned long __r2 asm("r2"); \
+ register unsigned long __l asm("r1") = __limit; \
register int __e asm("r0"); \
switch (sizeof(*(__p))) { \
case 1: \
- __get_user_x(__r2, __p, __e, 1, "lr"); \
- break; \
+ __get_user_x(__r2, __p, __e, __l, 1); \
+ break; \
case 2: \
- __get_user_x(__r2, __p, __e, 2, "r3", "lr"); \
+ __get_user_x(__r2, __p, __e, __l, 2); \
break; \
case 4: \
- __get_user_x(__r2, __p, __e, 4, "lr"); \
+ __get_user_x(__r2, __p, __e, __l, 4); \
break; \
default: __e = __get_user_bad(); break; \
} \
@@ -130,42 +141,57 @@
__e; \
})
+#define get_user(x,p) \
+ ({ \
+ might_fault(); \
+ __get_user_check(x,p); \
+ })
+
extern int __put_user_1(void *, unsigned int);
extern int __put_user_2(void *, unsigned int);
extern int __put_user_4(void *, unsigned int);
extern int __put_user_8(void *, unsigned long long);
-#define __put_user_x(__r2,__p,__e,__s) \
+#define __put_user_x(__r2,__p,__e,__l,__s) \
__asm__ __volatile__ ( \
__asmeq("%0", "r0") __asmeq("%2", "r2") \
+ __asmeq("%3", "r1") \
"bl __put_user_" #__s \
: "=&r" (__e) \
- : "0" (__p), "r" (__r2) \
+ : "0" (__p), "r" (__r2), "r" (__l) \
: "ip", "lr", "cc")
-#define put_user(x,p) \
+#define __put_user_check(x,p) \
({ \
+ unsigned long __limit = current_thread_info()->addr_limit - 1; \
register const typeof(*(p)) __r2 asm("r2") = (x); \
register const typeof(*(p)) __user *__p asm("r0") = (p);\
+ register unsigned long __l asm("r1") = __limit; \
register int __e asm("r0"); \
switch (sizeof(*(__p))) { \
case 1: \
- __put_user_x(__r2, __p, __e, 1); \
+ __put_user_x(__r2, __p, __e, __l, 1); \
break; \
case 2: \
- __put_user_x(__r2, __p, __e, 2); \
+ __put_user_x(__r2, __p, __e, __l, 2); \
break; \
case 4: \
- __put_user_x(__r2, __p, __e, 4); \
+ __put_user_x(__r2, __p, __e, __l, 4); \
break; \
case 8: \
- __put_user_x(__r2, __p, __e, 8); \
+ __put_user_x(__r2, __p, __e, __l, 8); \
break; \
default: __e = __put_user_bad(); break; \
} \
__e; \
})
+#define put_user(x,p) \
+ ({ \
+ might_fault(); \
+ __put_user_check(x,p); \
+ })
+
#else /* CONFIG_MMU */
/*
@@ -219,6 +245,7 @@
unsigned long __gu_addr = (unsigned long)(ptr); \
unsigned long __gu_val; \
__chk_user_ptr(ptr); \
+ might_fault(); \
switch (sizeof(*(ptr))) { \
case 1: __get_user_asm_byte(__gu_val,__gu_addr,err); break; \
case 2: __get_user_asm_half(__gu_val,__gu_addr,err); break; \
@@ -300,6 +327,7 @@
unsigned long __pu_addr = (unsigned long)(ptr); \
__typeof__(*(ptr)) __pu_val = (x); \
__chk_user_ptr(ptr); \
+ might_fault(); \
switch (sizeof(*(ptr))) { \
case 1: __put_user_asm_byte(__pu_val,__pu_addr,err); break; \
case 2: __put_user_asm_half(__pu_val,__pu_addr,err); break; \
diff --git a/arch/arm/include/asm/unistd.h b/arch/arm/include/asm/unistd.h
index 0cab47d..2fde5fd 100644
--- a/arch/arm/include/asm/unistd.h
+++ b/arch/arm/include/asm/unistd.h
@@ -404,6 +404,7 @@
#define __NR_setns (__NR_SYSCALL_BASE+375)
#define __NR_process_vm_readv (__NR_SYSCALL_BASE+376)
#define __NR_process_vm_writev (__NR_SYSCALL_BASE+377)
+ /* 378 for kcmp */
/*
* The following SWIs are ARM private.
@@ -483,6 +484,7 @@
*/
#define __IGNORE_fadvise64_64
#define __IGNORE_migrate_pages
+#define __IGNORE_kcmp
#endif /* __KERNEL__ */
#endif /* __ASM_ARM_UNISTD_H */
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 2b2f25e..9cf16b8 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -270,15 +270,6 @@
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ITE, PCI_DEVICE_ID_ITE_8152, pci_fixup_it8152);
-
-
-void __devinit pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- if (debug_pci)
- printk("PCI: Assigning IRQ %02d to %s\n", irq, pci_name(dev));
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
/*
* If the bus contains any of these devices, then we must not turn on
* parity checking of any kind. Currently this is CyberPro 20x0 only.
diff --git a/arch/arm/kernel/calls.S b/arch/arm/kernel/calls.S
index 463ff4a..e337879 100644
--- a/arch/arm/kernel/calls.S
+++ b/arch/arm/kernel/calls.S
@@ -387,6 +387,7 @@
/* 375 */ CALL(sys_setns)
CALL(sys_process_vm_readv)
CALL(sys_process_vm_writev)
+ CALL(sys_ni_syscall) /* reserved for sys_kcmp */
#ifndef syscalls_counted
.equ syscalls_padding, ((NR_syscalls + 3) & ~3) - NR_syscalls
#define syscalls_counted
diff --git a/arch/arm/kernel/hw_breakpoint.c b/arch/arm/kernel/hw_breakpoint.c
index ba386bd..281bf33 100644
--- a/arch/arm/kernel/hw_breakpoint.c
+++ b/arch/arm/kernel/hw_breakpoint.c
@@ -159,6 +159,12 @@
arch >= ARM_DEBUG_ARCH_V7_1;
}
+/* Can we determine the watchpoint access type from the fsr? */
+static int debug_exception_updates_fsr(void)
+{
+ return 0;
+}
+
/* Determine number of WRP registers available. */
static int get_num_wrp_resources(void)
{
@@ -604,13 +610,14 @@
/* Aligned */
break;
case 1:
- /* Allow single byte watchpoint. */
- if (info->ctrl.len == ARM_BREAKPOINT_LEN_1)
- break;
case 2:
/* Allow halfword watchpoints and breakpoints. */
if (info->ctrl.len == ARM_BREAKPOINT_LEN_2)
break;
+ case 3:
+ /* Allow single byte watchpoint. */
+ if (info->ctrl.len == ARM_BREAKPOINT_LEN_1)
+ break;
default:
ret = -EINVAL;
goto out;
@@ -619,18 +626,35 @@
info->address &= ~alignment_mask;
info->ctrl.len <<= offset;
- /*
- * Currently we rely on an overflow handler to take
- * care of single-stepping the breakpoint when it fires.
- * In the case of userspace breakpoints on a core with V7 debug,
- * we can use the mismatch feature as a poor-man's hardware
- * single-step, but this only works for per-task breakpoints.
- */
- if (!bp->overflow_handler && (arch_check_bp_in_kernelspace(bp) ||
- !core_has_mismatch_brps() || !bp->hw.bp_target)) {
- pr_warning("overflow handler required but none found\n");
- ret = -EINVAL;
+ if (!bp->overflow_handler) {
+ /*
+ * Mismatch breakpoints are required for single-stepping
+ * breakpoints.
+ */
+ if (!core_has_mismatch_brps())
+ return -EINVAL;
+
+ /* We don't allow mismatch breakpoints in kernel space. */
+ if (arch_check_bp_in_kernelspace(bp))
+ return -EPERM;
+
+ /*
+ * Per-cpu breakpoints are not supported by our stepping
+ * mechanism.
+ */
+ if (!bp->hw.bp_target)
+ return -EINVAL;
+
+ /*
+ * We only support specific access types if the fsr
+ * reports them.
+ */
+ if (!debug_exception_updates_fsr() &&
+ (info->ctrl.type == ARM_BREAKPOINT_LOAD ||
+ info->ctrl.type == ARM_BREAKPOINT_STORE))
+ return -EINVAL;
}
+
out:
return ret;
}
@@ -706,10 +730,12 @@
goto unlock;
/* Check that the access type matches. */
- access = (fsr & ARM_FSR_ACCESS_MASK) ? HW_BREAKPOINT_W :
- HW_BREAKPOINT_R;
- if (!(access & hw_breakpoint_type(wp)))
- goto unlock;
+ if (debug_exception_updates_fsr()) {
+ access = (fsr & ARM_FSR_ACCESS_MASK) ?
+ HW_BREAKPOINT_W : HW_BREAKPOINT_R;
+ if (!(access & hw_breakpoint_type(wp)))
+ goto unlock;
+ }
/* We have a winner. */
info->trigger = addr;
diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c
index fef42b2..e1f9069 100644
--- a/arch/arm/kernel/smp_twd.c
+++ b/arch/arm/kernel/smp_twd.c
@@ -11,7 +11,6 @@
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/clk.h>
-#include <linux/cpufreq.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/err.h>
@@ -96,7 +95,52 @@
disable_percpu_irq(clk->irq);
}
-#ifdef CONFIG_CPU_FREQ
+#ifdef CONFIG_COMMON_CLK
+
+/*
+ * Updates clockevent frequency when the cpu frequency changes.
+ * Called on the cpu that is changing frequency with interrupts disabled.
+ */
+static void twd_update_frequency(void *new_rate)
+{
+ twd_timer_rate = *((unsigned long *) new_rate);
+
+ clockevents_update_freq(*__this_cpu_ptr(twd_evt), twd_timer_rate);
+}
+
+static int twd_rate_change(struct notifier_block *nb,
+ unsigned long flags, void *data)
+{
+ struct clk_notifier_data *cnd = data;
+
+ /*
+ * The twd clock events must be reprogrammed to account for the new
+ * frequency. The timer is local to a cpu, so cross-call to the
+ * changing cpu.
+ */
+ if (flags == POST_RATE_CHANGE)
+ smp_call_function(twd_update_frequency,
+ (void *)&cnd->new_rate, 1);
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block twd_clk_nb = {
+ .notifier_call = twd_rate_change,
+};
+
+static int twd_clk_init(void)
+{
+ if (twd_evt && *__this_cpu_ptr(twd_evt) && !IS_ERR(twd_clk))
+ return clk_notifier_register(twd_clk, &twd_clk_nb);
+
+ return 0;
+}
+core_initcall(twd_clk_init);
+
+#elif defined (CONFIG_CPU_FREQ)
+
+#include <linux/cpufreq.h>
/*
* Updates clockevent frequency when the cpu frequency changes.
diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c
index f794521..b0179b8 100644
--- a/arch/arm/kernel/traps.c
+++ b/arch/arm/kernel/traps.c
@@ -420,20 +420,23 @@
#endif
instr = *(u32 *) pc;
} else if (thumb_mode(regs)) {
- get_user(instr, (u16 __user *)pc);
+ if (get_user(instr, (u16 __user *)pc))
+ goto die_sig;
if (is_wide_instruction(instr)) {
unsigned int instr2;
- get_user(instr2, (u16 __user *)pc+1);
+ if (get_user(instr2, (u16 __user *)pc+1))
+ goto die_sig;
instr <<= 16;
instr |= instr2;
}
- } else {
- get_user(instr, (u32 __user *)pc);
+ } else if (get_user(instr, (u32 __user *)pc)) {
+ goto die_sig;
}
if (call_undef_hook(regs, instr) == 0)
return;
+die_sig:
#ifdef CONFIG_DEBUG_USER
if (user_debug & UDBG_UNDEFINED) {
printk(KERN_INFO "%s (%d): undefined instruction: pc=%p\n",
diff --git a/arch/arm/lib/delay.c b/arch/arm/lib/delay.c
index d6dacc6..395d5fb 100644
--- a/arch/arm/lib/delay.c
+++ b/arch/arm/lib/delay.c
@@ -59,6 +59,7 @@
{
pr_info("Switching to timer-based delay loop\n");
lpj_fine = freq / HZ;
+ loops_per_jiffy = lpj_fine;
arm_delay_ops.delay = __timer_delay;
arm_delay_ops.const_udelay = __timer_const_udelay;
arm_delay_ops.udelay = __timer_udelay;
diff --git a/arch/arm/lib/getuser.S b/arch/arm/lib/getuser.S
index 11093a7..9b06bb4 100644
--- a/arch/arm/lib/getuser.S
+++ b/arch/arm/lib/getuser.S
@@ -16,8 +16,9 @@
* __get_user_X
*
* Inputs: r0 contains the address
+ * r1 contains the address limit, which must be preserved
* Outputs: r0 is the error code
- * r2, r3 contains the zero-extended value
+ * r2 contains the zero-extended value
* lr corrupted
*
* No other registers must be altered. (see <asm/uaccess.h>
@@ -27,33 +28,39 @@
* Note also that it is intended that __get_user_bad is not global.
*/
#include <linux/linkage.h>
+#include <asm/assembler.h>
#include <asm/errno.h>
#include <asm/domain.h>
ENTRY(__get_user_1)
+ check_uaccess r0, 1, r1, r2, __get_user_bad
1: TUSER(ldrb) r2, [r0]
mov r0, #0
mov pc, lr
ENDPROC(__get_user_1)
ENTRY(__get_user_2)
-#ifdef CONFIG_THUMB2_KERNEL
-2: TUSER(ldrb) r2, [r0]
-3: TUSER(ldrb) r3, [r0, #1]
+ check_uaccess r0, 2, r1, r2, __get_user_bad
+#ifdef CONFIG_CPU_USE_DOMAINS
+rb .req ip
+2: ldrbt r2, [r0], #1
+3: ldrbt rb, [r0], #0
#else
-2: TUSER(ldrb) r2, [r0], #1
-3: TUSER(ldrb) r3, [r0]
+rb .req r0
+2: ldrb r2, [r0]
+3: ldrb rb, [r0, #1]
#endif
#ifndef __ARMEB__
- orr r2, r2, r3, lsl #8
+ orr r2, r2, rb, lsl #8
#else
- orr r2, r3, r2, lsl #8
+ orr r2, rb, r2, lsl #8
#endif
mov r0, #0
mov pc, lr
ENDPROC(__get_user_2)
ENTRY(__get_user_4)
+ check_uaccess r0, 4, r1, r2, __get_user_bad
4: TUSER(ldr) r2, [r0]
mov r0, #0
mov pc, lr
diff --git a/arch/arm/lib/putuser.S b/arch/arm/lib/putuser.S
index 7db2599..3d73dcb 100644
--- a/arch/arm/lib/putuser.S
+++ b/arch/arm/lib/putuser.S
@@ -16,6 +16,7 @@
* __put_user_X
*
* Inputs: r0 contains the address
+ * r1 contains the address limit, which must be preserved
* r2, r3 contains the value
* Outputs: r0 is the error code
* lr corrupted
@@ -27,16 +28,19 @@
* Note also that it is intended that __put_user_bad is not global.
*/
#include <linux/linkage.h>
+#include <asm/assembler.h>
#include <asm/errno.h>
#include <asm/domain.h>
ENTRY(__put_user_1)
+ check_uaccess r0, 1, r1, ip, __put_user_bad
1: TUSER(strb) r2, [r0]
mov r0, #0
mov pc, lr
ENDPROC(__put_user_1)
ENTRY(__put_user_2)
+ check_uaccess r0, 2, r1, ip, __put_user_bad
mov ip, r2, lsr #8
#ifdef CONFIG_THUMB2_KERNEL
#ifndef __ARMEB__
@@ -60,12 +64,14 @@
ENDPROC(__put_user_2)
ENTRY(__put_user_4)
+ check_uaccess r0, 4, r1, ip, __put_user_bad
4: TUSER(str) r2, [r0]
mov r0, #0
mov pc, lr
ENDPROC(__put_user_4)
ENTRY(__put_user_8)
+ check_uaccess r0, 8, r1, ip, __put_user_bad
#ifdef CONFIG_THUMB2_KERNEL
5: TUSER(str) r2, [r0]
6: TUSER(str) r3, [r0, #4]
diff --git a/arch/arm/mach-at91/at91rm9200_time.c b/arch/arm/mach-at91/at91rm9200_time.c
index 104ca40..aaa443b 100644
--- a/arch/arm/mach-at91/at91rm9200_time.c
+++ b/arch/arm/mach-at91/at91rm9200_time.c
@@ -197,7 +197,7 @@
at91_st_read(AT91_ST_SR);
/* Make IRQs happen for the system timer */
- setup_irq(AT91_ID_SYS, &at91rm9200_timer_irq);
+ setup_irq(NR_IRQS_LEGACY + AT91_ID_SYS, &at91rm9200_timer_irq);
/* The 32KiHz "Slow Clock" (tick every 30517.58 nanoseconds) is used
* directly for the clocksource and all clockevents, after adjusting
diff --git a/arch/arm/mach-at91/at91sam9260_devices.c b/arch/arm/mach-at91/at91sam9260_devices.c
index 7b9c2ba..bce572a 100644
--- a/arch/arm/mach-at91/at91sam9260_devices.c
+++ b/arch/arm/mach-at91/at91sam9260_devices.c
@@ -726,6 +726,8 @@
.flags = IORESOURCE_MEM,
}, {
.flags = IORESOURCE_MEM,
+ }, {
+ .flags = IORESOURCE_IRQ,
},
};
@@ -744,10 +746,12 @@
* The second resource is needed:
* GPBR will serve as the storage for RTC time offset
*/
- at91sam9260_rtt_device.num_resources = 2;
+ at91sam9260_rtt_device.num_resources = 3;
rtt_resources[1].start = AT91SAM9260_BASE_GPBR +
4 * CONFIG_RTC_DRV_AT91SAM9_GPBR;
rtt_resources[1].end = rtt_resources[1].start + 3;
+ rtt_resources[2].start = NR_IRQS_LEGACY + AT91_ID_SYS;
+ rtt_resources[2].end = NR_IRQS_LEGACY + AT91_ID_SYS;
}
#else
static void __init at91_add_device_rtt_rtc(void)
diff --git a/arch/arm/mach-at91/at91sam9261_devices.c b/arch/arm/mach-at91/at91sam9261_devices.c
index 8df5c1bd..bc2590d 100644
--- a/arch/arm/mach-at91/at91sam9261_devices.c
+++ b/arch/arm/mach-at91/at91sam9261_devices.c
@@ -609,6 +609,8 @@
.flags = IORESOURCE_MEM,
}, {
.flags = IORESOURCE_MEM,
+ }, {
+ .flags = IORESOURCE_IRQ,
}
};
@@ -626,10 +628,12 @@
* The second resource is needed:
* GPBR will serve as the storage for RTC time offset
*/
- at91sam9261_rtt_device.num_resources = 2;
+ at91sam9261_rtt_device.num_resources = 3;
rtt_resources[1].start = AT91SAM9261_BASE_GPBR +
4 * CONFIG_RTC_DRV_AT91SAM9_GPBR;
rtt_resources[1].end = rtt_resources[1].start + 3;
+ rtt_resources[2].start = NR_IRQS_LEGACY + AT91_ID_SYS;
+ rtt_resources[2].end = NR_IRQS_LEGACY + AT91_ID_SYS;
}
#else
static void __init at91_add_device_rtt_rtc(void)
diff --git a/arch/arm/mach-at91/at91sam9263_devices.c b/arch/arm/mach-at91/at91sam9263_devices.c
index eb6bbf8..9b6ca73 100644
--- a/arch/arm/mach-at91/at91sam9263_devices.c
+++ b/arch/arm/mach-at91/at91sam9263_devices.c
@@ -990,6 +990,8 @@
.flags = IORESOURCE_MEM,
}, {
.flags = IORESOURCE_MEM,
+ }, {
+ .flags = IORESOURCE_IRQ,
}
};
@@ -1006,6 +1008,8 @@
.flags = IORESOURCE_MEM,
}, {
.flags = IORESOURCE_MEM,
+ }, {
+ .flags = IORESOURCE_IRQ,
}
};
@@ -1027,14 +1031,14 @@
* The second resource is needed only for the chosen RTT:
* GPBR will serve as the storage for RTC time offset
*/
- at91sam9263_rtt0_device.num_resources = 2;
+ at91sam9263_rtt0_device.num_resources = 3;
at91sam9263_rtt1_device.num_resources = 1;
pdev = &at91sam9263_rtt0_device;
r = rtt0_resources;
break;
case 1:
at91sam9263_rtt0_device.num_resources = 1;
- at91sam9263_rtt1_device.num_resources = 2;
+ at91sam9263_rtt1_device.num_resources = 3;
pdev = &at91sam9263_rtt1_device;
r = rtt1_resources;
break;
@@ -1047,6 +1051,8 @@
pdev->name = "rtc-at91sam9";
r[1].start = AT91SAM9263_BASE_GPBR + 4 * CONFIG_RTC_DRV_AT91SAM9_GPBR;
r[1].end = r[1].start + 3;
+ r[2].start = NR_IRQS_LEGACY + AT91_ID_SYS;
+ r[2].end = NR_IRQS_LEGACY + AT91_ID_SYS;
}
#else
static void __init at91_add_device_rtt_rtc(void)
diff --git a/arch/arm/mach-at91/at91sam9g45_devices.c b/arch/arm/mach-at91/at91sam9g45_devices.c
index 0607399..1b47319 100644
--- a/arch/arm/mach-at91/at91sam9g45_devices.c
+++ b/arch/arm/mach-at91/at91sam9g45_devices.c
@@ -1293,6 +1293,8 @@
.flags = IORESOURCE_MEM,
}, {
.flags = IORESOURCE_MEM,
+ }, {
+ .flags = IORESOURCE_IRQ,
}
};
@@ -1310,10 +1312,12 @@
* The second resource is needed:
* GPBR will serve as the storage for RTC time offset
*/
- at91sam9g45_rtt_device.num_resources = 2;
+ at91sam9g45_rtt_device.num_resources = 3;
rtt_resources[1].start = AT91SAM9G45_BASE_GPBR +
4 * CONFIG_RTC_DRV_AT91SAM9_GPBR;
rtt_resources[1].end = rtt_resources[1].start + 3;
+ rtt_resources[2].start = NR_IRQS_LEGACY + AT91_ID_SYS;
+ rtt_resources[2].end = NR_IRQS_LEGACY + AT91_ID_SYS;
}
#else
static void __init at91_add_device_rtt_rtc(void)
diff --git a/arch/arm/mach-at91/at91sam9rl_devices.c b/arch/arm/mach-at91/at91sam9rl_devices.c
index f09fff9..b3d365d 100644
--- a/arch/arm/mach-at91/at91sam9rl_devices.c
+++ b/arch/arm/mach-at91/at91sam9rl_devices.c
@@ -688,6 +688,8 @@
.flags = IORESOURCE_MEM,
}, {
.flags = IORESOURCE_MEM,
+ }, {
+ .flags = IORESOURCE_IRQ,
}
};
@@ -705,10 +707,12 @@
* The second resource is needed:
* GPBR will serve as the storage for RTC time offset
*/
- at91sam9rl_rtt_device.num_resources = 2;
+ at91sam9rl_rtt_device.num_resources = 3;
rtt_resources[1].start = AT91SAM9RL_BASE_GPBR +
4 * CONFIG_RTC_DRV_AT91SAM9_GPBR;
rtt_resources[1].end = rtt_resources[1].start + 3;
+ rtt_resources[2].start = NR_IRQS_LEGACY + AT91_ID_SYS;
+ rtt_resources[2].end = NR_IRQS_LEGACY + AT91_ID_SYS;
}
#else
static void __init at91_add_device_rtt_rtc(void)
diff --git a/arch/arm/mach-at91/clock.c b/arch/arm/mach-at91/clock.c
index de2ec6b..188c829 100644
--- a/arch/arm/mach-at91/clock.c
+++ b/arch/arm/mach-at91/clock.c
@@ -63,6 +63,12 @@
#define cpu_has_300M_plla() (cpu_is_at91sam9g10())
+#define cpu_has_240M_plla() (cpu_is_at91sam9261() \
+ || cpu_is_at91sam9263() \
+ || cpu_is_at91sam9rl())
+
+#define cpu_has_210M_plla() (cpu_is_at91sam9260())
+
#define cpu_has_pllb() (!(cpu_is_at91sam9rl() \
|| cpu_is_at91sam9g45() \
|| cpu_is_at91sam9x5() \
@@ -706,6 +712,12 @@
} else if (cpu_has_800M_plla()) {
if (plla.rate_hz > 800000000)
pll_overclock = true;
+ } else if (cpu_has_240M_plla()) {
+ if (plla.rate_hz > 240000000)
+ pll_overclock = true;
+ } else if (cpu_has_210M_plla()) {
+ if (plla.rate_hz > 210000000)
+ pll_overclock = true;
} else {
if (plla.rate_hz > 209000000)
pll_overclock = true;
diff --git a/arch/arm/mach-dove/common.c b/arch/arm/mach-dove/common.c
index 4db5de5..6321567 100644
--- a/arch/arm/mach-dove/common.c
+++ b/arch/arm/mach-dove/common.c
@@ -102,7 +102,8 @@
void __init dove_ge00_init(struct mv643xx_eth_platform_data *eth_data)
{
orion_ge00_init(eth_data, DOVE_GE00_PHYS_BASE,
- IRQ_DOVE_GE00_SUM, IRQ_DOVE_GE00_ERR);
+ IRQ_DOVE_GE00_SUM, IRQ_DOVE_GE00_ERR,
+ 1600);
}
/*****************************************************************************
diff --git a/arch/arm/mach-exynos/mach-origen.c b/arch/arm/mach-exynos/mach-origen.c
index 5ca8030..4e574c2 100644
--- a/arch/arm/mach-exynos/mach-origen.c
+++ b/arch/arm/mach-exynos/mach-origen.c
@@ -42,6 +42,7 @@
#include <plat/backlight.h>
#include <plat/fb.h>
#include <plat/mfc.h>
+#include <plat/hdmi.h>
#include <mach/ohci.h>
#include <mach/map.h>
@@ -734,6 +735,11 @@
s3c_gpio_setpull(EXYNOS4_GPX2(2), S3C_GPIO_PULL_NONE);
}
+/* I2C module and id for HDMIPHY */
+static struct i2c_board_info hdmiphy_info = {
+ I2C_BOARD_INFO("hdmiphy-exynos4210", 0x38),
+};
+
static void s5p_tv_setup(void)
{
/* Direct HPD to HDMI chip */
@@ -781,6 +787,7 @@
s5p_tv_setup();
s5p_i2c_hdmiphy_set_platdata(NULL);
+ s5p_hdmi_set_platdata(&hdmiphy_info, NULL, 0);
#ifdef CONFIG_DRM_EXYNOS
s5p_device_fimd0.dev.platform_data = &drm_fimd_pdata;
diff --git a/arch/arm/mach-exynos/mach-smdkv310.c b/arch/arm/mach-exynos/mach-smdkv310.c
index 3cfa688..73f2bce 100644
--- a/arch/arm/mach-exynos/mach-smdkv310.c
+++ b/arch/arm/mach-exynos/mach-smdkv310.c
@@ -40,6 +40,7 @@
#include <plat/mfc.h>
#include <plat/ehci.h>
#include <plat/clock.h>
+#include <plat/hdmi.h>
#include <mach/map.h>
#include <mach/ohci.h>
@@ -354,6 +355,11 @@
.pwm_period_ns = 1000,
};
+/* I2C module and id for HDMIPHY */
+static struct i2c_board_info hdmiphy_info = {
+ I2C_BOARD_INFO("hdmiphy-exynos4210", 0x38),
+};
+
static void s5p_tv_setup(void)
{
/* direct HPD to HDMI chip */
@@ -388,6 +394,7 @@
s5p_tv_setup();
s5p_i2c_hdmiphy_set_platdata(NULL);
+ s5p_hdmi_set_platdata(&hdmiphy_info, NULL, 0);
samsung_keypad_set_platdata(&smdkv310_keypad_data);
diff --git a/arch/arm/mach-gemini/irq.c b/arch/arm/mach-gemini/irq.c
index ca70e5f..020852d 100644
--- a/arch/arm/mach-gemini/irq.c
+++ b/arch/arm/mach-gemini/irq.c
@@ -17,6 +17,7 @@
#include <linux/sched.h>
#include <asm/irq.h>
#include <asm/mach/irq.h>
+#include <asm/system_misc.h>
#include <mach/hardware.h>
#define IRQ_SOURCE(base_addr) (base_addr + 0x00)
diff --git a/arch/arm/mach-imx/Makefile b/arch/arm/mach-imx/Makefile
index 07f7c22..d004d37 100644
--- a/arch/arm/mach-imx/Makefile
+++ b/arch/arm/mach-imx/Makefile
@@ -9,7 +9,8 @@
obj-$(CONFIG_SOC_IMX31) += mm-imx3.o cpu-imx31.o clk-imx31.o iomux-imx31.o ehci-imx31.o pm-imx3.o
obj-$(CONFIG_SOC_IMX35) += mm-imx3.o cpu-imx35.o clk-imx35.o ehci-imx35.o pm-imx3.o
-obj-$(CONFIG_SOC_IMX5) += cpu-imx5.o mm-imx5.o clk-imx51-imx53.o ehci-imx5.o pm-imx5.o cpu_op-mx51.o
+imx5-pm-$(CONFIG_PM) += pm-imx5.o
+obj-$(CONFIG_SOC_IMX5) += cpu-imx5.o mm-imx5.o clk-imx51-imx53.o ehci-imx5.o $(imx5-pm-y) cpu_op-mx51.o
obj-$(CONFIG_COMMON_CLK) += clk-pllv1.o clk-pllv2.o clk-pllv3.o clk-gate2.o \
clk-pfd.o clk-busy.o
@@ -70,14 +71,13 @@
obj-$(CONFIG_HAVE_IMX_GPC) += gpc.o
obj-$(CONFIG_HAVE_IMX_MMDC) += mmdc.o
obj-$(CONFIG_HAVE_IMX_SRC) += src.o
-obj-$(CONFIG_CPU_V7) += head-v7.o
-AFLAGS_head-v7.o :=-Wa,-march=armv7-a
-obj-$(CONFIG_SMP) += platsmp.o
+AFLAGS_headsmp.o :=-Wa,-march=armv7-a
+obj-$(CONFIG_SMP) += headsmp.o platsmp.o
obj-$(CONFIG_HOTPLUG_CPU) += hotplug.o
obj-$(CONFIG_SOC_IMX6Q) += clk-imx6q.o mach-imx6q.o
ifeq ($(CONFIG_PM),y)
-obj-$(CONFIG_SOC_IMX6Q) += pm-imx6q.o
+obj-$(CONFIG_SOC_IMX6Q) += pm-imx6q.o headsmp.o
endif
# i.MX5 based machines
diff --git a/arch/arm/mach-imx/clk-imx25.c b/arch/arm/mach-imx/clk-imx25.c
index fdd8cc8..d20d479 100644
--- a/arch/arm/mach-imx/clk-imx25.c
+++ b/arch/arm/mach-imx/clk-imx25.c
@@ -222,10 +222,8 @@
clk_register_clkdev(clk[lcdc_ipg], "ipg", "imx-fb.0");
clk_register_clkdev(clk[lcdc_ahb], "ahb", "imx-fb.0");
clk_register_clkdev(clk[wdt_ipg], NULL, "imx2-wdt.0");
- clk_register_clkdev(clk[ssi1_ipg_per], "per", "imx-ssi.0");
- clk_register_clkdev(clk[ssi1_ipg], "ipg", "imx-ssi.0");
- clk_register_clkdev(clk[ssi2_ipg_per], "per", "imx-ssi.1");
- clk_register_clkdev(clk[ssi2_ipg], "ipg", "imx-ssi.1");
+ clk_register_clkdev(clk[ssi1_ipg], NULL, "imx-ssi.0");
+ clk_register_clkdev(clk[ssi2_ipg], NULL, "imx-ssi.1");
clk_register_clkdev(clk[esdhc1_ipg_per], "per", "sdhci-esdhc-imx25.0");
clk_register_clkdev(clk[esdhc1_ipg], "ipg", "sdhci-esdhc-imx25.0");
clk_register_clkdev(clk[esdhc1_ahb], "ahb", "sdhci-esdhc-imx25.0");
@@ -243,6 +241,6 @@
clk_register_clkdev(clk[sdma_ahb], "ahb", "imx35-sdma");
clk_register_clkdev(clk[iim_ipg], "iim", NULL);
- mxc_timer_init(MX25_IO_ADDRESS(MX25_GPT1_BASE_ADDR), 54);
+ mxc_timer_init(MX25_IO_ADDRESS(MX25_GPT1_BASE_ADDR), MX25_INT_GPT1);
return 0;
}
diff --git a/arch/arm/mach-imx/clk-imx35.c b/arch/arm/mach-imx/clk-imx35.c
index c6422fb..65fb8bc 100644
--- a/arch/arm/mach-imx/clk-imx35.c
+++ b/arch/arm/mach-imx/clk-imx35.c
@@ -230,10 +230,8 @@
clk_register_clkdev(clk[ipu_gate], NULL, "mx3_sdc_fb");
clk_register_clkdev(clk[owire_gate], NULL, "mxc_w1");
clk_register_clkdev(clk[sdma_gate], NULL, "imx35-sdma");
- clk_register_clkdev(clk[ipg], "ipg", "imx-ssi.0");
- clk_register_clkdev(clk[ssi1_div_post], "per", "imx-ssi.0");
- clk_register_clkdev(clk[ipg], "ipg", "imx-ssi.1");
- clk_register_clkdev(clk[ssi2_div_post], "per", "imx-ssi.1");
+ clk_register_clkdev(clk[ssi1_gate], NULL, "imx-ssi.0");
+ clk_register_clkdev(clk[ssi2_gate], NULL, "imx-ssi.1");
/* i.mx35 has the i.mx21 type uart */
clk_register_clkdev(clk[uart1_gate], "per", "imx21-uart.0");
clk_register_clkdev(clk[ipg], "ipg", "imx21-uart.0");
diff --git a/arch/arm/mach-imx/clk-imx6q.c b/arch/arm/mach-imx/clk-imx6q.c
index ea89520..4233d9e 100644
--- a/arch/arm/mach-imx/clk-imx6q.c
+++ b/arch/arm/mach-imx/clk-imx6q.c
@@ -152,7 +152,7 @@
ssi2, ssi3, uart_ipg, uart_serial, usboh3, usdhc1, usdhc2, usdhc3,
usdhc4, vdo_axi, vpu_axi, cko1, pll1_sys, pll2_bus, pll3_usb_otg,
pll4_audio, pll5_video, pll6_mlb, pll7_usb_host, pll8_enet, ssi1_ipg,
- ssi2_ipg, ssi3_ipg, rom, usbphy1, usbphy2,
+ ssi2_ipg, ssi3_ipg, rom, usbphy1, usbphy2, ldb_di0_div_3_5, ldb_di1_div_3_5,
clk_max
};
@@ -288,8 +288,10 @@
clk[gpu3d_shader] = imx_clk_divider("gpu3d_shader", "gpu3d_shader_sel", base + 0x18, 29, 3);
clk[ipu1_podf] = imx_clk_divider("ipu1_podf", "ipu1_sel", base + 0x3c, 11, 3);
clk[ipu2_podf] = imx_clk_divider("ipu2_podf", "ipu2_sel", base + 0x3c, 16, 3);
- clk[ldb_di0_podf] = imx_clk_divider("ldb_di0_podf", "ldb_di0_sel", base + 0x20, 10, 1);
- clk[ldb_di1_podf] = imx_clk_divider("ldb_di1_podf", "ldb_di1_sel", base + 0x20, 11, 1);
+ clk[ldb_di0_div_3_5] = imx_clk_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
+ clk[ldb_di0_podf] = imx_clk_divider("ldb_di0_podf", "ldb_di0_div_3_5", base + 0x20, 10, 1);
+ clk[ldb_di1_div_3_5] = imx_clk_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
+ clk[ldb_di1_podf] = imx_clk_divider("ldb_di1_podf", "ldb_di1_div_3_5", base + 0x20, 11, 1);
clk[ipu1_di0_pre] = imx_clk_divider("ipu1_di0_pre", "ipu1_di0_pre_sel", base + 0x34, 3, 3);
clk[ipu1_di1_pre] = imx_clk_divider("ipu1_di1_pre", "ipu1_di1_pre_sel", base + 0x34, 12, 3);
clk[ipu2_di0_pre] = imx_clk_divider("ipu2_di0_pre", "ipu2_di0_pre_sel", base + 0x38, 3, 3);
diff --git a/arch/arm/mach-imx/head-v7.S b/arch/arm/mach-imx/headsmp.S
similarity index 100%
rename from arch/arm/mach-imx/head-v7.S
rename to arch/arm/mach-imx/headsmp.S
diff --git a/arch/arm/mach-imx/hotplug.c b/arch/arm/mach-imx/hotplug.c
index 20ed2d5..f8f7437 100644
--- a/arch/arm/mach-imx/hotplug.c
+++ b/arch/arm/mach-imx/hotplug.c
@@ -42,22 +42,6 @@
: "cc");
}
-static inline void cpu_leave_lowpower(void)
-{
- unsigned int v;
-
- asm volatile(
- "mrc p15, 0, %0, c1, c0, 0\n"
- " orr %0, %0, %1\n"
- " mcr p15, 0, %0, c1, c0, 0\n"
- " mrc p15, 0, %0, c1, c0, 1\n"
- " orr %0, %0, %2\n"
- " mcr p15, 0, %0, c1, c0, 1\n"
- : "=&r" (v)
- : "Ir" (CR_C), "Ir" (0x40)
- : "cc");
-}
-
/*
* platform-specific code to shutdown a CPU
*
@@ -67,11 +51,10 @@
{
cpu_enter_lowpower();
imx_enable_cpu(cpu, false);
- cpu_do_idle();
- cpu_leave_lowpower();
- /* We should never return from idle */
- panic("cpu %d unexpectedly exit from shutdown\n", cpu);
+ /* spin here until hardware takes it down */
+ while (1)
+ ;
}
int platform_cpu_disable(unsigned int cpu)
diff --git a/arch/arm/mach-imx/mach-armadillo5x0.c b/arch/arm/mach-imx/mach-armadillo5x0.c
index 2c6ab32..5985ed1 100644
--- a/arch/arm/mach-imx/mach-armadillo5x0.c
+++ b/arch/arm/mach-imx/mach-armadillo5x0.c
@@ -526,7 +526,8 @@
imx31_add_mxc_nand(&armadillo5x0_nand_board_info);
/* set NAND page size to 2k if not configured via boot mode pins */
- __raw_writel(__raw_readl(MXC_CCM_RCSR) | (1 << 30), MXC_CCM_RCSR);
+ __raw_writel(__raw_readl(mx3_ccm_base + MXC_CCM_RCSR) |
+ (1 << 30), mx3_ccm_base + MXC_CCM_RCSR);
/* RTC */
/* Get RTC IRQ and register the chip */
diff --git a/arch/arm/mach-imx/mach-imx6q.c b/arch/arm/mach-imx/mach-imx6q.c
index 5ec0608..045b3f6 100644
--- a/arch/arm/mach-imx/mach-imx6q.c
+++ b/arch/arm/mach-imx/mach-imx6q.c
@@ -71,7 +71,7 @@
/* For imx6q sabrelite board: set KSZ9021RN RGMII pad skew */
static int ksz9021rn_phy_fixup(struct phy_device *phydev)
{
- if (IS_ENABLED(CONFIG_PHYLIB)) {
+ if (IS_BUILTIN(CONFIG_PHYLIB)) {
/* min rx data delay */
phy_write(phydev, 0x0b, 0x8105);
phy_write(phydev, 0x0c, 0x0000);
@@ -112,7 +112,7 @@
static void __init imx6q_sabrelite_init(void)
{
- if (IS_ENABLED(CONFIG_PHYLIB))
+ if (IS_BUILTIN(CONFIG_PHYLIB))
phy_register_fixup_for_uid(PHY_ID_KSZ9021, MICREL_PHY_ID_MASK,
ksz9021rn_phy_fixup);
imx6q_sabrelite_cko1_setup();
diff --git a/arch/arm/mach-kirkwood/Makefile.boot b/arch/arm/mach-kirkwood/Makefile.boot
index a571755..a13299d 100644
--- a/arch/arm/mach-kirkwood/Makefile.boot
+++ b/arch/arm/mach-kirkwood/Makefile.boot
@@ -7,7 +7,8 @@
dtb-$(CONFIG_MACH_DLINK_KIRKWOOD_DT) += kirkwood-dns325.dtb
dtb-$(CONFIG_MACH_ICONNECT_DT) += kirkwood-iconnect.dtb
dtb-$(CONFIG_MACH_IB62X0_DT) += kirkwood-ib62x0.dtb
-dtb-$(CONFIG_MACH_TS219_DT) += kirkwood-qnap-ts219.dtb
+dtb-$(CONFIG_MACH_TS219_DT) += kirkwood-ts219-6281.dtb
+dtb-$(CONFIG_MACH_TS219_DT) += kirkwood-ts219-6282.dtb
dtb-$(CONFIG_MACH_GOFLEXNET_DT) += kirkwood-goflexnet.dtb
dtb-$(CONFIG_MACH_LSXL_DT) += kirkwood-lschlv2.dtb
dtb-$(CONFIG_MACH_LSXL_DT) += kirkwood-lsxhl.dtb
diff --git a/arch/arm/mach-kirkwood/common.c b/arch/arm/mach-kirkwood/common.c
index c4b64ad..1201191 100644
--- a/arch/arm/mach-kirkwood/common.c
+++ b/arch/arm/mach-kirkwood/common.c
@@ -301,7 +301,7 @@
{
orion_ge00_init(eth_data,
GE00_PHYS_BASE, IRQ_KIRKWOOD_GE00_SUM,
- IRQ_KIRKWOOD_GE00_ERR);
+ IRQ_KIRKWOOD_GE00_ERR, 1600);
/* The interface forgets the MAC address assigned by u-boot if
the clock is turned off, so claim the clk now. */
clk_prepare_enable(ge0);
@@ -315,7 +315,7 @@
{
orion_ge01_init(eth_data,
GE01_PHYS_BASE, IRQ_KIRKWOOD_GE01_SUM,
- IRQ_KIRKWOOD_GE01_ERR);
+ IRQ_KIRKWOOD_GE01_ERR, 1600);
clk_prepare_enable(ge1);
}
@@ -517,6 +517,13 @@
void __init kirkwood_init_early(void)
{
orion_time_set_base(TIMER_VIRT_BASE);
+
+ /*
+ * Some Kirkwood devices allocate their coherent buffers from atomic
+ * context. Increase size of atomic coherent pool to make sure such
+ * the allocations won't fail.
+ */
+ init_dma_coherent_pool_size(SZ_1M);
}
int kirkwood_tclk;
diff --git a/arch/arm/mach-kirkwood/db88f6281-bp-setup.c b/arch/arm/mach-kirkwood/db88f6281-bp-setup.c
index d933593..be90b7d 100644
--- a/arch/arm/mach-kirkwood/db88f6281-bp-setup.c
+++ b/arch/arm/mach-kirkwood/db88f6281-bp-setup.c
@@ -10,6 +10,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/sizes.h>
#include <linux/platform_device.h>
#include <linux/mtd/partitions.h>
#include <linux/ata_platform.h>
diff --git a/arch/arm/mach-mmp/Kconfig b/arch/arm/mach-mmp/Kconfig
index 7fddd01..d697d07 100644
--- a/arch/arm/mach-mmp/Kconfig
+++ b/arch/arm/mach-mmp/Kconfig
@@ -108,18 +108,21 @@
config CPU_PXA168
bool
select CPU_MOHAWK
+ select COMMON_CLK
help
Select code specific to PXA168
config CPU_PXA910
bool
select CPU_MOHAWK
+ select COMMON_CLK
help
Select code specific to PXA910
config CPU_MMP2
bool
select CPU_PJ4
+ select COMMON_CLK
help
Select code specific to MMP2. MMP2 is ARMv7 compatible.
diff --git a/arch/arm/mach-mmp/sram.c b/arch/arm/mach-mmp/sram.c
index 4304f95..7e8a5a2 100644
--- a/arch/arm/mach-mmp/sram.c
+++ b/arch/arm/mach-mmp/sram.c
@@ -68,7 +68,7 @@
struct resource *res;
int ret = 0;
- if (!pdata && !pdata->pool_name)
+ if (!pdata || !pdata->pool_name)
return -ENODEV;
info = kzalloc(sizeof(*info), GFP_KERNEL);
diff --git a/arch/arm/mach-mv78xx0/addr-map.c b/arch/arm/mach-mv78xx0/addr-map.c
index 62b53d7..a9bc841 100644
--- a/arch/arm/mach-mv78xx0/addr-map.c
+++ b/arch/arm/mach-mv78xx0/addr-map.c
@@ -37,7 +37,7 @@
#define WIN0_OFF(n) (BRIDGE_VIRT_BASE + 0x0000 + ((n) << 4))
#define WIN8_OFF(n) (BRIDGE_VIRT_BASE + 0x0900 + (((n) - 8) << 4))
-static void __init __iomem *win_cfg_base(int win)
+static void __init __iomem *win_cfg_base(const struct orion_addr_map_cfg *cfg, int win)
{
/*
* Find the control register base address for this window.
diff --git a/arch/arm/mach-mv78xx0/common.c b/arch/arm/mach-mv78xx0/common.c
index b4c53b8..3057f7d 100644
--- a/arch/arm/mach-mv78xx0/common.c
+++ b/arch/arm/mach-mv78xx0/common.c
@@ -213,7 +213,8 @@
{
orion_ge00_init(eth_data,
GE00_PHYS_BASE, IRQ_MV78XX0_GE00_SUM,
- IRQ_MV78XX0_GE_ERR);
+ IRQ_MV78XX0_GE_ERR,
+ MV643XX_TX_CSUM_DEFAULT_LIMIT);
}
@@ -224,7 +225,8 @@
{
orion_ge01_init(eth_data,
GE01_PHYS_BASE, IRQ_MV78XX0_GE01_SUM,
- NO_IRQ);
+ NO_IRQ,
+ MV643XX_TX_CSUM_DEFAULT_LIMIT);
}
diff --git a/arch/arm/mach-mxs/mach-mxs.c b/arch/arm/mach-mxs/mach-mxs.c
index 8dabfe8..ff886e0 100644
--- a/arch/arm/mach-mxs/mach-mxs.c
+++ b/arch/arm/mach-mxs/mach-mxs.c
@@ -261,7 +261,7 @@
enable_clk_enet_out();
if (IS_BUILTIN(CONFIG_PHYLIB))
- phy_register_fixup_for_uid(PHY_ID_KS8051, MICREL_PHY_ID_MASK,
+ phy_register_fixup_for_uid(PHY_ID_KSZ8051, MICREL_PHY_ID_MASK,
apx4devkit_phy_fixup);
mxsfb_pdata.mode_list = apx4devkit_video_modes;
diff --git a/arch/arm/mach-omap2/Kconfig b/arch/arm/mach-omap2/Kconfig
index dd2db02..346fd26 100644
--- a/arch/arm/mach-omap2/Kconfig
+++ b/arch/arm/mach-omap2/Kconfig
@@ -62,13 +62,14 @@
select PM_OPP if PM
select USB_ARCH_HAS_EHCI if USB_SUPPORT
select ARM_CPU_SUSPEND if PM
- select ARCH_NEEDS_CPU_IDLE_COUPLED
+ select ARCH_NEEDS_CPU_IDLE_COUPLED if SMP
config SOC_OMAP5
bool "TI OMAP5"
select CPU_V7
select ARM_GIC
select HAVE_SMP
+ select ARM_CPU_SUSPEND if PM
comment "OMAP Core Type"
depends on ARCH_OMAP2
@@ -231,10 +232,11 @@
select OMAP_PACKAGE_CBB
select REGULATOR_FIXED_VOLTAGE if REGULATOR
-config MACH_OMAP3_TOUCHBOOK
+config MACH_TOUCHBOOK
bool "OMAP3 Touch Book"
depends on ARCH_OMAP3
default y
+ select OMAP_PACKAGE_CBB
config MACH_OMAP_3430SDP
bool "OMAP 3430 SDP board"
diff --git a/arch/arm/mach-omap2/Makefile b/arch/arm/mach-omap2/Makefile
index f6a24b3..34c2c7f 100644
--- a/arch/arm/mach-omap2/Makefile
+++ b/arch/arm/mach-omap2/Makefile
@@ -255,7 +255,7 @@
obj-$(CONFIG_MACH_CM_T35) += board-cm-t35.o
obj-$(CONFIG_MACH_CM_T3517) += board-cm-t3517.o
obj-$(CONFIG_MACH_IGEP0020) += board-igep0020.o
-obj-$(CONFIG_MACH_OMAP3_TOUCHBOOK) += board-omap3touchbook.o
+obj-$(CONFIG_MACH_TOUCHBOOK) += board-omap3touchbook.o
obj-$(CONFIG_MACH_OMAP_4430SDP) += board-4430sdp.o
obj-$(CONFIG_MACH_OMAP4_PANDA) += board-omap4panda.o
diff --git a/arch/arm/mach-omap2/board-igep0020.c b/arch/arm/mach-omap2/board-igep0020.c
index 7491529..2821448 100644
--- a/arch/arm/mach-omap2/board-igep0020.c
+++ b/arch/arm/mach-omap2/board-igep0020.c
@@ -554,6 +554,8 @@
#ifdef CONFIG_OMAP_MUX
static struct omap_board_mux board_mux[] __initdata = {
+ /* SMSC9221 LAN Controller ETH IRQ (GPIO_176) */
+ OMAP3_MUX(MCSPI1_CS2, OMAP_MUX_MODE4 | OMAP_PIN_INPUT),
{ .reg_offset = OMAP_MUX_TERMINATOR },
};
#endif
diff --git a/arch/arm/mach-omap2/board-omap3evm.c b/arch/arm/mach-omap2/board-omap3evm.c
index ef230a0..0d362e9 100644
--- a/arch/arm/mach-omap2/board-omap3evm.c
+++ b/arch/arm/mach-omap2/board-omap3evm.c
@@ -58,6 +58,7 @@
#include "hsmmc.h"
#include "common-board-devices.h"
+#define OMAP3_EVM_TS_GPIO 175
#define OMAP3_EVM_EHCI_VBUS 22
#define OMAP3_EVM_EHCI_SELECT 61
diff --git a/arch/arm/mach-omap2/clock33xx_data.c b/arch/arm/mach-omap2/clock33xx_data.c
index 25bbcc7..ae27de8 100644
--- a/arch/arm/mach-omap2/clock33xx_data.c
+++ b/arch/arm/mach-omap2/clock33xx_data.c
@@ -1036,13 +1036,13 @@
CLK(NULL, "mmu_fck", &mmu_fck, CK_AM33XX),
CLK(NULL, "smartreflex0_fck", &smartreflex0_fck, CK_AM33XX),
CLK(NULL, "smartreflex1_fck", &smartreflex1_fck, CK_AM33XX),
- CLK(NULL, "gpt1_fck", &timer1_fck, CK_AM33XX),
- CLK(NULL, "gpt2_fck", &timer2_fck, CK_AM33XX),
- CLK(NULL, "gpt3_fck", &timer3_fck, CK_AM33XX),
- CLK(NULL, "gpt4_fck", &timer4_fck, CK_AM33XX),
- CLK(NULL, "gpt5_fck", &timer5_fck, CK_AM33XX),
- CLK(NULL, "gpt6_fck", &timer6_fck, CK_AM33XX),
- CLK(NULL, "gpt7_fck", &timer7_fck, CK_AM33XX),
+ CLK(NULL, "timer1_fck", &timer1_fck, CK_AM33XX),
+ CLK(NULL, "timer2_fck", &timer2_fck, CK_AM33XX),
+ CLK(NULL, "timer3_fck", &timer3_fck, CK_AM33XX),
+ CLK(NULL, "timer4_fck", &timer4_fck, CK_AM33XX),
+ CLK(NULL, "timer5_fck", &timer5_fck, CK_AM33XX),
+ CLK(NULL, "timer6_fck", &timer6_fck, CK_AM33XX),
+ CLK(NULL, "timer7_fck", &timer7_fck, CK_AM33XX),
CLK(NULL, "usbotg_fck", &usbotg_fck, CK_AM33XX),
CLK(NULL, "ieee5000_fck", &ieee5000_fck, CK_AM33XX),
CLK(NULL, "wdt1_fck", &wdt1_fck, CK_AM33XX),
diff --git a/arch/arm/mach-omap2/clockdomain2xxx_3xxx.c b/arch/arm/mach-omap2/clockdomain2xxx_3xxx.c
index a0d68db..f99e65c 100644
--- a/arch/arm/mach-omap2/clockdomain2xxx_3xxx.c
+++ b/arch/arm/mach-omap2/clockdomain2xxx_3xxx.c
@@ -241,6 +241,52 @@
_clkdm_del_autodeps(clkdm);
}
+static int omap3xxx_clkdm_clk_enable(struct clockdomain *clkdm)
+{
+ bool hwsup = false;
+
+ if (!clkdm->clktrctrl_mask)
+ return 0;
+
+ hwsup = omap2_cm_is_clkdm_in_hwsup(clkdm->pwrdm.ptr->prcm_offs,
+ clkdm->clktrctrl_mask);
+
+ if (hwsup) {
+ /* Disable HW transitions when we are changing deps */
+ _disable_hwsup(clkdm);
+ _clkdm_add_autodeps(clkdm);
+ _enable_hwsup(clkdm);
+ } else {
+ if (clkdm->flags & CLKDM_CAN_FORCE_WAKEUP)
+ omap3_clkdm_wakeup(clkdm);
+ }
+
+ return 0;
+}
+
+static int omap3xxx_clkdm_clk_disable(struct clockdomain *clkdm)
+{
+ bool hwsup = false;
+
+ if (!clkdm->clktrctrl_mask)
+ return 0;
+
+ hwsup = omap2_cm_is_clkdm_in_hwsup(clkdm->pwrdm.ptr->prcm_offs,
+ clkdm->clktrctrl_mask);
+
+ if (hwsup) {
+ /* Disable HW transitions when we are changing deps */
+ _disable_hwsup(clkdm);
+ _clkdm_del_autodeps(clkdm);
+ _enable_hwsup(clkdm);
+ } else {
+ if (clkdm->flags & CLKDM_CAN_FORCE_SLEEP)
+ omap3_clkdm_sleep(clkdm);
+ }
+
+ return 0;
+}
+
struct clkdm_ops omap2_clkdm_operations = {
.clkdm_add_wkdep = omap2_clkdm_add_wkdep,
.clkdm_del_wkdep = omap2_clkdm_del_wkdep,
@@ -267,6 +313,6 @@
.clkdm_wakeup = omap3_clkdm_wakeup,
.clkdm_allow_idle = omap3_clkdm_allow_idle,
.clkdm_deny_idle = omap3_clkdm_deny_idle,
- .clkdm_clk_enable = omap2_clkdm_clk_enable,
- .clkdm_clk_disable = omap2_clkdm_clk_disable,
+ .clkdm_clk_enable = omap3xxx_clkdm_clk_enable,
+ .clkdm_clk_disable = omap3xxx_clkdm_clk_disable,
};
diff --git a/arch/arm/mach-omap2/cm-regbits-34xx.h b/arch/arm/mach-omap2/cm-regbits-34xx.h
index 766338f..975f6bd 100644
--- a/arch/arm/mach-omap2/cm-regbits-34xx.h
+++ b/arch/arm/mach-omap2/cm-regbits-34xx.h
@@ -67,6 +67,7 @@
#define OMAP3430_EN_IVA2_DPLL_MASK (0x7 << 0)
/* CM_IDLEST_IVA2 */
+#define OMAP3430_ST_IVA2_SHIFT 0
#define OMAP3430_ST_IVA2_MASK (1 << 0)
/* CM_IDLEST_PLL_IVA2 */
diff --git a/arch/arm/mach-omap2/common-board-devices.c b/arch/arm/mach-omap2/common-board-devices.c
index 1473474..c187586 100644
--- a/arch/arm/mach-omap2/common-board-devices.c
+++ b/arch/arm/mach-omap2/common-board-devices.c
@@ -35,16 +35,6 @@
.turbo_mode = 0,
};
-/*
- * ADS7846 driver maybe request a gpio according to the value
- * of pdata->get_pendown_state, but we have done this. So set
- * get_pendown_state to avoid twice gpio requesting.
- */
-static int omap3_get_pendown_state(void)
-{
- return !gpio_get_value(OMAP3_EVM_TS_GPIO);
-}
-
static struct ads7846_platform_data ads7846_config = {
.x_max = 0x0fff,
.y_max = 0x0fff,
@@ -55,7 +45,6 @@
.debounce_rep = 1,
.gpio_pendown = -EINVAL,
.keep_vref_on = 1,
- .get_pendown_state = &omap3_get_pendown_state,
};
static struct spi_board_info ads7846_spi_board_info __initdata = {
diff --git a/arch/arm/mach-omap2/common-board-devices.h b/arch/arm/mach-omap2/common-board-devices.h
index 4c4ef6a..a0b4a428 100644
--- a/arch/arm/mach-omap2/common-board-devices.h
+++ b/arch/arm/mach-omap2/common-board-devices.h
@@ -4,7 +4,6 @@
#include "twl-common.h"
#define NAND_BLOCK_SIZE SZ_128K
-#define OMAP3_EVM_TS_GPIO 175
struct mtd_partition;
struct ads7846_platform_data;
diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index ee05e19..288bee6 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -238,8 +238,9 @@
for_each_cpu(cpu_id, cpu_online_mask) {
dev = &per_cpu(omap4_idle_dev, cpu_id);
dev->cpu = cpu_id;
+#ifdef CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED
dev->coupled_cpus = *cpu_online_mask;
-
+#endif
cpuidle_register_driver(&omap4_idle_driver);
if (cpuidle_register_device(dev)) {
diff --git a/arch/arm/mach-omap2/mux.h b/arch/arm/mach-omap2/mux.h
index 471e62a7..76f9b3c 100644
--- a/arch/arm/mach-omap2/mux.h
+++ b/arch/arm/mach-omap2/mux.h
@@ -127,7 +127,6 @@
* @gpio: GPIO number
* @muxnames: available signal modes for a ball
* @balls: available balls on the package
- * @partition: mux partition
*/
struct omap_mux {
u16 reg_offset;
diff --git a/arch/arm/mach-omap2/omap-wakeupgen.c b/arch/arm/mach-omap2/omap-wakeupgen.c
index 05fdebf..330d4c6 100644
--- a/arch/arm/mach-omap2/omap-wakeupgen.c
+++ b/arch/arm/mach-omap2/omap-wakeupgen.c
@@ -46,7 +46,7 @@
static void __iomem *wakeupgen_base;
static void __iomem *sar_base;
static DEFINE_SPINLOCK(wakeupgen_lock);
-static unsigned int irq_target_cpu[NR_IRQS];
+static unsigned int irq_target_cpu[MAX_IRQS];
static unsigned int irq_banks = MAX_NR_REG_BANKS;
static unsigned int max_irqs = MAX_IRQS;
static unsigned int omap_secure_apis;
diff --git a/arch/arm/mach-omap2/omap_hwmod.c b/arch/arm/mach-omap2/omap_hwmod.c
index 6ca8e51..37afbd1 100644
--- a/arch/arm/mach-omap2/omap_hwmod.c
+++ b/arch/arm/mach-omap2/omap_hwmod.c
@@ -1889,6 +1889,7 @@
_enable_sysc(oh);
}
} else {
+ _omap4_disable_module(oh);
_disable_clocks(oh);
pr_debug("omap_hwmod: %s: _wait_target_ready: %d\n",
oh->name, r);
diff --git a/arch/arm/mach-omap2/omap_hwmod_3xxx_data.c b/arch/arm/mach-omap2/omap_hwmod_3xxx_data.c
index c9e3820..ce7e606 100644
--- a/arch/arm/mach-omap2/omap_hwmod_3xxx_data.c
+++ b/arch/arm/mach-omap2/omap_hwmod_3xxx_data.c
@@ -100,9 +100,9 @@
/* IVA2 (IVA2) */
static struct omap_hwmod_rst_info omap3xxx_iva_resets[] = {
- { .name = "logic", .rst_shift = 0 },
- { .name = "seq0", .rst_shift = 1 },
- { .name = "seq1", .rst_shift = 2 },
+ { .name = "logic", .rst_shift = 0, .st_shift = 8 },
+ { .name = "seq0", .rst_shift = 1, .st_shift = 9 },
+ { .name = "seq1", .rst_shift = 2, .st_shift = 10 },
};
static struct omap_hwmod omap3xxx_iva_hwmod = {
@@ -112,6 +112,15 @@
.rst_lines = omap3xxx_iva_resets,
.rst_lines_cnt = ARRAY_SIZE(omap3xxx_iva_resets),
.main_clk = "iva2_ck",
+ .prcm = {
+ .omap2 = {
+ .module_offs = OMAP3430_IVA2_MOD,
+ .prcm_reg_id = 1,
+ .module_bit = OMAP3430_CM_FCLKEN_IVA2_EN_IVA2_SHIFT,
+ .idlest_reg_id = 1,
+ .idlest_idle_bit = OMAP3430_ST_IVA2_SHIFT,
+ }
+ },
};
/* timer class */
diff --git a/arch/arm/mach-omap2/omap_hwmod_44xx_data.c b/arch/arm/mach-omap2/omap_hwmod_44xx_data.c
index 242aee4..afb6091 100644
--- a/arch/arm/mach-omap2/omap_hwmod_44xx_data.c
+++ b/arch/arm/mach-omap2/omap_hwmod_44xx_data.c
@@ -4210,7 +4210,7 @@
};
/* dsp -> sl2if */
-static struct omap_hwmod_ocp_if omap44xx_dsp__sl2if = {
+static struct omap_hwmod_ocp_if __maybe_unused omap44xx_dsp__sl2if = {
.master = &omap44xx_dsp_hwmod,
.slave = &omap44xx_sl2if_hwmod,
.clk = "dpll_iva_m5x2_ck",
@@ -4828,7 +4828,7 @@
};
/* iva -> sl2if */
-static struct omap_hwmod_ocp_if omap44xx_iva__sl2if = {
+static struct omap_hwmod_ocp_if __maybe_unused omap44xx_iva__sl2if = {
.master = &omap44xx_iva_hwmod,
.slave = &omap44xx_sl2if_hwmod,
.clk = "dpll_iva_m5x2_ck",
@@ -5362,7 +5362,7 @@
};
/* l3_main_2 -> sl2if */
-static struct omap_hwmod_ocp_if omap44xx_l3_main_2__sl2if = {
+static struct omap_hwmod_ocp_if __maybe_unused omap44xx_l3_main_2__sl2if = {
.master = &omap44xx_l3_main_2_hwmod,
.slave = &omap44xx_sl2if_hwmod,
.clk = "l3_div_ck",
@@ -6032,7 +6032,7 @@
&omap44xx_l4_abe__dmic,
&omap44xx_l4_abe__dmic_dma,
&omap44xx_dsp__iva,
- &omap44xx_dsp__sl2if,
+ /* &omap44xx_dsp__sl2if, */
&omap44xx_l4_cfg__dsp,
&omap44xx_l3_main_2__dss,
&omap44xx_l4_per__dss,
@@ -6068,7 +6068,7 @@
&omap44xx_l4_per__i2c4,
&omap44xx_l3_main_2__ipu,
&omap44xx_l3_main_2__iss,
- &omap44xx_iva__sl2if,
+ /* &omap44xx_iva__sl2if, */
&omap44xx_l3_main_2__iva,
&omap44xx_l4_wkup__kbd,
&omap44xx_l4_cfg__mailbox,
@@ -6099,7 +6099,7 @@
&omap44xx_l4_cfg__cm_core,
&omap44xx_l4_wkup__prm,
&omap44xx_l4_wkup__scrm,
- &omap44xx_l3_main_2__sl2if,
+ /* &omap44xx_l3_main_2__sl2if, */
&omap44xx_l4_abe__slimbus1,
&omap44xx_l4_abe__slimbus1_dma,
&omap44xx_l4_per__slimbus2,
diff --git a/arch/arm/mach-omap2/opp4xxx_data.c b/arch/arm/mach-omap2/opp4xxx_data.c
index 2293ba2..c95415d 100644
--- a/arch/arm/mach-omap2/opp4xxx_data.c
+++ b/arch/arm/mach-omap2/opp4xxx_data.c
@@ -94,7 +94,7 @@
{
int r = -ENODEV;
- if (!cpu_is_omap44xx())
+ if (!cpu_is_omap443x())
return r;
r = omap_init_opp_table(omap44xx_opp_def_list,
diff --git a/arch/arm/mach-omap2/pm34xx.c b/arch/arm/mach-omap2/pm34xx.c
index e4fc88c..05bd8f0 100644
--- a/arch/arm/mach-omap2/pm34xx.c
+++ b/arch/arm/mach-omap2/pm34xx.c
@@ -272,21 +272,16 @@
per_next_state = pwrdm_read_next_pwrst(per_pwrdm);
core_next_state = pwrdm_read_next_pwrst(core_pwrdm);
- if (mpu_next_state < PWRDM_POWER_ON) {
- pwrdm_pre_transition(mpu_pwrdm);
- pwrdm_pre_transition(neon_pwrdm);
- }
+ pwrdm_pre_transition(NULL);
/* PER */
if (per_next_state < PWRDM_POWER_ON) {
- pwrdm_pre_transition(per_pwrdm);
per_going_off = (per_next_state == PWRDM_POWER_OFF) ? 1 : 0;
omap2_gpio_prepare_for_idle(per_going_off);
}
/* CORE */
if (core_next_state < PWRDM_POWER_ON) {
- pwrdm_pre_transition(core_pwrdm);
if (core_next_state == PWRDM_POWER_OFF) {
omap3_core_save_context();
omap3_cm_save_context();
@@ -339,20 +334,14 @@
omap2_prm_clear_mod_reg_bits(OMAP3430_AUTO_OFF_MASK,
OMAP3430_GR_MOD,
OMAP3_PRM_VOLTCTRL_OFFSET);
- pwrdm_post_transition(core_pwrdm);
}
omap3_intc_resume_idle();
- /* PER */
- if (per_next_state < PWRDM_POWER_ON) {
- omap2_gpio_resume_after_idle();
- pwrdm_post_transition(per_pwrdm);
- }
+ pwrdm_post_transition(NULL);
- if (mpu_next_state < PWRDM_POWER_ON) {
- pwrdm_post_transition(mpu_pwrdm);
- pwrdm_post_transition(neon_pwrdm);
- }
+ /* PER */
+ if (per_next_state < PWRDM_POWER_ON)
+ omap2_gpio_resume_after_idle();
}
static void omap3_pm_idle(void)
diff --git a/arch/arm/mach-omap2/sleep44xx.S b/arch/arm/mach-omap2/sleep44xx.S
index 9f6b83d..91e71d8 100644
--- a/arch/arm/mach-omap2/sleep44xx.S
+++ b/arch/arm/mach-omap2/sleep44xx.S
@@ -56,9 +56,13 @@
* The restore function pointer is stored at CPUx_WAKEUP_NS_PA_ADDR_OFFSET.
* It returns to the caller for CPU INACTIVE and ON power states or in case
* CPU failed to transition to targeted OFF/DORMANT state.
+ *
+ * omap4_finish_suspend() calls v7_flush_dcache_all() which doesn't save
+ * stack frame and it expects the caller to take care of it. Hence the entire
+ * stack frame is saved to avoid possible stack corruption.
*/
ENTRY(omap4_finish_suspend)
- stmfd sp!, {lr}
+ stmfd sp!, {r4-r12, lr}
cmp r0, #0x0
beq do_WFI @ No lowpower state, jump to WFI
@@ -226,7 +230,7 @@
skip_scu_gp_clear:
isb
dsb
- ldmfd sp!, {pc}
+ ldmfd sp!, {r4-r12, pc}
ENDPROC(omap4_finish_suspend)
/*
diff --git a/arch/arm/mach-omap2/timer.c b/arch/arm/mach-omap2/timer.c
index 2ff6d41..2ba4f57 100644
--- a/arch/arm/mach-omap2/timer.c
+++ b/arch/arm/mach-omap2/timer.c
@@ -260,6 +260,7 @@
return 0;
}
+#ifdef CONFIG_OMAP_32K_TIMER
/* Setup free-running counter for clocksource */
static int __init omap2_sync32k_clocksource_init(void)
{
@@ -299,6 +300,12 @@
return ret;
}
+#else
+static inline int omap2_sync32k_clocksource_init(void)
+{
+ return -ENODEV;
+}
+#endif
static void __init omap2_gptimer_clocksource_init(int gptimer_id,
const char *fck_source)
diff --git a/arch/arm/mach-omap2/twl-common.c b/arch/arm/mach-omap2/twl-common.c
index de47f17..db5ff66 100644
--- a/arch/arm/mach-omap2/twl-common.c
+++ b/arch/arm/mach-omap2/twl-common.c
@@ -67,6 +67,7 @@
const char *pmic_type, int pmic_irq,
struct twl4030_platform_data *pmic_data)
{
+ omap_mux_init_signal("sys_nirq", OMAP_PIN_INPUT_PULLUP | OMAP_PIN_OFF_WAKEUPENABLE);
strncpy(pmic_i2c_board_info.type, pmic_type,
sizeof(pmic_i2c_board_info.type));
pmic_i2c_board_info.irq = pmic_irq;
diff --git a/arch/arm/mach-orion5x/common.c b/arch/arm/mach-orion5x/common.c
index 9148b22..a6cd14a 100644
--- a/arch/arm/mach-orion5x/common.c
+++ b/arch/arm/mach-orion5x/common.c
@@ -109,7 +109,8 @@
{
orion_ge00_init(eth_data,
ORION5X_ETH_PHYS_BASE, IRQ_ORION5X_ETH_SUM,
- IRQ_ORION5X_ETH_ERR);
+ IRQ_ORION5X_ETH_ERR,
+ MV643XX_TX_CSUM_DEFAULT_LIMIT);
}
@@ -203,6 +204,13 @@
void __init orion5x_init_early(void)
{
orion_time_set_base(TIMER_VIRT_BASE);
+
+ /*
+ * Some Orion5x devices allocate their coherent buffers from atomic
+ * context. Increase size of atomic coherent pool to make sure such
+ * the allocations won't fail.
+ */
+ init_dma_coherent_pool_size(SZ_1M);
}
int orion5x_tclk;
diff --git a/arch/arm/mach-prima2/Makefile b/arch/arm/mach-prima2/Makefile
index 13dd160..841847d 100644
--- a/arch/arm/mach-prima2/Makefile
+++ b/arch/arm/mach-prima2/Makefile
@@ -1,6 +1,5 @@
obj-y := timer.o
obj-y += irq.o
-obj-y += clock.o
obj-y += rstc.o
obj-y += prima2.o
obj-y += rtciobrg.o
diff --git a/arch/arm/mach-prima2/clock.c b/arch/arm/mach-prima2/clock.c
deleted file mode 100644
index aebad7e..0000000
--- a/arch/arm/mach-prima2/clock.c
+++ /dev/null
@@ -1,510 +0,0 @@
-/*
- * Clock tree for CSR SiRFprimaII
- *
- * Copyright (c) 2011 Cambridge Silicon Radio Limited, a CSR plc group company.
- *
- * Licensed under GPLv2 or later.
- */
-
-#include <linux/module.h>
-#include <linux/bitops.h>
-#include <linux/err.h>
-#include <linux/errno.h>
-#include <linux/io.h>
-#include <linux/clkdev.h>
-#include <linux/clk.h>
-#include <linux/spinlock.h>
-#include <linux/of.h>
-#include <linux/of_address.h>
-#include <asm/mach/map.h>
-#include <mach/map.h>
-
-#define SIRFSOC_CLKC_CLK_EN0 0x0000
-#define SIRFSOC_CLKC_CLK_EN1 0x0004
-#define SIRFSOC_CLKC_REF_CFG 0x0014
-#define SIRFSOC_CLKC_CPU_CFG 0x0018
-#define SIRFSOC_CLKC_MEM_CFG 0x001c
-#define SIRFSOC_CLKC_SYS_CFG 0x0020
-#define SIRFSOC_CLKC_IO_CFG 0x0024
-#define SIRFSOC_CLKC_DSP_CFG 0x0028
-#define SIRFSOC_CLKC_GFX_CFG 0x002c
-#define SIRFSOC_CLKC_MM_CFG 0x0030
-#define SIRFSOC_LKC_LCD_CFG 0x0034
-#define SIRFSOC_CLKC_MMC_CFG 0x0038
-#define SIRFSOC_CLKC_PLL1_CFG0 0x0040
-#define SIRFSOC_CLKC_PLL2_CFG0 0x0044
-#define SIRFSOC_CLKC_PLL3_CFG0 0x0048
-#define SIRFSOC_CLKC_PLL1_CFG1 0x004c
-#define SIRFSOC_CLKC_PLL2_CFG1 0x0050
-#define SIRFSOC_CLKC_PLL3_CFG1 0x0054
-#define SIRFSOC_CLKC_PLL1_CFG2 0x0058
-#define SIRFSOC_CLKC_PLL2_CFG2 0x005c
-#define SIRFSOC_CLKC_PLL3_CFG2 0x0060
-
-#define SIRFSOC_CLOCK_VA_BASE SIRFSOC_VA(0x005000)
-
-#define KHZ 1000
-#define MHZ (KHZ * KHZ)
-
-struct clk_ops {
- unsigned long (*get_rate)(struct clk *clk);
- long (*round_rate)(struct clk *clk, unsigned long rate);
- int (*set_rate)(struct clk *clk, unsigned long rate);
- int (*enable)(struct clk *clk);
- int (*disable)(struct clk *clk);
- struct clk *(*get_parent)(struct clk *clk);
- int (*set_parent)(struct clk *clk, struct clk *parent);
-};
-
-struct clk {
- struct clk *parent; /* parent clk */
- unsigned long rate; /* clock rate in Hz */
- signed char usage; /* clock enable count */
- signed char enable_bit; /* enable bit: 0 ~ 63 */
- unsigned short regofs; /* register offset */
- struct clk_ops *ops; /* clock operation */
-};
-
-static DEFINE_SPINLOCK(clocks_lock);
-
-static inline unsigned long clkc_readl(unsigned reg)
-{
- return readl(SIRFSOC_CLOCK_VA_BASE + reg);
-}
-
-static inline void clkc_writel(u32 val, unsigned reg)
-{
- writel(val, SIRFSOC_CLOCK_VA_BASE + reg);
-}
-
-/*
- * osc_rtc - real time oscillator - 32.768KHz
- * osc_sys - high speed oscillator - 26MHz
- */
-
-static struct clk clk_rtc = {
- .rate = 32768,
-};
-
-static struct clk clk_osc = {
- .rate = 26 * MHZ,
-};
-
-/*
- * std pll
- */
-static unsigned long std_pll_get_rate(struct clk *clk)
-{
- unsigned long fin = clk_get_rate(clk->parent);
- u32 regcfg2 = clk->regofs + SIRFSOC_CLKC_PLL1_CFG2 -
- SIRFSOC_CLKC_PLL1_CFG0;
-
- if (clkc_readl(regcfg2) & BIT(2)) {
- /* pll bypass mode */
- clk->rate = fin;
- } else {
- /* fout = fin * nf / nr / od */
- u32 cfg0 = clkc_readl(clk->regofs);
- u32 nf = (cfg0 & (BIT(13) - 1)) + 1;
- u32 nr = ((cfg0 >> 13) & (BIT(6) - 1)) + 1;
- u32 od = ((cfg0 >> 19) & (BIT(4) - 1)) + 1;
- WARN_ON(fin % MHZ);
- clk->rate = fin / MHZ * nf / nr / od * MHZ;
- }
-
- return clk->rate;
-}
-
-static int std_pll_set_rate(struct clk *clk, unsigned long rate)
-{
- unsigned long fin, nf, nr, od, reg;
-
- /*
- * fout = fin * nf / (nr * od);
- * set od = 1, nr = fin/MHz, so fout = nf * MHz
- */
-
- nf = rate / MHZ;
- if (unlikely((rate % MHZ) || nf > BIT(13) || nf < 1))
- return -EINVAL;
-
- fin = clk_get_rate(clk->parent);
- BUG_ON(fin < MHZ);
-
- nr = fin / MHZ;
- BUG_ON((fin % MHZ) || nr > BIT(6));
-
- od = 1;
-
- reg = (nf - 1) | ((nr - 1) << 13) | ((od - 1) << 19);
- clkc_writel(reg, clk->regofs);
-
- reg = clk->regofs + SIRFSOC_CLKC_PLL1_CFG1 - SIRFSOC_CLKC_PLL1_CFG0;
- clkc_writel((nf >> 1) - 1, reg);
-
- reg = clk->regofs + SIRFSOC_CLKC_PLL1_CFG2 - SIRFSOC_CLKC_PLL1_CFG0;
- while (!(clkc_readl(reg) & BIT(6)))
- cpu_relax();
-
- clk->rate = 0; /* set to zero will force recalculation */
- return 0;
-}
-
-static struct clk_ops std_pll_ops = {
- .get_rate = std_pll_get_rate,
- .set_rate = std_pll_set_rate,
-};
-
-static struct clk clk_pll1 = {
- .parent = &clk_osc,
- .regofs = SIRFSOC_CLKC_PLL1_CFG0,
- .ops = &std_pll_ops,
-};
-
-static struct clk clk_pll2 = {
- .parent = &clk_osc,
- .regofs = SIRFSOC_CLKC_PLL2_CFG0,
- .ops = &std_pll_ops,
-};
-
-static struct clk clk_pll3 = {
- .parent = &clk_osc,
- .regofs = SIRFSOC_CLKC_PLL3_CFG0,
- .ops = &std_pll_ops,
-};
-
-/*
- * clock domains - cpu, mem, sys/io
- */
-
-static struct clk clk_mem;
-
-static struct clk *dmn_get_parent(struct clk *clk)
-{
- struct clk *clks[] = {
- &clk_osc, &clk_rtc, &clk_pll1, &clk_pll2, &clk_pll3
- };
- u32 cfg = clkc_readl(clk->regofs);
- WARN_ON((cfg & (BIT(3) - 1)) > 4);
- return clks[cfg & (BIT(3) - 1)];
-}
-
-static int dmn_set_parent(struct clk *clk, struct clk *parent)
-{
- const struct clk *clks[] = {
- &clk_osc, &clk_rtc, &clk_pll1, &clk_pll2, &clk_pll3
- };
- u32 cfg = clkc_readl(clk->regofs);
- int i;
- for (i = 0; i < ARRAY_SIZE(clks); i++) {
- if (clks[i] == parent) {
- cfg &= ~(BIT(3) - 1);
- clkc_writel(cfg | i, clk->regofs);
- /* BIT(3) - switching status: 1 - busy, 0 - done */
- while (clkc_readl(clk->regofs) & BIT(3))
- cpu_relax();
- return 0;
- }
- }
- return -EINVAL;
-}
-
-static unsigned long dmn_get_rate(struct clk *clk)
-{
- unsigned long fin = clk_get_rate(clk->parent);
- u32 cfg = clkc_readl(clk->regofs);
- if (cfg & BIT(24)) {
- /* fcd bypass mode */
- clk->rate = fin;
- } else {
- /*
- * wait count: bit[19:16], hold count: bit[23:20]
- */
- u32 wait = (cfg >> 16) & (BIT(4) - 1);
- u32 hold = (cfg >> 20) & (BIT(4) - 1);
-
- clk->rate = fin / (wait + hold + 2);
- }
-
- return clk->rate;
-}
-
-static int dmn_set_rate(struct clk *clk, unsigned long rate)
-{
- unsigned long fin;
- unsigned ratio, wait, hold, reg;
- unsigned bits = (clk == &clk_mem) ? 3 : 4;
-
- fin = clk_get_rate(clk->parent);
- ratio = fin / rate;
-
- if (unlikely(ratio < 2 || ratio > BIT(bits + 1)))
- return -EINVAL;
-
- WARN_ON(fin % rate);
-
- wait = (ratio >> 1) - 1;
- hold = ratio - wait - 2;
-
- reg = clkc_readl(clk->regofs);
- reg &= ~(((BIT(bits) - 1) << 16) | ((BIT(bits) - 1) << 20));
- reg |= (wait << 16) | (hold << 20) | BIT(25);
- clkc_writel(reg, clk->regofs);
-
- /* waiting FCD been effective */
- while (clkc_readl(clk->regofs) & BIT(25))
- cpu_relax();
-
- clk->rate = 0; /* set to zero will force recalculation */
-
- return 0;
-}
-
-/*
- * cpu clock has no FCD register in Prima2, can only change pll
- */
-static int cpu_set_rate(struct clk *clk, unsigned long rate)
-{
- int ret1, ret2;
- struct clk *cur_parent, *tmp_parent;
-
- cur_parent = dmn_get_parent(clk);
- BUG_ON(cur_parent == NULL || cur_parent->usage > 1);
-
- /* switch to tmp pll before setting parent clock's rate */
- tmp_parent = cur_parent == &clk_pll1 ? &clk_pll2 : &clk_pll1;
- ret1 = dmn_set_parent(clk, tmp_parent);
- BUG_ON(ret1);
-
- ret2 = clk_set_rate(cur_parent, rate);
-
- ret1 = dmn_set_parent(clk, cur_parent);
-
- clk->rate = 0; /* set to zero will force recalculation */
-
- return ret2 ? ret2 : ret1;
-}
-
-static struct clk_ops cpu_ops = {
- .get_parent = dmn_get_parent,
- .set_parent = dmn_set_parent,
- .set_rate = cpu_set_rate,
-};
-
-static struct clk clk_cpu = {
- .parent = &clk_pll1,
- .regofs = SIRFSOC_CLKC_CPU_CFG,
- .ops = &cpu_ops,
-};
-
-
-static struct clk_ops msi_ops = {
- .set_rate = dmn_set_rate,
- .get_rate = dmn_get_rate,
- .set_parent = dmn_set_parent,
- .get_parent = dmn_get_parent,
-};
-
-static struct clk clk_mem = {
- .parent = &clk_pll2,
- .regofs = SIRFSOC_CLKC_MEM_CFG,
- .ops = &msi_ops,
-};
-
-static struct clk clk_sys = {
- .parent = &clk_pll3,
- .regofs = SIRFSOC_CLKC_SYS_CFG,
- .ops = &msi_ops,
-};
-
-static struct clk clk_io = {
- .parent = &clk_pll3,
- .regofs = SIRFSOC_CLKC_IO_CFG,
- .ops = &msi_ops,
-};
-
-/*
- * on-chip clock sets
- */
-static struct clk_lookup onchip_clks[] = {
- {
- .dev_id = "rtc",
- .clk = &clk_rtc,
- }, {
- .dev_id = "osc",
- .clk = &clk_osc,
- }, {
- .dev_id = "pll1",
- .clk = &clk_pll1,
- }, {
- .dev_id = "pll2",
- .clk = &clk_pll2,
- }, {
- .dev_id = "pll3",
- .clk = &clk_pll3,
- }, {
- .dev_id = "cpu",
- .clk = &clk_cpu,
- }, {
- .dev_id = "mem",
- .clk = &clk_mem,
- }, {
- .dev_id = "sys",
- .clk = &clk_sys,
- }, {
- .dev_id = "io",
- .clk = &clk_io,
- },
-};
-
-int clk_enable(struct clk *clk)
-{
- unsigned long flags;
-
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return -EINVAL;
-
- if (clk->parent)
- clk_enable(clk->parent);
-
- spin_lock_irqsave(&clocks_lock, flags);
- if (!clk->usage++ && clk->ops && clk->ops->enable)
- clk->ops->enable(clk);
- spin_unlock_irqrestore(&clocks_lock, flags);
- return 0;
-}
-EXPORT_SYMBOL(clk_enable);
-
-void clk_disable(struct clk *clk)
-{
- unsigned long flags;
-
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return;
-
- WARN_ON(!clk->usage);
-
- spin_lock_irqsave(&clocks_lock, flags);
- if (--clk->usage == 0 && clk->ops && clk->ops->disable)
- clk->ops->disable(clk);
- spin_unlock_irqrestore(&clocks_lock, flags);
-
- if (clk->parent)
- clk_disable(clk->parent);
-}
-EXPORT_SYMBOL(clk_disable);
-
-unsigned long clk_get_rate(struct clk *clk)
-{
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return 0;
-
- if (clk->rate)
- return clk->rate;
-
- if (clk->ops && clk->ops->get_rate)
- return clk->ops->get_rate(clk);
-
- return clk_get_rate(clk->parent);
-}
-EXPORT_SYMBOL(clk_get_rate);
-
-long clk_round_rate(struct clk *clk, unsigned long rate)
-{
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return 0;
-
- if (clk->ops && clk->ops->round_rate)
- return clk->ops->round_rate(clk, rate);
-
- return 0;
-}
-EXPORT_SYMBOL(clk_round_rate);
-
-int clk_set_rate(struct clk *clk, unsigned long rate)
-{
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return -EINVAL;
-
- if (!clk->ops || !clk->ops->set_rate)
- return -EINVAL;
-
- return clk->ops->set_rate(clk, rate);
-}
-EXPORT_SYMBOL(clk_set_rate);
-
-int clk_set_parent(struct clk *clk, struct clk *parent)
-{
- int ret;
- unsigned long flags;
-
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return -EINVAL;
-
- if (!clk->ops || !clk->ops->set_parent)
- return -EINVAL;
-
- spin_lock_irqsave(&clocks_lock, flags);
- ret = clk->ops->set_parent(clk, parent);
- if (!ret) {
- parent->usage += clk->usage;
- clk->parent->usage -= clk->usage;
- BUG_ON(clk->parent->usage < 0);
- clk->parent = parent;
- }
- spin_unlock_irqrestore(&clocks_lock, flags);
- return ret;
-}
-EXPORT_SYMBOL(clk_set_parent);
-
-struct clk *clk_get_parent(struct clk *clk)
-{
- unsigned long flags;
-
- if (unlikely(IS_ERR_OR_NULL(clk)))
- return NULL;
-
- if (!clk->ops || !clk->ops->get_parent)
- return clk->parent;
-
- spin_lock_irqsave(&clocks_lock, flags);
- clk->parent = clk->ops->get_parent(clk);
- spin_unlock_irqrestore(&clocks_lock, flags);
- return clk->parent;
-}
-EXPORT_SYMBOL(clk_get_parent);
-
-static void __init sirfsoc_clk_init(void)
-{
- clkdev_add_table(onchip_clks, ARRAY_SIZE(onchip_clks));
-}
-
-static struct of_device_id clkc_ids[] = {
- { .compatible = "sirf,prima2-clkc" },
- {},
-};
-
-void __init sirfsoc_of_clk_init(void)
-{
- struct device_node *np;
- struct resource res;
- struct map_desc sirfsoc_clkc_iodesc = {
- .virtual = SIRFSOC_CLOCK_VA_BASE,
- .type = MT_DEVICE,
- };
-
- np = of_find_matching_node(NULL, clkc_ids);
- if (!np)
- panic("unable to find compatible clkc node in dtb\n");
-
- if (of_address_to_resource(np, 0, &res))
- panic("unable to find clkc range in dtb");
- of_node_put(np);
-
- sirfsoc_clkc_iodesc.pfn = __phys_to_pfn(res.start);
- sirfsoc_clkc_iodesc.length = 1 + res.end - res.start;
-
- iotable_init(&sirfsoc_clkc_iodesc, 1);
-
- sirfsoc_clk_init();
-}
diff --git a/arch/arm/mach-prima2/prima2.c b/arch/arm/mach-prima2/prima2.c
index 8f0429d..e9a17ae 100644
--- a/arch/arm/mach-prima2/prima2.c
+++ b/arch/arm/mach-prima2/prima2.c
@@ -38,7 +38,6 @@
MACHINE_START(PRIMA2_EVB, "prima2cb")
/* Maintainer: Barry Song <baohua.song@csr.com> */
.atag_offset = 0x100,
- .init_early = sirfsoc_of_clk_init,
.map_io = sirfsoc_map_lluart,
.init_irq = sirfsoc_of_irq_init,
.timer = &sirfsoc_timer,
diff --git a/arch/arm/mach-prima2/timer.c b/arch/arm/mach-prima2/timer.c
index f224107..d95bf25 100644
--- a/arch/arm/mach-prima2/timer.c
+++ b/arch/arm/mach-prima2/timer.c
@@ -21,6 +21,8 @@
#include <asm/sched_clock.h>
#include <asm/mach/time.h>
+#include "common.h"
+
#define SIRFSOC_TIMER_COUNTER_LO 0x0000
#define SIRFSOC_TIMER_COUNTER_HI 0x0004
#define SIRFSOC_TIMER_MATCH_0 0x0008
@@ -188,9 +190,13 @@
static void __init sirfsoc_timer_init(void)
{
unsigned long rate;
+ struct clk *clk;
+
+ /* initialize clocking early, we want to set the OS timer */
+ sirfsoc_of_clk_init();
/* timer's input clock is io clock */
- struct clk *clk = clk_get_sys("io", NULL);
+ clk = clk_get_sys("io", NULL);
BUG_ON(IS_ERR(clk));
diff --git a/arch/arm/mach-realview/core.c b/arch/arm/mach-realview/core.c
index 45868bb..ff007d1 100644
--- a/arch/arm/mach-realview/core.c
+++ b/arch/arm/mach-realview/core.c
@@ -30,7 +30,6 @@
#include <linux/ata_platform.h>
#include <linux/amba/mmci.h>
#include <linux/gfp.h>
-#include <linux/clkdev.h>
#include <linux/mtd/physmap.h>
#include <mach/hardware.h>
@@ -226,115 +225,10 @@
.cd_invert = true,
};
-/*
- * Clock handling
- */
-static const struct icst_params realview_oscvco_params = {
- .ref = 24000000,
- .vco_max = ICST307_VCO_MAX,
- .vco_min = ICST307_VCO_MIN,
- .vd_min = 4 + 8,
- .vd_max = 511 + 8,
- .rd_min = 1 + 2,
- .rd_max = 127 + 2,
- .s2div = icst307_s2div,
- .idx2s = icst307_idx2s,
-};
-
-static void realview_oscvco_set(struct clk *clk, struct icst_vco vco)
-{
- void __iomem *sys_lock = __io_address(REALVIEW_SYS_BASE) + REALVIEW_SYS_LOCK_OFFSET;
- u32 val;
-
- val = readl(clk->vcoreg) & ~0x7ffff;
- val |= vco.v | (vco.r << 9) | (vco.s << 16);
-
- writel(0xa05f, sys_lock);
- writel(val, clk->vcoreg);
- writel(0, sys_lock);
-}
-
-static const struct clk_ops oscvco_clk_ops = {
- .round = icst_clk_round,
- .set = icst_clk_set,
- .setvco = realview_oscvco_set,
-};
-
-static struct clk oscvco_clk = {
- .ops = &oscvco_clk_ops,
- .params = &realview_oscvco_params,
-};
-
-/*
- * These are fixed clocks.
- */
-static struct clk ref24_clk = {
- .rate = 24000000,
-};
-
-static struct clk sp804_clk = {
- .rate = 1000000,
-};
-
-static struct clk dummy_apb_pclk;
-
-static struct clk_lookup lookups[] = {
- { /* Bus clock */
- .con_id = "apb_pclk",
- .clk = &dummy_apb_pclk,
- }, { /* UART0 */
- .dev_id = "dev:uart0",
- .clk = &ref24_clk,
- }, { /* UART1 */
- .dev_id = "dev:uart1",
- .clk = &ref24_clk,
- }, { /* UART2 */
- .dev_id = "dev:uart2",
- .clk = &ref24_clk,
- }, { /* UART3 */
- .dev_id = "fpga:uart3",
- .clk = &ref24_clk,
- }, { /* UART3 is on the dev chip in PB1176 */
- .dev_id = "dev:uart3",
- .clk = &ref24_clk,
- }, { /* UART4 only exists in PB1176 */
- .dev_id = "fpga:uart4",
- .clk = &ref24_clk,
- }, { /* KMI0 */
- .dev_id = "fpga:kmi0",
- .clk = &ref24_clk,
- }, { /* KMI1 */
- .dev_id = "fpga:kmi1",
- .clk = &ref24_clk,
- }, { /* MMC0 */
- .dev_id = "fpga:mmc0",
- .clk = &ref24_clk,
- }, { /* CLCD is in the PB1176 and EB DevChip */
- .dev_id = "dev:clcd",
- .clk = &oscvco_clk,
- }, { /* PB:CLCD */
- .dev_id = "issp:clcd",
- .clk = &oscvco_clk,
- }, { /* SSP */
- .dev_id = "dev:ssp0",
- .clk = &ref24_clk,
- }, { /* SP804 timers */
- .dev_id = "sp804",
- .clk = &sp804_clk,
- },
-};
-
void __init realview_init_early(void)
{
void __iomem *sys = __io_address(REALVIEW_SYS_BASE);
- if (machine_is_realview_pb1176())
- oscvco_clk.vcoreg = sys + REALVIEW_SYS_OSC0_OFFSET;
- else
- oscvco_clk.vcoreg = sys + REALVIEW_SYS_OSC4_OFFSET;
-
- clkdev_add_table(lookups, ARRAY_SIZE(lookups));
-
versatile_sched_clock_init(sys + REALVIEW_SYS_24MHz_OFFSET, 24000000);
}
diff --git a/arch/arm/mach-realview/include/mach/clkdev.h b/arch/arm/mach-realview/include/mach/clkdev.h
deleted file mode 100644
index e58d077..0000000
--- a/arch/arm/mach-realview/include/mach/clkdev.h
+++ /dev/null
@@ -1,16 +0,0 @@
-#ifndef __ASM_MACH_CLKDEV_H
-#define __ASM_MACH_CLKDEV_H
-
-#include <plat/clock.h>
-
-struct clk {
- unsigned long rate;
- const struct clk_ops *ops;
- const struct icst_params *params;
- void __iomem *vcoreg;
-};
-
-#define __clk_get(clk) ({ 1; })
-#define __clk_put(clk) do { } while (0)
-
-#endif
diff --git a/arch/arm/mach-realview/realview_eb.c b/arch/arm/mach-realview/realview_eb.c
index baf382c..a33e33b 100644
--- a/arch/arm/mach-realview/realview_eb.c
+++ b/arch/arm/mach-realview/realview_eb.c
@@ -27,6 +27,7 @@
#include <linux/amba/mmci.h>
#include <linux/amba/pl022.h>
#include <linux/io.h>
+#include <linux/platform_data/clk-realview.h>
#include <mach/hardware.h>
#include <asm/irq.h>
@@ -414,6 +415,7 @@
else
timer_irq = IRQ_EB_TIMER0_1;
+ realview_clk_init(__io_address(REALVIEW_SYS_BASE), false);
realview_timer_init(timer_irq);
realview_eb_twd_init();
}
diff --git a/arch/arm/mach-realview/realview_pb1176.c b/arch/arm/mach-realview/realview_pb1176.c
index b1d7caf..f0298cb 100644
--- a/arch/arm/mach-realview/realview_pb1176.c
+++ b/arch/arm/mach-realview/realview_pb1176.c
@@ -29,6 +29,7 @@
#include <linux/mtd/physmap.h>
#include <linux/mtd/partitions.h>
#include <linux/io.h>
+#include <linux/platform_data/clk-realview.h>
#include <mach/hardware.h>
#include <asm/irq.h>
@@ -326,6 +327,7 @@
timer2_va_base = __io_address(REALVIEW_PB1176_TIMER2_3_BASE);
timer3_va_base = __io_address(REALVIEW_PB1176_TIMER2_3_BASE) + 0x20;
+ realview_clk_init(__io_address(REALVIEW_SYS_BASE), true);
realview_timer_init(IRQ_DC1176_TIMER0);
}
diff --git a/arch/arm/mach-realview/realview_pb11mp.c b/arch/arm/mach-realview/realview_pb11mp.c
index a98c536..1f019f7 100644
--- a/arch/arm/mach-realview/realview_pb11mp.c
+++ b/arch/arm/mach-realview/realview_pb11mp.c
@@ -27,6 +27,7 @@
#include <linux/amba/mmci.h>
#include <linux/amba/pl022.h>
#include <linux/io.h>
+#include <linux/platform_data/clk-realview.h>
#include <mach/hardware.h>
#include <asm/irq.h>
@@ -312,6 +313,7 @@
timer2_va_base = __io_address(REALVIEW_PB11MP_TIMER2_3_BASE);
timer3_va_base = __io_address(REALVIEW_PB11MP_TIMER2_3_BASE) + 0x20;
+ realview_clk_init(__io_address(REALVIEW_SYS_BASE), false);
realview_timer_init(IRQ_TC11MP_TIMER0_1);
realview_pb11mp_twd_init();
}
diff --git a/arch/arm/mach-realview/realview_pba8.c b/arch/arm/mach-realview/realview_pba8.c
index 5965017..5032775 100644
--- a/arch/arm/mach-realview/realview_pba8.c
+++ b/arch/arm/mach-realview/realview_pba8.c
@@ -27,6 +27,7 @@
#include <linux/amba/mmci.h>
#include <linux/amba/pl022.h>
#include <linux/io.h>
+#include <linux/platform_data/clk-realview.h>
#include <asm/irq.h>
#include <asm/leds.h>
@@ -261,6 +262,7 @@
timer2_va_base = __io_address(REALVIEW_PBA8_TIMER2_3_BASE);
timer3_va_base = __io_address(REALVIEW_PBA8_TIMER2_3_BASE) + 0x20;
+ realview_clk_init(__io_address(REALVIEW_SYS_BASE), false);
realview_timer_init(IRQ_PBA8_TIMER0_1);
}
diff --git a/arch/arm/mach-realview/realview_pbx.c b/arch/arm/mach-realview/realview_pbx.c
index 3f2f605..de64ba0 100644
--- a/arch/arm/mach-realview/realview_pbx.c
+++ b/arch/arm/mach-realview/realview_pbx.c
@@ -26,6 +26,7 @@
#include <linux/amba/mmci.h>
#include <linux/amba/pl022.h>
#include <linux/io.h>
+#include <linux/platform_data/clk-realview.h>
#include <asm/irq.h>
#include <asm/leds.h>
@@ -320,6 +321,7 @@
timer2_va_base = __io_address(REALVIEW_PBX_TIMER2_3_BASE);
timer3_va_base = __io_address(REALVIEW_PBX_TIMER2_3_BASE) + 0x20;
+ realview_clk_init(__io_address(REALVIEW_SYS_BASE), false);
realview_timer_init(IRQ_PBX_TIMER0_1);
realview_pbx_twd_init();
}
diff --git a/arch/arm/mach-s3c24xx/include/mach/dma.h b/arch/arm/mach-s3c24xx/include/mach/dma.h
index 454831b..ee99fd5 100644
--- a/arch/arm/mach-s3c24xx/include/mach/dma.h
+++ b/arch/arm/mach-s3c24xx/include/mach/dma.h
@@ -24,7 +24,8 @@
*/
enum dma_ch {
- DMACH_XD0,
+ DMACH_DT_PROP = -1, /* not yet supported, do not use */
+ DMACH_XD0 = 0,
DMACH_XD1,
DMACH_SDI,
DMACH_SPI0,
diff --git a/arch/arm/mach-shmobile/board-armadillo800eva.c b/arch/arm/mach-shmobile/board-armadillo800eva.c
index cf10f92..453a6e5 100644
--- a/arch/arm/mach-shmobile/board-armadillo800eva.c
+++ b/arch/arm/mach-shmobile/board-armadillo800eva.c
@@ -520,13 +520,14 @@
};
/* GPIO KEY */
-#define GPIO_KEY(c, g, d) { .code = c, .gpio = g, .desc = d, .active_low = 1 }
+#define GPIO_KEY(c, g, d, ...) \
+ { .code = c, .gpio = g, .desc = d, .active_low = 1, __VA_ARGS__ }
static struct gpio_keys_button gpio_buttons[] = {
- GPIO_KEY(KEY_POWER, GPIO_PORT99, "SW1"),
- GPIO_KEY(KEY_BACK, GPIO_PORT100, "SW2"),
- GPIO_KEY(KEY_MENU, GPIO_PORT97, "SW3"),
- GPIO_KEY(KEY_HOME, GPIO_PORT98, "SW4"),
+ GPIO_KEY(KEY_POWER, GPIO_PORT99, "SW3", .wakeup = 1),
+ GPIO_KEY(KEY_BACK, GPIO_PORT100, "SW4"),
+ GPIO_KEY(KEY_MENU, GPIO_PORT97, "SW5"),
+ GPIO_KEY(KEY_HOME, GPIO_PORT98, "SW6"),
};
static struct gpio_keys_platform_data gpio_key_info = {
@@ -901,8 +902,8 @@
&camera_device,
&ceu0_device,
&fsi_device,
- &fsi_hdmi_device,
&fsi_wm8978_device,
+ &fsi_hdmi_device,
};
static void __init eva_clock_init(void)
diff --git a/arch/arm/mach-shmobile/board-kzm9g.c b/arch/arm/mach-shmobile/board-kzm9g.c
index 53b7ea9..3b8a017 100644
--- a/arch/arm/mach-shmobile/board-kzm9g.c
+++ b/arch/arm/mach-shmobile/board-kzm9g.c
@@ -346,11 +346,11 @@
.flags = IORESOURCE_MEM,
},
[1] = {
- .start = gic_spi(141),
+ .start = gic_spi(140),
.flags = IORESOURCE_IRQ,
},
[2] = {
- .start = gic_spi(140),
+ .start = gic_spi(141),
.flags = IORESOURCE_IRQ,
},
};
diff --git a/arch/arm/mach-shmobile/board-mackerel.c b/arch/arm/mach-shmobile/board-mackerel.c
index 7ea2b31..c129542 100644
--- a/arch/arm/mach-shmobile/board-mackerel.c
+++ b/arch/arm/mach-shmobile/board-mackerel.c
@@ -695,6 +695,7 @@
* - J30 "open"
* - modify usbhs1_get_id() USBHS_HOST -> USBHS_GADGET
* - add .get_vbus = usbhs_get_vbus in usbhs1_private
+ * - check usbhs0_device(pio)/usbhs1_device(irq) order in mackerel_devices.
*/
#define IRQ8 evt2irq(0x0300)
#define USB_PHY_MODE (1 << 4)
@@ -1325,8 +1326,8 @@
&nor_flash_device,
&smc911x_device,
&lcdc_device,
- &usbhs1_device,
&usbhs0_device,
+ &usbhs1_device,
&leds_device,
&fsi_device,
&fsi_ak4643_device,
diff --git a/arch/arm/mach-shmobile/board-marzen.c b/arch/arm/mach-shmobile/board-marzen.c
index 3a528cf..fcf5a47 100644
--- a/arch/arm/mach-shmobile/board-marzen.c
+++ b/arch/arm/mach-shmobile/board-marzen.c
@@ -67,7 +67,7 @@
static struct platform_device eth_device = {
.name = "smsc911x",
- .id = 0,
+ .id = -1,
.dev = {
.platform_data = &smsc911x_platdata,
},
diff --git a/arch/arm/mach-shmobile/intc-sh73a0.c b/arch/arm/mach-shmobile/intc-sh73a0.c
index ee44740..588555a 100644
--- a/arch/arm/mach-shmobile/intc-sh73a0.c
+++ b/arch/arm/mach-shmobile/intc-sh73a0.c
@@ -259,9 +259,9 @@
return 0; /* always allow wakeup */
}
-#define RELOC_BASE 0x1000
+#define RELOC_BASE 0x1200
-/* INTCA IRQ pins at INTCS + 0x1000 to make space for GIC+INTC handling */
+/* INTCA IRQ pins at INTCS + RELOC_BASE to make space for GIC+INTC handling */
#define INTCS_VECT_RELOC(n, vect) INTCS_VECT((n), (vect) + RELOC_BASE)
INTC_IRQ_PINS_32(intca_irq_pins, 0xe6900000,
diff --git a/arch/arm/mach-tegra/board-harmony-power.c b/arch/arm/mach-tegra/board-harmony-power.c
index b7344be..94486e7 100644
--- a/arch/arm/mach-tegra/board-harmony-power.c
+++ b/arch/arm/mach-tegra/board-harmony-power.c
@@ -67,6 +67,13 @@
}, \
}
+static struct regulator_init_data sys_data = {
+ .supply_regulator = "vdd_5v0",
+ .constraints = {
+ .name = "vdd_sys",
+ },
+};
+
HARMONY_REGULATOR_INIT(sm0, "vdd_sm0", "vdd_sys", 725, 1500, 1);
HARMONY_REGULATOR_INIT(sm1, "vdd_sm1", "vdd_sys", 725, 1500, 1);
HARMONY_REGULATOR_INIT(sm2, "vdd_sm2", "vdd_sys", 3000, 4550, 1);
@@ -74,7 +81,7 @@
HARMONY_REGULATOR_INIT(ldo2, "vdd_ldo2", "vdd_sm2", 725, 1500, 0);
HARMONY_REGULATOR_INIT(ldo3, "vdd_ldo3", "vdd_sm2", 1250, 3300, 1);
HARMONY_REGULATOR_INIT(ldo4, "vdd_ldo4", "vdd_sm2", 1700, 2475, 1);
-HARMONY_REGULATOR_INIT(ldo5, "vdd_ldo5", NULL, 1250, 3300, 1);
+HARMONY_REGULATOR_INIT(ldo5, "vdd_ldo5", "vdd_sys", 1250, 3300, 1);
HARMONY_REGULATOR_INIT(ldo6, "vdd_ldo6", "vdd_sm2", 1250, 3300, 0);
HARMONY_REGULATOR_INIT(ldo7, "vdd_ldo7", "vdd_sm2", 1250, 3300, 0);
HARMONY_REGULATOR_INIT(ldo8, "vdd_ldo8", "vdd_sm2", 1250, 3300, 0);
@@ -88,6 +95,7 @@
}
static struct tps6586x_subdev_info tps_devs[] = {
+ TPS_REG(SYS, &sys_data),
TPS_REG(SM_0, &sm0_data),
TPS_REG(SM_1, &sm1_data),
TPS_REG(SM_2, &sm2_data),
@@ -120,7 +128,7 @@
int __init harmony_regulator_init(void)
{
- regulator_register_always_on(0, "vdd_sys",
+ regulator_register_always_on(0, "vdd_5v0",
NULL, 0, 5000000);
if (machine_is_harmony()) {
diff --git a/arch/arm/mach-tegra/pcie.c b/arch/arm/mach-tegra/pcie.c
index d3ad515..c25a2a4 100644
--- a/arch/arm/mach-tegra/pcie.c
+++ b/arch/arm/mach-tegra/pcie.c
@@ -367,17 +367,7 @@
/* Tegra PCIE requires relaxed ordering */
static void __devinit tegra_pcie_relax_enable(struct pci_dev *dev)
{
- u16 val16;
- int pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
-
- if (pos <= 0) {
- dev_err(&dev->dev, "skipping relaxed ordering fixup\n");
- return;
- }
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &val16);
- val16 |= PCI_EXP_DEVCTL_RELAX_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, val16);
+ pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_RELAX_EN);
}
DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID, PCI_ANY_ID, tegra_pcie_relax_enable);
diff --git a/arch/arm/mach-ux500/Kconfig b/arch/arm/mach-ux500/Kconfig
index c013bbf..a258996 100644
--- a/arch/arm/mach-ux500/Kconfig
+++ b/arch/arm/mach-ux500/Kconfig
@@ -11,6 +11,7 @@
select CACHE_L2X0
select PINCTRL
select PINCTRL_NOMADIK
+ select COMMON_CLK
config UX500_SOC_DB8500
bool
@@ -41,7 +42,6 @@
config MACH_SNOWBALL
bool "U8500 Snowball platform"
select MACH_MOP500
- select LEDS_GPIO
help
Include support for the snowball development platform.
diff --git a/arch/arm/mach-ux500/Makefile b/arch/arm/mach-ux500/Makefile
index 026086f..5691ef6 100644
--- a/arch/arm/mach-ux500/Makefile
+++ b/arch/arm/mach-ux500/Makefile
@@ -2,7 +2,7 @@
# Makefile for the linux kernel, U8500 machine.
#
-obj-y := clock.o cpu.o devices.o devices-common.o \
+obj-y := cpu.o devices.o devices-common.o \
id.o usb.o timer.o
obj-$(CONFIG_CPU_IDLE) += cpuidle.o
obj-$(CONFIG_CACHE_L2X0) += cache-l2x0.o
diff --git a/arch/arm/mach-ux500/board-mop500-msp.c b/arch/arm/mach-ux500/board-mop500-msp.c
index 9960480..df15646 100644
--- a/arch/arm/mach-ux500/board-mop500-msp.c
+++ b/arch/arm/mach-ux500/board-mop500-msp.c
@@ -191,9 +191,9 @@
return pdev;
}
-/* Platform device for ASoC U8500 machine */
-static struct platform_device snd_soc_u8500 = {
- .name = "snd-soc-u8500",
+/* Platform device for ASoC MOP500 machine */
+static struct platform_device snd_soc_mop500 = {
+ .name = "snd-soc-mop500",
.id = 0,
.dev = {
.platform_data = NULL,
@@ -227,8 +227,8 @@
{
struct platform_device *msp1;
- pr_info("%s: Register platform-device 'snd-soc-u8500'.\n", __func__);
- platform_device_register(&snd_soc_u8500);
+ pr_info("%s: Register platform-device 'snd-soc-mop500'.\n", __func__);
+ platform_device_register(&snd_soc_mop500);
pr_info("Initialize MSP I2S-devices.\n");
db8500_add_msp_i2s(parent, 0, U8500_MSP0_BASE, IRQ_DB8500_MSP0,
diff --git a/arch/arm/mach-ux500/board-mop500.c b/arch/arm/mach-ux500/board-mop500.c
index 8674a89..a534d88 100644
--- a/arch/arm/mach-ux500/board-mop500.c
+++ b/arch/arm/mach-ux500/board-mop500.c
@@ -797,6 +797,7 @@
ARRAY_SIZE(mop500_platform_devs));
mop500_sdi_init(parent);
+ mop500_msp_init(parent);
i2c0_devs = ARRAY_SIZE(mop500_i2c0_devices);
i2c_register_board_info(0, mop500_i2c0_devices, i2c0_devs);
i2c_register_board_info(2, mop500_i2c2_devices,
@@ -804,6 +805,8 @@
mop500_uib_init();
+ } else if (of_machine_is_compatible("calaosystems,snowball-a9500")) {
+ mop500_msp_init(parent);
} else if (of_machine_is_compatible("st-ericsson,hrefv60+")) {
/*
* The HREFv60 board removed a GPIO expander and routed
@@ -815,6 +818,7 @@
ARRAY_SIZE(mop500_platform_devs));
hrefv60_sdi_init(parent);
+ mop500_msp_init(parent);
i2c0_devs = ARRAY_SIZE(mop500_i2c0_devices);
i2c0_devs -= NUM_PRE_V60_I2C0_DEVICES;
diff --git a/arch/arm/mach-ux500/clock.c b/arch/arm/mach-ux500/clock.c
deleted file mode 100644
index 8d73b06..0000000
--- a/arch/arm/mach-ux500/clock.c
+++ /dev/null
@@ -1,715 +0,0 @@
-/*
- * Copyright (C) 2009 ST-Ericsson
- * Copyright (C) 2009 STMicroelectronics
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/list.h>
-#include <linux/errno.h>
-#include <linux/err.h>
-#include <linux/clk.h>
-#include <linux/io.h>
-#include <linux/clkdev.h>
-#include <linux/cpufreq.h>
-
-#include <plat/mtu.h>
-#include <mach/hardware.h>
-#include "clock.h"
-
-#ifdef CONFIG_DEBUG_FS
-#include <linux/debugfs.h>
-#include <linux/uaccess.h> /* for copy_from_user */
-static LIST_HEAD(clk_list);
-#endif
-
-#define PRCC_PCKEN 0x00
-#define PRCC_PCKDIS 0x04
-#define PRCC_KCKEN 0x08
-#define PRCC_KCKDIS 0x0C
-
-#define PRCM_YYCLKEN0_MGT_SET 0x510
-#define PRCM_YYCLKEN1_MGT_SET 0x514
-#define PRCM_YYCLKEN0_MGT_CLR 0x518
-#define PRCM_YYCLKEN1_MGT_CLR 0x51C
-#define PRCM_YYCLKEN0_MGT_VAL 0x520
-#define PRCM_YYCLKEN1_MGT_VAL 0x524
-
-#define PRCM_SVAMMDSPCLK_MGT 0x008
-#define PRCM_SIAMMDSPCLK_MGT 0x00C
-#define PRCM_SGACLK_MGT 0x014
-#define PRCM_UARTCLK_MGT 0x018
-#define PRCM_MSP02CLK_MGT 0x01C
-#define PRCM_MSP1CLK_MGT 0x288
-#define PRCM_I2CCLK_MGT 0x020
-#define PRCM_SDMMCCLK_MGT 0x024
-#define PRCM_SLIMCLK_MGT 0x028
-#define PRCM_PER1CLK_MGT 0x02C
-#define PRCM_PER2CLK_MGT 0x030
-#define PRCM_PER3CLK_MGT 0x034
-#define PRCM_PER5CLK_MGT 0x038
-#define PRCM_PER6CLK_MGT 0x03C
-#define PRCM_PER7CLK_MGT 0x040
-#define PRCM_LCDCLK_MGT 0x044
-#define PRCM_BMLCLK_MGT 0x04C
-#define PRCM_HSITXCLK_MGT 0x050
-#define PRCM_HSIRXCLK_MGT 0x054
-#define PRCM_HDMICLK_MGT 0x058
-#define PRCM_APEATCLK_MGT 0x05C
-#define PRCM_APETRACECLK_MGT 0x060
-#define PRCM_MCDECLK_MGT 0x064
-#define PRCM_IPI2CCLK_MGT 0x068
-#define PRCM_DSIALTCLK_MGT 0x06C
-#define PRCM_DMACLK_MGT 0x074
-#define PRCM_B2R2CLK_MGT 0x078
-#define PRCM_TVCLK_MGT 0x07C
-#define PRCM_TCR 0x1C8
-#define PRCM_TCR_STOPPED (1 << 16)
-#define PRCM_TCR_DOZE_MODE (1 << 17)
-#define PRCM_UNIPROCLK_MGT 0x278
-#define PRCM_SSPCLK_MGT 0x280
-#define PRCM_RNGCLK_MGT 0x284
-#define PRCM_UICCCLK_MGT 0x27C
-
-#define PRCM_MGT_ENABLE (1 << 8)
-
-static DEFINE_SPINLOCK(clocks_lock);
-
-static void __clk_enable(struct clk *clk)
-{
- if (clk->enabled++ == 0) {
- if (clk->parent_cluster)
- __clk_enable(clk->parent_cluster);
-
- if (clk->parent_periph)
- __clk_enable(clk->parent_periph);
-
- if (clk->ops && clk->ops->enable)
- clk->ops->enable(clk);
- }
-}
-
-int clk_enable(struct clk *clk)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&clocks_lock, flags);
- __clk_enable(clk);
- spin_unlock_irqrestore(&clocks_lock, flags);
-
- return 0;
-}
-EXPORT_SYMBOL(clk_enable);
-
-static void __clk_disable(struct clk *clk)
-{
- if (--clk->enabled == 0) {
- if (clk->ops && clk->ops->disable)
- clk->ops->disable(clk);
-
- if (clk->parent_periph)
- __clk_disable(clk->parent_periph);
-
- if (clk->parent_cluster)
- __clk_disable(clk->parent_cluster);
- }
-}
-
-void clk_disable(struct clk *clk)
-{
- unsigned long flags;
-
- WARN_ON(!clk->enabled);
-
- spin_lock_irqsave(&clocks_lock, flags);
- __clk_disable(clk);
- spin_unlock_irqrestore(&clocks_lock, flags);
-}
-EXPORT_SYMBOL(clk_disable);
-
-/*
- * The MTU has a separate, rather complex muxing setup
- * with alternative parents (peripheral cluster or
- * ULP or fixed 32768 Hz) depending on settings
- */
-static unsigned long clk_mtu_get_rate(struct clk *clk)
-{
- void __iomem *addr;
- u32 tcr;
- int mtu = (int) clk->data;
- /*
- * One of these is selected eventually
- * TODO: Replace the constant with a reference
- * to the ULP source once this is modeled.
- */
- unsigned long clk32k = 32768;
- unsigned long mturate;
- unsigned long retclk;
-
- if (cpu_is_u8500_family())
- addr = __io_address(U8500_PRCMU_BASE);
- else
- ux500_unknown_soc();
-
- /*
- * On a startup, always conifgure the TCR to the doze mode;
- * bootloaders do it for us. Do this in the kernel too.
- */
- writel(PRCM_TCR_DOZE_MODE, addr + PRCM_TCR);
-
- tcr = readl(addr + PRCM_TCR);
-
- /* Get the rate from the parent as a default */
- if (clk->parent_periph)
- mturate = clk_get_rate(clk->parent_periph);
- else if (clk->parent_cluster)
- mturate = clk_get_rate(clk->parent_cluster);
- else
- /* We need to be connected SOMEWHERE */
- BUG();
-
- /* Return the clock selected for this MTU */
- if (tcr & (1 << mtu))
- retclk = clk32k;
- else
- retclk = mturate;
-
- pr_info("MTU%d clock rate: %lu Hz\n", mtu, retclk);
- return retclk;
-}
-
-unsigned long clk_get_rate(struct clk *clk)
-{
- unsigned long rate;
-
- /*
- * If there is a custom getrate callback for this clock,
- * it will take precedence.
- */
- if (clk->get_rate)
- return clk->get_rate(clk);
-
- if (clk->ops && clk->ops->get_rate)
- return clk->ops->get_rate(clk);
-
- rate = clk->rate;
- if (!rate) {
- if (clk->parent_periph)
- rate = clk_get_rate(clk->parent_periph);
- else if (clk->parent_cluster)
- rate = clk_get_rate(clk->parent_cluster);
- }
-
- return rate;
-}
-EXPORT_SYMBOL(clk_get_rate);
-
-long clk_round_rate(struct clk *clk, unsigned long rate)
-{
- /*TODO*/
- return rate;
-}
-EXPORT_SYMBOL(clk_round_rate);
-
-int clk_set_rate(struct clk *clk, unsigned long rate)
-{
- clk->rate = rate;
- return 0;
-}
-EXPORT_SYMBOL(clk_set_rate);
-
-int clk_set_parent(struct clk *clk, struct clk *parent)
-{
- /*TODO*/
- return -ENOSYS;
-}
-EXPORT_SYMBOL(clk_set_parent);
-
-static void clk_prcmu_enable(struct clk *clk)
-{
- void __iomem *cg_set_reg = __io_address(U8500_PRCMU_BASE)
- + PRCM_YYCLKEN0_MGT_SET + clk->prcmu_cg_off;
-
- writel(1 << clk->prcmu_cg_bit, cg_set_reg);
-}
-
-static void clk_prcmu_disable(struct clk *clk)
-{
- void __iomem *cg_clr_reg = __io_address(U8500_PRCMU_BASE)
- + PRCM_YYCLKEN0_MGT_CLR + clk->prcmu_cg_off;
-
- writel(1 << clk->prcmu_cg_bit, cg_clr_reg);
-}
-
-static struct clkops clk_prcmu_ops = {
- .enable = clk_prcmu_enable,
- .disable = clk_prcmu_disable,
-};
-
-static unsigned int clkrst_base[] = {
- [1] = U8500_CLKRST1_BASE,
- [2] = U8500_CLKRST2_BASE,
- [3] = U8500_CLKRST3_BASE,
- [5] = U8500_CLKRST5_BASE,
- [6] = U8500_CLKRST6_BASE,
-};
-
-static void clk_prcc_enable(struct clk *clk)
-{
- void __iomem *addr = __io_address(clkrst_base[clk->cluster]);
-
- if (clk->prcc_kernel != -1)
- writel(1 << clk->prcc_kernel, addr + PRCC_KCKEN);
-
- if (clk->prcc_bus != -1)
- writel(1 << clk->prcc_bus, addr + PRCC_PCKEN);
-}
-
-static void clk_prcc_disable(struct clk *clk)
-{
- void __iomem *addr = __io_address(clkrst_base[clk->cluster]);
-
- if (clk->prcc_bus != -1)
- writel(1 << clk->prcc_bus, addr + PRCC_PCKDIS);
-
- if (clk->prcc_kernel != -1)
- writel(1 << clk->prcc_kernel, addr + PRCC_KCKDIS);
-}
-
-static struct clkops clk_prcc_ops = {
- .enable = clk_prcc_enable,
- .disable = clk_prcc_disable,
-};
-
-static struct clk clk_32khz = {
- .name = "clk_32khz",
- .rate = 32000,
-};
-
-/*
- * PRCMU level clock gating
- */
-
-/* Bank 0 */
-static DEFINE_PRCMU_CLK(svaclk, 0x0, 2, SVAMMDSPCLK);
-static DEFINE_PRCMU_CLK(siaclk, 0x0, 3, SIAMMDSPCLK);
-static DEFINE_PRCMU_CLK(sgaclk, 0x0, 4, SGACLK);
-static DEFINE_PRCMU_CLK_RATE(uartclk, 0x0, 5, UARTCLK, 38400000);
-static DEFINE_PRCMU_CLK(msp02clk, 0x0, 6, MSP02CLK);
-static DEFINE_PRCMU_CLK(msp1clk, 0x0, 7, MSP1CLK); /* v1 */
-static DEFINE_PRCMU_CLK_RATE(i2cclk, 0x0, 8, I2CCLK, 48000000);
-static DEFINE_PRCMU_CLK_RATE(sdmmcclk, 0x0, 9, SDMMCCLK, 100000000);
-static DEFINE_PRCMU_CLK(slimclk, 0x0, 10, SLIMCLK);
-static DEFINE_PRCMU_CLK(per1clk, 0x0, 11, PER1CLK);
-static DEFINE_PRCMU_CLK(per2clk, 0x0, 12, PER2CLK);
-static DEFINE_PRCMU_CLK(per3clk, 0x0, 13, PER3CLK);
-static DEFINE_PRCMU_CLK(per5clk, 0x0, 14, PER5CLK);
-static DEFINE_PRCMU_CLK_RATE(per6clk, 0x0, 15, PER6CLK, 133330000);
-static DEFINE_PRCMU_CLK(lcdclk, 0x0, 17, LCDCLK);
-static DEFINE_PRCMU_CLK(bmlclk, 0x0, 18, BMLCLK);
-static DEFINE_PRCMU_CLK(hsitxclk, 0x0, 19, HSITXCLK);
-static DEFINE_PRCMU_CLK(hsirxclk, 0x0, 20, HSIRXCLK);
-static DEFINE_PRCMU_CLK(hdmiclk, 0x0, 21, HDMICLK);
-static DEFINE_PRCMU_CLK(apeatclk, 0x0, 22, APEATCLK);
-static DEFINE_PRCMU_CLK(apetraceclk, 0x0, 23, APETRACECLK);
-static DEFINE_PRCMU_CLK(mcdeclk, 0x0, 24, MCDECLK);
-static DEFINE_PRCMU_CLK(ipi2clk, 0x0, 25, IPI2CCLK);
-static DEFINE_PRCMU_CLK(dsialtclk, 0x0, 26, DSIALTCLK); /* v1 */
-static DEFINE_PRCMU_CLK(dmaclk, 0x0, 27, DMACLK);
-static DEFINE_PRCMU_CLK(b2r2clk, 0x0, 28, B2R2CLK);
-static DEFINE_PRCMU_CLK(tvclk, 0x0, 29, TVCLK);
-static DEFINE_PRCMU_CLK(uniproclk, 0x0, 30, UNIPROCLK); /* v1 */
-static DEFINE_PRCMU_CLK_RATE(sspclk, 0x0, 31, SSPCLK, 48000000); /* v1 */
-
-/* Bank 1 */
-static DEFINE_PRCMU_CLK(rngclk, 0x4, 0, RNGCLK); /* v1 */
-static DEFINE_PRCMU_CLK(uiccclk, 0x4, 1, UICCCLK); /* v1 */
-
-/*
- * PRCC level clock gating
- * Format: per#, clk, PCKEN bit, KCKEN bit, parent
- */
-
-/* Peripheral Cluster #1 */
-static DEFINE_PRCC_CLK(1, msp3, 11, 10, &clk_msp1clk);
-static DEFINE_PRCC_CLK(1, i2c4, 10, 9, &clk_i2cclk);
-static DEFINE_PRCC_CLK(1, gpio0, 9, -1, NULL);
-static DEFINE_PRCC_CLK(1, slimbus0, 8, 8, &clk_slimclk);
-static DEFINE_PRCC_CLK(1, spi3, 7, -1, NULL);
-static DEFINE_PRCC_CLK(1, i2c2, 6, 6, &clk_i2cclk);
-static DEFINE_PRCC_CLK(1, sdi0, 5, 5, &clk_sdmmcclk);
-static DEFINE_PRCC_CLK(1, msp1, 4, 4, &clk_msp1clk);
-static DEFINE_PRCC_CLK(1, msp0, 3, 3, &clk_msp02clk);
-static DEFINE_PRCC_CLK(1, i2c1, 2, 2, &clk_i2cclk);
-static DEFINE_PRCC_CLK(1, uart1, 1, 1, &clk_uartclk);
-static DEFINE_PRCC_CLK(1, uart0, 0, 0, &clk_uartclk);
-
-/* Peripheral Cluster #2 */
-static DEFINE_PRCC_CLK(2, gpio1, 11, -1, NULL);
-static DEFINE_PRCC_CLK(2, ssitx, 10, 7, NULL);
-static DEFINE_PRCC_CLK(2, ssirx, 9, 6, NULL);
-static DEFINE_PRCC_CLK(2, spi0, 8, -1, NULL);
-static DEFINE_PRCC_CLK(2, sdi3, 7, 5, &clk_sdmmcclk);
-static DEFINE_PRCC_CLK(2, sdi1, 6, 4, &clk_sdmmcclk);
-static DEFINE_PRCC_CLK(2, msp2, 5, 3, &clk_msp02clk);
-static DEFINE_PRCC_CLK(2, sdi4, 4, 2, &clk_sdmmcclk);
-static DEFINE_PRCC_CLK(2, pwl, 3, 1, NULL);
-static DEFINE_PRCC_CLK(2, spi1, 2, -1, NULL);
-static DEFINE_PRCC_CLK(2, spi2, 1, -1, NULL);
-static DEFINE_PRCC_CLK(2, i2c3, 0, 0, &clk_i2cclk);
-
-/* Peripheral Cluster #3 */
-static DEFINE_PRCC_CLK(3, gpio2, 8, -1, NULL);
-static DEFINE_PRCC_CLK(3, sdi5, 7, 7, &clk_sdmmcclk);
-static DEFINE_PRCC_CLK(3, uart2, 6, 6, &clk_uartclk);
-static DEFINE_PRCC_CLK(3, ske, 5, 5, &clk_32khz);
-static DEFINE_PRCC_CLK(3, sdi2, 4, 4, &clk_sdmmcclk);
-static DEFINE_PRCC_CLK(3, i2c0, 3, 3, &clk_i2cclk);
-static DEFINE_PRCC_CLK(3, ssp1, 2, 2, &clk_sspclk);
-static DEFINE_PRCC_CLK(3, ssp0, 1, 1, &clk_sspclk);
-static DEFINE_PRCC_CLK(3, fsmc, 0, -1, NULL);
-
-/* Peripheral Cluster #4 is in the always on domain */
-
-/* Peripheral Cluster #5 */
-static DEFINE_PRCC_CLK(5, gpio3, 1, -1, NULL);
-static DEFINE_PRCC_CLK(5, usb, 0, 0, NULL);
-
-/* Peripheral Cluster #6 */
-
-/* MTU ID in data */
-static DEFINE_PRCC_CLK_CUSTOM(6, mtu1, 9, -1, NULL, clk_mtu_get_rate, 1);
-static DEFINE_PRCC_CLK_CUSTOM(6, mtu0, 8, -1, NULL, clk_mtu_get_rate, 0);
-static DEFINE_PRCC_CLK(6, cfgreg, 7, 7, NULL);
-static DEFINE_PRCC_CLK(6, hash1, 6, -1, NULL);
-static DEFINE_PRCC_CLK(6, unipro, 5, 1, &clk_uniproclk);
-static DEFINE_PRCC_CLK(6, pka, 4, -1, NULL);
-static DEFINE_PRCC_CLK(6, hash0, 3, -1, NULL);
-static DEFINE_PRCC_CLK(6, cryp0, 2, -1, NULL);
-static DEFINE_PRCC_CLK(6, cryp1, 1, -1, NULL);
-static DEFINE_PRCC_CLK(6, rng, 0, 0, &clk_rngclk);
-
-static struct clk clk_dummy_apb_pclk = {
- .name = "apb_pclk",
-};
-
-static struct clk_lookup u8500_clks[] = {
- CLK(dummy_apb_pclk, NULL, "apb_pclk"),
-
- /* Peripheral Cluster #1 */
- CLK(gpio0, "gpio.0", NULL),
- CLK(gpio0, "gpio.1", NULL),
- CLK(slimbus0, "slimbus0", NULL),
- CLK(i2c2, "nmk-i2c.2", NULL),
- CLK(sdi0, "sdi0", NULL),
- CLK(msp0, "ux500-msp-i2s.0", NULL),
- CLK(i2c1, "nmk-i2c.1", NULL),
- CLK(uart1, "uart1", NULL),
- CLK(uart0, "uart0", NULL),
-
- /* Peripheral Cluster #3 */
- CLK(gpio2, "gpio.2", NULL),
- CLK(gpio2, "gpio.3", NULL),
- CLK(gpio2, "gpio.4", NULL),
- CLK(gpio2, "gpio.5", NULL),
- CLK(sdi5, "sdi5", NULL),
- CLK(uart2, "uart2", NULL),
- CLK(ske, "ske", NULL),
- CLK(ske, "nmk-ske-keypad", NULL),
- CLK(sdi2, "sdi2", NULL),
- CLK(i2c0, "nmk-i2c.0", NULL),
- CLK(fsmc, "fsmc", NULL),
-
- /* Peripheral Cluster #5 */
- CLK(gpio3, "gpio.8", NULL),
-
- /* Peripheral Cluster #6 */
- CLK(hash1, "hash1", NULL),
- CLK(pka, "pka", NULL),
- CLK(hash0, "hash0", NULL),
- CLK(cryp0, "cryp0", NULL),
- CLK(cryp1, "cryp1", NULL),
-
- /* PRCMU level clock gating */
-
- /* Bank 0 */
- CLK(svaclk, "sva", NULL),
- CLK(siaclk, "sia", NULL),
- CLK(sgaclk, "sga", NULL),
- CLK(slimclk, "slim", NULL),
- CLK(lcdclk, "lcd", NULL),
- CLK(bmlclk, "bml", NULL),
- CLK(hsitxclk, "stm-hsi.0", NULL),
- CLK(hsirxclk, "stm-hsi.1", NULL),
- CLK(hdmiclk, "hdmi", NULL),
- CLK(apeatclk, "apeat", NULL),
- CLK(apetraceclk, "apetrace", NULL),
- CLK(mcdeclk, "mcde", NULL),
- CLK(ipi2clk, "ipi2", NULL),
- CLK(dmaclk, "dma40.0", NULL),
- CLK(b2r2clk, "b2r2", NULL),
- CLK(tvclk, "tv", NULL),
-
- /* Peripheral Cluster #1 */
- CLK(i2c4, "nmk-i2c.4", NULL),
- CLK(spi3, "spi3", NULL),
- CLK(msp1, "ux500-msp-i2s.1", NULL),
- CLK(msp3, "ux500-msp-i2s.3", NULL),
-
- /* Peripheral Cluster #2 */
- CLK(gpio1, "gpio.6", NULL),
- CLK(gpio1, "gpio.7", NULL),
- CLK(ssitx, "ssitx", NULL),
- CLK(ssirx, "ssirx", NULL),
- CLK(spi0, "spi0", NULL),
- CLK(sdi3, "sdi3", NULL),
- CLK(sdi1, "sdi1", NULL),
- CLK(msp2, "ux500-msp-i2s.2", NULL),
- CLK(sdi4, "sdi4", NULL),
- CLK(pwl, "pwl", NULL),
- CLK(spi1, "spi1", NULL),
- CLK(spi2, "spi2", NULL),
- CLK(i2c3, "nmk-i2c.3", NULL),
-
- /* Peripheral Cluster #3 */
- CLK(ssp1, "ssp1", NULL),
- CLK(ssp0, "ssp0", NULL),
-
- /* Peripheral Cluster #5 */
- CLK(usb, "musb-ux500.0", "usb"),
-
- /* Peripheral Cluster #6 */
- CLK(mtu1, "mtu1", NULL),
- CLK(mtu0, "mtu0", NULL),
- CLK(cfgreg, "cfgreg", NULL),
- CLK(hash1, "hash1", NULL),
- CLK(unipro, "unipro", NULL),
- CLK(rng, "rng", NULL),
-
- /* PRCMU level clock gating */
-
- /* Bank 0 */
- CLK(uniproclk, "uniproclk", NULL),
- CLK(dsialtclk, "dsialt", NULL),
-
- /* Bank 1 */
- CLK(rngclk, "rng", NULL),
- CLK(uiccclk, "uicc", NULL),
-};
-
-#ifdef CONFIG_DEBUG_FS
-/*
- * debugfs support to trace clock tree hierarchy and attributes with
- * powerdebug
- */
-static struct dentry *clk_debugfs_root;
-
-void __init clk_debugfs_add_table(struct clk_lookup *cl, size_t num)
-{
- while (num--) {
- /* Check that the clock has not been already registered */
- if (!(cl->clk->list.prev != cl->clk->list.next))
- list_add_tail(&cl->clk->list, &clk_list);
-
- cl++;
- }
-}
-
-static ssize_t usecount_dbg_read(struct file *file, char __user *buf,
- size_t size, loff_t *off)
-{
- struct clk *clk = file->f_dentry->d_inode->i_private;
- char cusecount[128];
- unsigned int len;
-
- len = sprintf(cusecount, "%u\n", clk->enabled);
- return simple_read_from_buffer(buf, size, off, cusecount, len);
-}
-
-static ssize_t rate_dbg_read(struct file *file, char __user *buf,
- size_t size, loff_t *off)
-{
- struct clk *clk = file->f_dentry->d_inode->i_private;
- char crate[128];
- unsigned int rate;
- unsigned int len;
-
- rate = clk_get_rate(clk);
- len = sprintf(crate, "%u\n", rate);
- return simple_read_from_buffer(buf, size, off, crate, len);
-}
-
-static const struct file_operations usecount_fops = {
- .read = usecount_dbg_read,
-};
-
-static const struct file_operations set_rate_fops = {
- .read = rate_dbg_read,
-};
-
-static struct dentry *clk_debugfs_register_dir(struct clk *c,
- struct dentry *p_dentry)
-{
- struct dentry *d, *clk_d;
- const char *p = c->name;
-
- if (!p)
- p = "BUG";
-
- clk_d = debugfs_create_dir(p, p_dentry);
- if (!clk_d)
- return NULL;
-
- d = debugfs_create_file("usecount", S_IRUGO,
- clk_d, c, &usecount_fops);
- if (!d)
- goto err_out;
- d = debugfs_create_file("rate", S_IRUGO,
- clk_d, c, &set_rate_fops);
- if (!d)
- goto err_out;
- /*
- * TODO : not currently available in ux500
- * d = debugfs_create_x32("flags", S_IRUGO, clk_d, (u32 *)&c->flags);
- * if (!d)
- * goto err_out;
- */
-
- return clk_d;
-
-err_out:
- debugfs_remove_recursive(clk_d);
- return NULL;
-}
-
-static int clk_debugfs_register_one(struct clk *c)
-{
- struct clk *pa = c->parent_periph;
- struct clk *bpa = c->parent_cluster;
-
- if (!(bpa && !pa)) {
- c->dent = clk_debugfs_register_dir(c,
- pa ? pa->dent : clk_debugfs_root);
- if (!c->dent)
- return -ENOMEM;
- }
-
- if (bpa) {
- c->dent_bus = clk_debugfs_register_dir(c,
- bpa->dent_bus ? bpa->dent_bus : bpa->dent);
- if ((!c->dent_bus) && (c->dent)) {
- debugfs_remove_recursive(c->dent);
- c->dent = NULL;
- return -ENOMEM;
- }
- }
- return 0;
-}
-
-static int clk_debugfs_register(struct clk *c)
-{
- int err;
- struct clk *pa = c->parent_periph;
- struct clk *bpa = c->parent_cluster;
-
- if (pa && (!pa->dent && !pa->dent_bus)) {
- err = clk_debugfs_register(pa);
- if (err)
- return err;
- }
-
- if (bpa && (!bpa->dent && !bpa->dent_bus)) {
- err = clk_debugfs_register(bpa);
- if (err)
- return err;
- }
-
- if ((!c->dent) && (!c->dent_bus)) {
- err = clk_debugfs_register_one(c);
- if (err)
- return err;
- }
- return 0;
-}
-
-int __init clk_debugfs_init(void)
-{
- struct clk *c;
- struct dentry *d;
- int err;
-
- d = debugfs_create_dir("clock", NULL);
- if (!d)
- return -ENOMEM;
- clk_debugfs_root = d;
-
- list_for_each_entry(c, &clk_list, list) {
- err = clk_debugfs_register(c);
- if (err)
- goto err_out;
- }
- return 0;
-err_out:
- debugfs_remove_recursive(clk_debugfs_root);
- return err;
-}
-
-#endif /* defined(CONFIG_DEBUG_FS) */
-
-unsigned long clk_smp_twd_rate = 500000000;
-
-unsigned long clk_smp_twd_get_rate(struct clk *clk)
-{
- return clk_smp_twd_rate;
-}
-
-static struct clk clk_smp_twd = {
- .get_rate = clk_smp_twd_get_rate,
- .name = "smp_twd",
-};
-
-static struct clk_lookup clk_smp_twd_lookup = {
- .dev_id = "smp_twd",
- .clk = &clk_smp_twd,
-};
-
-#ifdef CONFIG_CPU_FREQ
-
-static int clk_twd_cpufreq_transition(struct notifier_block *nb,
- unsigned long state, void *data)
-{
- struct cpufreq_freqs *f = data;
-
- if (state == CPUFREQ_PRECHANGE) {
- /* Save frequency in simple Hz */
- clk_smp_twd_rate = (f->new * 1000) / 2;
- }
-
- return NOTIFY_OK;
-}
-
-static struct notifier_block clk_twd_cpufreq_nb = {
- .notifier_call = clk_twd_cpufreq_transition,
-};
-
-int clk_init_smp_twd_cpufreq(void)
-{
- return cpufreq_register_notifier(&clk_twd_cpufreq_nb,
- CPUFREQ_TRANSITION_NOTIFIER);
-}
-
-#endif
-
-int __init clk_init(void)
-{
- clkdev_add_table(u8500_clks, ARRAY_SIZE(u8500_clks));
- clkdev_add(&clk_smp_twd_lookup);
-
-#ifdef CONFIG_DEBUG_FS
- clk_debugfs_add_table(u8500_clks, ARRAY_SIZE(u8500_clks));
-#endif
- return 0;
-}
diff --git a/arch/arm/mach-ux500/clock.h b/arch/arm/mach-ux500/clock.h
deleted file mode 100644
index 65d27a1..0000000
--- a/arch/arm/mach-ux500/clock.h
+++ /dev/null
@@ -1,164 +0,0 @@
-/*
- * Copyright (C) 2010 ST-Ericsson
- * Copyright (C) 2009 STMicroelectronics
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-/**
- * struct clkops - ux500 clock operations
- * @enable: function to enable the clock
- * @disable: function to disable the clock
- * @get_rate: function to get the current clock rate
- *
- * This structure contains function pointers to functions that will be used to
- * control the clock. All of these functions are optional. If get_rate is
- * NULL, the rate in the struct clk will be used.
- */
-struct clkops {
- void (*enable) (struct clk *);
- void (*disable) (struct clk *);
- unsigned long (*get_rate) (struct clk *);
- int (*set_parent)(struct clk *, struct clk *);
-};
-
-/**
- * struct clk - ux500 clock structure
- * @ops: pointer to clkops struct used to control this clock
- * @name: name, for debugging
- * @enabled: refcount. positive if enabled, zero if disabled
- * @get_rate: custom callback for getting the clock rate
- * @data: custom per-clock data for example for the get_rate
- * callback
- * @rate: fixed rate for clocks which don't implement
- * ops->getrate
- * @prcmu_cg_off: address offset of the combined enable/disable register
- * (used on u8500v1)
- * @prcmu_cg_bit: bit in the combined enable/disable register (used on
- * u8500v1)
- * @prcmu_cg_mgt: address of the enable/disable register (used on
- * u8500ed)
- * @cluster: peripheral cluster number
- * @prcc_bus: bit for the bus clock in the peripheral's CLKRST
- * @prcc_kernel: bit for the kernel clock in the peripheral's CLKRST.
- * -1 if no kernel clock exists.
- * @parent_cluster: pointer to parent's cluster clk struct
- * @parent_periph: pointer to parent's peripheral clk struct
- *
- * Peripherals are organised into clusters, and each cluster has an associated
- * bus clock. Some peripherals also have a parent peripheral clock.
- *
- * In order to enable a clock for a peripheral, we need to enable:
- * (1) the parent cluster (bus) clock at the PRCMU level
- * (2) the parent peripheral clock (if any) at the PRCMU level
- * (3) the peripheral's bus & kernel clock at the PRCC level
- *
- * (1) and (2) are handled by defining clk structs (DEFINE_PRCMU_CLK) for each
- * of the cluster and peripheral clocks, and hooking these as the parents of
- * the individual peripheral clocks.
- *
- * (3) is handled by specifying the bits in the PRCC control registers required
- * to enable these clocks and modifying them in the ->enable and
- * ->disable callbacks of the peripheral clocks (DEFINE_PRCC_CLK).
- *
- * This structure describes both the PRCMU-level clocks and PRCC-level clocks.
- * The prcmu_* fields are only used for the PRCMU clocks, and the cluster,
- * prcc, and parent pointers are only used for the PRCC-level clocks.
- */
-struct clk {
- const struct clkops *ops;
- const char *name;
- unsigned int enabled;
- unsigned long (*get_rate)(struct clk *);
- void *data;
-
- unsigned long rate;
- struct list_head list;
-
- /* These three are only for PRCMU clks */
-
- unsigned int prcmu_cg_off;
- unsigned int prcmu_cg_bit;
- unsigned int prcmu_cg_mgt;
-
- /* The rest are only for PRCC clks */
-
- int cluster;
- unsigned int prcc_bus;
- unsigned int prcc_kernel;
-
- struct clk *parent_cluster;
- struct clk *parent_periph;
-#if defined(CONFIG_DEBUG_FS)
- struct dentry *dent; /* For visible tree hierarchy */
- struct dentry *dent_bus; /* For visible tree hierarchy */
-#endif
-};
-
-#define DEFINE_PRCMU_CLK(_name, _cg_off, _cg_bit, _reg) \
-struct clk clk_##_name = { \
- .name = #_name, \
- .ops = &clk_prcmu_ops, \
- .prcmu_cg_off = _cg_off, \
- .prcmu_cg_bit = _cg_bit, \
- .prcmu_cg_mgt = PRCM_##_reg##_MGT \
- }
-
-#define DEFINE_PRCMU_CLK_RATE(_name, _cg_off, _cg_bit, _reg, _rate) \
-struct clk clk_##_name = { \
- .name = #_name, \
- .ops = &clk_prcmu_ops, \
- .prcmu_cg_off = _cg_off, \
- .prcmu_cg_bit = _cg_bit, \
- .rate = _rate, \
- .prcmu_cg_mgt = PRCM_##_reg##_MGT \
- }
-
-#define DEFINE_PRCC_CLK(_pclust, _name, _bus_en, _kernel_en, _kernclk) \
-struct clk clk_##_name = { \
- .name = #_name, \
- .ops = &clk_prcc_ops, \
- .cluster = _pclust, \
- .prcc_bus = _bus_en, \
- .prcc_kernel = _kernel_en, \
- .parent_cluster = &clk_per##_pclust##clk, \
- .parent_periph = _kernclk \
- }
-
-#define DEFINE_PRCC_CLK_CUSTOM(_pclust, _name, _bus_en, _kernel_en, _kernclk, _callback, _data) \
-struct clk clk_##_name = { \
- .name = #_name, \
- .ops = &clk_prcc_ops, \
- .cluster = _pclust, \
- .prcc_bus = _bus_en, \
- .prcc_kernel = _kernel_en, \
- .parent_cluster = &clk_per##_pclust##clk, \
- .parent_periph = _kernclk, \
- .get_rate = _callback, \
- .data = (void *) _data \
- }
-
-
-#define CLK(_clk, _devname, _conname) \
- { \
- .clk = &clk_##_clk, \
- .dev_id = _devname, \
- .con_id = _conname, \
- }
-
-int __init clk_db8500_ed_fixup(void);
-int __init clk_init(void);
-
-#ifdef CONFIG_DEBUG_FS
-int clk_debugfs_init(void);
-#else
-static inline int clk_debugfs_init(void) { return 0; }
-#endif
-
-#ifdef CONFIG_CPU_FREQ
-int clk_init_smp_twd_cpufreq(void);
-#else
-static inline int clk_init_smp_twd_cpufreq(void) { return 0; }
-#endif
diff --git a/arch/arm/mach-ux500/cpu.c b/arch/arm/mach-ux500/cpu.c
index e2360e7..17a78ec 100644
--- a/arch/arm/mach-ux500/cpu.c
+++ b/arch/arm/mach-ux500/cpu.c
@@ -8,7 +8,6 @@
#include <linux/platform_device.h>
#include <linux/io.h>
-#include <linux/clk.h>
#include <linux/mfd/db8500-prcmu.h>
#include <linux/clksrc-dbx500-prcmu.h>
#include <linux/sys_soc.h>
@@ -17,6 +16,7 @@
#include <linux/stat.h>
#include <linux/of.h>
#include <linux/of_irq.h>
+#include <linux/platform_data/clk-ux500.h>
#include <asm/hardware/gic.h>
#include <asm/mach/map.h>
@@ -25,8 +25,6 @@
#include <mach/setup.h>
#include <mach/devices.h>
-#include "clock.h"
-
void __iomem *_PRCMU_BASE;
/*
@@ -70,13 +68,17 @@
*/
if (cpu_is_u8500_family())
db8500_prcmu_early_init();
- clk_init();
+
+ if (cpu_is_u8500_family())
+ u8500_clk_init();
+ else if (cpu_is_u9540())
+ u9540_clk_init();
+ else if (cpu_is_u8540())
+ u8540_clk_init();
}
void __init ux500_init_late(void)
{
- clk_debugfs_init();
- clk_init_smp_twd_cpufreq();
}
static const char * __init ux500_get_machine(void)
diff --git a/arch/arm/mm/context.c b/arch/arm/mm/context.c
index 119bc52..4e07eec 100644
--- a/arch/arm/mm/context.c
+++ b/arch/arm/mm/context.c
@@ -63,10 +63,11 @@
pid = task_pid_nr(thread->task) << ASID_BITS;
asm volatile(
" mrc p15, 0, %0, c13, c0, 1\n"
- " bfi %1, %0, #0, %2\n"
- " mcr p15, 0, %1, c13, c0, 1\n"
+ " and %0, %0, %2\n"
+ " orr %0, %0, %1\n"
+ " mcr p15, 0, %0, c13, c0, 1\n"
: "=r" (contextidr), "+r" (pid)
- : "I" (ASID_BITS));
+ : "I" (~ASID_MASK));
isb();
return NOTIFY_OK;
diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 4e7d118..13f555d 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -267,17 +267,19 @@
vunmap(cpu_addr);
}
+#define DEFAULT_DMA_COHERENT_POOL_SIZE SZ_256K
+
struct dma_pool {
size_t size;
spinlock_t lock;
unsigned long *bitmap;
unsigned long nr_pages;
void *vaddr;
- struct page *page;
+ struct page **pages;
};
static struct dma_pool atomic_pool = {
- .size = SZ_256K,
+ .size = DEFAULT_DMA_COHERENT_POOL_SIZE,
};
static int __init early_coherent_pool(char *p)
@@ -287,6 +289,21 @@
}
early_param("coherent_pool", early_coherent_pool);
+void __init init_dma_coherent_pool_size(unsigned long size)
+{
+ /*
+ * Catch any attempt to set the pool size too late.
+ */
+ BUG_ON(atomic_pool.vaddr);
+
+ /*
+ * Set architecture specific coherent pool size only if
+ * it has not been changed by kernel command line parameter.
+ */
+ if (atomic_pool.size == DEFAULT_DMA_COHERENT_POOL_SIZE)
+ atomic_pool.size = size;
+}
+
/*
* Initialise the coherent pool for atomic allocations.
*/
@@ -297,6 +314,7 @@
unsigned long nr_pages = pool->size >> PAGE_SHIFT;
unsigned long *bitmap;
struct page *page;
+ struct page **pages;
void *ptr;
int bitmap_size = BITS_TO_LONGS(nr_pages) * sizeof(long);
@@ -304,21 +322,33 @@
if (!bitmap)
goto no_bitmap;
+ pages = kzalloc(nr_pages * sizeof(struct page *), GFP_KERNEL);
+ if (!pages)
+ goto no_pages;
+
if (IS_ENABLED(CONFIG_CMA))
ptr = __alloc_from_contiguous(NULL, pool->size, prot, &page);
else
ptr = __alloc_remap_buffer(NULL, pool->size, GFP_KERNEL, prot,
&page, NULL);
if (ptr) {
+ int i;
+
+ for (i = 0; i < nr_pages; i++)
+ pages[i] = page + i;
+
spin_lock_init(&pool->lock);
pool->vaddr = ptr;
- pool->page = page;
+ pool->pages = pages;
pool->bitmap = bitmap;
pool->nr_pages = nr_pages;
pr_info("DMA: preallocated %u KiB pool for atomic coherent allocations\n",
(unsigned)pool->size / 1024);
return 0;
}
+
+ kfree(pages);
+no_pages:
kfree(bitmap);
no_bitmap:
pr_err("DMA: failed to allocate %u KiB pool for atomic coherent allocation\n",
@@ -443,27 +473,45 @@
if (pageno < pool->nr_pages) {
bitmap_set(pool->bitmap, pageno, count);
ptr = pool->vaddr + PAGE_SIZE * pageno;
- *ret_page = pool->page + pageno;
+ *ret_page = pool->pages[pageno];
+ } else {
+ pr_err_once("ERROR: %u KiB atomic DMA coherent pool is too small!\n"
+ "Please increase it with coherent_pool= kernel parameter!\n",
+ (unsigned)pool->size / 1024);
}
spin_unlock_irqrestore(&pool->lock, flags);
return ptr;
}
+static bool __in_atomic_pool(void *start, size_t size)
+{
+ struct dma_pool *pool = &atomic_pool;
+ void *end = start + size;
+ void *pool_start = pool->vaddr;
+ void *pool_end = pool->vaddr + pool->size;
+
+ if (start < pool_start || start >= pool_end)
+ return false;
+
+ if (end <= pool_end)
+ return true;
+
+ WARN(1, "Wrong coherent size(%p-%p) from atomic pool(%p-%p)\n",
+ start, end - 1, pool_start, pool_end - 1);
+
+ return false;
+}
+
static int __free_from_pool(void *start, size_t size)
{
struct dma_pool *pool = &atomic_pool;
unsigned long pageno, count;
unsigned long flags;
- if (start < pool->vaddr || start > pool->vaddr + pool->size)
+ if (!__in_atomic_pool(start, size))
return 0;
- if (start + size > pool->vaddr + pool->size) {
- WARN(1, "freeing wrong coherent size from pool\n");
- return 0;
- }
-
pageno = (start - pool->vaddr) >> PAGE_SHIFT;
count = size >> PAGE_SHIFT;
@@ -1090,10 +1138,22 @@
return 0;
}
+static struct page **__atomic_get_pages(void *addr)
+{
+ struct dma_pool *pool = &atomic_pool;
+ struct page **pages = pool->pages;
+ int offs = (addr - pool->vaddr) >> PAGE_SHIFT;
+
+ return pages + offs;
+}
+
static struct page **__iommu_get_pages(void *cpu_addr, struct dma_attrs *attrs)
{
struct vm_struct *area;
+ if (__in_atomic_pool(cpu_addr, PAGE_SIZE))
+ return __atomic_get_pages(cpu_addr);
+
if (dma_get_attr(DMA_ATTR_NO_KERNEL_MAPPING, attrs))
return cpu_addr;
@@ -1103,6 +1163,34 @@
return NULL;
}
+static void *__iommu_alloc_atomic(struct device *dev, size_t size,
+ dma_addr_t *handle)
+{
+ struct page *page;
+ void *addr;
+
+ addr = __alloc_from_pool(size, &page);
+ if (!addr)
+ return NULL;
+
+ *handle = __iommu_create_mapping(dev, &page, size);
+ if (*handle == DMA_ERROR_CODE)
+ goto err_mapping;
+
+ return addr;
+
+err_mapping:
+ __free_from_pool(addr, size);
+ return NULL;
+}
+
+static void __iommu_free_atomic(struct device *dev, struct page **pages,
+ dma_addr_t handle, size_t size)
+{
+ __iommu_remove_mapping(dev, handle, size);
+ __free_from_pool(page_address(pages[0]), size);
+}
+
static void *arm_iommu_alloc_attrs(struct device *dev, size_t size,
dma_addr_t *handle, gfp_t gfp, struct dma_attrs *attrs)
{
@@ -1113,6 +1201,9 @@
*handle = DMA_ERROR_CODE;
size = PAGE_ALIGN(size);
+ if (gfp & GFP_ATOMIC)
+ return __iommu_alloc_atomic(dev, size, handle);
+
pages = __iommu_alloc_buffer(dev, size, gfp);
if (!pages)
return NULL;
@@ -1179,6 +1270,11 @@
return;
}
+ if (__in_atomic_pool(cpu_addr, size)) {
+ __iommu_free_atomic(dev, pages, handle, size);
+ return;
+ }
+
if (!dma_get_attr(DMA_ATTR_NO_KERNEL_MAPPING, attrs)) {
unmap_kernel_range((unsigned long)cpu_addr, size);
vunmap(cpu_addr);
diff --git a/arch/arm/mm/mm.h b/arch/arm/mm/mm.h
index 6776160..a8ee92d 100644
--- a/arch/arm/mm/mm.h
+++ b/arch/arm/mm/mm.h
@@ -55,6 +55,9 @@
/* permanent static mappings from iotable_init() */
#define VM_ARM_STATIC_MAPPING 0x40000000
+/* empty mapping */
+#define VM_ARM_EMPTY_MAPPING 0x20000000
+
/* mapping type (attributes) for permanent static mappings */
#define VM_ARM_MTYPE(mt) ((mt) << 20)
#define VM_ARM_MTYPE_MASK (0x1f << 20)
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index 4c2d045..c2fa21d 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -807,7 +807,7 @@
vm = early_alloc_aligned(sizeof(*vm), __alignof__(*vm));
vm->addr = (void *)addr;
vm->size = SECTION_SIZE;
- vm->flags = VM_IOREMAP | VM_ARM_STATIC_MAPPING;
+ vm->flags = VM_IOREMAP | VM_ARM_EMPTY_MAPPING;
vm->caller = pmd_empty_section_gap;
vm_area_add_early(vm);
}
@@ -820,7 +820,7 @@
/* we're still single threaded hence no lock needed here */
for (vm = vmlist; vm; vm = vm->next) {
- if (!(vm->flags & VM_ARM_STATIC_MAPPING))
+ if (!(vm->flags & (VM_ARM_STATIC_MAPPING | VM_ARM_EMPTY_MAPPING)))
continue;
addr = (unsigned long)vm->addr;
if (addr < next)
@@ -961,8 +961,8 @@
* Check whether this memory bank would partially overlap
* the vmalloc area.
*/
- if (__va(bank->start + bank->size) > vmalloc_min ||
- __va(bank->start + bank->size) < __va(bank->start)) {
+ if (__va(bank->start + bank->size - 1) >= vmalloc_min ||
+ __va(bank->start + bank->size - 1) <= __va(bank->start)) {
unsigned long newsize = vmalloc_min - __va(bank->start);
printk(KERN_NOTICE "Truncating RAM at %.8llx-%.8llx "
"to -%.8llx (vmalloc region overlap).\n",
diff --git a/arch/arm/plat-mxc/include/mach/mx25.h b/arch/arm/plat-mxc/include/mach/mx25.h
index 627d94f..ec46640 100644
--- a/arch/arm/plat-mxc/include/mach/mx25.h
+++ b/arch/arm/plat-mxc/include/mach/mx25.h
@@ -98,6 +98,7 @@
#define MX25_INT_UART1 (NR_IRQS_LEGACY + 45)
#define MX25_INT_GPIO2 (NR_IRQS_LEGACY + 51)
#define MX25_INT_GPIO1 (NR_IRQS_LEGACY + 52)
+#define MX25_INT_GPT1 (NR_IRQS_LEGACY + 54)
#define MX25_INT_FEC (NR_IRQS_LEGACY + 57)
#define MX25_DMA_REQ_SSI2_RX1 22
diff --git a/arch/arm/plat-omap/dmtimer.c b/arch/arm/plat-omap/dmtimer.c
index 626ad8c..938b50a 100644
--- a/arch/arm/plat-omap/dmtimer.c
+++ b/arch/arm/plat-omap/dmtimer.c
@@ -189,6 +189,7 @@
timer->reserved = 1;
break;
}
+ spin_unlock_irqrestore(&dm_timer_lock, flags);
if (timer) {
ret = omap_dm_timer_prepare(timer);
@@ -197,7 +198,6 @@
timer = NULL;
}
}
- spin_unlock_irqrestore(&dm_timer_lock, flags);
if (!timer)
pr_debug("%s: timer request failed!\n", __func__);
@@ -220,6 +220,7 @@
break;
}
}
+ spin_unlock_irqrestore(&dm_timer_lock, flags);
if (timer) {
ret = omap_dm_timer_prepare(timer);
@@ -228,7 +229,6 @@
timer = NULL;
}
}
- spin_unlock_irqrestore(&dm_timer_lock, flags);
if (!timer)
pr_debug("%s: timer%d request failed!\n", __func__, id);
@@ -258,7 +258,7 @@
void omap_dm_timer_disable(struct omap_dm_timer *timer)
{
- pm_runtime_put(&timer->pdev->dev);
+ pm_runtime_put_sync(&timer->pdev->dev);
}
EXPORT_SYMBOL_GPL(omap_dm_timer_disable);
diff --git a/arch/arm/plat-omap/include/plat/cpu.h b/arch/arm/plat-omap/include/plat/cpu.h
index 68b180e..bb5d08a 100644
--- a/arch/arm/plat-omap/include/plat/cpu.h
+++ b/arch/arm/plat-omap/include/plat/cpu.h
@@ -372,7 +372,8 @@
#define cpu_class_is_omap1() (cpu_is_omap7xx() || cpu_is_omap15xx() || \
cpu_is_omap16xx())
#define cpu_class_is_omap2() (cpu_is_omap24xx() || cpu_is_omap34xx() || \
- cpu_is_omap44xx() || soc_is_omap54xx())
+ cpu_is_omap44xx() || soc_is_omap54xx() || \
+ soc_is_am33xx())
/* Various silicon revisions for omap2 */
#define OMAP242X_CLASS 0x24200024
diff --git a/arch/arm/plat-omap/include/plat/multi.h b/arch/arm/plat-omap/include/plat/multi.h
index 045e320..324d31b 100644
--- a/arch/arm/plat-omap/include/plat/multi.h
+++ b/arch/arm/plat-omap/include/plat/multi.h
@@ -108,4 +108,13 @@
# endif
#endif
+#ifdef CONFIG_SOC_AM33XX
+# ifdef OMAP_NAME
+# undef MULTI_OMAP2
+# define MULTI_OMAP2
+# else
+# define OMAP_NAME am33xx
+# endif
+#endif
+
#endif /* __PLAT_OMAP_MULTI_H */
diff --git a/arch/arm/plat-omap/include/plat/uncompress.h b/arch/arm/plat-omap/include/plat/uncompress.h
index b8d19a1..7f7b112 100644
--- a/arch/arm/plat-omap/include/plat/uncompress.h
+++ b/arch/arm/plat-omap/include/plat/uncompress.h
@@ -110,7 +110,7 @@
_DEBUG_LL_ENTRY(mach, AM33XX_UART##p##_BASE, OMAP_PORT_SHIFT, \
AM33XXUART##p)
-static inline void __arch_decomp_setup(unsigned long arch_id)
+static inline void arch_decomp_setup(void)
{
int port = 0;
@@ -198,8 +198,6 @@
} while (0);
}
-#define arch_decomp_setup() __arch_decomp_setup(arch_id)
-
/*
* nothing to do
*/
diff --git a/arch/arm/plat-omap/sram.c b/arch/arm/plat-omap/sram.c
index 766181c..024f3b0 100644
--- a/arch/arm/plat-omap/sram.c
+++ b/arch/arm/plat-omap/sram.c
@@ -68,6 +68,7 @@
static unsigned long omap_sram_start;
static void __iomem *omap_sram_base;
+static unsigned long omap_sram_skip;
static unsigned long omap_sram_size;
static void __iomem *omap_sram_ceil;
@@ -106,6 +107,7 @@
*/
static void __init omap_detect_sram(void)
{
+ omap_sram_skip = SRAM_BOOTLOADER_SZ;
if (cpu_class_is_omap2()) {
if (is_sram_locked()) {
if (cpu_is_omap34xx()) {
@@ -113,6 +115,7 @@
if ((omap_type() == OMAP2_DEVICE_TYPE_EMU) ||
(omap_type() == OMAP2_DEVICE_TYPE_SEC)) {
omap_sram_size = 0x7000; /* 28K */
+ omap_sram_skip += SZ_16K;
} else {
omap_sram_size = 0x8000; /* 32K */
}
@@ -175,8 +178,10 @@
return;
#ifdef CONFIG_OMAP4_ERRATA_I688
+ if (cpu_is_omap44xx()) {
omap_sram_start += PAGE_SIZE;
omap_sram_size -= SZ_16K;
+ }
#endif
if (cpu_is_omap34xx()) {
/*
@@ -203,8 +208,8 @@
* Looks like we need to preserve some bootloader code at the
* beginning of SRAM for jumping to flash for reboot to work...
*/
- memset_io(omap_sram_base + SRAM_BOOTLOADER_SZ, 0,
- omap_sram_size - SRAM_BOOTLOADER_SZ);
+ memset_io(omap_sram_base + omap_sram_skip, 0,
+ omap_sram_size - omap_sram_skip);
}
/*
@@ -218,7 +223,7 @@
{
unsigned long available, new_ceil = (unsigned long)omap_sram_ceil;
- available = omap_sram_ceil - (omap_sram_base + SRAM_BOOTLOADER_SZ);
+ available = omap_sram_ceil - (omap_sram_base + omap_sram_skip);
if (size > available) {
pr_err("Not enough space in SRAM\n");
diff --git a/arch/arm/plat-orion/common.c b/arch/arm/plat-orion/common.c
index d245a87..b8b747a 100644
--- a/arch/arm/plat-orion/common.c
+++ b/arch/arm/plat-orion/common.c
@@ -291,10 +291,12 @@
void __init orion_ge00_init(struct mv643xx_eth_platform_data *eth_data,
unsigned long mapbase,
unsigned long irq,
- unsigned long irq_err)
+ unsigned long irq_err,
+ unsigned int tx_csum_limit)
{
fill_resources(&orion_ge00_shared, orion_ge00_shared_resources,
mapbase + 0x2000, SZ_16K - 1, irq_err);
+ orion_ge00_shared_data.tx_csum_limit = tx_csum_limit;
ge_complete(&orion_ge00_shared_data,
orion_ge00_resources, irq, &orion_ge00_shared,
eth_data, &orion_ge00);
@@ -343,10 +345,12 @@
void __init orion_ge01_init(struct mv643xx_eth_platform_data *eth_data,
unsigned long mapbase,
unsigned long irq,
- unsigned long irq_err)
+ unsigned long irq_err,
+ unsigned int tx_csum_limit)
{
fill_resources(&orion_ge01_shared, orion_ge01_shared_resources,
mapbase + 0x2000, SZ_16K - 1, irq_err);
+ orion_ge01_shared_data.tx_csum_limit = tx_csum_limit;
ge_complete(&orion_ge01_shared_data,
orion_ge01_resources, irq, &orion_ge01_shared,
eth_data, &orion_ge01);
diff --git a/arch/arm/plat-orion/include/plat/common.h b/arch/arm/plat-orion/include/plat/common.h
index e00fdb2..ae2377e 100644
--- a/arch/arm/plat-orion/include/plat/common.h
+++ b/arch/arm/plat-orion/include/plat/common.h
@@ -39,12 +39,14 @@
void __init orion_ge00_init(struct mv643xx_eth_platform_data *eth_data,
unsigned long mapbase,
unsigned long irq,
- unsigned long irq_err);
+ unsigned long irq_err,
+ unsigned int tx_csum_limit);
void __init orion_ge01_init(struct mv643xx_eth_platform_data *eth_data,
unsigned long mapbase,
unsigned long irq,
- unsigned long irq_err);
+ unsigned long irq_err,
+ unsigned int tx_csum_limit);
void __init orion_ge10_init(struct mv643xx_eth_platform_data *eth_data,
unsigned long mapbase,
diff --git a/arch/arm/plat-s3c24xx/dma.c b/arch/arm/plat-s3c24xx/dma.c
index 28f898f..db98e70 100644
--- a/arch/arm/plat-s3c24xx/dma.c
+++ b/arch/arm/plat-s3c24xx/dma.c
@@ -430,7 +430,7 @@
* when necessary.
*/
-int s3c2410_dma_enqueue(unsigned int channel, void *id,
+int s3c2410_dma_enqueue(enum dma_ch channel, void *id,
dma_addr_t data, int size)
{
struct s3c2410_dma_chan *chan = s3c_dma_lookup_channel(channel);
diff --git a/arch/arm/plat-samsung/clock.c b/arch/arm/plat-samsung/clock.c
index 65c5eca..d1116e2 100644
--- a/arch/arm/plat-samsung/clock.c
+++ b/arch/arm/plat-samsung/clock.c
@@ -144,6 +144,7 @@
int clk_set_rate(struct clk *clk, unsigned long rate)
{
+ unsigned long flags;
int ret;
if (IS_ERR(clk))
@@ -159,9 +160,9 @@
if (clk->ops == NULL || clk->ops->set_rate == NULL)
return -EINVAL;
- spin_lock(&clocks_lock);
+ spin_lock_irqsave(&clocks_lock, flags);
ret = (clk->ops->set_rate)(clk, rate);
- spin_unlock(&clocks_lock);
+ spin_unlock_irqrestore(&clocks_lock, flags);
return ret;
}
@@ -173,17 +174,18 @@
int clk_set_parent(struct clk *clk, struct clk *parent)
{
+ unsigned long flags;
int ret = 0;
if (IS_ERR(clk))
return -EINVAL;
- spin_lock(&clocks_lock);
+ spin_lock_irqsave(&clocks_lock, flags);
if (clk->ops && clk->ops->set_parent)
ret = (clk->ops->set_parent)(clk, parent);
- spin_unlock(&clocks_lock);
+ spin_unlock_irqrestore(&clocks_lock, flags);
return ret;
}
diff --git a/arch/arm/plat-samsung/devs.c b/arch/arm/plat-samsung/devs.c
index 74e31ce..fc49f3d 100644
--- a/arch/arm/plat-samsung/devs.c
+++ b/arch/arm/plat-samsung/devs.c
@@ -32,6 +32,8 @@
#include <linux/platform_data/s3c-hsudc.h>
#include <linux/platform_data/s3c-hsotg.h>
+#include <media/s5p_hdmi.h>
+
#include <asm/irq.h>
#include <asm/pmu.h>
#include <asm/mach/arch.h>
@@ -748,7 +750,8 @@
if (!pd) {
pd = &default_i2c_data;
- if (soc_is_exynos4210())
+ if (soc_is_exynos4210() ||
+ soc_is_exynos4212() || soc_is_exynos4412())
pd->bus_num = 8;
else if (soc_is_s5pv210())
pd->bus_num = 3;
@@ -759,6 +762,30 @@
npd = s3c_set_platdata(pd, sizeof(struct s3c2410_platform_i2c),
&s5p_device_i2c_hdmiphy);
}
+
+struct s5p_hdmi_platform_data s5p_hdmi_def_platdata;
+
+void __init s5p_hdmi_set_platdata(struct i2c_board_info *hdmiphy_info,
+ struct i2c_board_info *mhl_info, int mhl_bus)
+{
+ struct s5p_hdmi_platform_data *pd = &s5p_hdmi_def_platdata;
+
+ if (soc_is_exynos4210() ||
+ soc_is_exynos4212() || soc_is_exynos4412())
+ pd->hdmiphy_bus = 8;
+ else if (soc_is_s5pv210())
+ pd->hdmiphy_bus = 3;
+ else
+ pd->hdmiphy_bus = 0;
+
+ pd->hdmiphy_info = hdmiphy_info;
+ pd->mhl_info = mhl_info;
+ pd->mhl_bus = mhl_bus;
+
+ s3c_set_platdata(pd, sizeof(struct s5p_hdmi_platform_data),
+ &s5p_device_hdmi);
+}
+
#endif /* CONFIG_S5P_DEV_I2C_HDMIPHY */
/* I2S */
diff --git a/arch/arm/plat-samsung/include/plat/hdmi.h b/arch/arm/plat-samsung/include/plat/hdmi.h
new file mode 100644
index 0000000..331d046
--- /dev/null
+++ b/arch/arm/plat-samsung/include/plat/hdmi.h
@@ -0,0 +1,16 @@
+/*
+ * Copyright (C) 2012 Samsung Electronics Co.Ltd
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef __PLAT_SAMSUNG_HDMI_H
+#define __PLAT_SAMSUNG_HDMI_H __FILE__
+
+extern void s5p_hdmi_set_platdata(struct i2c_board_info *hdmiphy_info,
+ struct i2c_board_info *mhl_info, int mhl_bus);
+
+#endif /* __PLAT_SAMSUNG_HDMI_H */
diff --git a/arch/arm/plat-samsung/pm.c b/arch/arm/plat-samsung/pm.c
index 64ab65f..1507028 100644
--- a/arch/arm/plat-samsung/pm.c
+++ b/arch/arm/plat-samsung/pm.c
@@ -74,7 +74,7 @@
#ifdef CONFIG_SAMSUNG_PM_DEBUG
-struct pm_uart_save uart_save[CONFIG_SERIAL_SAMSUNG_UARTS];
+static struct pm_uart_save uart_save[CONFIG_SERIAL_SAMSUNG_UARTS];
static void s3c_pm_save_uart(unsigned int uart, struct pm_uart_save *save)
{
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
new file mode 100644
index 0000000..767ba56
--- /dev/null
+++ b/arch/arm64/Kconfig
@@ -0,0 +1,222 @@
+config ARM64
+ def_bool y
+ select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
+ select GENERIC_CLOCKEVENTS
+ select GENERIC_HARDIRQS_NO_DEPRECATED
+ select GENERIC_IOMAP
+ select GENERIC_IRQ_PROBE
+ select GENERIC_IRQ_SHOW
+ select GENERIC_SMP_IDLE_THREAD
+ select GENERIC_TIME_VSYSCALL
+ select HARDIRQS_SW_RESEND
+ select HAVE_ARCH_TRACEHOOK
+ select HAVE_DMA_API_DEBUG
+ select HAVE_DMA_ATTRS
+ select HAVE_GENERIC_DMA_COHERENT
+ select HAVE_GENERIC_HARDIRQS
+ select HAVE_HW_BREAKPOINT if PERF_EVENTS
+ select HAVE_IRQ_WORK
+ select HAVE_MEMBLOCK
+ select HAVE_PERF_EVENTS
+ select HAVE_SPARSE_IRQ
+ select IRQ_DOMAIN
+ select NO_BOOTMEM
+ select OF
+ select OF_EARLY_FLATTREE
+ select PERF_USE_VMALLOC
+ select RTC_LIB
+ select SPARSE_IRQ
+ help
+ ARM 64-bit (AArch64) Linux support.
+
+config 64BIT
+ def_bool y
+
+config ARCH_PHYS_ADDR_T_64BIT
+ def_bool y
+
+config MMU
+ def_bool y
+
+config NO_IOPORT
+ def_bool y
+
+config STACKTRACE_SUPPORT
+ def_bool y
+
+config LOCKDEP_SUPPORT
+ def_bool y
+
+config TRACE_IRQFLAGS_SUPPORT
+ def_bool y
+
+config GENERIC_LOCKBREAK
+ def_bool y
+ depends on SMP && PREEMPT
+
+config RWSEM_GENERIC_SPINLOCK
+ def_bool y
+
+config GENERIC_HWEIGHT
+ def_bool y
+
+config GENERIC_CSUM
+ def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+ def_bool y
+
+config ZONE_DMA32
+ def_bool y
+
+config ARCH_DMA_ADDR_T_64BIT
+ def_bool y
+
+config NEED_DMA_MAP_STATE
+ def_bool y
+
+config NEED_SG_DMA_LENGTH
+ def_bool y
+
+config SWIOTLB
+ def_bool y
+
+config IOMMU_HELPER
+ def_bool SWIOTLB
+
+source "init/Kconfig"
+
+source "kernel/Kconfig.freezer"
+
+menu "System Type"
+
+endmenu
+
+menu "Bus support"
+
+config ARM_AMBA
+ bool
+
+endmenu
+
+menu "Kernel Features"
+
+source "kernel/time/Kconfig"
+
+config ARM64_64K_PAGES
+ bool "Enable 64KB pages support"
+ help
+ This feature enables 64KB pages support (4KB by default)
+ allowing only two levels of page tables and faster TLB
+ look-up. AArch32 emulation is not available when this feature
+ is enabled.
+
+config SMP
+ bool "Symmetric Multi-Processing"
+ select USE_GENERIC_SMP_HELPERS
+ help
+ This enables support for systems with more than one CPU. If
+ you say N here, the kernel will run on single and
+ multiprocessor machines, but will use only one CPU of a
+ multiprocessor machine. If you say Y here, the kernel will run
+ on many, but not all, single processor machines. On a single
+ processor machine, the kernel will run faster if you say N
+ here.
+
+ If you don't know what to do here, say N.
+
+config NR_CPUS
+ int "Maximum number of CPUs (2-32)"
+ range 2 32
+ depends on SMP
+ default "4"
+
+source kernel/Kconfig.preempt
+
+config HZ
+ int
+ default 100
+
+config ARCH_HAS_HOLES_MEMORYMODEL
+ def_bool y if SPARSEMEM
+
+config ARCH_SPARSEMEM_ENABLE
+ def_bool y
+ select SPARSEMEM_VMEMMAP_ENABLE
+
+config ARCH_SPARSEMEM_DEFAULT
+ def_bool ARCH_SPARSEMEM_ENABLE
+
+config ARCH_SELECT_MEMORY_MODEL
+ def_bool ARCH_SPARSEMEM_ENABLE
+
+config HAVE_ARCH_PFN_VALID
+ def_bool ARCH_HAS_HOLES_MEMORYMODEL || !SPARSEMEM
+
+config HW_PERF_EVENTS
+ bool "Enable hardware performance counter support for perf events"
+ depends on PERF_EVENTS
+ default y
+ help
+ Enable hardware performance counter support for perf events. If
+ disabled, perf events will use software events only.
+
+source "mm/Kconfig"
+
+endmenu
+
+menu "Boot options"
+
+config CMDLINE
+ string "Default kernel command string"
+ default ""
+ help
+ Provide a set of default command-line options at build time by
+ entering them here. As a minimum, you should specify the the
+ root device (e.g. root=/dev/nfs).
+
+config CMDLINE_FORCE
+ bool "Always use the default kernel command string"
+ help
+ Always use the default kernel command string, even if the boot
+ loader passes other arguments to the kernel.
+ This is useful if you cannot or don't want to change the
+ command-line options your boot loader passes to the kernel.
+
+endmenu
+
+menu "Userspace binary formats"
+
+source "fs/Kconfig.binfmt"
+
+config COMPAT
+ bool "Kernel support for 32-bit EL0"
+ depends on !ARM64_64K_PAGES
+ select COMPAT_BINFMT_ELF
+ help
+ This option enables support for a 32-bit EL0 running under a 64-bit
+ kernel at EL1. AArch32-specific components such as system calls,
+ the user helper functions, VFP support and the ptrace interface are
+ handled appropriately by the kernel.
+
+ If you want to execute 32-bit userspace applications, say Y.
+
+config SYSVIPC_COMPAT
+ def_bool y
+ depends on COMPAT && SYSVIPC
+
+endmenu
+
+source "net/Kconfig"
+
+source "drivers/Kconfig"
+
+source "fs/Kconfig"
+
+source "arch/arm64/Kconfig.debug"
+
+source "security/Kconfig"
+
+source "crypto/Kconfig"
+
+source "lib/Kconfig"
diff --git a/arch/arm64/Kconfig.debug b/arch/arm64/Kconfig.debug
new file mode 100644
index 0000000..d7553f2
--- /dev/null
+++ b/arch/arm64/Kconfig.debug
@@ -0,0 +1,27 @@
+menu "Kernel hacking"
+
+source "lib/Kconfig.debug"
+
+config FRAME_POINTER
+ bool
+ default y
+
+config DEBUG_ERRORS
+ bool "Verbose kernel error messages"
+ depends on DEBUG_KERNEL
+ help
+ This option controls verbose debugging information which can be
+ printed when the kernel detects an internal error. This debugging
+ information is useful to kernel hackers when tracking down problems,
+ but mostly meaningless to other people. It's safe to say Y unless
+ you are concerned with the code size or don't want to see these
+ messages.
+
+config DEBUG_STACK_USAGE
+ bool "Enable stack utilization instrumentation"
+ depends on DEBUG_KERNEL
+ help
+ Enables the display of the minimum amount of free stack which each
+ task has ever had available in the sysrq-T output.
+
+endmenu
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
new file mode 100644
index 0000000..364191f
--- /dev/null
+++ b/arch/arm64/Makefile
@@ -0,0 +1,71 @@
+#
+# arch/arm64/Makefile
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License. See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 1995-2001 by Russell King
+
+LDFLAGS_vmlinux :=-p --no-undefined -X
+CPPFLAGS_vmlinux.lds = -DTEXT_OFFSET=$(TEXT_OFFSET)
+OBJCOPYFLAGS :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
+GZFLAGS :=-9
+
+LIBGCC := $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name)
+
+KBUILD_DEFCONFIG := defconfig
+
+KBUILD_CFLAGS += -mgeneral-regs-only
+KBUILD_CPPFLAGS += -mlittle-endian
+AS += -EL
+LD += -EL
+
+comma = ,
+
+CHECKFLAGS += -D__aarch64__
+
+# Default value
+head-y := arch/arm64/kernel/head.o
+
+# The byte offset of the kernel image in RAM from the start of RAM.
+TEXT_OFFSET := 0x00080000
+
+export TEXT_OFFSET GZFLAGS
+
+core-y += arch/arm64/kernel/ arch/arm64/mm/
+libs-y := arch/arm64/lib/ $(libs-y)
+libs-y += $(LIBGCC)
+
+# Default target when executing plain make
+KBUILD_IMAGE := Image.gz
+
+all: $(KBUILD_IMAGE)
+
+boot := arch/arm64/boot
+
+Image Image.gz: vmlinux
+ $(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $(boot)/$@
+
+zinstall install: vmlinux
+ $(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $@
+
+%.dtb:
+ $(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $(boot)/$@
+
+# We use MRPROPER_FILES and CLEAN_FILES now
+archclean:
+ $(Q)$(MAKE) $(clean)=$(boot)
+
+define archhelp
+ echo '* Image.gz - Compressed kernel image (arch/$(ARCH)/boot/Image.gz)'
+ echo ' Image - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
+ echo ' install - Install uncompressed kernel'
+ echo ' zinstall - Install compressed kernel'
+ echo ' Install using (your) ~/bin/installkernel or'
+ echo ' (distribution) /sbin/installkernel or'
+ echo ' install to $$(INSTALL_PATH) and run lilo'
+endef
diff --git a/arch/arm64/boot/.gitignore b/arch/arm64/boot/.gitignore
new file mode 100644
index 0000000..8dab0bb
--- /dev/null
+++ b/arch/arm64/boot/.gitignore
@@ -0,0 +1,2 @@
+Image
+Image.gz
diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
new file mode 100644
index 0000000..eca209b
--- /dev/null
+++ b/arch/arm64/boot/Makefile
@@ -0,0 +1,36 @@
+#
+# arch/arm64/boot/Makefile
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License. See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 2012, ARM Ltd.
+# Author: Will Deacon <will.deacon@arm.com>
+#
+# Based on the ia64 boot/Makefile.
+#
+
+targets := Image Image.gz
+
+$(obj)/Image: vmlinux FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/Image.gz: $(obj)/Image FORCE
+ $(call if_changed,gzip)
+
+$(obj)/%.dtb: $(src)/dts/%.dts
+ $(call cmd,dtc)
+
+install: $(obj)/Image
+ $(CONFIG_SHELL) $(srctree)/$(src)/install.sh $(KERNELRELEASE) \
+ $(obj)/Image System.map "$(INSTALL_PATH)"
+
+zinstall: $(obj)/Image.gz
+ $(CONFIG_SHELL) $(srctree)/$(src)/install.sh $(KERNELRELEASE) \
+ $(obj)/Image.gz System.map "$(INSTALL_PATH)"
+
+clean-files += *.dtb
diff --git a/arch/arm64/boot/install.sh b/arch/arm64/boot/install.sh
new file mode 100644
index 0000000..12ed78a
--- /dev/null
+++ b/arch/arm64/boot/install.sh
@@ -0,0 +1,46 @@
+#!/bin/sh
+#
+# arch/arm64/boot/install.sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License. See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 1995 by Linus Torvalds
+#
+# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin
+# Adapted from code in arch/i386/boot/install.sh by Russell King
+#
+# "make install" script for the AArch64 Linux port
+#
+# Arguments:
+# $1 - kernel version
+# $2 - kernel image file
+# $3 - kernel map file
+# $4 - default install path (blank if root directory)
+#
+
+# User may have a custom install script
+if [ -x ~/bin/${INSTALLKERNEL} ]; then exec ~/bin/${INSTALLKERNEL} "$@"; fi
+if [ -x /sbin/${INSTALLKERNEL} ]; then exec /sbin/${INSTALLKERNEL} "$@"; fi
+
+if [ "$(basename $2)" = "Image.gz" ]; then
+# Compressed install
+ echo "Installing compressed kernel"
+ base=vmlinuz
+else
+# Normal install
+ echo "Installing normal kernel"
+ base=vmlinux
+fi
+
+if [ -f $4/$base-$1 ]; then
+ mv $4/$base-$1 $4/$base-$1.old
+fi
+cat $2 > $4/$base-$1
+
+# Install system map file
+if [ -f $4/System.map-$1 ]; then
+ mv $4/System.map-$1 $4/System.map-$1.old
+fi
+cp $3 $4/System.map-$1
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
new file mode 100644
index 0000000..9212c78
--- /dev/null
+++ b/arch/arm64/configs/defconfig
@@ -0,0 +1,85 @@
+CONFIG_EXPERIMENTAL=y
+# CONFIG_LOCALVERSION_AUTO is not set
+# CONFIG_SWAP is not set
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_BSD_PROCESS_ACCT_V3=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_LOG_BUF_SHIFT=14
+# CONFIG_UTS_NS is not set
+# CONFIG_IPC_NS is not set
+# CONFIG_PID_NS is not set
+# CONFIG_NET_NS is not set
+CONFIG_SCHED_AUTOGROUP=y
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_KALLSYMS_ALL=y
+# CONFIG_COMPAT_BRK is not set
+CONFIG_PROFILING=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_IOSCHED_DEADLINE is not set
+CONFIG_SMP=y
+CONFIG_PREEMPT_VOLUNTARY=y
+CONFIG_CMDLINE="console=ttyAMA0"
+# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
+CONFIG_COMPAT=y
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_IP_PNP_BOOTP=y
+# CONFIG_INET_LRO is not set
+# CONFIG_IPV6 is not set
+# CONFIG_WIRELESS is not set
+CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
+# CONFIG_BLK_DEV is not set
+CONFIG_SCSI=y
+# CONFIG_SCSI_PROC_FS is not set
+CONFIG_BLK_DEV_SD=y
+# CONFIG_SCSI_LOWLEVEL is not set
+CONFIG_NETDEVICES=y
+CONFIG_MII=y
+# CONFIG_WLAN is not set
+CONFIG_INPUT_EVDEV=y
+# CONFIG_SERIO_I8042 is not set
+# CONFIG_SERIO_SERPORT is not set
+CONFIG_LEGACY_PTY_COUNT=16
+# CONFIG_HW_RANDOM is not set
+# CONFIG_HWMON is not set
+CONFIG_FB=y
+# CONFIG_VGA_CONSOLE is not set
+CONFIG_FRAMEBUFFER_CONSOLE=y
+CONFIG_LOGO=y
+# CONFIG_LOGO_LINUX_MONO is not set
+# CONFIG_LOGO_LINUX_VGA16 is not set
+# CONFIG_USB_SUPPORT is not set
+# CONFIG_IOMMU_SUPPORT is not set
+CONFIG_EXT2_FS=y
+CONFIG_EXT3_FS=y
+# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
+# CONFIG_EXT3_FS_XATTR is not set
+CONFIG_FUSE_FS=y
+CONFIG_CUSE=y
+CONFIG_VFAT_FS=y
+CONFIG_TMPFS=y
+# CONFIG_MISC_FILESYSTEMS is not set
+CONFIG_NFS_FS=y
+CONFIG_ROOT_NFS=y
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_DEBUG_FS=y
+CONFIG_DEBUG_KERNEL=y
+# CONFIG_SCHED_DEBUG is not set
+CONFIG_DEBUG_INFO=y
+# CONFIG_FTRACE is not set
+CONFIG_ATOMIC64_SELFTEST=y
+CONFIG_DEBUG_ERRORS=y
diff --git a/arch/arm64/include/asm/Kbuild b/arch/arm64/include/asm/Kbuild
new file mode 100644
index 0000000..35924a5
--- /dev/null
+++ b/arch/arm64/include/asm/Kbuild
@@ -0,0 +1,51 @@
+include include/asm-generic/Kbuild.asm
+
+header-y += hwcap.h
+
+generic-y += bug.h
+generic-y += bugs.h
+generic-y += checksum.h
+generic-y += cputime.h
+generic-y += current.h
+generic-y += delay.h
+generic-y += div64.h
+generic-y += dma.h
+generic-y += emergency-restart.h
+generic-y += errno.h
+generic-y += ftrace.h
+generic-y += hw_irq.h
+generic-y += ioctl.h
+generic-y += ioctls.h
+generic-y += ipcbuf.h
+generic-y += irq_regs.h
+generic-y += kdebug.h
+generic-y += kmap_types.h
+generic-y += linkage.h
+generic-y += local.h
+generic-y += local64.h
+generic-y += mman.h
+generic-y += msgbuf.h
+generic-y += mutex.h
+generic-y += pci.h
+generic-y += percpu.h
+generic-y += poll.h
+generic-y += posix_types.h
+generic-y += resource.h
+generic-y += scatterlist.h
+generic-y += sections.h
+generic-y += segment.h
+generic-y += sembuf.h
+generic-y += serial.h
+generic-y += shmbuf.h
+generic-y += sizes.h
+generic-y += socket.h
+generic-y += sockios.h
+generic-y += string.h
+generic-y += switch_to.h
+generic-y += swab.h
+generic-y += termbits.h
+generic-y += termios.h
+generic-y += topology.h
+generic-y += types.h
+generic-y += unaligned.h
+generic-y += user.h
diff --git a/arch/arm64/include/asm/arm_generic.h b/arch/arm64/include/asm/arm_generic.h
new file mode 100644
index 0000000..e4cec9d
--- /dev/null
+++ b/arch/arm64/include/asm/arm_generic.h
@@ -0,0 +1,100 @@
+/*
+ * arch/arm64/include/asm/arm_generic.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_ARM_GENERIC_H
+#define __ASM_ARM_GENERIC_H
+
+#include <linux/clocksource.h>
+
+#define ARCH_TIMER_CTRL_ENABLE (1 << 0)
+#define ARCH_TIMER_CTRL_IMASK (1 << 1)
+#define ARCH_TIMER_CTRL_ISTATUS (1 << 2)
+
+#define ARCH_TIMER_REG_CTRL 0
+#define ARCH_TIMER_REG_FREQ 1
+#define ARCH_TIMER_REG_TVAL 2
+
+static inline void arch_timer_reg_write(int reg, u32 val)
+{
+ switch (reg) {
+ case ARCH_TIMER_REG_CTRL:
+ asm volatile("msr cntp_ctl_el0, %0" : : "r" (val));
+ break;
+ case ARCH_TIMER_REG_TVAL:
+ asm volatile("msr cntp_tval_el0, %0" : : "r" (val));
+ break;
+ default:
+ BUILD_BUG();
+ }
+
+ isb();
+}
+
+static inline u32 arch_timer_reg_read(int reg)
+{
+ u32 val;
+
+ switch (reg) {
+ case ARCH_TIMER_REG_CTRL:
+ asm volatile("mrs %0, cntp_ctl_el0" : "=r" (val));
+ break;
+ case ARCH_TIMER_REG_FREQ:
+ asm volatile("mrs %0, cntfrq_el0" : "=r" (val));
+ break;
+ case ARCH_TIMER_REG_TVAL:
+ asm volatile("mrs %0, cntp_tval_el0" : "=r" (val));
+ break;
+ default:
+ BUILD_BUG();
+ }
+
+ return val;
+}
+
+static inline void __cpuinit arch_counter_enable_user_access(void)
+{
+ u32 cntkctl;
+
+ /* Disable user access to the timers and the virtual counter. */
+ asm volatile("mrs %0, cntkctl_el1" : "=r" (cntkctl));
+ cntkctl &= ~((3 << 8) | (1 << 1));
+
+ /* Enable user access to the physical counter and frequency. */
+ cntkctl |= 1;
+ asm volatile("msr cntkctl_el1, %0" : : "r" (cntkctl));
+}
+
+static inline cycle_t arch_counter_get_cntpct(void)
+{
+ cycle_t cval;
+
+ asm volatile("mrs %0, cntpct_el0" : "=r" (cval));
+
+ return cval;
+}
+
+static inline cycle_t arch_counter_get_cntvct(void)
+{
+ cycle_t cval;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (cval));
+
+ return cval;
+}
+
+#endif
diff --git a/arch/arm64/include/asm/asm-offsets.h b/arch/arm64/include/asm/asm-offsets.h
new file mode 100644
index 0000000..d370ee3
--- /dev/null
+++ b/arch/arm64/include/asm/asm-offsets.h
@@ -0,0 +1 @@
+#include <generated/asm-offsets.h>
diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h
new file mode 100644
index 0000000..da2a13e
--- /dev/null
+++ b/arch/arm64/include/asm/assembler.h
@@ -0,0 +1,109 @@
+/*
+ * Based on arch/arm/include/asm/assembler.h
+ *
+ * Copyright (C) 1996-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASSEMBLY__
+#error "Only include this from assembly code"
+#endif
+
+#include <asm/ptrace.h>
+
+/*
+ * Stack pushing/popping (register pairs only). Equivalent to store decrement
+ * before, load increment after.
+ */
+ .macro push, xreg1, xreg2
+ stp \xreg1, \xreg2, [sp, #-16]!
+ .endm
+
+ .macro pop, xreg1, xreg2
+ ldp \xreg1, \xreg2, [sp], #16
+ .endm
+
+/*
+ * Enable and disable interrupts.
+ */
+ .macro disable_irq
+ msr daifset, #2
+ .endm
+
+ .macro enable_irq
+ msr daifclr, #2
+ .endm
+
+/*
+ * Save/disable and restore interrupts.
+ */
+ .macro save_and_disable_irqs, olddaif
+ mrs \olddaif, daif
+ disable_irq
+ .endm
+
+ .macro restore_irqs, olddaif
+ msr daif, \olddaif
+ .endm
+
+/*
+ * Enable and disable debug exceptions.
+ */
+ .macro disable_dbg
+ msr daifset, #8
+ .endm
+
+ .macro enable_dbg
+ msr daifclr, #8
+ .endm
+
+ .macro disable_step, tmp
+ mrs \tmp, mdscr_el1
+ bic \tmp, \tmp, #1
+ msr mdscr_el1, \tmp
+ .endm
+
+ .macro enable_step, tmp
+ mrs \tmp, mdscr_el1
+ orr \tmp, \tmp, #1
+ msr mdscr_el1, \tmp
+ .endm
+
+ .macro enable_dbg_if_not_stepping, tmp
+ mrs \tmp, mdscr_el1
+ tbnz \tmp, #1, 9990f
+ enable_dbg
+9990:
+ .endm
+
+/*
+ * SMP data memory barrier
+ */
+ .macro smp_dmb, opt
+#ifdef CONFIG_SMP
+ dmb \opt
+#endif
+ .endm
+
+#define USER(l, x...) \
+9999: x; \
+ .section __ex_table,"a"; \
+ .align 3; \
+ .quad 9999b,l; \
+ .previous
+
+/*
+ * Register aliases.
+ */
+lr .req x30 // link register
diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h
new file mode 100644
index 0000000..407717b
--- /dev/null
+++ b/arch/arm64/include/asm/atomic.h
@@ -0,0 +1,305 @@
+/*
+ * Based on arch/arm/include/asm/atomic.h
+ *
+ * Copyright (C) 1996 Russell King.
+ * Copyright (C) 2002 Deep Blue Solutions Ltd.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_ATOMIC_H
+#define __ASM_ATOMIC_H
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+#include <asm/barrier.h>
+#include <asm/cmpxchg.h>
+
+#define ATOMIC_INIT(i) { (i) }
+
+#ifdef __KERNEL__
+
+/*
+ * On ARM, ordinary assignment (str instruction) doesn't clear the local
+ * strex/ldrex monitor on some implementations. The reason we can use it for
+ * atomic_set() is the clrex or dummy strex done on every exception return.
+ */
+#define atomic_read(v) (*(volatile int *)&(v)->counter)
+#define atomic_set(v,i) (((v)->counter) = (i))
+
+/*
+ * AArch64 UP and SMP safe atomic ops. We use load exclusive and
+ * store exclusive to ensure that these are atomic. We may loop
+ * to ensure that the update happens.
+ */
+static inline void atomic_add(int i, atomic_t *v)
+{
+ unsigned long tmp;
+ int result;
+
+ asm volatile("// atomic_add\n"
+"1: ldxr %w0, [%3]\n"
+" add %w0, %w0, %w4\n"
+" stxr %w1, %w0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+}
+
+static inline int atomic_add_return(int i, atomic_t *v)
+{
+ unsigned long tmp;
+ int result;
+
+ asm volatile("// atomic_add_return\n"
+"1: ldaxr %w0, [%3]\n"
+" add %w0, %w0, %w4\n"
+" stlxr %w1, %w0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+
+ return result;
+}
+
+static inline void atomic_sub(int i, atomic_t *v)
+{
+ unsigned long tmp;
+ int result;
+
+ asm volatile("// atomic_sub\n"
+"1: ldxr %w0, [%3]\n"
+" sub %w0, %w0, %w4\n"
+" stxr %w1, %w0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+}
+
+static inline int atomic_sub_return(int i, atomic_t *v)
+{
+ unsigned long tmp;
+ int result;
+
+ asm volatile("// atomic_sub_return\n"
+"1: ldaxr %w0, [%3]\n"
+" sub %w0, %w0, %w4\n"
+" stlxr %w1, %w0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+
+ return result;
+}
+
+static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new)
+{
+ unsigned long tmp;
+ int oldval;
+
+ asm volatile("// atomic_cmpxchg\n"
+"1: ldaxr %w1, [%3]\n"
+" cmp %w1, %w4\n"
+" b.ne 2f\n"
+" stlxr %w0, %w5, [%3]\n"
+" cbnz %w0, 1b\n"
+"2:"
+ : "=&r" (tmp), "=&r" (oldval), "+o" (ptr->counter)
+ : "r" (&ptr->counter), "Ir" (old), "r" (new)
+ : "cc");
+
+ return oldval;
+}
+
+static inline void atomic_clear_mask(unsigned long mask, unsigned long *addr)
+{
+ unsigned long tmp, tmp2;
+
+ asm volatile("// atomic_clear_mask\n"
+"1: ldxr %0, [%3]\n"
+" bic %0, %0, %4\n"
+" stxr %w1, %0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (tmp), "=&r" (tmp2), "+o" (*addr)
+ : "r" (addr), "Ir" (mask)
+ : "cc");
+}
+
+#define atomic_xchg(v, new) (xchg(&((v)->counter), new))
+
+static inline int __atomic_add_unless(atomic_t *v, int a, int u)
+{
+ int c, old;
+
+ c = atomic_read(v);
+ while (c != u && (old = atomic_cmpxchg((v), c, c + a)) != c)
+ c = old;
+ return c;
+}
+
+#define atomic_inc(v) atomic_add(1, v)
+#define atomic_dec(v) atomic_sub(1, v)
+
+#define atomic_inc_and_test(v) (atomic_add_return(1, v) == 0)
+#define atomic_dec_and_test(v) (atomic_sub_return(1, v) == 0)
+#define atomic_inc_return(v) (atomic_add_return(1, v))
+#define atomic_dec_return(v) (atomic_sub_return(1, v))
+#define atomic_sub_and_test(i, v) (atomic_sub_return(i, v) == 0)
+
+#define atomic_add_negative(i,v) (atomic_add_return(i, v) < 0)
+
+#define smp_mb__before_atomic_dec() smp_mb()
+#define smp_mb__after_atomic_dec() smp_mb()
+#define smp_mb__before_atomic_inc() smp_mb()
+#define smp_mb__after_atomic_inc() smp_mb()
+
+/*
+ * 64-bit atomic operations.
+ */
+#define ATOMIC64_INIT(i) { (i) }
+
+#define atomic64_read(v) (*(volatile long long *)&(v)->counter)
+#define atomic64_set(v,i) (((v)->counter) = (i))
+
+static inline void atomic64_add(u64 i, atomic64_t *v)
+{
+ long result;
+ unsigned long tmp;
+
+ asm volatile("// atomic64_add\n"
+"1: ldxr %0, [%3]\n"
+" add %0, %0, %4\n"
+" stxr %w1, %0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+}
+
+static inline long atomic64_add_return(long i, atomic64_t *v)
+{
+ long result;
+ unsigned long tmp;
+
+ asm volatile("// atomic64_add_return\n"
+"1: ldaxr %0, [%3]\n"
+" add %0, %0, %4\n"
+" stlxr %w1, %0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+
+ return result;
+}
+
+static inline void atomic64_sub(u64 i, atomic64_t *v)
+{
+ long result;
+ unsigned long tmp;
+
+ asm volatile("// atomic64_sub\n"
+"1: ldxr %0, [%3]\n"
+" sub %0, %0, %4\n"
+" stxr %w1, %0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+}
+
+static inline long atomic64_sub_return(long i, atomic64_t *v)
+{
+ long result;
+ unsigned long tmp;
+
+ asm volatile("// atomic64_sub_return\n"
+"1: ldaxr %0, [%3]\n"
+" sub %0, %0, %4\n"
+" stlxr %w1, %0, [%3]\n"
+" cbnz %w1, 1b"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter), "Ir" (i)
+ : "cc");
+
+ return result;
+}
+
+static inline long atomic64_cmpxchg(atomic64_t *ptr, long old, long new)
+{
+ long oldval;
+ unsigned long res;
+
+ asm volatile("// atomic64_cmpxchg\n"
+"1: ldaxr %1, [%3]\n"
+" cmp %1, %4\n"
+" b.ne 2f\n"
+" stlxr %w0, %5, [%3]\n"
+" cbnz %w0, 1b\n"
+"2:"
+ : "=&r" (res), "=&r" (oldval), "+o" (ptr->counter)
+ : "r" (&ptr->counter), "Ir" (old), "r" (new)
+ : "cc");
+
+ return oldval;
+}
+
+#define atomic64_xchg(v, new) (xchg(&((v)->counter), new))
+
+static inline long atomic64_dec_if_positive(atomic64_t *v)
+{
+ long result;
+ unsigned long tmp;
+
+ asm volatile("// atomic64_dec_if_positive\n"
+"1: ldaxr %0, [%3]\n"
+" subs %0, %0, #1\n"
+" b.mi 2f\n"
+" stlxr %w1, %0, [%3]\n"
+" cbnz %w1, 1b\n"
+"2:"
+ : "=&r" (result), "=&r" (tmp), "+o" (v->counter)
+ : "r" (&v->counter)
+ : "cc");
+
+ return result;
+}
+
+static inline int atomic64_add_unless(atomic64_t *v, long a, long u)
+{
+ long c, old;
+
+ c = atomic64_read(v);
+ while (c != u && (old = atomic64_cmpxchg((v), c, c + a)) != c)
+ c = old;
+
+ return c != u;
+}
+
+#define atomic64_add_negative(a, v) (atomic64_add_return((a), (v)) < 0)
+#define atomic64_inc(v) atomic64_add(1LL, (v))
+#define atomic64_inc_return(v) atomic64_add_return(1LL, (v))
+#define atomic64_inc_and_test(v) (atomic64_inc_return(v) == 0)
+#define atomic64_sub_and_test(a, v) (atomic64_sub_return((a), (v)) == 0)
+#define atomic64_dec(v) atomic64_sub(1LL, (v))
+#define atomic64_dec_return(v) atomic64_sub_return(1LL, (v))
+#define atomic64_dec_and_test(v) (atomic64_dec_return((v)) == 0)
+#define atomic64_inc_not_zero(v) atomic64_add_unless((v), 1LL, 0LL)
+
+#endif
+#endif
diff --git a/arch/arm64/include/asm/auxvec.h b/arch/arm64/include/asm/auxvec.h
new file mode 100644
index 0000000..22d6d88
--- /dev/null
+++ b/arch/arm64/include/asm/auxvec.h
@@ -0,0 +1,22 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_AUXVEC_H
+#define __ASM_AUXVEC_H
+
+/* vDSO location */
+#define AT_SYSINFO_EHDR 33
+
+#endif
diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h
new file mode 100644
index 0000000..d4a6333
--- /dev/null
+++ b/arch/arm64/include/asm/barrier.h
@@ -0,0 +1,52 @@
+/*
+ * Based on arch/arm/include/asm/barrier.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_BARRIER_H
+#define __ASM_BARRIER_H
+
+#ifndef __ASSEMBLY__
+
+#define sev() asm volatile("sev" : : : "memory")
+#define wfe() asm volatile("wfe" : : : "memory")
+#define wfi() asm volatile("wfi" : : : "memory")
+
+#define isb() asm volatile("isb" : : : "memory")
+#define dsb() asm volatile("dsb sy" : : : "memory")
+
+#define mb() dsb()
+#define rmb() asm volatile("dsb ld" : : : "memory")
+#define wmb() asm volatile("dsb st" : : : "memory")
+
+#ifndef CONFIG_SMP
+#define smp_mb() barrier()
+#define smp_rmb() barrier()
+#define smp_wmb() barrier()
+#else
+#define smp_mb() asm volatile("dmb ish" : : : "memory")
+#define smp_rmb() asm volatile("dmb ishld" : : : "memory")
+#define smp_wmb() asm volatile("dmb ishst" : : : "memory")
+#endif
+
+#define read_barrier_depends() do { } while(0)
+#define smp_read_barrier_depends() do { } while(0)
+
+#define set_mb(var, value) do { var = value; smp_mb(); } while (0)
+#define nop() asm volatile("nop");
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_BARRIER_H */
diff --git a/arch/arm64/include/asm/bitops.h b/arch/arm64/include/asm/bitops.h
new file mode 100644
index 0000000..5e69307
--- /dev/null
+++ b/arch/arm64/include/asm/bitops.h
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_BITOPS_H
+#define __ASM_BITOPS_H
+
+#include <linux/compiler.h>
+
+#include <asm/barrier.h>
+
+/*
+ * clear_bit may not imply a memory barrier
+ */
+#ifndef smp_mb__before_clear_bit
+#define smp_mb__before_clear_bit() smp_mb()
+#define smp_mb__after_clear_bit() smp_mb()
+#endif
+
+#ifndef _LINUX_BITOPS_H
+#error only <linux/bitops.h> can be included directly
+#endif
+
+#include <asm-generic/bitops/builtin-__ffs.h>
+#include <asm-generic/bitops/builtin-ffs.h>
+#include <asm-generic/bitops/builtin-__fls.h>
+#include <asm-generic/bitops/builtin-fls.h>
+
+#include <asm-generic/bitops/ffz.h>
+#include <asm-generic/bitops/fls64.h>
+#include <asm-generic/bitops/find.h>
+
+#include <asm-generic/bitops/sched.h>
+#include <asm-generic/bitops/hweight.h>
+#include <asm-generic/bitops/lock.h>
+
+#include <asm-generic/bitops/atomic.h>
+#include <asm-generic/bitops/non-atomic.h>
+#include <asm-generic/bitops/le.h>
+#include <asm-generic/bitops/ext2-atomic.h>
+
+#endif /* __ASM_BITOPS_H */
diff --git a/arch/arm64/include/asm/bitsperlong.h b/arch/arm64/include/asm/bitsperlong.h
new file mode 100644
index 0000000..fce9c29
--- /dev/null
+++ b/arch/arm64/include/asm/bitsperlong.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_BITSPERLONG_H
+#define __ASM_BITSPERLONG_H
+
+#define __BITS_PER_LONG 64
+
+#include <asm-generic/bitsperlong.h>
+
+#endif /* __ASM_BITSPERLONG_H */
diff --git a/arch/arm64/include/asm/byteorder.h b/arch/arm64/include/asm/byteorder.h
new file mode 100644
index 0000000..2b92046
--- /dev/null
+++ b/arch/arm64/include/asm/byteorder.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_BYTEORDER_H
+#define __ASM_BYTEORDER_H
+
+#include <linux/byteorder/little_endian.h>
+
+#endif /* __ASM_BYTEORDER_H */
diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h
new file mode 100644
index 0000000..390308a
--- /dev/null
+++ b/arch/arm64/include/asm/cache.h
@@ -0,0 +1,32 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_CACHE_H
+#define __ASM_CACHE_H
+
+#define L1_CACHE_SHIFT 6
+#define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT)
+
+/*
+ * Memory returned by kmalloc() may be used for DMA, so we must make
+ * sure that all such allocations are cache aligned. Otherwise,
+ * unrelated code may cause parts of the buffer to be read into the
+ * cache before the transfer is done, causing old data to be seen by
+ * the CPU.
+ */
+#define ARCH_DMA_MINALIGN L1_CACHE_BYTES
+#define ARCH_SLAB_MINALIGN 8
+
+#endif
diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h
new file mode 100644
index 0000000..aa3132a
--- /dev/null
+++ b/arch/arm64/include/asm/cacheflush.h
@@ -0,0 +1,148 @@
+/*
+ * Based on arch/arm/include/asm/cacheflush.h
+ *
+ * Copyright (C) 1999-2002 Russell King.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_CACHEFLUSH_H
+#define __ASM_CACHEFLUSH_H
+
+#include <linux/mm.h>
+
+/*
+ * This flag is used to indicate that the page pointed to by a pte is clean
+ * and does not require cleaning before returning it to the user.
+ */
+#define PG_dcache_clean PG_arch_1
+
+/*
+ * MM Cache Management
+ * ===================
+ *
+ * The arch/arm64/mm/cache.S implements these methods.
+ *
+ * Start addresses are inclusive and end addresses are exclusive; start
+ * addresses should be rounded down, end addresses up.
+ *
+ * See Documentation/cachetlb.txt for more information. Please note that
+ * the implementation assumes non-aliasing VIPT D-cache and (aliasing)
+ * VIPT or ASID-tagged VIVT I-cache.
+ *
+ * flush_cache_all()
+ *
+ * Unconditionally clean and invalidate the entire cache.
+ *
+ * flush_cache_mm(mm)
+ *
+ * Clean and invalidate all user space cache entries
+ * before a change of page tables.
+ *
+ * flush_icache_range(start, end)
+ *
+ * Ensure coherency between the I-cache and the D-cache in the
+ * region described by start, end.
+ * - start - virtual start address
+ * - end - virtual end address
+ *
+ * __flush_cache_user_range(start, end)
+ *
+ * Ensure coherency between the I-cache and the D-cache in the
+ * region described by start, end.
+ * - start - virtual start address
+ * - end - virtual end address
+ *
+ * __flush_dcache_area(kaddr, size)
+ *
+ * Ensure that the data held in page is written back.
+ * - kaddr - page address
+ * - size - region size
+ */
+extern void flush_cache_all(void);
+extern void flush_cache_mm(struct mm_struct *mm);
+extern void flush_cache_range(struct vm_area_struct *vma, unsigned long start, unsigned long end);
+extern void flush_cache_page(struct vm_area_struct *vma, unsigned long user_addr, unsigned long pfn);
+extern void flush_icache_range(unsigned long start, unsigned long end);
+extern void __flush_dcache_area(void *addr, size_t len);
+extern void __flush_cache_user_range(unsigned long start, unsigned long end);
+
+/*
+ * Copy user data from/to a page which is mapped into a different
+ * processes address space. Really, we want to allow our "user
+ * space" model to handle this.
+ */
+extern void copy_to_user_page(struct vm_area_struct *, struct page *,
+ unsigned long, void *, const void *, unsigned long);
+#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
+ do { \
+ memcpy(dst, src, len); \
+ } while (0)
+
+#define flush_cache_dup_mm(mm) flush_cache_mm(mm)
+
+/*
+ * flush_dcache_page is used when the kernel has written to the page
+ * cache page at virtual address page->virtual.
+ *
+ * If this page isn't mapped (ie, page_mapping == NULL), or it might
+ * have userspace mappings, then we _must_ always clean + invalidate
+ * the dcache entries associated with the kernel mapping.
+ *
+ * Otherwise we can defer the operation, and clean the cache when we are
+ * about to change to user space. This is the same method as used on SPARC64.
+ * See update_mmu_cache for the user space part.
+ */
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1
+extern void flush_dcache_page(struct page *);
+
+static inline void __flush_icache_all(void)
+{
+ asm("ic ialluis");
+}
+
+#define flush_dcache_mmap_lock(mapping) \
+ spin_lock_irq(&(mapping)->tree_lock)
+#define flush_dcache_mmap_unlock(mapping) \
+ spin_unlock_irq(&(mapping)->tree_lock)
+
+#define flush_icache_user_range(vma,page,addr,len) \
+ flush_dcache_page(page)
+
+/*
+ * We don't appear to need to do anything here. In fact, if we did, we'd
+ * duplicate cache flushing elsewhere performed by flush_dcache_page().
+ */
+#define flush_icache_page(vma,page) do { } while (0)
+
+/*
+ * flush_cache_vmap() is used when creating mappings (eg, via vmap,
+ * vmalloc, ioremap etc) in kernel space for pages. On non-VIPT
+ * caches, since the direct-mappings of these pages may contain cached
+ * data, we need to do a full cache flush to ensure that writebacks
+ * don't corrupt data placed into these pages via the new mappings.
+ */
+static inline void flush_cache_vmap(unsigned long start, unsigned long end)
+{
+ /*
+ * set_pte_at() called from vmap_pte_range() does not
+ * have a DSB after cleaning the cache line.
+ */
+ dsb();
+}
+
+static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
+{
+}
+
+#endif
diff --git a/arch/arm64/include/asm/cachetype.h b/arch/arm64/include/asm/cachetype.h
new file mode 100644
index 0000000..85f5f51
--- /dev/null
+++ b/arch/arm64/include/asm/cachetype.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_CACHETYPE_H
+#define __ASM_CACHETYPE_H
+
+#include <asm/cputype.h>
+
+#define CTR_L1IP_SHIFT 14
+#define CTR_L1IP_MASK 3
+
+#define ICACHE_POLICY_RESERVED 0
+#define ICACHE_POLICY_AIVIVT 1
+#define ICACHE_POLICY_VIPT 2
+#define ICACHE_POLICY_PIPT 3
+
+static inline u32 icache_policy(void)
+{
+ return (read_cpuid_cachetype() >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK;
+}
+
+/*
+ * Whilst the D-side always behaves as PIPT on AArch64, aliasing is
+ * permitted in the I-cache.
+ */
+static inline int icache_is_aliasing(void)
+{
+ return icache_policy() != ICACHE_POLICY_PIPT;
+}
+
+static inline int icache_is_aivivt(void)
+{
+ return icache_policy() == ICACHE_POLICY_AIVIVT;
+}
+
+#endif /* __ASM_CACHETYPE_H */
diff --git a/arch/arm64/include/asm/cmpxchg.h b/arch/arm64/include/asm/cmpxchg.h
new file mode 100644
index 0000000..e0e65b0
--- /dev/null
+++ b/arch/arm64/include/asm/cmpxchg.h
@@ -0,0 +1,173 @@
+/*
+ * Based on arch/arm/include/asm/cmpxchg.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_CMPXCHG_H
+#define __ASM_CMPXCHG_H
+
+#include <linux/bug.h>
+
+#include <asm/barrier.h>
+
+static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size)
+{
+ unsigned long ret, tmp;
+
+ switch (size) {
+ case 1:
+ asm volatile("// __xchg1\n"
+ "1: ldaxrb %w0, [%3]\n"
+ " stlxrb %w1, %w2, [%3]\n"
+ " cbnz %w1, 1b\n"
+ : "=&r" (ret), "=&r" (tmp)
+ : "r" (x), "r" (ptr)
+ : "memory", "cc");
+ break;
+ case 2:
+ asm volatile("// __xchg2\n"
+ "1: ldaxrh %w0, [%3]\n"
+ " stlxrh %w1, %w2, [%3]\n"
+ " cbnz %w1, 1b\n"
+ : "=&r" (ret), "=&r" (tmp)
+ : "r" (x), "r" (ptr)
+ : "memory", "cc");
+ break;
+ case 4:
+ asm volatile("// __xchg4\n"
+ "1: ldaxr %w0, [%3]\n"
+ " stlxr %w1, %w2, [%3]\n"
+ " cbnz %w1, 1b\n"
+ : "=&r" (ret), "=&r" (tmp)
+ : "r" (x), "r" (ptr)
+ : "memory", "cc");
+ break;
+ case 8:
+ asm volatile("// __xchg8\n"
+ "1: ldaxr %0, [%3]\n"
+ " stlxr %w1, %2, [%3]\n"
+ " cbnz %w1, 1b\n"
+ : "=&r" (ret), "=&r" (tmp)
+ : "r" (x), "r" (ptr)
+ : "memory", "cc");
+ break;
+ default:
+ BUILD_BUG();
+ }
+
+ return ret;
+}
+
+#define xchg(ptr,x) \
+ ((__typeof__(*(ptr)))__xchg((unsigned long)(x),(ptr),sizeof(*(ptr))))
+
+static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
+ unsigned long new, int size)
+{
+ unsigned long oldval = 0, res;
+
+ switch (size) {
+ case 1:
+ do {
+ asm volatile("// __cmpxchg1\n"
+ " ldxrb %w1, [%2]\n"
+ " mov %w0, #0\n"
+ " cmp %w1, %w3\n"
+ " b.ne 1f\n"
+ " stxrb %w0, %w4, [%2]\n"
+ "1:\n"
+ : "=&r" (res), "=&r" (oldval)
+ : "r" (ptr), "Ir" (old), "r" (new)
+ : "cc");
+ } while (res);
+ break;
+
+ case 2:
+ do {
+ asm volatile("// __cmpxchg2\n"
+ " ldxrh %w1, [%2]\n"
+ " mov %w0, #0\n"
+ " cmp %w1, %w3\n"
+ " b.ne 1f\n"
+ " stxrh %w0, %w4, [%2]\n"
+ "1:\n"
+ : "=&r" (res), "=&r" (oldval)
+ : "r" (ptr), "Ir" (old), "r" (new)
+ : "memory", "cc");
+ } while (res);
+ break;
+
+ case 4:
+ do {
+ asm volatile("// __cmpxchg4\n"
+ " ldxr %w1, [%2]\n"
+ " mov %w0, #0\n"
+ " cmp %w1, %w3\n"
+ " b.ne 1f\n"
+ " stxr %w0, %w4, [%2]\n"
+ "1:\n"
+ : "=&r" (res), "=&r" (oldval)
+ : "r" (ptr), "Ir" (old), "r" (new)
+ : "cc");
+ } while (res);
+ break;
+
+ case 8:
+ do {
+ asm volatile("// __cmpxchg8\n"
+ " ldxr %1, [%2]\n"
+ " mov %w0, #0\n"
+ " cmp %1, %3\n"
+ " b.ne 1f\n"
+ " stxr %w0, %4, [%2]\n"
+ "1:\n"
+ : "=&r" (res), "=&r" (oldval)
+ : "r" (ptr), "Ir" (old), "r" (new)
+ : "cc");
+ } while (res);
+ break;
+
+ default:
+ BUILD_BUG();
+ }
+
+ return oldval;
+}
+
+static inline unsigned long __cmpxchg_mb(volatile void *ptr, unsigned long old,
+ unsigned long new, int size)
+{
+ unsigned long ret;
+
+ smp_mb();
+ ret = __cmpxchg(ptr, old, new, size);
+ smp_mb();
+
+ return ret;
+}
+
+#define cmpxchg(ptr,o,n) \
+ ((__typeof__(*(ptr)))__cmpxchg_mb((ptr), \
+ (unsigned long)(o), \
+ (unsigned long)(n), \
+ sizeof(*(ptr))))
+
+#define cmpxchg_local(ptr,o,n) \
+ ((__typeof__(*(ptr)))__cmpxchg((ptr), \
+ (unsigned long)(o), \
+ (unsigned long)(n), \
+ sizeof(*(ptr))))
+
+#endif /* __ASM_CMPXCHG_H */
diff --git a/arch/arm64/include/asm/compat.h b/arch/arm64/include/asm/compat.h
new file mode 100644
index 0000000..a670a33
--- /dev/null
+++ b/arch/arm64/include/asm/compat.h
@@ -0,0 +1,242 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_COMPAT_H
+#define __ASM_COMPAT_H
+#ifdef __KERNEL__
+#ifdef CONFIG_COMPAT
+
+/*
+ * Architecture specific compatibility types
+ */
+#include <linux/types.h>
+#include <linux/sched.h>
+
+#define COMPAT_USER_HZ 100
+#define COMPAT_UTS_MACHINE "armv8l\0\0"
+
+typedef u32 compat_size_t;
+typedef s32 compat_ssize_t;
+typedef s32 compat_time_t;
+typedef s32 compat_clock_t;
+typedef s32 compat_pid_t;
+typedef u32 __compat_uid_t;
+typedef u32 __compat_gid_t;
+typedef u32 __compat_uid32_t;
+typedef u32 __compat_gid32_t;
+typedef u32 compat_mode_t;
+typedef u32 compat_ino_t;
+typedef u32 compat_dev_t;
+typedef s32 compat_off_t;
+typedef s64 compat_loff_t;
+typedef s16 compat_nlink_t;
+typedef u16 compat_ipc_pid_t;
+typedef s32 compat_daddr_t;
+typedef u32 compat_caddr_t;
+typedef __kernel_fsid_t compat_fsid_t;
+typedef s32 compat_key_t;
+typedef s32 compat_timer_t;
+
+typedef s32 compat_int_t;
+typedef s32 compat_long_t;
+typedef s64 compat_s64;
+typedef u32 compat_uint_t;
+typedef u32 compat_ulong_t;
+typedef u64 compat_u64;
+
+struct compat_timespec {
+ compat_time_t tv_sec;
+ s32 tv_nsec;
+};
+
+struct compat_timeval {
+ compat_time_t tv_sec;
+ s32 tv_usec;
+};
+
+struct compat_stat {
+ compat_dev_t st_dev;
+ compat_ino_t st_ino;
+ compat_mode_t st_mode;
+ compat_nlink_t st_nlink;
+ __compat_uid32_t st_uid;
+ __compat_gid32_t st_gid;
+ compat_dev_t st_rdev;
+ compat_off_t st_size;
+ compat_off_t st_blksize;
+ compat_off_t st_blocks;
+ compat_time_t st_atime;
+ u32 st_atime_nsec;
+ compat_time_t st_mtime;
+ u32 st_mtime_nsec;
+ compat_time_t st_ctime;
+ u32 st_ctime_nsec;
+ u32 __unused4[2];
+};
+
+struct compat_flock {
+ short l_type;
+ short l_whence;
+ compat_off_t l_start;
+ compat_off_t l_len;
+ compat_pid_t l_pid;
+};
+
+#define F_GETLK64 12 /* using 'struct flock64' */
+#define F_SETLK64 13
+#define F_SETLKW64 14
+
+struct compat_flock64 {
+ short l_type;
+ short l_whence;
+ compat_loff_t l_start;
+ compat_loff_t l_len;
+ compat_pid_t l_pid;
+};
+
+struct compat_statfs {
+ int f_type;
+ int f_bsize;
+ int f_blocks;
+ int f_bfree;
+ int f_bavail;
+ int f_files;
+ int f_ffree;
+ compat_fsid_t f_fsid;
+ int f_namelen; /* SunOS ignores this field. */
+ int f_frsize;
+ int f_flags;
+ int f_spare[4];
+};
+
+#define COMPAT_RLIM_INFINITY 0xffffffff
+
+typedef u32 compat_old_sigset_t;
+
+#define _COMPAT_NSIG 64
+#define _COMPAT_NSIG_BPW 32
+
+typedef u32 compat_sigset_word;
+
+#define COMPAT_OFF_T_MAX 0x7fffffff
+#define COMPAT_LOFF_T_MAX 0x7fffffffffffffffL
+
+/*
+ * A pointer passed in from user mode. This should not
+ * be used for syscall parameters, just declare them
+ * as pointers because the syscall entry code will have
+ * appropriately converted them already.
+ */
+typedef u32 compat_uptr_t;
+
+static inline void __user *compat_ptr(compat_uptr_t uptr)
+{
+ return (void __user *)(unsigned long)uptr;
+}
+
+static inline compat_uptr_t ptr_to_compat(void __user *uptr)
+{
+ return (u32)(unsigned long)uptr;
+}
+
+static inline void __user *arch_compat_alloc_user_space(long len)
+{
+ struct pt_regs *regs = task_pt_regs(current);
+ return (void __user *)regs->compat_sp - len;
+}
+
+struct compat_ipc64_perm {
+ compat_key_t key;
+ __compat_uid32_t uid;
+ __compat_gid32_t gid;
+ __compat_uid32_t cuid;
+ __compat_gid32_t cgid;
+ unsigned short mode;
+ unsigned short __pad1;
+ unsigned short seq;
+ unsigned short __pad2;
+ compat_ulong_t unused1;
+ compat_ulong_t unused2;
+};
+
+struct compat_semid64_ds {
+ struct compat_ipc64_perm sem_perm;
+ compat_time_t sem_otime;
+ compat_ulong_t __unused1;
+ compat_time_t sem_ctime;
+ compat_ulong_t __unused2;
+ compat_ulong_t sem_nsems;
+ compat_ulong_t __unused3;
+ compat_ulong_t __unused4;
+};
+
+struct compat_msqid64_ds {
+ struct compat_ipc64_perm msg_perm;
+ compat_time_t msg_stime;
+ compat_ulong_t __unused1;
+ compat_time_t msg_rtime;
+ compat_ulong_t __unused2;
+ compat_time_t msg_ctime;
+ compat_ulong_t __unused3;
+ compat_ulong_t msg_cbytes;
+ compat_ulong_t msg_qnum;
+ compat_ulong_t msg_qbytes;
+ compat_pid_t msg_lspid;
+ compat_pid_t msg_lrpid;
+ compat_ulong_t __unused4;
+ compat_ulong_t __unused5;
+};
+
+struct compat_shmid64_ds {
+ struct compat_ipc64_perm shm_perm;
+ compat_size_t shm_segsz;
+ compat_time_t shm_atime;
+ compat_ulong_t __unused1;
+ compat_time_t shm_dtime;
+ compat_ulong_t __unused2;
+ compat_time_t shm_ctime;
+ compat_ulong_t __unused3;
+ compat_pid_t shm_cpid;
+ compat_pid_t shm_lpid;
+ compat_ulong_t shm_nattch;
+ compat_ulong_t __unused4;
+ compat_ulong_t __unused5;
+};
+
+static inline int is_compat_task(void)
+{
+ return test_thread_flag(TIF_32BIT);
+}
+
+static inline int is_compat_thread(struct thread_info *thread)
+{
+ return test_ti_thread_flag(thread, TIF_32BIT);
+}
+
+#else /* !CONFIG_COMPAT */
+
+static inline int is_compat_task(void)
+{
+ return 0;
+}
+
+static inline int is_compat_thread(struct thread_info *thread)
+{
+ return 0;
+}
+
+#endif /* CONFIG_COMPAT */
+#endif /* __KERNEL__ */
+#endif /* __ASM_COMPAT_H */
diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h
new file mode 100644
index 0000000..ee35fd0
--- /dev/null
+++ b/arch/arm64/include/asm/compiler.h
@@ -0,0 +1,30 @@
+/*
+ * Based on arch/arm/include/asm/compiler.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_COMPILER_H
+#define __ASM_COMPILER_H
+
+/*
+ * This is used to ensure the compiler did actually allocate the register we
+ * asked it for some inline assembly sequences. Apparently we can't trust the
+ * compiler from one version to another so a bit of paranoia won't hurt. This
+ * string is meant to be concatenated with the inline asm string and will
+ * cause compilation to stop on mismatch. (for details, see gcc PR 15089)
+ */
+#define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t"
+
+#endif /* __ASM_COMPILER_H */
diff --git a/arch/arm64/include/asm/cputable.h b/arch/arm64/include/asm/cputable.h
new file mode 100644
index 0000000..e3bd983
--- /dev/null
+++ b/arch/arm64/include/asm/cputable.h
@@ -0,0 +1,30 @@
+/*
+ * arch/arm64/include/asm/cputable.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_CPUTABLE_H
+#define __ASM_CPUTABLE_H
+
+struct cpu_info {
+ unsigned int cpu_id_val;
+ unsigned int cpu_id_mask;
+ const char *cpu_name;
+ unsigned long (*cpu_setup)(void);
+};
+
+extern struct cpu_info *lookup_processor_type(unsigned int);
+
+#endif
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
new file mode 100644
index 0000000..ef54125
--- /dev/null
+++ b/arch/arm64/include/asm/cputype.h
@@ -0,0 +1,49 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_CPUTYPE_H
+#define __ASM_CPUTYPE_H
+
+#define ID_MIDR_EL1 "midr_el1"
+#define ID_CTR_EL0 "ctr_el0"
+
+#define ID_AA64PFR0_EL1 "id_aa64pfr0_el1"
+#define ID_AA64DFR0_EL1 "id_aa64dfr0_el1"
+#define ID_AA64AFR0_EL1 "id_aa64afr0_el1"
+#define ID_AA64ISAR0_EL1 "id_aa64isar0_el1"
+#define ID_AA64MMFR0_EL1 "id_aa64mmfr0_el1"
+
+#define read_cpuid(reg) ({ \
+ u64 __val; \
+ asm("mrs %0, " reg : "=r" (__val)); \
+ __val; \
+})
+
+/*
+ * The CPU ID never changes at run time, so we might as well tell the
+ * compiler that it's constant. Use this function to read the CPU ID
+ * rather than directly reading processor_id or read_cpuid() directly.
+ */
+static inline u32 __attribute_const__ read_cpuid_id(void)
+{
+ return read_cpuid(ID_MIDR_EL1);
+}
+
+static inline u32 __attribute_const__ read_cpuid_cachetype(void)
+{
+ return read_cpuid(ID_CTR_EL0);
+}
+
+#endif
diff --git a/arch/arm64/include/asm/debug-monitors.h b/arch/arm64/include/asm/debug-monitors.h
new file mode 100644
index 0000000..7eaa0b3
--- /dev/null
+++ b/arch/arm64/include/asm/debug-monitors.h
@@ -0,0 +1,88 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_DEBUG_MONITORS_H
+#define __ASM_DEBUG_MONITORS_H
+
+#ifdef __KERNEL__
+
+#define DBG_ESR_EVT(x) (((x) >> 27) & 0x7)
+
+/* AArch64 */
+#define DBG_ESR_EVT_HWBP 0x0
+#define DBG_ESR_EVT_HWSS 0x1
+#define DBG_ESR_EVT_HWWP 0x2
+#define DBG_ESR_EVT_BRK 0x6
+
+enum debug_el {
+ DBG_ACTIVE_EL0 = 0,
+ DBG_ACTIVE_EL1,
+};
+
+/* AArch32 */
+#define DBG_ESR_EVT_BKPT 0x4
+#define DBG_ESR_EVT_VECC 0x5
+
+#define AARCH32_BREAK_ARM 0x07f001f0
+#define AARCH32_BREAK_THUMB 0xde01
+#define AARCH32_BREAK_THUMB2_LO 0xf7f0
+#define AARCH32_BREAK_THUMB2_HI 0xa000
+
+#ifndef __ASSEMBLY__
+struct task_struct;
+
+#define local_dbg_save(flags) \
+ do { \
+ typecheck(unsigned long, flags); \
+ asm volatile( \
+ "mrs %0, daif // local_dbg_save\n" \
+ "msr daifset, #8" \
+ : "=r" (flags) : : "memory"); \
+ } while (0)
+
+#define local_dbg_restore(flags) \
+ do { \
+ typecheck(unsigned long, flags); \
+ asm volatile( \
+ "msr daif, %0 // local_dbg_restore\n" \
+ : : "r" (flags) : "memory"); \
+ } while (0)
+
+#define DBG_ARCH_ID_RESERVED 0 /* In case of ptrace ABI updates. */
+
+u8 debug_monitors_arch(void);
+
+void enable_debug_monitors(enum debug_el el);
+void disable_debug_monitors(enum debug_el el);
+
+void user_rewind_single_step(struct task_struct *task);
+void user_fastforward_single_step(struct task_struct *task);
+
+void kernel_enable_single_step(struct pt_regs *regs);
+void kernel_disable_single_step(void);
+int kernel_active_single_step(void);
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+int reinstall_suspended_bps(struct pt_regs *regs);
+#else
+static inline int reinstall_suspended_bps(struct pt_regs *regs)
+{
+ return -ENODEV;
+}
+#endif
+
+#endif /* __ASSEMBLY */
+#endif /* __KERNEL__ */
+#endif /* __ASM_DEBUG_MONITORS_H */
diff --git a/arch/arm64/include/asm/device.h b/arch/arm64/include/asm/device.h
new file mode 100644
index 0000000..0d8453c
--- /dev/null
+++ b/arch/arm64/include/asm/device.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_DEVICE_H
+#define __ASM_DEVICE_H
+
+struct dev_archdata {
+ struct dma_map_ops *dma_ops;
+};
+
+struct pdev_archdata {
+};
+
+#endif
diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h
new file mode 100644
index 0000000..538f4b4
--- /dev/null
+++ b/arch/arm64/include/asm/dma-mapping.h
@@ -0,0 +1,124 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_DMA_MAPPING_H
+#define __ASM_DMA_MAPPING_H
+
+#ifdef __KERNEL__
+
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include <asm-generic/dma-coherent.h>
+
+#define ARCH_HAS_DMA_GET_REQUIRED_MASK
+
+extern struct dma_map_ops *dma_ops;
+
+static inline struct dma_map_ops *get_dma_ops(struct device *dev)
+{
+ if (unlikely(!dev) || !dev->archdata.dma_ops)
+ return dma_ops;
+ else
+ return dev->archdata.dma_ops;
+}
+
+#include <asm-generic/dma-mapping-common.h>
+
+static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr)
+{
+ return (dma_addr_t)paddr;
+}
+
+static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr)
+{
+ return (phys_addr_t)dev_addr;
+}
+
+static inline int dma_mapping_error(struct device *dev, dma_addr_t dev_addr)
+{
+ struct dma_map_ops *ops = get_dma_ops(dev);
+ return ops->mapping_error(dev, dev_addr);
+}
+
+static inline int dma_supported(struct device *dev, u64 mask)
+{
+ struct dma_map_ops *ops = get_dma_ops(dev);
+ return ops->dma_supported(dev, mask);
+}
+
+static inline int dma_set_mask(struct device *dev, u64 mask)
+{
+ if (!dev->dma_mask || !dma_supported(dev, mask))
+ return -EIO;
+ *dev->dma_mask = mask;
+
+ return 0;
+}
+
+static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size)
+{
+ if (!dev->dma_mask)
+ return 0;
+
+ return addr + size - 1 <= *dev->dma_mask;
+}
+
+static inline void dma_mark_clean(void *addr, size_t size)
+{
+}
+
+static inline void *dma_alloc_coherent(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t flags)
+{
+ struct dma_map_ops *ops = get_dma_ops(dev);
+ void *vaddr;
+
+ if (dma_alloc_from_coherent(dev, size, dma_handle, &vaddr))
+ return vaddr;
+
+ vaddr = ops->alloc(dev, size, dma_handle, flags, NULL);
+ debug_dma_alloc_coherent(dev, size, *dma_handle, vaddr);
+ return vaddr;
+}
+
+static inline void dma_free_coherent(struct device *dev, size_t size,
+ void *vaddr, dma_addr_t dev_addr)
+{
+ struct dma_map_ops *ops = get_dma_ops(dev);
+
+ if (dma_release_from_coherent(dev, get_order(size), vaddr))
+ return;
+
+ debug_dma_free_coherent(dev, size, vaddr, dev_addr);
+ ops->free(dev, size, vaddr, dev_addr, NULL);
+}
+
+/*
+ * There is no dma_cache_sync() implementation, so just return NULL here.
+ */
+static inline void *dma_alloc_noncoherent(struct device *dev, size_t size,
+ dma_addr_t *handle, gfp_t flags)
+{
+ return NULL;
+}
+
+static inline void dma_free_noncoherent(struct device *dev, size_t size,
+ void *cpu_addr, dma_addr_t handle)
+{
+}
+
+#endif /* __KERNEL__ */
+#endif /* __ASM_DMA_MAPPING_H */
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
new file mode 100644
index 0000000..cf28464
--- /dev/null
+++ b/arch/arm64/include/asm/elf.h
@@ -0,0 +1,179 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_ELF_H
+#define __ASM_ELF_H
+
+#include <asm/hwcap.h>
+
+/*
+ * ELF register definitions..
+ */
+#include <asm/ptrace.h>
+#include <asm/user.h>
+
+typedef unsigned long elf_greg_t;
+typedef unsigned long elf_freg_t[3];
+
+#define ELF_NGREG (sizeof (struct pt_regs) / sizeof(elf_greg_t))
+typedef elf_greg_t elf_gregset_t[ELF_NGREG];
+
+typedef struct user_fp elf_fpregset_t;
+
+#define EM_AARCH64 183
+
+/*
+ * AArch64 static relocation types.
+ */
+
+/* Miscellaneous. */
+#define R_ARM_NONE 0
+#define R_AARCH64_NONE 256
+
+/* Data. */
+#define R_AARCH64_ABS64 257
+#define R_AARCH64_ABS32 258
+#define R_AARCH64_ABS16 259
+#define R_AARCH64_PREL64 260
+#define R_AARCH64_PREL32 261
+#define R_AARCH64_PREL16 262
+
+/* Instructions. */
+#define R_AARCH64_MOVW_UABS_G0 263
+#define R_AARCH64_MOVW_UABS_G0_NC 264
+#define R_AARCH64_MOVW_UABS_G1 265
+#define R_AARCH64_MOVW_UABS_G1_NC 266
+#define R_AARCH64_MOVW_UABS_G2 267
+#define R_AARCH64_MOVW_UABS_G2_NC 268
+#define R_AARCH64_MOVW_UABS_G3 269
+
+#define R_AARCH64_MOVW_SABS_G0 270
+#define R_AARCH64_MOVW_SABS_G1 271
+#define R_AARCH64_MOVW_SABS_G2 272
+
+#define R_AARCH64_LD_PREL_LO19 273
+#define R_AARCH64_ADR_PREL_LO21 274
+#define R_AARCH64_ADR_PREL_PG_HI21 275
+#define R_AARCH64_ADR_PREL_PG_HI21_NC 276
+#define R_AARCH64_ADD_ABS_LO12_NC 277
+#define R_AARCH64_LDST8_ABS_LO12_NC 278
+
+#define R_AARCH64_TSTBR14 279
+#define R_AARCH64_CONDBR19 280
+#define R_AARCH64_JUMP26 282
+#define R_AARCH64_CALL26 283
+#define R_AARCH64_LDST16_ABS_LO12_NC 284
+#define R_AARCH64_LDST32_ABS_LO12_NC 285
+#define R_AARCH64_LDST64_ABS_LO12_NC 286
+#define R_AARCH64_LDST128_ABS_LO12_NC 299
+
+#define R_AARCH64_MOVW_PREL_G0 287
+#define R_AARCH64_MOVW_PREL_G0_NC 288
+#define R_AARCH64_MOVW_PREL_G1 289
+#define R_AARCH64_MOVW_PREL_G1_NC 290
+#define R_AARCH64_MOVW_PREL_G2 291
+#define R_AARCH64_MOVW_PREL_G2_NC 292
+#define R_AARCH64_MOVW_PREL_G3 293
+
+
+/*
+ * These are used to set parameters in the core dumps.
+ */
+#define ELF_CLASS ELFCLASS64
+#define ELF_DATA ELFDATA2LSB
+#define ELF_ARCH EM_AARCH64
+
+#define ELF_PLATFORM_SIZE 16
+#define ELF_PLATFORM ("aarch64")
+
+/*
+ * This is used to ensure we don't load something for the wrong architecture.
+ */
+#define elf_check_arch(x) ((x)->e_machine == EM_AARCH64)
+
+#define elf_read_implies_exec(ex,stk) (stk != EXSTACK_DISABLE_X)
+
+#define CORE_DUMP_USE_REGSET
+#define ELF_EXEC_PAGESIZE PAGE_SIZE
+
+/*
+ * This is the location that an ET_DYN program is loaded if exec'ed. Typical
+ * use of this is to invoke "./ld.so someprog" to test out a new version of
+ * the loader. We need to make sure that it is out of the way of the program
+ * that it will "exec", and that there is sufficient room for the brk.
+ */
+extern unsigned long randomize_et_dyn(unsigned long base);
+#define ELF_ET_DYN_BASE (randomize_et_dyn(2 * TASK_SIZE_64 / 3))
+
+/*
+ * When the program starts, a1 contains a pointer to a function to be
+ * registered with atexit, as per the SVR4 ABI. A value of 0 means we have no
+ * such handler.
+ */
+#define ELF_PLAT_INIT(_r, load_addr) (_r)->regs[0] = 0
+
+#define SET_PERSONALITY(ex) clear_thread_flag(TIF_32BIT);
+
+#define ARCH_DLINFO \
+do { \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (elf_addr_t)current->mm->context.vdso); \
+} while (0)
+
+#define ARCH_HAS_SETUP_ADDITIONAL_PAGES
+struct linux_binprm;
+extern int arch_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
+
+/* 1GB of VA */
+#ifdef CONFIG_COMPAT
+#define STACK_RND_MASK (test_thread_flag(TIF_32BIT) ? \
+ 0x7ff >> (PAGE_SHIFT - 12) : \
+ 0x3ffff >> (PAGE_SHIFT - 12))
+#else
+#define STACK_RND_MASK (0x3ffff >> (PAGE_SHIFT - 12))
+#endif
+
+struct mm_struct;
+extern unsigned long arch_randomize_brk(struct mm_struct *mm);
+#define arch_randomize_brk arch_randomize_brk
+
+#ifdef CONFIG_COMPAT
+#define EM_ARM 40
+#define COMPAT_ELF_PLATFORM ("v8l")
+
+#define COMPAT_ELF_ET_DYN_BASE (randomize_et_dyn(2 * TASK_SIZE_32 / 3))
+
+/* AArch32 registers. */
+#define COMPAT_ELF_NGREG 18
+typedef unsigned int compat_elf_greg_t;
+typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
+
+/* AArch32 EABI. */
+#define EF_ARM_EABI_MASK 0xff000000
+#define compat_elf_check_arch(x) (((x)->e_machine == EM_ARM) && \
+ ((x)->e_flags & EF_ARM_EABI_MASK))
+
+#define compat_start_thread compat_start_thread
+#define COMPAT_SET_PERSONALITY(ex) set_thread_flag(TIF_32BIT);
+#define COMPAT_ARCH_DLINFO
+extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
+ int uses_interp);
+#define compat_arch_setup_additional_pages \
+ aarch32_setup_vectors_page
+
+#endif /* CONFIG_COMPAT */
+
+#endif
diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h
new file mode 100644
index 0000000..ac63519
--- /dev/null
+++ b/arch/arm64/include/asm/exception.h
@@ -0,0 +1,23 @@
+/*
+ * Based on arch/arm/include/asm/exception.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_EXCEPTION_H
+#define __ASM_EXCEPTION_H
+
+#define __exception __attribute__((section(".exception.text")))
+
+#endif /* __ASM_EXCEPTION_H */
diff --git a/arch/arm64/include/asm/exec.h b/arch/arm64/include/asm/exec.h
new file mode 100644
index 0000000..db0563c
--- /dev/null
+++ b/arch/arm64/include/asm/exec.h
@@ -0,0 +1,23 @@
+/*
+ * Based on arch/arm/include/asm/exec.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_EXEC_H
+#define __ASM_EXEC_H
+
+extern unsigned long arch_align_stack(unsigned long sp);
+
+#endif /* __ASM_EXEC_H */
diff --git a/arch/arm64/include/asm/fb.h b/arch/arm64/include/asm/fb.h
new file mode 100644
index 0000000..adb88a6
--- /dev/null
+++ b/arch/arm64/include/asm/fb.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_FB_H_
+#define __ASM_FB_H_
+
+#include <linux/fb.h>
+#include <linux/fs.h>
+#include <asm/page.h>
+
+static inline void fb_pgprotect(struct file *file, struct vm_area_struct *vma,
+ unsigned long off)
+{
+ vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+}
+
+static inline int fb_is_primary_device(struct fb_info *info)
+{
+ return 0;
+}
+
+#endif /* __ASM_FB_H_ */
diff --git a/arch/arm64/include/asm/fcntl.h b/arch/arm64/include/asm/fcntl.h
new file mode 100644
index 0000000..cd2e630
--- /dev/null
+++ b/arch/arm64/include/asm/fcntl.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_FCNTL_H
+#define __ASM_FCNTL_H
+
+/*
+ * Using our own definitions for AArch32 (compat) support.
+ */
+#define O_DIRECTORY 040000 /* must be a directory */
+#define O_NOFOLLOW 0100000 /* don't follow links */
+#define O_DIRECT 0200000 /* direct disk access hint - currently ignored */
+#define O_LARGEFILE 0400000
+
+#include <asm-generic/fcntl.h>
+
+#endif
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
new file mode 100644
index 0000000..b42fab9
--- /dev/null
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -0,0 +1,64 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_FP_H
+#define __ASM_FP_H
+
+#include <asm/ptrace.h>
+
+#ifndef __ASSEMBLY__
+
+/*
+ * FP/SIMD storage area has:
+ * - FPSR and FPCR
+ * - 32 128-bit data registers
+ *
+ * Note that user_fp forms a prefix of this structure, which is relied
+ * upon in the ptrace FP/SIMD accessors. struct user_fpsimd_state must
+ * form a prefix of struct fpsimd_state.
+ */
+struct fpsimd_state {
+ union {
+ struct user_fpsimd_state user_fpsimd;
+ struct {
+ __uint128_t vregs[32];
+ u32 fpsr;
+ u32 fpcr;
+ };
+ };
+};
+
+#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
+/* Masks for extracting the FPSR and FPCR from the FPSCR */
+#define VFP_FPSCR_STAT_MASK 0xf800009f
+#define VFP_FPSCR_CTRL_MASK 0x07f79f00
+/*
+ * The VFP state has 32x64-bit registers and a single 32-bit
+ * control/status register.
+ */
+#define VFP_STATE_SIZE ((32 * 8) + 4)
+#endif
+
+struct task_struct;
+
+extern void fpsimd_save_state(struct fpsimd_state *state);
+extern void fpsimd_load_state(struct fpsimd_state *state);
+
+extern void fpsimd_thread_switch(struct task_struct *next);
+extern void fpsimd_flush_thread(void);
+
+#endif
+
+#endif
diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h
new file mode 100644
index 0000000..3468ae8
--- /dev/null
+++ b/arch/arm64/include/asm/futex.h
@@ -0,0 +1,136 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_FUTEX_H
+#define __ASM_FUTEX_H
+
+#ifdef __KERNEL__
+
+#include <linux/futex.h>
+#include <linux/uaccess.h>
+#include <asm/errno.h>
+
+#define __futex_atomic_op(insn, ret, oldval, uaddr, tmp, oparg) \
+ asm volatile( \
+"1: ldaxr %w1, %2\n" \
+ insn "\n" \
+"2: stlxr %w3, %w0, %2\n" \
+" cbnz %w3, 1b\n" \
+"3:\n" \
+" .pushsection .fixup,\"ax\"\n" \
+"4: mov %w0, %w5\n" \
+" b 3b\n" \
+" .popsection\n" \
+" .pushsection __ex_table,\"a\"\n" \
+" .align 3\n" \
+" .quad 1b, 4b, 2b, 4b\n" \
+" .popsection\n" \
+ : "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp) \
+ : "r" (oparg), "Ir" (-EFAULT) \
+ : "cc")
+
+static inline int
+futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr)
+{
+ int op = (encoded_op >> 28) & 7;
+ int cmp = (encoded_op >> 24) & 15;
+ int oparg = (encoded_op << 8) >> 20;
+ int cmparg = (encoded_op << 20) >> 20;
+ int oldval = 0, ret, tmp;
+
+ if (encoded_op & (FUTEX_OP_OPARG_SHIFT << 28))
+ oparg = 1 << oparg;
+
+ if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32)))
+ return -EFAULT;
+
+ pagefault_disable(); /* implies preempt_disable() */
+
+ switch (op) {
+ case FUTEX_OP_SET:
+ __futex_atomic_op("mov %w0, %w4",
+ ret, oldval, uaddr, tmp, oparg);
+ break;
+ case FUTEX_OP_ADD:
+ __futex_atomic_op("add %w0, %w1, %w4",
+ ret, oldval, uaddr, tmp, oparg);
+ break;
+ case FUTEX_OP_OR:
+ __futex_atomic_op("orr %w0, %w1, %w4",
+ ret, oldval, uaddr, tmp, oparg);
+ break;
+ case FUTEX_OP_ANDN:
+ __futex_atomic_op("and %w0, %w1, %w4",
+ ret, oldval, uaddr, tmp, ~oparg);
+ break;
+ case FUTEX_OP_XOR:
+ __futex_atomic_op("eor %w0, %w1, %w4",
+ ret, oldval, uaddr, tmp, oparg);
+ break;
+ default:
+ ret = -ENOSYS;
+ }
+
+ pagefault_enable(); /* subsumes preempt_enable() */
+
+ if (!ret) {
+ switch (cmp) {
+ case FUTEX_OP_CMP_EQ: ret = (oldval == cmparg); break;
+ case FUTEX_OP_CMP_NE: ret = (oldval != cmparg); break;
+ case FUTEX_OP_CMP_LT: ret = (oldval < cmparg); break;
+ case FUTEX_OP_CMP_GE: ret = (oldval >= cmparg); break;
+ case FUTEX_OP_CMP_LE: ret = (oldval <= cmparg); break;
+ case FUTEX_OP_CMP_GT: ret = (oldval > cmparg); break;
+ default: ret = -ENOSYS;
+ }
+ }
+ return ret;
+}
+
+static inline int
+futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
+ u32 oldval, u32 newval)
+{
+ int ret = 0;
+ u32 val, tmp;
+
+ if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32)))
+ return -EFAULT;
+
+ asm volatile("// futex_atomic_cmpxchg_inatomic\n"
+"1: ldaxr %w1, %2\n"
+" sub %w3, %w1, %w4\n"
+" cbnz %w3, 3f\n"
+"2: stlxr %w3, %w5, %2\n"
+" cbnz %w3, 1b\n"
+"3:\n"
+" .pushsection .fixup,\"ax\"\n"
+"4: mov %w0, %w6\n"
+" b 3b\n"
+" .popsection\n"
+" .pushsection __ex_table,\"a\"\n"
+" .align 3\n"
+" .quad 1b, 4b, 2b, 4b\n"
+" .popsection\n"
+ : "+r" (ret), "=&r" (val), "+Q" (*uaddr), "=&r" (tmp)
+ : "r" (oldval), "r" (newval), "Ir" (-EFAULT)
+ : "cc", "memory");
+
+ *uval = val;
+ return ret;
+}
+
+#endif /* __KERNEL__ */
+#endif /* __ASM_FUTEX_H */
diff --git a/arch/arm64/include/asm/hardirq.h b/arch/arm64/include/asm/hardirq.h
new file mode 100644
index 0000000..5075463
--- /dev/null
+++ b/arch/arm64/include/asm/hardirq.h
@@ -0,0 +1,52 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_HARDIRQ_H
+#define __ASM_HARDIRQ_H
+
+#include <linux/cache.h>
+#include <linux/threads.h>
+#include <asm/irq.h>
+
+#define NR_IPI 4
+
+typedef struct {
+ unsigned int __softirq_pending;
+#ifdef CONFIG_SMP
+ unsigned int ipi_irqs[NR_IPI];
+#endif
+} ____cacheline_aligned irq_cpustat_t;
+
+#include <linux/irq_cpustat.h> /* Standard mappings for irq_cpustat_t above */
+
+#define __inc_irq_stat(cpu, member) __IRQ_STAT(cpu, member)++
+#define __get_irq_stat(cpu, member) __IRQ_STAT(cpu, member)
+
+#ifdef CONFIG_SMP
+u64 smp_irq_stat_cpu(unsigned int cpu);
+#define arch_irq_stat_cpu smp_irq_stat_cpu
+#endif
+
+#define __ARCH_IRQ_EXIT_IRQS_DISABLED 1
+
+static inline void ack_bad_irq(unsigned int irq)
+{
+ extern unsigned long irq_err_count;
+ irq_err_count++;
+}
+
+extern void handle_IRQ(unsigned int, struct pt_regs *);
+
+#endif /* __ASM_HARDIRQ_H */
diff --git a/arch/arm64/include/asm/hw_breakpoint.h b/arch/arm64/include/asm/hw_breakpoint.h
new file mode 100644
index 0000000..d064047
--- /dev/null
+++ b/arch/arm64/include/asm/hw_breakpoint.h
@@ -0,0 +1,137 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_HW_BREAKPOINT_H
+#define __ASM_HW_BREAKPOINT_H
+
+#ifdef __KERNEL__
+
+struct arch_hw_breakpoint_ctrl {
+ u32 __reserved : 19,
+ len : 8,
+ type : 2,
+ privilege : 2,
+ enabled : 1;
+};
+
+struct arch_hw_breakpoint {
+ u64 address;
+ u64 trigger;
+ struct arch_hw_breakpoint_ctrl ctrl;
+};
+
+static inline u32 encode_ctrl_reg(struct arch_hw_breakpoint_ctrl ctrl)
+{
+ return (ctrl.len << 5) | (ctrl.type << 3) | (ctrl.privilege << 1) |
+ ctrl.enabled;
+}
+
+static inline void decode_ctrl_reg(u32 reg,
+ struct arch_hw_breakpoint_ctrl *ctrl)
+{
+ ctrl->enabled = reg & 0x1;
+ reg >>= 1;
+ ctrl->privilege = reg & 0x3;
+ reg >>= 2;
+ ctrl->type = reg & 0x3;
+ reg >>= 2;
+ ctrl->len = reg & 0xff;
+}
+
+/* Breakpoint */
+#define ARM_BREAKPOINT_EXECUTE 0
+
+/* Watchpoints */
+#define ARM_BREAKPOINT_LOAD 1
+#define ARM_BREAKPOINT_STORE 2
+#define AARCH64_ESR_ACCESS_MASK (1 << 6)
+
+/* Privilege Levels */
+#define AARCH64_BREAKPOINT_EL1 1
+#define AARCH64_BREAKPOINT_EL0 2
+
+/* Lengths */
+#define ARM_BREAKPOINT_LEN_1 0x1
+#define ARM_BREAKPOINT_LEN_2 0x3
+#define ARM_BREAKPOINT_LEN_4 0xf
+#define ARM_BREAKPOINT_LEN_8 0xff
+
+/* Kernel stepping */
+#define ARM_KERNEL_STEP_NONE 0
+#define ARM_KERNEL_STEP_ACTIVE 1
+#define ARM_KERNEL_STEP_SUSPEND 2
+
+/*
+ * Limits.
+ * Changing these will require modifications to the register accessors.
+ */
+#define ARM_MAX_BRP 16
+#define ARM_MAX_WRP 16
+#define ARM_MAX_HBP_SLOTS (ARM_MAX_BRP + ARM_MAX_WRP)
+
+/* Virtual debug register bases. */
+#define AARCH64_DBG_REG_BVR 0
+#define AARCH64_DBG_REG_BCR (AARCH64_DBG_REG_BVR + ARM_MAX_BRP)
+#define AARCH64_DBG_REG_WVR (AARCH64_DBG_REG_BCR + ARM_MAX_BRP)
+#define AARCH64_DBG_REG_WCR (AARCH64_DBG_REG_WVR + ARM_MAX_WRP)
+
+/* Debug register names. */
+#define AARCH64_DBG_REG_NAME_BVR "bvr"
+#define AARCH64_DBG_REG_NAME_BCR "bcr"
+#define AARCH64_DBG_REG_NAME_WVR "wvr"
+#define AARCH64_DBG_REG_NAME_WCR "wcr"
+
+/* Accessor macros for the debug registers. */
+#define AARCH64_DBG_READ(N, REG, VAL) do {\
+ asm volatile("mrs %0, dbg" REG #N "_el1" : "=r" (VAL));\
+} while (0)
+
+#define AARCH64_DBG_WRITE(N, REG, VAL) do {\
+ asm volatile("msr dbg" REG #N "_el1, %0" :: "r" (VAL));\
+} while (0)
+
+struct task_struct;
+struct notifier_block;
+struct perf_event;
+struct pmu;
+
+extern int arch_bp_generic_fields(struct arch_hw_breakpoint_ctrl ctrl,
+ int *gen_len, int *gen_type);
+extern int arch_check_bp_in_kernelspace(struct perf_event *bp);
+extern int arch_validate_hwbkpt_settings(struct perf_event *bp);
+extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
+ unsigned long val, void *data);
+
+extern int arch_install_hw_breakpoint(struct perf_event *bp);
+extern void arch_uninstall_hw_breakpoint(struct perf_event *bp);
+extern void hw_breakpoint_pmu_read(struct perf_event *bp);
+extern int hw_breakpoint_slots(int type);
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+extern void hw_breakpoint_thread_switch(struct task_struct *next);
+extern void ptrace_hw_copy_thread(struct task_struct *task);
+#else
+static inline void hw_breakpoint_thread_switch(struct task_struct *next)
+{
+}
+static inline void ptrace_hw_copy_thread(struct task_struct *task)
+{
+}
+#endif
+
+extern struct pmu perf_ops_bp;
+
+#endif /* __KERNEL__ */
+#endif /* __ASM_BREAKPOINT_H */
diff --git a/arch/arm64/include/asm/hwcap.h b/arch/arm64/include/asm/hwcap.h
new file mode 100644
index 0000000..f8190ba
--- /dev/null
+++ b/arch/arm64/include/asm/hwcap.h
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_HWCAP_H
+#define __ASM_HWCAP_H
+
+/*
+ * HWCAP flags - for elf_hwcap (in kernel) and AT_HWCAP
+ */
+#define HWCAP_FP (1 << 0)
+#define HWCAP_ASIMD (1 << 1)
+
+#define COMPAT_HWCAP_HALF (1 << 1)
+#define COMPAT_HWCAP_THUMB (1 << 2)
+#define COMPAT_HWCAP_FAST_MULT (1 << 4)
+#define COMPAT_HWCAP_VFP (1 << 6)
+#define COMPAT_HWCAP_EDSP (1 << 7)
+#define COMPAT_HWCAP_NEON (1 << 12)
+#define COMPAT_HWCAP_VFPv3 (1 << 13)
+#define COMPAT_HWCAP_TLS (1 << 15)
+#define COMPAT_HWCAP_VFPv4 (1 << 16)
+#define COMPAT_HWCAP_IDIVA (1 << 17)
+#define COMPAT_HWCAP_IDIVT (1 << 18)
+#define COMPAT_HWCAP_IDIV (COMPAT_HWCAP_IDIVA|COMPAT_HWCAP_IDIVT)
+
+#if defined(__KERNEL__) && !defined(__ASSEMBLY__)
+/*
+ * This yields a mask that user programs can use to figure out what
+ * instruction set this cpu supports.
+ */
+#define ELF_HWCAP (elf_hwcap)
+#define COMPAT_ELF_HWCAP (COMPAT_HWCAP_HALF|COMPAT_HWCAP_THUMB|\
+ COMPAT_HWCAP_FAST_MULT|COMPAT_HWCAP_EDSP|\
+ COMPAT_HWCAP_TLS|COMPAT_HWCAP_VFP|\
+ COMPAT_HWCAP_VFPv3|COMPAT_HWCAP_VFPv4|\
+ COMPAT_HWCAP_NEON|COMPAT_HWCAP_IDIV)
+
+extern unsigned int elf_hwcap;
+#endif
+
+#endif
diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
new file mode 100644
index 0000000..74a2a7d
--- /dev/null
+++ b/arch/arm64/include/asm/io.h
@@ -0,0 +1,258 @@
+/*
+ * Based on arch/arm/include/asm/io.h
+ *
+ * Copyright (C) 1996-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_IO_H
+#define __ASM_IO_H
+
+#ifdef __KERNEL__
+
+#include <linux/types.h>
+
+#include <asm/byteorder.h>
+#include <asm/barrier.h>
+#include <asm/pgtable.h>
+
+/*
+ * Generic IO read/write. These perform native-endian accesses.
+ */
+static inline void __raw_writeb(u8 val, volatile void __iomem *addr)
+{
+ asm volatile("strb %w0, [%1]" : : "r" (val), "r" (addr));
+}
+
+static inline void __raw_writew(u16 val, volatile void __iomem *addr)
+{
+ asm volatile("strh %w0, [%1]" : : "r" (val), "r" (addr));
+}
+
+static inline void __raw_writel(u32 val, volatile void __iomem *addr)
+{
+ asm volatile("str %w0, [%1]" : : "r" (val), "r" (addr));
+}
+
+static inline void __raw_writeq(u64 val, volatile void __iomem *addr)
+{
+ asm volatile("str %0, [%1]" : : "r" (val), "r" (addr));
+}
+
+static inline u8 __raw_readb(const volatile void __iomem *addr)
+{
+ u8 val;
+ asm volatile("ldrb %w0, [%1]" : "=r" (val) : "r" (addr));
+ return val;
+}
+
+static inline u16 __raw_readw(const volatile void __iomem *addr)
+{
+ u16 val;
+ asm volatile("ldrh %w0, [%1]" : "=r" (val) : "r" (addr));
+ return val;
+}
+
+static inline u32 __raw_readl(const volatile void __iomem *addr)
+{
+ u32 val;
+ asm volatile("ldr %w0, [%1]" : "=r" (val) : "r" (addr));
+ return val;
+}
+
+static inline u64 __raw_readq(const volatile void __iomem *addr)
+{
+ u64 val;
+ asm volatile("ldr %0, [%1]" : "=r" (val) : "r" (addr));
+ return val;
+}
+
+/* IO barriers */
+#define __iormb() rmb()
+#define __iowmb() wmb()
+
+#define mmiowb() do { } while (0)
+
+/*
+ * Relaxed I/O memory access primitives. These follow the Device memory
+ * ordering rules but do not guarantee any ordering relative to Normal memory
+ * accesses.
+ */
+#define readb_relaxed(c) ({ u8 __v = __raw_readb(c); __v; })
+#define readw_relaxed(c) ({ u16 __v = le16_to_cpu((__force __le16)__raw_readw(c)); __v; })
+#define readl_relaxed(c) ({ u32 __v = le32_to_cpu((__force __le32)__raw_readl(c)); __v; })
+
+#define writeb_relaxed(v,c) ((void)__raw_writeb((v),(c)))
+#define writew_relaxed(v,c) ((void)__raw_writew((__force u16)cpu_to_le16(v),(c)))
+#define writel_relaxed(v,c) ((void)__raw_writel((__force u32)cpu_to_le32(v),(c)))
+
+/*
+ * I/O memory access primitives. Reads are ordered relative to any
+ * following Normal memory access. Writes are ordered relative to any prior
+ * Normal memory access.
+ */
+#define readb(c) ({ u8 __v = readb_relaxed(c); __iormb(); __v; })
+#define readw(c) ({ u16 __v = readw_relaxed(c); __iormb(); __v; })
+#define readl(c) ({ u32 __v = readl_relaxed(c); __iormb(); __v; })
+
+#define writeb(v,c) ({ __iowmb(); writeb_relaxed((v),(c)); })
+#define writew(v,c) ({ __iowmb(); writew_relaxed((v),(c)); })
+#define writel(v,c) ({ __iowmb(); writel_relaxed((v),(c)); })
+
+/*
+ * I/O port access primitives.
+ */
+#define IO_SPACE_LIMIT 0xffff
+#define PCI_IOBASE ((void __iomem *)0xffffffbbfffe0000UL)
+
+static inline u8 inb(unsigned long addr)
+{
+ return readb(addr + PCI_IOBASE);
+}
+
+static inline u16 inw(unsigned long addr)
+{
+ return readw(addr + PCI_IOBASE);
+}
+
+static inline u32 inl(unsigned long addr)
+{
+ return readl(addr + PCI_IOBASE);
+}
+
+static inline void outb(u8 b, unsigned long addr)
+{
+ writeb(b, addr + PCI_IOBASE);
+}
+
+static inline void outw(u16 b, unsigned long addr)
+{
+ writew(b, addr + PCI_IOBASE);
+}
+
+static inline void outl(u32 b, unsigned long addr)
+{
+ writel(b, addr + PCI_IOBASE);
+}
+
+#define inb_p(addr) inb(addr)
+#define inw_p(addr) inw(addr)
+#define inl_p(addr) inl(addr)
+
+#define outb_p(x, addr) outb((x), (addr))
+#define outw_p(x, addr) outw((x), (addr))
+#define outl_p(x, addr) outl((x), (addr))
+
+static inline void insb(unsigned long addr, void *buffer, int count)
+{
+ u8 *buf = buffer;
+ while (count--)
+ *buf++ = __raw_readb(addr + PCI_IOBASE);
+}
+
+static inline void insw(unsigned long addr, void *buffer, int count)
+{
+ u16 *buf = buffer;
+ while (count--)
+ *buf++ = __raw_readw(addr + PCI_IOBASE);
+}
+
+static inline void insl(unsigned long addr, void *buffer, int count)
+{
+ u32 *buf = buffer;
+ while (count--)
+ *buf++ = __raw_readl(addr + PCI_IOBASE);
+}
+
+static inline void outsb(unsigned long addr, const void *buffer, int count)
+{
+ const u8 *buf = buffer;
+ while (count--)
+ __raw_writeb(*buf++, addr + PCI_IOBASE);
+}
+
+static inline void outsw(unsigned long addr, const void *buffer, int count)
+{
+ const u16 *buf = buffer;
+ while (count--)
+ __raw_writew(*buf++, addr + PCI_IOBASE);
+}
+
+static inline void outsl(unsigned long addr, const void *buffer, int count)
+{
+ const u32 *buf = buffer;
+ while (count--)
+ __raw_writel(*buf++, addr + PCI_IOBASE);
+}
+
+#define insb_p(port,to,len) insb(port,to,len)
+#define insw_p(port,to,len) insw(port,to,len)
+#define insl_p(port,to,len) insl(port,to,len)
+
+#define outsb_p(port,from,len) outsb(port,from,len)
+#define outsw_p(port,from,len) outsw(port,from,len)
+#define outsl_p(port,from,len) outsl(port,from,len)
+
+/*
+ * String version of I/O memory access operations.
+ */
+extern void __memcpy_fromio(void *, const volatile void __iomem *, size_t);
+extern void __memcpy_toio(volatile void __iomem *, const void *, size_t);
+extern void __memset_io(volatile void __iomem *, int, size_t);
+
+#define memset_io(c,v,l) __memset_io((c),(v),(l))
+#define memcpy_fromio(a,c,l) __memcpy_fromio((a),(c),(l))
+#define memcpy_toio(c,a,l) __memcpy_toio((c),(a),(l))
+
+/*
+ * I/O memory mapping functions.
+ */
+extern void __iomem *__ioremap(phys_addr_t phys_addr, size_t size, pgprot_t prot);
+extern void __iounmap(volatile void __iomem *addr);
+
+#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_DIRTY)
+#define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_XN | PTE_ATTRINDX(MT_DEVICE_nGnRE))
+#define PROT_NORMAL_NC (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL_NC))
+
+#define ioremap(addr, size) __ioremap((addr), (size), PROT_DEVICE_nGnRE)
+#define ioremap_nocache(addr, size) __ioremap((addr), (size), PROT_DEVICE_nGnRE)
+#define ioremap_wc(addr, size) __ioremap((addr), (size), PROT_NORMAL_NC)
+#define iounmap __iounmap
+
+#define ARCH_HAS_IOREMAP_WC
+#include <asm-generic/iomap.h>
+
+/*
+ * More restrictive address range checking than the default implementation
+ * (PHYS_OFFSET and PHYS_MASK taken into account).
+ */
+#define ARCH_HAS_VALID_PHYS_ADDR_RANGE
+extern int valid_phys_addr_range(unsigned long addr, size_t size);
+extern int valid_mmap_phys_addr_range(unsigned long pfn, size_t size);
+
+extern int devmem_is_allowed(unsigned long pfn);
+
+/*
+ * Convert a physical pointer to a virtual kernel pointer for /dev/mem
+ * access
+ */
+#define xlate_dev_mem_ptr(p) __va(p)
+
+/*
+ * Convert a virtual cached pointer to an uncached pointer
+ */
+#define xlate_dev_kmem_ptr(p) p
+
+#endif /* __KERNEL__ */
+#endif /* __ASM_IO_H */
diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h
new file mode 100644
index 0000000..a4e1cad
--- /dev/null
+++ b/arch/arm64/include/asm/irq.h
@@ -0,0 +1,8 @@
+#ifndef __ASM_IRQ_H
+#define __ASM_IRQ_H
+
+#include <asm-generic/irq.h>
+
+extern void (*handle_arch_irq)(struct pt_regs *);
+
+#endif
diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h
new file mode 100644
index 0000000..aa11943
--- /dev/null
+++ b/arch/arm64/include/asm/irqflags.h
@@ -0,0 +1,91 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_IRQFLAGS_H
+#define __ASM_IRQFLAGS_H
+
+#ifdef __KERNEL__
+
+#include <asm/ptrace.h>
+
+/*
+ * CPU interrupt mask handling.
+ */
+static inline unsigned long arch_local_irq_save(void)
+{
+ unsigned long flags;
+ asm volatile(
+ "mrs %0, daif // arch_local_irq_save\n"
+ "msr daifset, #2"
+ : "=r" (flags)
+ :
+ : "memory");
+ return flags;
+}
+
+static inline void arch_local_irq_enable(void)
+{
+ asm volatile(
+ "msr daifclr, #2 // arch_local_irq_enable"
+ :
+ :
+ : "memory");
+}
+
+static inline void arch_local_irq_disable(void)
+{
+ asm volatile(
+ "msr daifset, #2 // arch_local_irq_disable"
+ :
+ :
+ : "memory");
+}
+
+#define local_fiq_enable() asm("msr daifclr, #1" : : : "memory")
+#define local_fiq_disable() asm("msr daifset, #1" : : : "memory")
+
+/*
+ * Save the current interrupt enable state.
+ */
+static inline unsigned long arch_local_save_flags(void)
+{
+ unsigned long flags;
+ asm volatile(
+ "mrs %0, daif // arch_local_save_flags"
+ : "=r" (flags)
+ :
+ : "memory");
+ return flags;
+}
+
+/*
+ * restore saved IRQ state
+ */
+static inline void arch_local_irq_restore(unsigned long flags)
+{
+ asm volatile(
+ "msr daif, %0 // arch_local_irq_restore"
+ :
+ : "r" (flags)
+ : "memory");
+}
+
+static inline int arch_irqs_disabled_flags(unsigned long flags)
+{
+ return flags & PSR_I_BIT;
+}
+
+#endif
+#endif
diff --git a/arch/arm64/include/asm/memblock.h b/arch/arm64/include/asm/memblock.h
new file mode 100644
index 0000000..6afeed2
--- /dev/null
+++ b/arch/arm64/include/asm/memblock.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_MEMBLOCK_H
+#define __ASM_MEMBLOCK_H
+
+extern void arm64_memblock_init(void);
+
+#endif
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
new file mode 100644
index 0000000..1cac16a
--- /dev/null
+++ b/arch/arm64/include/asm/memory.h
@@ -0,0 +1,144 @@
+/*
+ * Based on arch/arm/include/asm/memory.h
+ *
+ * Copyright (C) 2000-2002 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Note: this file should not be included by non-asm/.h files
+ */
+#ifndef __ASM_MEMORY_H
+#define __ASM_MEMORY_H
+
+#include <linux/compiler.h>
+#include <linux/const.h>
+#include <linux/types.h>
+#include <asm/sizes.h>
+
+/*
+ * Allow for constants defined here to be used from assembly code
+ * by prepending the UL suffix only with actual C code compilation.
+ */
+#define UL(x) _AC(x, UL)
+
+/*
+ * PAGE_OFFSET - the virtual address of the start of the kernel image.
+ * VA_BITS - the maximum number of bits for virtual addresses.
+ * TASK_SIZE - the maximum size of a user space task.
+ * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area.
+ * The module space lives between the addresses given by TASK_SIZE
+ * and PAGE_OFFSET - it must be within 128MB of the kernel text.
+ */
+#define PAGE_OFFSET UL(0xffffffc000000000)
+#define MODULES_END (PAGE_OFFSET)
+#define MODULES_VADDR (MODULES_END - SZ_64M)
+#define VA_BITS (39)
+#define TASK_SIZE_64 (UL(1) << VA_BITS)
+
+#ifdef CONFIG_COMPAT
+#define TASK_SIZE_32 UL(0x100000000)
+#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \
+ TASK_SIZE_32 : TASK_SIZE_64)
+#else
+#define TASK_SIZE TASK_SIZE_64
+#endif /* CONFIG_COMPAT */
+
+#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4))
+
+#if TASK_SIZE_64 > MODULES_VADDR
+#error Top of 64-bit user space clashes with start of module space
+#endif
+
+/*
+ * Physical vs virtual RAM address space conversion. These are
+ * private definitions which should NOT be used outside memory.h
+ * files. Use virt_to_phys/phys_to_virt/__pa/__va instead.
+ */
+#define __virt_to_phys(x) (((phys_addr_t)(x) - PAGE_OFFSET + PHYS_OFFSET))
+#define __phys_to_virt(x) ((unsigned long)((x) - PHYS_OFFSET + PAGE_OFFSET))
+
+/*
+ * Convert a physical address to a Page Frame Number and back
+ */
+#define __phys_to_pfn(paddr) ((unsigned long)((paddr) >> PAGE_SHIFT))
+#define __pfn_to_phys(pfn) ((phys_addr_t)(pfn) << PAGE_SHIFT)
+
+/*
+ * Convert a page to/from a physical address
+ */
+#define page_to_phys(page) (__pfn_to_phys(page_to_pfn(page)))
+#define phys_to_page(phys) (pfn_to_page(__phys_to_pfn(phys)))
+
+/*
+ * Memory types available.
+ */
+#define MT_DEVICE_nGnRnE 0
+#define MT_DEVICE_nGnRE 1
+#define MT_DEVICE_GRE 2
+#define MT_NORMAL_NC 3
+#define MT_NORMAL 4
+
+#ifndef __ASSEMBLY__
+
+extern phys_addr_t memstart_addr;
+/* PHYS_OFFSET - the physical address of the start of memory. */
+#define PHYS_OFFSET ({ memstart_addr; })
+
+/*
+ * PFNs are used to describe any physical page; this means
+ * PFN 0 == physical address 0.
+ *
+ * This is the PFN of the first RAM page in the kernel
+ * direct-mapped view. We assume this is the first page
+ * of RAM in the mem_map as well.
+ */
+#define PHYS_PFN_OFFSET (PHYS_OFFSET >> PAGE_SHIFT)
+
+/*
+ * Note: Drivers should NOT use these. They are the wrong
+ * translation for translating DMA addresses. Use the driver
+ * DMA support - see dma-mapping.h.
+ */
+static inline phys_addr_t virt_to_phys(const volatile void *x)
+{
+ return __virt_to_phys((unsigned long)(x));
+}
+
+static inline void *phys_to_virt(phys_addr_t x)
+{
+ return (void *)(__phys_to_virt(x));
+}
+
+/*
+ * Drivers should NOT use these either.
+ */
+#define __pa(x) __virt_to_phys((unsigned long)(x))
+#define __va(x) ((void *)__phys_to_virt((phys_addr_t)(x)))
+#define pfn_to_kaddr(pfn) __va((pfn) << PAGE_SHIFT)
+
+/*
+ * virt_to_page(k) convert a _valid_ virtual address to struct page *
+ * virt_addr_valid(k) indicates whether a virtual address is valid
+ */
+#define ARCH_PFN_OFFSET PHYS_PFN_OFFSET
+
+#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)
+#define virt_addr_valid(kaddr) (((void *)(kaddr) >= (void *)PAGE_OFFSET) && \
+ ((void *)(kaddr) < (void *)high_memory))
+
+#endif
+
+#include <asm-generic/memory_model.h>
+
+#endif
diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
new file mode 100644
index 0000000..d4f7fd5
--- /dev/null
+++ b/arch/arm64/include/asm/mmu.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_MMU_H
+#define __ASM_MMU_H
+
+typedef struct {
+ unsigned int id;
+ raw_spinlock_t id_lock;
+ void *vdso;
+} mm_context_t;
+
+#define ASID(mm) ((mm)->context.id & 0xffff)
+
+extern void paging_init(void);
+extern void setup_mm_for_reboot(void);
+
+#endif
diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h
new file mode 100644
index 0000000..f68465d
--- /dev/null
+++ b/arch/arm64/include/asm/mmu_context.h
@@ -0,0 +1,152 @@
+/*
+ * Based on arch/arm/include/asm/mmu_context.h
+ *
+ * Copyright (C) 1996 Russell King.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_MMU_CONTEXT_H
+#define __ASM_MMU_CONTEXT_H
+
+#include <linux/compiler.h>
+#include <linux/sched.h>
+
+#include <asm/cacheflush.h>
+#include <asm/proc-fns.h>
+#include <asm-generic/mm_hooks.h>
+#include <asm/cputype.h>
+#include <asm/pgtable.h>
+
+#define MAX_ASID_BITS 16
+
+extern unsigned int cpu_last_asid;
+
+void __init_new_context(struct task_struct *tsk, struct mm_struct *mm);
+void __new_context(struct mm_struct *mm);
+
+/*
+ * Set TTBR0 to empty_zero_page. No translations will be possible via TTBR0.
+ */
+static inline void cpu_set_reserved_ttbr0(void)
+{
+ unsigned long ttbr = page_to_phys(empty_zero_page);
+
+ asm(
+ " msr ttbr0_el1, %0 // set TTBR0\n"
+ " isb"
+ :
+ : "r" (ttbr));
+}
+
+static inline void switch_new_context(struct mm_struct *mm)
+{
+ unsigned long flags;
+
+ __new_context(mm);
+
+ local_irq_save(flags);
+ cpu_switch_mm(mm->pgd, mm);
+ local_irq_restore(flags);
+}
+
+static inline void check_and_switch_context(struct mm_struct *mm,
+ struct task_struct *tsk)
+{
+ /*
+ * Required during context switch to avoid speculative page table
+ * walking with the wrong TTBR.
+ */
+ cpu_set_reserved_ttbr0();
+
+ if (!((mm->context.id ^ cpu_last_asid) >> MAX_ASID_BITS))
+ /*
+ * The ASID is from the current generation, just switch to the
+ * new pgd. This condition is only true for calls from
+ * context_switch() and interrupts are already disabled.
+ */
+ cpu_switch_mm(mm->pgd, mm);
+ else if (irqs_disabled())
+ /*
+ * Defer the new ASID allocation until after the context
+ * switch critical region since __new_context() cannot be
+ * called with interrupts disabled.
+ */
+ set_ti_thread_flag(task_thread_info(tsk), TIF_SWITCH_MM);
+ else
+ /*
+ * That is a direct call to switch_mm() or activate_mm() with
+ * interrupts enabled and a new context.
+ */
+ switch_new_context(mm);
+}
+
+#define init_new_context(tsk,mm) (__init_new_context(tsk,mm),0)
+#define destroy_context(mm) do { } while(0)
+
+#define finish_arch_post_lock_switch \
+ finish_arch_post_lock_switch
+static inline void finish_arch_post_lock_switch(void)
+{
+ if (test_and_clear_thread_flag(TIF_SWITCH_MM)) {
+ struct mm_struct *mm = current->mm;
+ unsigned long flags;
+
+ __new_context(mm);
+
+ local_irq_save(flags);
+ cpu_switch_mm(mm->pgd, mm);
+ local_irq_restore(flags);
+ }
+}
+
+/*
+ * This is called when "tsk" is about to enter lazy TLB mode.
+ *
+ * mm: describes the currently active mm context
+ * tsk: task which is entering lazy tlb
+ * cpu: cpu number which is entering lazy tlb
+ *
+ * tsk->mm will be NULL
+ */
+static inline void
+enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
+{
+}
+
+/*
+ * This is the actual mm switch as far as the scheduler
+ * is concerned. No registers are touched. We avoid
+ * calling the CPU specific function when the mm hasn't
+ * actually changed.
+ */
+static inline void
+switch_mm(struct mm_struct *prev, struct mm_struct *next,
+ struct task_struct *tsk)
+{
+ unsigned int cpu = smp_processor_id();
+
+#ifdef CONFIG_SMP
+ /* check for possible thread migration */
+ if (!cpumask_empty(mm_cpumask(next)) &&
+ !cpumask_test_cpu(cpu, mm_cpumask(next)))
+ __flush_icache_all();
+#endif
+ if (!cpumask_test_and_set_cpu(cpu, mm_cpumask(next)) || prev != next)
+ check_and_switch_context(next, tsk);
+}
+
+#define deactivate_mm(tsk,mm) do { } while (0)
+#define activate_mm(prev,next) switch_mm(prev, next, NULL)
+
+#endif
diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
new file mode 100644
index 0000000..e80e232
--- /dev/null
+++ b/arch/arm64/include/asm/module.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_MODULE_H
+#define __ASM_MODULE_H
+
+#include <asm-generic/module.h>
+
+#define MODULE_ARCH_VERMAGIC "aarch64"
+
+#endif /* __ASM_MODULE_H */
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
new file mode 100644
index 0000000..46bf666
--- /dev/null
+++ b/arch/arm64/include/asm/page.h
@@ -0,0 +1,67 @@
+/*
+ * Based on arch/arm/include/asm/page.h
+ *
+ * Copyright (C) 1995-2003 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PAGE_H
+#define __ASM_PAGE_H
+
+/* PAGE_SHIFT determines the page size */
+#ifdef CONFIG_ARM64_64K_PAGES
+#define PAGE_SHIFT 16
+#else
+#define PAGE_SHIFT 12
+#endif
+#define PAGE_SIZE (_AC(1,UL) << PAGE_SHIFT)
+#define PAGE_MASK (~(PAGE_SIZE-1))
+
+/* We do define AT_SYSINFO_EHDR but don't use the gate mechanism */
+#define __HAVE_ARCH_GATE_AREA 1
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#include <asm/pgtable-2level-types.h>
+#else
+#include <asm/pgtable-3level-types.h>
+#endif
+
+extern void __cpu_clear_user_page(void *p, unsigned long user);
+extern void __cpu_copy_user_page(void *to, const void *from,
+ unsigned long user);
+extern void copy_page(void *to, const void *from);
+extern void clear_page(void *to);
+
+#define clear_user_page(addr,vaddr,pg) __cpu_clear_user_page(addr, vaddr)
+#define copy_user_page(to,from,vaddr,pg) __cpu_copy_user_page(to, from, vaddr)
+
+typedef struct page *pgtable_t;
+
+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
+extern int pfn_valid(unsigned long);
+#endif
+
+#include <asm/memory.h>
+
+#endif /* !__ASSEMBLY__ */
+
+#define VM_DATA_DEFAULT_FLAGS \
+ (((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0) | \
+ VM_READ | VM_WRITE | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+
+#include <asm-generic/getorder.h>
+
+#endif
diff --git a/arch/arm64/include/asm/param.h b/arch/arm64/include/asm/param.h
new file mode 100644
index 0000000..8e3a281
--- /dev/null
+++ b/arch/arm64/include/asm/param.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PARAM_H
+#define __ASM_PARAM_H
+
+#define EXEC_PAGESIZE 65536
+
+#include <asm-generic/param.h>
+
+#endif
diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h
new file mode 100644
index 0000000..a6fffd5
--- /dev/null
+++ b/arch/arm64/include/asm/perf_event.h
@@ -0,0 +1,22 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_PERF_EVENT_H
+#define __ASM_PERF_EVENT_H
+
+/* It's quiet around here... */
+
+#endif
diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h
new file mode 100644
index 0000000..f214069
--- /dev/null
+++ b/arch/arm64/include/asm/pgalloc.h
@@ -0,0 +1,113 @@
+/*
+ * Based on arch/arm/include/asm/pgalloc.h
+ *
+ * Copyright (C) 2000-2001 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGALLOC_H
+#define __ASM_PGALLOC_H
+
+#include <asm/pgtable-hwdef.h>
+#include <asm/processor.h>
+#include <asm/cacheflush.h>
+#include <asm/tlbflush.h>
+
+#define check_pgt_cache() do { } while (0)
+
+#ifndef CONFIG_ARM64_64K_PAGES
+
+static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long addr)
+{
+ return (pmd_t *)get_zeroed_page(GFP_KERNEL | __GFP_REPEAT);
+}
+
+static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
+{
+ BUG_ON((unsigned long)pmd & (PAGE_SIZE-1));
+ free_page((unsigned long)pmd);
+}
+
+static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
+{
+ set_pud(pud, __pud(__pa(pmd) | PMD_TYPE_TABLE));
+}
+
+#endif /* CONFIG_ARM64_64K_PAGES */
+
+extern pgd_t *pgd_alloc(struct mm_struct *mm);
+extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);
+
+#define PGALLOC_GFP (GFP_KERNEL | __GFP_NOTRACK | __GFP_REPEAT | __GFP_ZERO)
+
+static inline pte_t *
+pte_alloc_one_kernel(struct mm_struct *mm, unsigned long addr)
+{
+ return (pte_t *)__get_free_page(PGALLOC_GFP);
+}
+
+static inline pgtable_t
+pte_alloc_one(struct mm_struct *mm, unsigned long addr)
+{
+ struct page *pte;
+
+ pte = alloc_pages(PGALLOC_GFP, 0);
+ if (pte)
+ pgtable_page_ctor(pte);
+
+ return pte;
+}
+
+/*
+ * Free a PTE table.
+ */
+static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+{
+ if (pte)
+ free_page((unsigned long)pte);
+}
+
+static inline void pte_free(struct mm_struct *mm, pgtable_t pte)
+{
+ pgtable_page_dtor(pte);
+ __free_page(pte);
+}
+
+static inline void __pmd_populate(pmd_t *pmdp, phys_addr_t pte,
+ pmdval_t prot)
+{
+ set_pmd(pmdp, __pmd(pte | prot));
+}
+
+/*
+ * Populate the pmdp entry with a pointer to the pte. This pmd is part
+ * of the mm address space.
+ */
+static inline void
+pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep)
+{
+ /*
+ * The pmd must be loaded with the physical address of the PTE table
+ */
+ __pmd_populate(pmdp, __pa(ptep), PMD_TYPE_TABLE);
+}
+
+static inline void
+pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep)
+{
+ __pmd_populate(pmdp, page_to_phys(ptep), PMD_TYPE_TABLE);
+}
+#define pmd_pgtable(pmd) pmd_page(pmd)
+
+#endif
diff --git a/arch/arm64/include/asm/pgtable-2level-hwdef.h b/arch/arm64/include/asm/pgtable-2level-hwdef.h
new file mode 100644
index 0000000..0a8ed3f
--- /dev/null
+++ b/arch/arm64/include/asm/pgtable-2level-hwdef.h
@@ -0,0 +1,43 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGTABLE_2LEVEL_HWDEF_H
+#define __ASM_PGTABLE_2LEVEL_HWDEF_H
+
+/*
+ * With LPAE and 64KB pages, there are 2 levels of page tables. Each level has
+ * 8192 entries of 8 bytes each, occupying a 64KB page. Levels 0 and 1 are not
+ * used. The 2nd level table (PGD for Linux) can cover a range of 4TB, each
+ * entry representing 512MB. The user and kernel address spaces are limited to
+ * 512GB and therefore we only use 1024 entries in the PGD.
+ */
+#define PTRS_PER_PTE 8192
+#define PTRS_PER_PGD 1024
+
+/*
+ * PGDIR_SHIFT determines the size a top-level page table entry can map.
+ */
+#define PGDIR_SHIFT 29
+#define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE-1))
+
+/*
+ * section address mask and size definitions.
+ */
+#define SECTION_SHIFT 29
+#define SECTION_SIZE (_AC(1, UL) << SECTION_SHIFT)
+#define SECTION_MASK (~(SECTION_SIZE-1))
+
+#endif
diff --git a/arch/arm64/include/asm/pgtable-2level-types.h b/arch/arm64/include/asm/pgtable-2level-types.h
new file mode 100644
index 0000000..3c3ca7d
--- /dev/null
+++ b/arch/arm64/include/asm/pgtable-2level-types.h
@@ -0,0 +1,60 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGTABLE_2LEVEL_TYPES_H
+#define __ASM_PGTABLE_2LEVEL_TYPES_H
+
+typedef u64 pteval_t;
+typedef u64 pgdval_t;
+typedef pgdval_t pmdval_t;
+
+#undef STRICT_MM_TYPECHECKS
+
+#ifdef STRICT_MM_TYPECHECKS
+
+/*
+ * These are used to make use of C type-checking..
+ */
+typedef struct { pteval_t pte; } pte_t;
+typedef struct { pgdval_t pgd; } pgd_t;
+typedef struct { pteval_t pgprot; } pgprot_t;
+
+#define pte_val(x) ((x).pte)
+#define pgd_val(x) ((x).pgd)
+#define pgprot_val(x) ((x).pgprot)
+
+#define __pte(x) ((pte_t) { (x) } )
+#define __pgd(x) ((pgd_t) { (x) } )
+#define __pgprot(x) ((pgprot_t) { (x) } )
+
+#else /* !STRICT_MM_TYPECHECKS */
+
+typedef pteval_t pte_t;
+typedef pgdval_t pgd_t;
+typedef pteval_t pgprot_t;
+
+#define pte_val(x) (x)
+#define pgd_val(x) (x)
+#define pgprot_val(x) (x)
+
+#define __pte(x) (x)
+#define __pgd(x) (x)
+#define __pgprot(x) (x)
+
+#endif /* STRICT_MM_TYPECHECKS */
+
+#include <asm-generic/pgtable-nopmd.h>
+
+#endif /* __ASM_PGTABLE_2LEVEL_TYPES_H */
diff --git a/arch/arm64/include/asm/pgtable-3level-hwdef.h b/arch/arm64/include/asm/pgtable-3level-hwdef.h
new file mode 100644
index 0000000..3dbf941
--- /dev/null
+++ b/arch/arm64/include/asm/pgtable-3level-hwdef.h
@@ -0,0 +1,50 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGTABLE_3LEVEL_HWDEF_H
+#define __ASM_PGTABLE_3LEVEL_HWDEF_H
+
+/*
+ * With LPAE and 4KB pages, there are 3 levels of page tables. Each level has
+ * 512 entries of 8 bytes each, occupying a 4K page. The first level table
+ * covers a range of 512GB, each entry representing 1GB. The user and kernel
+ * address spaces are limited to 512GB each.
+ */
+#define PTRS_PER_PTE 512
+#define PTRS_PER_PMD 512
+#define PTRS_PER_PGD 512
+
+/*
+ * PGDIR_SHIFT determines the size a top-level page table entry can map.
+ */
+#define PGDIR_SHIFT 30
+#define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT)
+#define PGDIR_MASK (~(PGDIR_SIZE-1))
+
+/*
+ * PMD_SHIFT determines the size a middle-level page table entry can map.
+ */
+#define PMD_SHIFT 21
+#define PMD_SIZE (_AC(1, UL) << PMD_SHIFT)
+#define PMD_MASK (~(PMD_SIZE-1))
+
+/*
+ * section address mask and size definitions.
+ */
+#define SECTION_SHIFT 21
+#define SECTION_SIZE (_AC(1, UL) << SECTION_SHIFT)
+#define SECTION_MASK (~(SECTION_SIZE-1))
+
+#endif
diff --git a/arch/arm64/include/asm/pgtable-3level-types.h b/arch/arm64/include/asm/pgtable-3level-types.h
new file mode 100644
index 0000000..4489615
--- /dev/null
+++ b/arch/arm64/include/asm/pgtable-3level-types.h
@@ -0,0 +1,66 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGTABLE_3LEVEL_TYPES_H
+#define __ASM_PGTABLE_3LEVEL_TYPES_H
+
+typedef u64 pteval_t;
+typedef u64 pmdval_t;
+typedef u64 pgdval_t;
+
+#undef STRICT_MM_TYPECHECKS
+
+#ifdef STRICT_MM_TYPECHECKS
+
+/*
+ * These are used to make use of C type-checking..
+ */
+typedef struct { pteval_t pte; } pte_t;
+typedef struct { pmdval_t pmd; } pmd_t;
+typedef struct { pgdval_t pgd; } pgd_t;
+typedef struct { pteval_t pgprot; } pgprot_t;
+
+#define pte_val(x) ((x).pte)
+#define pmd_val(x) ((x).pmd)
+#define pgd_val(x) ((x).pgd)
+#define pgprot_val(x) ((x).pgprot)
+
+#define __pte(x) ((pte_t) { (x) } )
+#define __pmd(x) ((pmd_t) { (x) } )
+#define __pgd(x) ((pgd_t) { (x) } )
+#define __pgprot(x) ((pgprot_t) { (x) } )
+
+#else /* !STRICT_MM_TYPECHECKS */
+
+typedef pteval_t pte_t;
+typedef pmdval_t pmd_t;
+typedef pgdval_t pgd_t;
+typedef pteval_t pgprot_t;
+
+#define pte_val(x) (x)
+#define pmd_val(x) (x)
+#define pgd_val(x) (x)
+#define pgprot_val(x) (x)
+
+#define __pte(x) (x)
+#define __pmd(x) (x)
+#define __pgd(x) (x)
+#define __pgprot(x) (x)
+
+#endif /* STRICT_MM_TYPECHECKS */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#endif /* __ASM_PGTABLE_3LEVEL_TYPES_H */
diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
new file mode 100644
index 0000000..0f3b458
--- /dev/null
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -0,0 +1,94 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGTABLE_HWDEF_H
+#define __ASM_PGTABLE_HWDEF_H
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#include <asm/pgtable-2level-hwdef.h>
+#else
+#include <asm/pgtable-3level-hwdef.h>
+#endif
+
+/*
+ * Hardware page table definitions.
+ *
+ * Level 2 descriptor (PMD).
+ */
+#define PMD_TYPE_MASK (_AT(pmdval_t, 3) << 0)
+#define PMD_TYPE_FAULT (_AT(pmdval_t, 0) << 0)
+#define PMD_TYPE_TABLE (_AT(pmdval_t, 3) << 0)
+#define PMD_TYPE_SECT (_AT(pmdval_t, 1) << 0)
+
+/*
+ * Section
+ */
+#define PMD_SECT_S (_AT(pmdval_t, 3) << 8)
+#define PMD_SECT_AF (_AT(pmdval_t, 1) << 10)
+#define PMD_SECT_NG (_AT(pmdval_t, 1) << 11)
+#define PMD_SECT_XN (_AT(pmdval_t, 1) << 54)
+
+/*
+ * AttrIndx[2:0] encoding (mapping attributes defined in the MAIR* registers).
+ */
+#define PMD_ATTRINDX(t) (_AT(pmdval_t, (t)) << 2)
+#define PMD_ATTRINDX_MASK (_AT(pmdval_t, 7) << 2)
+
+/*
+ * Level 3 descriptor (PTE).
+ */
+#define PTE_TYPE_MASK (_AT(pteval_t, 3) << 0)
+#define PTE_TYPE_FAULT (_AT(pteval_t, 0) << 0)
+#define PTE_TYPE_PAGE (_AT(pteval_t, 3) << 0)
+#define PTE_USER (_AT(pteval_t, 1) << 6) /* AP[1] */
+#define PTE_RDONLY (_AT(pteval_t, 1) << 7) /* AP[2] */
+#define PTE_SHARED (_AT(pteval_t, 3) << 8) /* SH[1:0], inner shareable */
+#define PTE_AF (_AT(pteval_t, 1) << 10) /* Access Flag */
+#define PTE_NG (_AT(pteval_t, 1) << 11) /* nG */
+#define PTE_XN (_AT(pteval_t, 1) << 54) /* XN */
+
+/*
+ * AttrIndx[2:0] encoding (mapping attributes defined in the MAIR* registers).
+ */
+#define PTE_ATTRINDX(t) (_AT(pteval_t, (t)) << 2)
+#define PTE_ATTRINDX_MASK (_AT(pteval_t, 7) << 2)
+
+/*
+ * 40-bit physical address supported.
+ */
+#define PHYS_MASK_SHIFT (40)
+#define PHYS_MASK ((UL(1) << PHYS_MASK_SHIFT) - 1)
+
+/*
+ * TCR flags.
+ */
+#define TCR_TxSZ(x) (((UL(64) - (x)) << 16) | ((UL(64) - (x)) << 0))
+#define TCR_IRGN_NC ((UL(0) << 8) | (UL(0) << 24))
+#define TCR_IRGN_WBWA ((UL(1) << 8) | (UL(1) << 24))
+#define TCR_IRGN_WT ((UL(2) << 8) | (UL(2) << 24))
+#define TCR_IRGN_WBnWA ((UL(3) << 8) | (UL(3) << 24))
+#define TCR_IRGN_MASK ((UL(3) << 8) | (UL(3) << 24))
+#define TCR_ORGN_NC ((UL(0) << 10) | (UL(0) << 26))
+#define TCR_ORGN_WBWA ((UL(1) << 10) | (UL(1) << 26))
+#define TCR_ORGN_WT ((UL(2) << 10) | (UL(2) << 26))
+#define TCR_ORGN_WBnWA ((UL(3) << 10) | (UL(3) << 26))
+#define TCR_ORGN_MASK ((UL(3) << 10) | (UL(3) << 26))
+#define TCR_SHARED ((UL(3) << 12) | (UL(3) << 28))
+#define TCR_TG0_64K (UL(1) << 14)
+#define TCR_TG1_64K (UL(1) << 30)
+#define TCR_IPS_40BIT (UL(2) << 32)
+#define TCR_ASID16 (UL(1) << 36)
+
+#endif
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
new file mode 100644
index 0000000..8960239
--- /dev/null
+++ b/arch/arm64/include/asm/pgtable.h
@@ -0,0 +1,328 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PGTABLE_H
+#define __ASM_PGTABLE_H
+
+#include <asm/proc-fns.h>
+
+#include <asm/memory.h>
+#include <asm/pgtable-hwdef.h>
+
+/*
+ * Software defined PTE bits definition.
+ */
+#define PTE_VALID (_AT(pteval_t, 1) << 0) /* pte_present() check */
+#define PTE_FILE (_AT(pteval_t, 1) << 2) /* only when !pte_present() */
+#define PTE_DIRTY (_AT(pteval_t, 1) << 55)
+#define PTE_SPECIAL (_AT(pteval_t, 1) << 56)
+
+/*
+ * VMALLOC and SPARSEMEM_VMEMMAP ranges.
+ */
+#define VMALLOC_START UL(0xffffff8000000000)
+#define VMALLOC_END (PAGE_OFFSET - UL(0x400000000) - SZ_64K)
+
+#define vmemmap ((struct page *)(VMALLOC_END + SZ_64K))
+
+#define FIRST_USER_ADDRESS 0
+
+#ifndef __ASSEMBLY__
+extern void __pte_error(const char *file, int line, unsigned long val);
+extern void __pmd_error(const char *file, int line, unsigned long val);
+extern void __pgd_error(const char *file, int line, unsigned long val);
+
+#define pte_ERROR(pte) __pte_error(__FILE__, __LINE__, pte_val(pte))
+#ifndef CONFIG_ARM64_64K_PAGES
+#define pmd_ERROR(pmd) __pmd_error(__FILE__, __LINE__, pmd_val(pmd))
+#endif
+#define pgd_ERROR(pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd))
+
+/*
+ * The pgprot_* and protection_map entries will be fixed up at runtime to
+ * include the cachable and bufferable bits based on memory policy, as well as
+ * any architecture dependent bits like global/ASID and SMP shared mapping
+ * bits.
+ */
+#define _PAGE_DEFAULT PTE_TYPE_PAGE | PTE_AF
+
+extern pgprot_t pgprot_default;
+
+#define _MOD_PROT(p, b) __pgprot(pgprot_val(p) | (b))
+
+#define PAGE_NONE _MOD_PROT(pgprot_default, PTE_NG | PTE_XN | PTE_RDONLY)
+#define PAGE_SHARED _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_XN)
+#define PAGE_SHARED_EXEC _MOD_PROT(pgprot_default, PTE_USER | PTE_NG)
+#define PAGE_COPY _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_XN | PTE_RDONLY)
+#define PAGE_COPY_EXEC _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_RDONLY)
+#define PAGE_READONLY _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_XN | PTE_RDONLY)
+#define PAGE_READONLY_EXEC _MOD_PROT(pgprot_default, PTE_USER | PTE_NG | PTE_RDONLY)
+#define PAGE_KERNEL _MOD_PROT(pgprot_default, PTE_XN | PTE_DIRTY)
+#define PAGE_KERNEL_EXEC _MOD_PROT(pgprot_default, PTE_DIRTY)
+
+#define __PAGE_NONE __pgprot(_PAGE_DEFAULT | PTE_NG | PTE_XN | PTE_RDONLY)
+#define __PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_XN)
+#define __PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG)
+#define __PAGE_COPY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_XN | PTE_RDONLY)
+#define __PAGE_COPY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_RDONLY)
+#define __PAGE_READONLY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_XN | PTE_RDONLY)
+#define __PAGE_READONLY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_RDONLY)
+
+#endif /* __ASSEMBLY__ */
+
+#define __P000 __PAGE_NONE
+#define __P001 __PAGE_READONLY
+#define __P010 __PAGE_COPY
+#define __P011 __PAGE_COPY
+#define __P100 __PAGE_READONLY_EXEC
+#define __P101 __PAGE_READONLY_EXEC
+#define __P110 __PAGE_COPY_EXEC
+#define __P111 __PAGE_COPY_EXEC
+
+#define __S000 __PAGE_NONE
+#define __S001 __PAGE_READONLY
+#define __S010 __PAGE_SHARED
+#define __S011 __PAGE_SHARED
+#define __S100 __PAGE_READONLY_EXEC
+#define __S101 __PAGE_READONLY_EXEC
+#define __S110 __PAGE_SHARED_EXEC
+#define __S111 __PAGE_SHARED_EXEC
+
+#ifndef __ASSEMBLY__
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern struct page *empty_zero_page;
+#define ZERO_PAGE(vaddr) (empty_zero_page)
+
+#define pte_pfn(pte) ((pte_val(pte) & PHYS_MASK) >> PAGE_SHIFT)
+
+#define pfn_pte(pfn,prot) (__pte(((phys_addr_t)(pfn) << PAGE_SHIFT) | pgprot_val(prot)))
+
+#define pte_none(pte) (!pte_val(pte))
+#define pte_clear(mm,addr,ptep) set_pte(ptep, __pte(0))
+#define pte_page(pte) (pfn_to_page(pte_pfn(pte)))
+#define pte_offset_kernel(dir,addr) (pmd_page_vaddr(*(dir)) + __pte_index(addr))
+
+#define pte_offset_map(dir,addr) pte_offset_kernel((dir), (addr))
+#define pte_offset_map_nested(dir,addr) pte_offset_kernel((dir), (addr))
+#define pte_unmap(pte) do { } while (0)
+#define pte_unmap_nested(pte) do { } while (0)
+
+/*
+ * The following only work if pte_present(). Undefined behaviour otherwise.
+ */
+#define pte_present(pte) (pte_val(pte) & PTE_VALID)
+#define pte_dirty(pte) (pte_val(pte) & PTE_DIRTY)
+#define pte_young(pte) (pte_val(pte) & PTE_AF)
+#define pte_special(pte) (pte_val(pte) & PTE_SPECIAL)
+#define pte_write(pte) (!(pte_val(pte) & PTE_RDONLY))
+#define pte_exec(pte) (!(pte_val(pte) & PTE_XN))
+
+#define pte_present_exec_user(pte) \
+ ((pte_val(pte) & (PTE_VALID | PTE_USER | PTE_XN)) == \
+ (PTE_VALID | PTE_USER))
+
+#define PTE_BIT_FUNC(fn,op) \
+static inline pte_t pte_##fn(pte_t pte) { pte_val(pte) op; return pte; }
+
+PTE_BIT_FUNC(wrprotect, |= PTE_RDONLY);
+PTE_BIT_FUNC(mkwrite, &= ~PTE_RDONLY);
+PTE_BIT_FUNC(mkclean, &= ~PTE_DIRTY);
+PTE_BIT_FUNC(mkdirty, |= PTE_DIRTY);
+PTE_BIT_FUNC(mkold, &= ~PTE_AF);
+PTE_BIT_FUNC(mkyoung, |= PTE_AF);
+PTE_BIT_FUNC(mkspecial, |= PTE_SPECIAL);
+
+static inline void set_pte(pte_t *ptep, pte_t pte)
+{
+ *ptep = pte;
+}
+
+extern void __sync_icache_dcache(pte_t pteval, unsigned long addr);
+
+static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
+ pte_t *ptep, pte_t pte)
+{
+ if (pte_present_exec_user(pte))
+ __sync_icache_dcache(pte, addr);
+ set_pte(ptep, pte);
+}
+
+/*
+ * Huge pte definitions.
+ */
+#define pte_huge(pte) ((pte_val(pte) & PTE_TYPE_MASK) == PTE_TYPE_HUGEPAGE)
+#define pte_mkhuge(pte) (__pte((pte_val(pte) & ~PTE_TYPE_MASK) | PTE_TYPE_HUGEPAGE))
+
+#define __pgprot_modify(prot,mask,bits) \
+ __pgprot((pgprot_val(prot) & ~(mask)) | (bits))
+
+#define __HAVE_ARCH_PTE_SPECIAL
+
+/*
+ * Mark the prot value as uncacheable and unbufferable.
+ */
+#define pgprot_noncached(prot) \
+ __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_DEVICE_nGnRnE))
+#define pgprot_writecombine(prot) \
+ __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_DEVICE_GRE))
+#define pgprot_dmacoherent(prot) \
+ __pgprot_modify(prot, PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_NORMAL_NC))
+#define __HAVE_PHYS_MEM_ACCESS_PROT
+struct file;
+extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
+ unsigned long size, pgprot_t vma_prot);
+
+#define pmd_none(pmd) (!pmd_val(pmd))
+#define pmd_present(pmd) (pmd_val(pmd))
+
+#define pmd_bad(pmd) (!(pmd_val(pmd) & 2))
+
+static inline void set_pmd(pmd_t *pmdp, pmd_t pmd)
+{
+ *pmdp = pmd;
+ dsb();
+}
+
+static inline void pmd_clear(pmd_t *pmdp)
+{
+ set_pmd(pmdp, __pmd(0));
+}
+
+static inline pte_t *pmd_page_vaddr(pmd_t pmd)
+{
+ return __va(pmd_val(pmd) & PHYS_MASK & (s32)PAGE_MASK);
+}
+
+#define pmd_page(pmd) pfn_to_page(__phys_to_pfn(pmd_val(pmd) & PHYS_MASK))
+
+/*
+ * Conversion functions: convert a page and protection to a page entry,
+ * and a page entry and page directory to the page they refer to.
+ */
+#define mk_pte(page,prot) pfn_pte(page_to_pfn(page),prot)
+
+#ifndef CONFIG_ARM64_64K_PAGES
+
+#define pud_none(pud) (!pud_val(pud))
+#define pud_bad(pud) (!(pud_val(pud) & 2))
+#define pud_present(pud) (pud_val(pud))
+
+static inline void set_pud(pud_t *pudp, pud_t pud)
+{
+ *pudp = pud;
+ dsb();
+}
+
+static inline void pud_clear(pud_t *pudp)
+{
+ set_pud(pudp, __pud(0));
+}
+
+static inline pmd_t *pud_page_vaddr(pud_t pud)
+{
+ return __va(pud_val(pud) & PHYS_MASK & (s32)PAGE_MASK);
+}
+
+#endif /* CONFIG_ARM64_64K_PAGES */
+
+/* to find an entry in a page-table-directory */
+#define pgd_index(addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1))
+
+#define pgd_offset(mm, addr) ((mm)->pgd+pgd_index(addr))
+
+/* to find an entry in a kernel page-table-directory */
+#define pgd_offset_k(addr) pgd_offset(&init_mm, addr)
+
+/* Find an entry in the second-level page table.. */
+#ifndef CONFIG_ARM64_64K_PAGES
+#define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
+static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr)
+{
+ return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(addr);
+}
+#endif
+
+/* Find an entry in the third-level page table.. */
+#define __pte_index(addr) (((addr) >> PAGE_SHIFT) & (PTRS_PER_PTE - 1))
+
+static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
+{
+ const pteval_t mask = PTE_USER | PTE_XN | PTE_RDONLY;
+ pte_val(pte) = (pte_val(pte) & ~mask) | (pgprot_val(newprot) & mask);
+ return pte;
+}
+
+extern pgd_t swapper_pg_dir[PTRS_PER_PGD];
+extern pgd_t idmap_pg_dir[PTRS_PER_PGD];
+
+#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE)
+#define IDMAP_DIR_SIZE (2 * PAGE_SIZE)
+
+/*
+ * Encode and decode a swap entry:
+ * bits 0-1: present (must be zero)
+ * bit 2: PTE_FILE
+ * bits 3-8: swap type
+ * bits 9-63: swap offset
+ */
+#define __SWP_TYPE_SHIFT 3
+#define __SWP_TYPE_BITS 6
+#define __SWP_TYPE_MASK ((1 << __SWP_TYPE_BITS) - 1)
+#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT)
+
+#define __swp_type(x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)
+#define __swp_offset(x) ((x).val >> __SWP_OFFSET_SHIFT)
+#define __swp_entry(type,offset) ((swp_entry_t) { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) })
+
+#define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(swp) ((pte_t) { (swp).val })
+
+/*
+ * Ensure that there are not more swap files than can be encoded in the kernel
+ * the PTEs.
+ */
+#define MAX_SWAPFILES_CHECK() BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > __SWP_TYPE_BITS)
+
+/*
+ * Encode and decode a file entry:
+ * bits 0-1: present (must be zero)
+ * bit 2: PTE_FILE
+ * bits 3-63: file offset / PAGE_SIZE
+ */
+#define pte_file(pte) (pte_val(pte) & PTE_FILE)
+#define pte_to_pgoff(x) (pte_val(x) >> 3)
+#define pgoff_to_pte(x) __pte(((x) << 3) | PTE_FILE)
+
+#define PTE_FILE_MAX_BITS 61
+
+extern int kern_addr_valid(unsigned long addr);
+
+#include <asm-generic/pgtable.h>
+
+/*
+ * remap a physical page `pfn' of size `size' with page protection `prot'
+ * into virtual address `from'
+ */
+#define io_remap_pfn_range(vma,from,pfn,size,prot) \
+ remap_pfn_range(vma, from, pfn, size, prot)
+
+#define pgtable_cache_init() do { } while (0)
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_PGTABLE_H */
diff --git a/arch/arm64/include/asm/pmu.h b/arch/arm64/include/asm/pmu.h
new file mode 100644
index 0000000..e6f0878
--- /dev/null
+++ b/arch/arm64/include/asm/pmu.h
@@ -0,0 +1,82 @@
+/*
+ * Based on arch/arm/include/asm/pmu.h
+ *
+ * Copyright (C) 2009 picoChip Designs Ltd, Jamie Iles
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PMU_H
+#define __ASM_PMU_H
+
+#ifdef CONFIG_HW_PERF_EVENTS
+
+/* The events for a given PMU register set. */
+struct pmu_hw_events {
+ /*
+ * The events that are active on the PMU for the given index.
+ */
+ struct perf_event **events;
+
+ /*
+ * A 1 bit for an index indicates that the counter is being used for
+ * an event. A 0 means that the counter can be used.
+ */
+ unsigned long *used_mask;
+
+ /*
+ * Hardware lock to serialize accesses to PMU registers. Needed for the
+ * read/modify/write sequences.
+ */
+ raw_spinlock_t pmu_lock;
+};
+
+struct arm_pmu {
+ struct pmu pmu;
+ cpumask_t active_irqs;
+ const char *name;
+ irqreturn_t (*handle_irq)(int irq_num, void *dev);
+ void (*enable)(struct hw_perf_event *evt, int idx);
+ void (*disable)(struct hw_perf_event *evt, int idx);
+ int (*get_event_idx)(struct pmu_hw_events *hw_events,
+ struct hw_perf_event *hwc);
+ int (*set_event_filter)(struct hw_perf_event *evt,
+ struct perf_event_attr *attr);
+ u32 (*read_counter)(int idx);
+ void (*write_counter)(int idx, u32 val);
+ void (*start)(void);
+ void (*stop)(void);
+ void (*reset)(void *);
+ int (*map_event)(struct perf_event *event);
+ int num_events;
+ atomic_t active_events;
+ struct mutex reserve_mutex;
+ u64 max_period;
+ struct platform_device *plat_device;
+ struct pmu_hw_events *(*get_hw_events)(void);
+};
+
+#define to_arm_pmu(p) (container_of(p, struct arm_pmu, pmu))
+
+int __init armpmu_register(struct arm_pmu *armpmu, char *name, int type);
+
+u64 armpmu_event_update(struct perf_event *event,
+ struct hw_perf_event *hwc,
+ int idx);
+
+int armpmu_event_set_period(struct perf_event *event,
+ struct hw_perf_event *hwc,
+ int idx);
+
+#endif /* CONFIG_HW_PERF_EVENTS */
+#endif /* __ASM_PMU_H */
diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h
new file mode 100644
index 0000000..7cdf466
--- /dev/null
+++ b/arch/arm64/include/asm/proc-fns.h
@@ -0,0 +1,50 @@
+/*
+ * Based on arch/arm/include/asm/proc-fns.h
+ *
+ * Copyright (C) 1997-1999 Russell King
+ * Copyright (C) 2000 Deep Blue Solutions Ltd
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PROCFNS_H
+#define __ASM_PROCFNS_H
+
+#ifdef __KERNEL__
+#ifndef __ASSEMBLY__
+
+#include <asm/page.h>
+
+struct mm_struct;
+
+extern void cpu_cache_off(void);
+extern void cpu_do_idle(void);
+extern void cpu_do_switch_mm(unsigned long pgd_phys, struct mm_struct *mm);
+extern void cpu_reset(unsigned long addr) __attribute__((noreturn));
+
+#include <asm/memory.h>
+
+#define cpu_switch_mm(pgd,mm) cpu_do_switch_mm(virt_to_phys(pgd),mm)
+
+#define cpu_get_pgd() \
+({ \
+ unsigned long pg; \
+ asm("mrs %0, ttbr0_el1\n" \
+ : "=r" (pg)); \
+ pg &= ~0xffff000000003ffful; \
+ (pgd_t *)phys_to_virt(pg); \
+})
+
+#endif /* __ASSEMBLY__ */
+#endif /* __KERNEL__ */
+#endif /* __ASM_PROCFNS_H */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
new file mode 100644
index 0000000..39a208a
--- /dev/null
+++ b/arch/arm64/include/asm/processor.h
@@ -0,0 +1,175 @@
+/*
+ * Based on arch/arm/include/asm/processor.h
+ *
+ * Copyright (C) 1995-1999 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PROCESSOR_H
+#define __ASM_PROCESSOR_H
+
+/*
+ * Default implementation of macro that returns current
+ * instruction pointer ("program counter").
+ */
+#define current_text_addr() ({ __label__ _l; _l: &&_l;})
+
+#ifdef __KERNEL__
+
+#include <linux/string.h>
+
+#include <asm/fpsimd.h>
+#include <asm/hw_breakpoint.h>
+#include <asm/ptrace.h>
+#include <asm/types.h>
+
+#ifdef __KERNEL__
+#define STACK_TOP_MAX TASK_SIZE_64
+#ifdef CONFIG_COMPAT
+#define AARCH32_VECTORS_BASE 0xffff0000
+#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
+ AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+#else
+#define STACK_TOP STACK_TOP_MAX
+#endif /* CONFIG_COMPAT */
+#endif /* __KERNEL__ */
+
+struct debug_info {
+ /* Have we suspended stepping by a debugger? */
+ int suspended_step;
+ /* Allow breakpoints and watchpoints to be disabled for this thread. */
+ int bps_disabled;
+ int wps_disabled;
+ /* Hardware breakpoints pinned to this task. */
+ struct perf_event *hbp_break[ARM_MAX_BRP];
+ struct perf_event *hbp_watch[ARM_MAX_WRP];
+};
+
+struct cpu_context {
+ unsigned long x19;
+ unsigned long x20;
+ unsigned long x21;
+ unsigned long x22;
+ unsigned long x23;
+ unsigned long x24;
+ unsigned long x25;
+ unsigned long x26;
+ unsigned long x27;
+ unsigned long x28;
+ unsigned long fp;
+ unsigned long sp;
+ unsigned long pc;
+};
+
+struct thread_struct {
+ struct cpu_context cpu_context; /* cpu context */
+ unsigned long tp_value;
+ struct fpsimd_state fpsimd_state;
+ unsigned long fault_address; /* fault info */
+ struct debug_info debug; /* debugging */
+};
+
+#define INIT_THREAD { }
+
+static inline void start_thread_common(struct pt_regs *regs, unsigned long pc)
+{
+ memset(regs, 0, sizeof(*regs));
+ regs->syscallno = ~0UL;
+ regs->pc = pc;
+}
+
+static inline void start_thread(struct pt_regs *regs, unsigned long pc,
+ unsigned long sp)
+{
+ unsigned long *stack = (unsigned long *)sp;
+
+ start_thread_common(regs, pc);
+ regs->pstate = PSR_MODE_EL0t;
+ regs->sp = sp;
+ regs->regs[2] = stack[2]; /* x2 (envp) */
+ regs->regs[1] = stack[1]; /* x1 (argv) */
+ regs->regs[0] = stack[0]; /* x0 (argc) */
+}
+
+#ifdef CONFIG_COMPAT
+static inline void compat_start_thread(struct pt_regs *regs, unsigned long pc,
+ unsigned long sp)
+{
+ unsigned int *stack = (unsigned int *)sp;
+
+ start_thread_common(regs, pc);
+ regs->pstate = COMPAT_PSR_MODE_USR;
+ if (pc & 1)
+ regs->pstate |= COMPAT_PSR_T_BIT;
+ regs->compat_sp = sp;
+ regs->regs[2] = stack[2]; /* x2 (envp) */
+ regs->regs[1] = stack[1]; /* x1 (argv) */
+ regs->regs[0] = stack[0]; /* x0 (argc) */
+}
+#endif
+
+/* Forward declaration, a strange C thing */
+struct task_struct;
+
+/* Free all resources held by a thread. */
+extern void release_thread(struct task_struct *);
+
+/* Prepare to copy thread state - unlazy all lazy status */
+#define prepare_to_copy(tsk) do { } while (0)
+
+unsigned long get_wchan(struct task_struct *p);
+
+#define cpu_relax() barrier()
+
+/* Thread switching */
+extern struct task_struct *cpu_switch_to(struct task_struct *prev,
+ struct task_struct *next);
+
+/*
+ * Create a new kernel thread
+ */
+extern int kernel_thread(int (*fn)(void *), void *arg, unsigned long flags);
+
+#define task_pt_regs(p) \
+ ((struct pt_regs *)(THREAD_START_SP + task_stack_page(p)) - 1)
+
+#define KSTK_EIP(tsk) task_pt_regs(tsk)->pc
+#define KSTK_ESP(tsk) task_pt_regs(tsk)->sp
+
+/*
+ * Prefetching support
+ */
+#define ARCH_HAS_PREFETCH
+static inline void prefetch(const void *ptr)
+{
+ asm volatile("prfm pldl1keep, %a0\n" : : "p" (ptr));
+}
+
+#define ARCH_HAS_PREFETCHW
+static inline void prefetchw(const void *ptr)
+{
+ asm volatile("prfm pstl1keep, %a0\n" : : "p" (ptr));
+}
+
+#define ARCH_HAS_SPINLOCK_PREFETCH
+static inline void spin_lock_prefetch(const void *x)
+{
+ prefetchw(x);
+}
+
+#define HAVE_ARCH_PICK_MMAP_LAYOUT
+
+#endif
+
+#endif /* __ASM_PROCESSOR_H */
diff --git a/arch/arm64/include/asm/prom.h b/arch/arm64/include/asm/prom.h
new file mode 100644
index 0000000..68b90e6
--- /dev/null
+++ b/arch/arm64/include/asm/prom.h
@@ -0,0 +1 @@
+/* Empty for now */
diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h
new file mode 100644
index 0000000..0fa5d6c
--- /dev/null
+++ b/arch/arm64/include/asm/ptrace.h
@@ -0,0 +1,207 @@
+/*
+ * Based on arch/arm/include/asm/ptrace.h
+ *
+ * Copyright (C) 1996-2003 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_PTRACE_H
+#define __ASM_PTRACE_H
+
+#include <linux/types.h>
+
+#include <asm/hwcap.h>
+
+/* AArch32-specific ptrace requests */
+#define COMPAT_PTRACE_GETREGS 12
+#define COMPAT_PTRACE_SETREGS 13
+#define COMPAT_PTRACE_GET_THREAD_AREA 22
+#define COMPAT_PTRACE_SET_SYSCALL 23
+#define COMPAT_PTRACE_GETVFPREGS 27
+#define COMPAT_PTRACE_SETVFPREGS 28
+#define COMPAT_PTRACE_GETHBPREGS 29
+#define COMPAT_PTRACE_SETHBPREGS 30
+
+/*
+ * PSR bits
+ */
+#define PSR_MODE_EL0t 0x00000000
+#define PSR_MODE_EL1t 0x00000004
+#define PSR_MODE_EL1h 0x00000005
+#define PSR_MODE_EL2t 0x00000008
+#define PSR_MODE_EL2h 0x00000009
+#define PSR_MODE_EL3t 0x0000000c
+#define PSR_MODE_EL3h 0x0000000d
+#define PSR_MODE_MASK 0x0000000f
+
+/* AArch32 CPSR bits */
+#define PSR_MODE32_BIT 0x00000010
+#define COMPAT_PSR_MODE_USR 0x00000010
+#define COMPAT_PSR_T_BIT 0x00000020
+#define COMPAT_PSR_IT_MASK 0x0600fc00 /* If-Then execution state mask */
+
+/* AArch64 SPSR bits */
+#define PSR_F_BIT 0x00000040
+#define PSR_I_BIT 0x00000080
+#define PSR_A_BIT 0x00000100
+#define PSR_D_BIT 0x00000200
+#define PSR_Q_BIT 0x08000000
+#define PSR_V_BIT 0x10000000
+#define PSR_C_BIT 0x20000000
+#define PSR_Z_BIT 0x40000000
+#define PSR_N_BIT 0x80000000
+
+/*
+ * Groups of PSR bits
+ */
+#define PSR_f 0xff000000 /* Flags */
+#define PSR_s 0x00ff0000 /* Status */
+#define PSR_x 0x0000ff00 /* Extension */
+#define PSR_c 0x000000ff /* Control */
+
+/*
+ * These are 'magic' values for PTRACE_PEEKUSR that return info about where a
+ * process is located in memory.
+ */
+#define PT_TEXT_ADDR 0x10000
+#define PT_DATA_ADDR 0x10004
+#define PT_TEXT_END_ADDR 0x10008
+
+#ifndef __ASSEMBLY__
+
+/*
+ * User structures for general purpose, floating point and debug registers.
+ */
+struct user_pt_regs {
+ __u64 regs[31];
+ __u64 sp;
+ __u64 pc;
+ __u64 pstate;
+};
+
+struct user_fpsimd_state {
+ __uint128_t vregs[32];
+ __u32 fpsr;
+ __u32 fpcr;
+};
+
+struct user_hwdebug_state {
+ __u32 dbg_info;
+ struct {
+ __u64 addr;
+ __u32 ctrl;
+ } dbg_regs[16];
+};
+
+#ifdef __KERNEL__
+
+/* sizeof(struct user) for AArch32 */
+#define COMPAT_USER_SZ 296
+/* AArch32 uses x13 as the stack pointer... */
+#define compat_sp regs[13]
+/* ... and x14 as the link register. */
+#define compat_lr regs[14]
+
+/*
+ * This struct defines the way the registers are stored on the stack during an
+ * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for
+ * stack alignment). struct user_pt_regs must form a prefix of struct pt_regs.
+ */
+struct pt_regs {
+ union {
+ struct user_pt_regs user_regs;
+ struct {
+ u64 regs[31];
+ u64 sp;
+ u64 pc;
+ u64 pstate;
+ };
+ };
+ u64 orig_x0;
+ u64 syscallno;
+};
+
+#define arch_has_single_step() (1)
+
+#ifdef CONFIG_COMPAT
+#define compat_thumb_mode(regs) \
+ (((regs)->pstate & COMPAT_PSR_T_BIT))
+#else
+#define compat_thumb_mode(regs) (0)
+#endif
+
+#define user_mode(regs) \
+ (((regs)->pstate & PSR_MODE_MASK) == PSR_MODE_EL0t)
+
+#define compat_user_mode(regs) \
+ (((regs)->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK)) == \
+ (PSR_MODE32_BIT | PSR_MODE_EL0t))
+
+#define processor_mode(regs) \
+ ((regs)->pstate & PSR_MODE_MASK)
+
+#define interrupts_enabled(regs) \
+ (!((regs)->pstate & PSR_I_BIT))
+
+#define fast_interrupts_enabled(regs) \
+ (!((regs)->pstate & PSR_F_BIT))
+
+#define user_stack_pointer(regs) \
+ ((regs)->sp)
+
+/*
+ * Are the current registers suitable for user mode? (used to maintain
+ * security in signal handlers)
+ */
+static inline int valid_user_regs(struct user_pt_regs *regs)
+{
+ if (user_mode(regs) && (regs->pstate & PSR_I_BIT) == 0) {
+ regs->pstate &= ~(PSR_F_BIT | PSR_A_BIT);
+
+ /* The T bit is reserved for AArch64 */
+ if (!(regs->pstate & PSR_MODE32_BIT))
+ regs->pstate &= ~COMPAT_PSR_T_BIT;
+
+ return 1;
+ }
+
+ /*
+ * Force PSR to something logical...
+ */
+ regs->pstate &= PSR_f | PSR_s | (PSR_x & ~PSR_A_BIT) | \
+ COMPAT_PSR_T_BIT | PSR_MODE32_BIT;
+
+ if (!(regs->pstate & PSR_MODE32_BIT)) {
+ regs->pstate &= ~COMPAT_PSR_T_BIT;
+ regs->pstate |= PSR_MODE_EL0t;
+ }
+
+ return 0;
+}
+
+#define instruction_pointer(regs) (regs)->pc
+
+#ifdef CONFIG_SMP
+extern unsigned long profile_pc(struct pt_regs *regs);
+#else
+#define profile_pc(regs) instruction_pointer(regs)
+#endif
+
+extern int aarch32_break_trap(struct pt_regs *regs);
+
+#endif /* __KERNEL__ */
+
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/arm64/include/asm/setup.h b/arch/arm64/include/asm/setup.h
new file mode 100644
index 0000000..9cf2e46
--- /dev/null
+++ b/arch/arm64/include/asm/setup.h
@@ -0,0 +1,26 @@
+/*
+ * Based on arch/arm/include/asm/setup.h
+ *
+ * Copyright (C) 1997-1999 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SETUP_H
+#define __ASM_SETUP_H
+
+#include <linux/types.h>
+
+#define COMMAND_LINE_SIZE 2048
+
+#endif
diff --git a/arch/arm64/include/asm/shmparam.h b/arch/arm64/include/asm/shmparam.h
new file mode 100644
index 0000000..4df608a
--- /dev/null
+++ b/arch/arm64/include/asm/shmparam.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SHMPARAM_H
+#define __ASM_SHMPARAM_H
+
+/*
+ * For IPC syscalls from compat tasks, we need to use the legacy 16k
+ * alignment value. Since we don't have aliasing D-caches, the rest of
+ * the time we can safely use PAGE_SIZE.
+ */
+#define COMPAT_SHMLBA 0x4000
+
+#include <asm-generic/shmparam.h>
+
+#endif /* __ASM_SHMPARAM_H */
diff --git a/arch/arm64/include/asm/sigcontext.h b/arch/arm64/include/asm/sigcontext.h
new file mode 100644
index 0000000..573cec7
--- /dev/null
+++ b/arch/arm64/include/asm/sigcontext.h
@@ -0,0 +1,69 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SIGCONTEXT_H
+#define __ASM_SIGCONTEXT_H
+
+#include <linux/types.h>
+
+/*
+ * Signal context structure - contains all info to do with the state
+ * before the signal handler was invoked.
+ */
+struct sigcontext {
+ __u64 fault_address;
+ /* AArch64 registers */
+ __u64 regs[31];
+ __u64 sp;
+ __u64 pc;
+ __u64 pstate;
+ /* 4K reserved for FP/SIMD state and future expansion */
+ __u8 __reserved[4096] __attribute__((__aligned__(16)));
+};
+
+/*
+ * Header to be used at the beginning of structures extending the user
+ * context. Such structures must be placed after the rt_sigframe on the stack
+ * and be 16-byte aligned. The last structure must be a dummy one with the
+ * magic and size set to 0.
+ */
+struct _aarch64_ctx {
+ __u32 magic;
+ __u32 size;
+};
+
+#define FPSIMD_MAGIC 0x46508001
+
+struct fpsimd_context {
+ struct _aarch64_ctx head;
+ __u32 fpsr;
+ __u32 fpcr;
+ __uint128_t vregs[32];
+};
+
+#ifdef __KERNEL__
+/*
+ * Auxiliary context saved in the sigcontext.__reserved array. Not exported to
+ * user space as it will change with the addition of new context. User space
+ * should check the magic/size information.
+ */
+struct aux_context {
+ struct fpsimd_context fpsimd;
+ /* additional context to be added before "end" */
+ struct _aarch64_ctx end;
+};
+#endif
+
+#endif
diff --git a/arch/arm64/include/asm/siginfo.h b/arch/arm64/include/asm/siginfo.h
new file mode 100644
index 0000000..5a74a08
--- /dev/null
+++ b/arch/arm64/include/asm/siginfo.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SIGINFO_H
+#define __ASM_SIGINFO_H
+
+#define __ARCH_SI_PREAMBLE_SIZE (4 * sizeof(int))
+
+#include <asm-generic/siginfo.h>
+
+#endif
diff --git a/arch/arm64/include/asm/signal.h b/arch/arm64/include/asm/signal.h
new file mode 100644
index 0000000..8d1e723
--- /dev/null
+++ b/arch/arm64/include/asm/signal.h
@@ -0,0 +1,24 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SIGNAL_H
+#define __ASM_SIGNAL_H
+
+/* Required for AArch32 compatibility. */
+#define SA_RESTORER 0x04000000
+
+#include <asm-generic/signal.h>
+
+#endif
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
new file mode 100644
index 0000000..7c275e3
--- /dev/null
+++ b/arch/arm64/include/asm/signal32.h
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SIGNAL32_H
+#define __ASM_SIGNAL32_H
+
+#ifdef __KERNEL__
+#ifdef CONFIG_COMPAT
+#include <linux/compat.h>
+
+#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
+
+extern const compat_ulong_t aarch32_sigret_code[6];
+
+int compat_setup_frame(int usig, struct k_sigaction *ka, sigset_t *set,
+ struct pt_regs *regs);
+int compat_setup_rt_frame(int usig, struct k_sigaction *ka, siginfo_t *info,
+ sigset_t *set, struct pt_regs *regs);
+
+void compat_setup_restart_syscall(struct pt_regs *regs);
+#else
+
+static inline int compat_setup_frame(int usid, struct k_sigaction *ka,
+ sigset_t *set, struct pt_regs *regs)
+{
+ return -ENOSYS;
+}
+
+static inline int compat_setup_rt_frame(int usig, struct k_sigaction *ka,
+ siginfo_t *info, sigset_t *set,
+ struct pt_regs *regs)
+{
+ return -ENOSYS;
+}
+
+static inline void compat_setup_restart_syscall(struct pt_regs *regs)
+{
+}
+#endif /* CONFIG_COMPAT */
+#endif /* __KERNEL__ */
+#endif /* __ASM_SIGNAL32_H */
diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h
new file mode 100644
index 0000000..7e34295
--- /dev/null
+++ b/arch/arm64/include/asm/smp.h
@@ -0,0 +1,69 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SMP_H
+#define __ASM_SMP_H
+
+#include <linux/threads.h>
+#include <linux/cpumask.h>
+#include <linux/thread_info.h>
+
+#ifndef CONFIG_SMP
+# error "<asm/smp.h> included in non-SMP build"
+#endif
+
+#define raw_smp_processor_id() (current_thread_info()->cpu)
+
+struct seq_file;
+
+/*
+ * generate IPI list text
+ */
+extern void show_ipi_list(struct seq_file *p, int prec);
+
+/*
+ * Called from C code, this handles an IPI.
+ */
+extern void handle_IPI(int ipinr, struct pt_regs *regs);
+
+/*
+ * Setup the set of possible CPUs (via set_cpu_possible)
+ */
+extern void smp_init_cpus(void);
+
+/*
+ * Provide a function to raise an IPI cross call on CPUs in callmap.
+ */
+extern void set_smp_cross_call(void (*)(const struct cpumask *, unsigned int));
+
+/*
+ * Called from the secondary holding pen, this is the secondary CPU entry point.
+ */
+asmlinkage void secondary_start_kernel(void);
+
+/*
+ * Initial data for bringing up a secondary CPU.
+ */
+struct secondary_data {
+ void *stack;
+};
+extern struct secondary_data secondary_data;
+extern void secondary_holding_pen(void);
+extern volatile unsigned long secondary_holding_pen_release;
+
+extern void arch_send_call_function_single_ipi(int cpu);
+extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
+
+#endif /* ifndef __ASM_SMP_H */
diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
new file mode 100644
index 0000000..1be62bc
--- /dev/null
+++ b/arch/arm64/include/asm/sparsemem.h
@@ -0,0 +1,24 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SPARSEMEM_H
+#define __ASM_SPARSEMEM_H
+
+#ifdef CONFIG_SPARSEMEM
+#define MAX_PHYSMEM_BITS 40
+#define SECTION_SIZE_BITS 30
+#endif
+
+#endif
diff --git a/arch/arm64/include/asm/spinlock.h b/arch/arm64/include/asm/spinlock.h
new file mode 100644
index 0000000..41112fe
--- /dev/null
+++ b/arch/arm64/include/asm/spinlock.h
@@ -0,0 +1,202 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SPINLOCK_H
+#define __ASM_SPINLOCK_H
+
+#include <asm/spinlock_types.h>
+#include <asm/processor.h>
+
+/*
+ * Spinlock implementation.
+ *
+ * The old value is read exclusively and the new one, if unlocked, is written
+ * exclusively. In case of failure, the loop is restarted.
+ *
+ * The memory barriers are implicit with the load-acquire and store-release
+ * instructions.
+ *
+ * Unlocked value: 0
+ * Locked value: 1
+ */
+
+#define arch_spin_is_locked(x) ((x)->lock != 0)
+#define arch_spin_unlock_wait(lock) \
+ do { while (arch_spin_is_locked(lock)) cpu_relax(); } while (0)
+
+#define arch_spin_lock_flags(lock, flags) arch_spin_lock(lock)
+
+static inline void arch_spin_lock(arch_spinlock_t *lock)
+{
+ unsigned int tmp;
+
+ asm volatile(
+ " sevl\n"
+ "1: wfe\n"
+ "2: ldaxr %w0, [%1]\n"
+ " cbnz %w0, 1b\n"
+ " stxr %w0, %w2, [%1]\n"
+ " cbnz %w0, 2b\n"
+ : "=&r" (tmp)
+ : "r" (&lock->lock), "r" (1)
+ : "memory");
+}
+
+static inline int arch_spin_trylock(arch_spinlock_t *lock)
+{
+ unsigned int tmp;
+
+ asm volatile(
+ " ldaxr %w0, [%1]\n"
+ " cbnz %w0, 1f\n"
+ " stxr %w0, %w2, [%1]\n"
+ "1:\n"
+ : "=&r" (tmp)
+ : "r" (&lock->lock), "r" (1)
+ : "memory");
+
+ return !tmp;
+}
+
+static inline void arch_spin_unlock(arch_spinlock_t *lock)
+{
+ asm volatile(
+ " stlr %w1, [%0]\n"
+ : : "r" (&lock->lock), "r" (0) : "memory");
+}
+
+/*
+ * Write lock implementation.
+ *
+ * Write locks set bit 31. Unlocking, is done by writing 0 since the lock is
+ * exclusively held.
+ *
+ * The memory barriers are implicit with the load-acquire and store-release
+ * instructions.
+ */
+
+static inline void arch_write_lock(arch_rwlock_t *rw)
+{
+ unsigned int tmp;
+
+ asm volatile(
+ " sevl\n"
+ "1: wfe\n"
+ "2: ldaxr %w0, [%1]\n"
+ " cbnz %w0, 1b\n"
+ " stxr %w0, %w2, [%1]\n"
+ " cbnz %w0, 2b\n"
+ : "=&r" (tmp)
+ : "r" (&rw->lock), "r" (0x80000000)
+ : "memory");
+}
+
+static inline int arch_write_trylock(arch_rwlock_t *rw)
+{
+ unsigned int tmp;
+
+ asm volatile(
+ " ldaxr %w0, [%1]\n"
+ " cbnz %w0, 1f\n"
+ " stxr %w0, %w2, [%1]\n"
+ "1:\n"
+ : "=&r" (tmp)
+ : "r" (&rw->lock), "r" (0x80000000)
+ : "memory");
+
+ return !tmp;
+}
+
+static inline void arch_write_unlock(arch_rwlock_t *rw)
+{
+ asm volatile(
+ " stlr %w1, [%0]\n"
+ : : "r" (&rw->lock), "r" (0) : "memory");
+}
+
+/* write_can_lock - would write_trylock() succeed? */
+#define arch_write_can_lock(x) ((x)->lock == 0)
+
+/*
+ * Read lock implementation.
+ *
+ * It exclusively loads the lock value, increments it and stores the new value
+ * back if positive and the CPU still exclusively owns the location. If the
+ * value is negative, the lock is already held.
+ *
+ * During unlocking there may be multiple active read locks but no write lock.
+ *
+ * The memory barriers are implicit with the load-acquire and store-release
+ * instructions.
+ */
+static inline void arch_read_lock(arch_rwlock_t *rw)
+{
+ unsigned int tmp, tmp2;
+
+ asm volatile(
+ " sevl\n"
+ "1: wfe\n"
+ "2: ldaxr %w0, [%2]\n"
+ " add %w0, %w0, #1\n"
+ " tbnz %w0, #31, 1b\n"
+ " stxr %w1, %w0, [%2]\n"
+ " cbnz %w1, 2b\n"
+ : "=&r" (tmp), "=&r" (tmp2)
+ : "r" (&rw->lock)
+ : "memory");
+}
+
+static inline void arch_read_unlock(arch_rwlock_t *rw)
+{
+ unsigned int tmp, tmp2;
+
+ asm volatile(
+ "1: ldxr %w0, [%2]\n"
+ " sub %w0, %w0, #1\n"
+ " stlxr %w1, %w0, [%2]\n"
+ " cbnz %w1, 1b\n"
+ : "=&r" (tmp), "=&r" (tmp2)
+ : "r" (&rw->lock)
+ : "memory");
+}
+
+static inline int arch_read_trylock(arch_rwlock_t *rw)
+{
+ unsigned int tmp, tmp2 = 1;
+
+ asm volatile(
+ " ldaxr %w0, [%2]\n"
+ " add %w0, %w0, #1\n"
+ " tbnz %w0, #31, 1f\n"
+ " stxr %w1, %w0, [%2]\n"
+ "1:\n"
+ : "=&r" (tmp), "+r" (tmp2)
+ : "r" (&rw->lock)
+ : "memory");
+
+ return !tmp2;
+}
+
+/* read_can_lock - would read_trylock() succeed? */
+#define arch_read_can_lock(x) ((x)->lock < 0x80000000)
+
+#define arch_read_lock_flags(lock, flags) arch_read_lock(lock)
+#define arch_write_lock_flags(lock, flags) arch_write_lock(lock)
+
+#define arch_spin_relax(lock) cpu_relax()
+#define arch_read_relax(lock) cpu_relax()
+#define arch_write_relax(lock) cpu_relax()
+
+#endif /* __ASM_SPINLOCK_H */
diff --git a/arch/arm64/include/asm/spinlock_types.h b/arch/arm64/include/asm/spinlock_types.h
new file mode 100644
index 0000000..9a49434
--- /dev/null
+++ b/arch/arm64/include/asm/spinlock_types.h
@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SPINLOCK_TYPES_H
+#define __ASM_SPINLOCK_TYPES_H
+
+#if !defined(__LINUX_SPINLOCK_TYPES_H) && !defined(__ASM_SPINLOCK_H)
+# error "please don't include this file directly"
+#endif
+
+/* We only require natural alignment for exclusive accesses. */
+#define __lock_aligned
+
+typedef struct {
+ volatile unsigned int lock;
+} arch_spinlock_t;
+
+#define __ARCH_SPIN_LOCK_UNLOCKED { 0 }
+
+typedef struct {
+ volatile unsigned int lock;
+} arch_rwlock_t;
+
+#define __ARCH_RW_LOCK_UNLOCKED { 0 }
+
+#endif
diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h
new file mode 100644
index 0000000..7318f6d
--- /dev/null
+++ b/arch/arm64/include/asm/stacktrace.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_STACKTRACE_H
+#define __ASM_STACKTRACE_H
+
+struct stackframe {
+ unsigned long fp;
+ unsigned long sp;
+ unsigned long pc;
+};
+
+extern int unwind_frame(struct stackframe *frame);
+extern void walk_stackframe(struct stackframe *frame,
+ int (*fn)(struct stackframe *, void *), void *data);
+
+#endif /* __ASM_STACKTRACE_H */
diff --git a/arch/arm64/include/asm/stat.h b/arch/arm64/include/asm/stat.h
new file mode 100644
index 0000000..d87225c
--- /dev/null
+++ b/arch/arm64/include/asm/stat.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_STAT_H
+#define __ASM_STAT_H
+
+#include <asm-generic/stat.h>
+
+#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
+
+#include <asm/compat.h>
+
+/*
+ * struct stat64 is needed for compat tasks only. Its definition is different
+ * from the generic struct stat64.
+ */
+struct stat64 {
+ compat_u64 st_dev;
+ unsigned char __pad0[4];
+
+#define STAT64_HAS_BROKEN_ST_INO 1
+ compat_ulong_t __st_ino;
+ compat_uint_t st_mode;
+ compat_uint_t st_nlink;
+
+ compat_ulong_t st_uid;
+ compat_ulong_t st_gid;
+
+ compat_u64 st_rdev;
+ unsigned char __pad3[4];
+
+ compat_s64 st_size;
+ compat_ulong_t st_blksize;
+ compat_u64 st_blocks; /* Number of 512-byte blocks allocated. */
+
+ compat_ulong_t st_atime;
+ compat_ulong_t st_atime_nsec;
+
+ compat_ulong_t st_mtime;
+ compat_ulong_t st_mtime_nsec;
+
+ compat_ulong_t st_ctime;
+ compat_ulong_t st_ctime_nsec;
+
+ compat_u64 st_ino;
+};
+
+#endif
+
+#endif
diff --git a/arch/arm64/include/asm/statfs.h b/arch/arm64/include/asm/statfs.h
new file mode 100644
index 0000000..6f62190
--- /dev/null
+++ b/arch/arm64/include/asm/statfs.h
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_STATFS_H
+#define __ASM_STATFS_H
+
+#define ARCH_PACK_COMPAT_STATFS64 __attribute__((packed,aligned(4)))
+
+#include <asm-generic/statfs.h>
+
+#endif
diff --git a/arch/arm64/include/asm/syscall.h b/arch/arm64/include/asm/syscall.h
new file mode 100644
index 0000000..89c047f
--- /dev/null
+++ b/arch/arm64/include/asm/syscall.h
@@ -0,0 +1,101 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SYSCALL_H
+#define __ASM_SYSCALL_H
+
+#include <linux/err.h>
+
+
+static inline int syscall_get_nr(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->syscallno;
+}
+
+static inline void syscall_rollback(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ regs->regs[0] = regs->orig_x0;
+}
+
+
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ unsigned long error = regs->regs[0];
+ return IS_ERR_VALUE(error) ? error : 0;
+}
+
+static inline long syscall_get_return_value(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->regs[0];
+}
+
+static inline void syscall_set_return_value(struct task_struct *task,
+ struct pt_regs *regs,
+ int error, long val)
+{
+ regs->regs[0] = (long) error ? error : val;
+}
+
+#define SYSCALL_MAX_ARGS 6
+
+static inline void syscall_get_arguments(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned int i, unsigned int n,
+ unsigned long *args)
+{
+ if (i + n > SYSCALL_MAX_ARGS) {
+ unsigned long *args_bad = args + SYSCALL_MAX_ARGS - i;
+ unsigned int n_bad = n + i - SYSCALL_MAX_ARGS;
+ pr_warning("%s called with max args %d, handling only %d\n",
+ __func__, i + n, SYSCALL_MAX_ARGS);
+ memset(args_bad, 0, n_bad * sizeof(args[0]));
+ }
+
+ if (i == 0) {
+ args[0] = regs->orig_x0;
+ args++;
+ i++;
+ n--;
+ }
+
+ memcpy(args, ®s->regs[i], n * sizeof(args[0]));
+}
+
+static inline void syscall_set_arguments(struct task_struct *task,
+ struct pt_regs *regs,
+ unsigned int i, unsigned int n,
+ const unsigned long *args)
+{
+ if (i + n > SYSCALL_MAX_ARGS) {
+ pr_warning("%s called with max args %d, handling only %d\n",
+ __func__, i + n, SYSCALL_MAX_ARGS);
+ n = SYSCALL_MAX_ARGS - i;
+ }
+
+ if (i == 0) {
+ regs->orig_x0 = args[0];
+ args++;
+ i++;
+ n--;
+ }
+
+ memcpy(®s->regs[i], args, n * sizeof(args[0]));
+}
+
+#endif /* __ASM_SYSCALL_H */
diff --git a/arch/arm64/include/asm/syscalls.h b/arch/arm64/include/asm/syscalls.h
new file mode 100644
index 0000000..09ff335
--- /dev/null
+++ b/arch/arm64/include/asm/syscalls.h
@@ -0,0 +1,40 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SYSCALLS_H
+#define __ASM_SYSCALLS_H
+
+#include <linux/linkage.h>
+#include <linux/compiler.h>
+#include <linux/signal.h>
+
+/*
+ * System call wrappers implemented in kernel/entry.S.
+ */
+asmlinkage long sys_execve_wrapper(const char __user *filename,
+ const char __user *const __user *argv,
+ const char __user *const __user *envp);
+asmlinkage long sys_clone_wrapper(unsigned long clone_flags,
+ unsigned long newsp,
+ void __user *parent_tid,
+ unsigned long tls_val,
+ void __user *child_tid);
+asmlinkage long sys_rt_sigreturn_wrapper(void);
+asmlinkage long sys_sigaltstack_wrapper(const stack_t __user *uss,
+ stack_t __user *uoss);
+
+#include <asm-generic/syscalls.h>
+
+#endif /* __ASM_SYSCALLS_H */
diff --git a/arch/arm64/include/asm/system_misc.h b/arch/arm64/include/asm/system_misc.h
new file mode 100644
index 0000000..95e4072
--- /dev/null
+++ b/arch/arm64/include/asm/system_misc.h
@@ -0,0 +1,54 @@
+/*
+ * Based on arch/arm/include/asm/system_misc.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_SYSTEM_MISC_H
+#define __ASM_SYSTEM_MISC_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <linux/irqflags.h>
+
+struct pt_regs;
+
+void die(const char *msg, struct pt_regs *regs, int err);
+
+struct siginfo;
+void arm64_notify_die(const char *str, struct pt_regs *regs,
+ struct siginfo *info, int err);
+
+void hook_debug_fault_code(int nr, int (*fn)(unsigned long, unsigned int,
+ struct pt_regs *),
+ int sig, int code, const char *name);
+
+struct mm_struct;
+extern void show_pte(struct mm_struct *mm, unsigned long addr);
+extern void __show_regs(struct pt_regs *);
+
+void soft_restart(unsigned long);
+extern void (*pm_restart)(const char *cmd);
+
+#define UDBG_UNDEFINED (1 << 0)
+#define UDBG_SYSCALL (1 << 1)
+#define UDBG_BADABORT (1 << 2)
+#define UDBG_SEGV (1 << 3)
+#define UDBG_BUS (1 << 4)
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_SYSTEM_MISC_H */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
new file mode 100644
index 0000000..3659e46
--- /dev/null
+++ b/arch/arm64/include/asm/thread_info.h
@@ -0,0 +1,127 @@
+/*
+ * Based on arch/arm/include/asm/thread_info.h
+ *
+ * Copyright (C) 2002 Russell King.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_THREAD_INFO_H
+#define __ASM_THREAD_INFO_H
+
+#ifdef __KERNEL__
+
+#include <linux/compiler.h>
+
+#ifndef CONFIG_ARM64_64K_PAGES
+#define THREAD_SIZE_ORDER 1
+#endif
+
+#define THREAD_SIZE 8192
+#define THREAD_START_SP (THREAD_SIZE - 16)
+
+#ifndef __ASSEMBLY__
+
+struct task_struct;
+struct exec_domain;
+
+#include <asm/types.h>
+
+typedef unsigned long mm_segment_t;
+
+/*
+ * low level task data that entry.S needs immediate access to.
+ * __switch_to() assumes cpu_context follows immediately after cpu_domain.
+ */
+struct thread_info {
+ unsigned long flags; /* low level flags */
+ mm_segment_t addr_limit; /* address limit */
+ struct task_struct *task; /* main task structure */
+ struct exec_domain *exec_domain; /* execution domain */
+ struct restart_block restart_block;
+ int preempt_count; /* 0 => preemptable, <0 => bug */
+ int cpu; /* cpu */
+};
+
+#define INIT_THREAD_INFO(tsk) \
+{ \
+ .task = &tsk, \
+ .exec_domain = &default_exec_domain, \
+ .flags = 0, \
+ .preempt_count = INIT_PREEMPT_COUNT, \
+ .addr_limit = KERNEL_DS, \
+ .restart_block = { \
+ .fn = do_no_restart_syscall, \
+ }, \
+}
+
+#define init_thread_info (init_thread_union.thread_info)
+#define init_stack (init_thread_union.stack)
+
+/*
+ * how to get the thread information struct from C
+ */
+static inline struct thread_info *current_thread_info(void) __attribute_const__;
+
+static inline struct thread_info *current_thread_info(void)
+{
+ register unsigned long sp asm ("sp");
+ return (struct thread_info *)(sp & ~(THREAD_SIZE - 1));
+}
+
+#define thread_saved_pc(tsk) \
+ ((unsigned long)(tsk->thread.cpu_context.pc))
+#define thread_saved_sp(tsk) \
+ ((unsigned long)(tsk->thread.cpu_context.sp))
+#define thread_saved_fp(tsk) \
+ ((unsigned long)(tsk->thread.cpu_context.fp))
+
+#endif
+
+/*
+ * We use bit 30 of the preempt_count to indicate that kernel
+ * preemption is occurring. See <asm/hardirq.h>.
+ */
+#define PREEMPT_ACTIVE 0x40000000
+
+/*
+ * thread information flags:
+ * TIF_SYSCALL_TRACE - syscall trace active
+ * TIF_SIGPENDING - signal pending
+ * TIF_NEED_RESCHED - rescheduling necessary
+ * TIF_NOTIFY_RESUME - callback before returning to user
+ * TIF_USEDFPU - FPU was used by this task this quantum (SMP)
+ * TIF_POLLING_NRFLAG - true if poll_idle() is polling TIF_NEED_RESCHED
+ */
+#define TIF_SIGPENDING 0
+#define TIF_NEED_RESCHED 1
+#define TIF_NOTIFY_RESUME 2 /* callback before returning to user */
+#define TIF_SYSCALL_TRACE 8
+#define TIF_POLLING_NRFLAG 16
+#define TIF_MEMDIE 18 /* is terminating due to OOM killer */
+#define TIF_FREEZE 19
+#define TIF_RESTORE_SIGMASK 20
+#define TIF_SINGLESTEP 21
+#define TIF_32BIT 22 /* 32bit process */
+#define TIF_SWITCH_MM 23 /* deferred switch_mm */
+
+#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
+#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
+#define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME)
+#define _TIF_32BIT (1 << TIF_32BIT)
+
+#define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \
+ _TIF_NOTIFY_RESUME)
+
+#endif /* __KERNEL__ */
+#endif /* __ASM_THREAD_INFO_H */
diff --git a/arch/arm64/include/asm/timex.h b/arch/arm64/include/asm/timex.h
new file mode 100644
index 0000000..b24a31a
--- /dev/null
+++ b/arch/arm64/include/asm/timex.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_TIMEX_H
+#define __ASM_TIMEX_H
+
+/*
+ * Use the current timer as a cycle counter since this is what we use for
+ * the delay loop.
+ */
+#define get_cycles() ({ cycles_t c; read_current_timer(&c); c; })
+
+#include <asm-generic/timex.h>
+
+#define ARCH_HAS_READ_CURRENT_TIMER
+
+#endif
diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h
new file mode 100644
index 0000000..654f096
--- /dev/null
+++ b/arch/arm64/include/asm/tlb.h
@@ -0,0 +1,190 @@
+/*
+ * Based on arch/arm/include/asm/tlb.h
+ *
+ * Copyright (C) 2002 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_TLB_H
+#define __ASM_TLB_H
+
+#include <linux/pagemap.h>
+#include <linux/swap.h>
+
+#include <asm/pgalloc.h>
+#include <asm/tlbflush.h>
+
+#define MMU_GATHER_BUNDLE 8
+
+/*
+ * TLB handling. This allows us to remove pages from the page
+ * tables, and efficiently handle the TLB issues.
+ */
+struct mmu_gather {
+ struct mm_struct *mm;
+ unsigned int fullmm;
+ struct vm_area_struct *vma;
+ unsigned long range_start;
+ unsigned long range_end;
+ unsigned int nr;
+ unsigned int max;
+ struct page **pages;
+ struct page *local[MMU_GATHER_BUNDLE];
+};
+
+/*
+ * This is unnecessarily complex. There's three ways the TLB shootdown
+ * code is used:
+ * 1. Unmapping a range of vmas. See zap_page_range(), unmap_region().
+ * tlb->fullmm = 0, and tlb_start_vma/tlb_end_vma will be called.
+ * tlb->vma will be non-NULL.
+ * 2. Unmapping all vmas. See exit_mmap().
+ * tlb->fullmm = 1, and tlb_start_vma/tlb_end_vma will be called.
+ * tlb->vma will be non-NULL. Additionally, page tables will be freed.
+ * 3. Unmapping argument pages. See shift_arg_pages().
+ * tlb->fullmm = 0, but tlb_start_vma/tlb_end_vma will not be called.
+ * tlb->vma will be NULL.
+ */
+static inline void tlb_flush(struct mmu_gather *tlb)
+{
+ if (tlb->fullmm || !tlb->vma)
+ flush_tlb_mm(tlb->mm);
+ else if (tlb->range_end > 0) {
+ flush_tlb_range(tlb->vma, tlb->range_start, tlb->range_end);
+ tlb->range_start = TASK_SIZE;
+ tlb->range_end = 0;
+ }
+}
+
+static inline void tlb_add_flush(struct mmu_gather *tlb, unsigned long addr)
+{
+ if (!tlb->fullmm) {
+ if (addr < tlb->range_start)
+ tlb->range_start = addr;
+ if (addr + PAGE_SIZE > tlb->range_end)
+ tlb->range_end = addr + PAGE_SIZE;
+ }
+}
+
+static inline void __tlb_alloc_page(struct mmu_gather *tlb)
+{
+ unsigned long addr = __get_free_pages(GFP_NOWAIT | __GFP_NOWARN, 0);
+
+ if (addr) {
+ tlb->pages = (void *)addr;
+ tlb->max = PAGE_SIZE / sizeof(struct page *);
+ }
+}
+
+static inline void tlb_flush_mmu(struct mmu_gather *tlb)
+{
+ tlb_flush(tlb);
+ free_pages_and_swap_cache(tlb->pages, tlb->nr);
+ tlb->nr = 0;
+ if (tlb->pages == tlb->local)
+ __tlb_alloc_page(tlb);
+}
+
+static inline void
+tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned int fullmm)
+{
+ tlb->mm = mm;
+ tlb->fullmm = fullmm;
+ tlb->vma = NULL;
+ tlb->max = ARRAY_SIZE(tlb->local);
+ tlb->pages = tlb->local;
+ tlb->nr = 0;
+ __tlb_alloc_page(tlb);
+}
+
+static inline void
+tlb_finish_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end)
+{
+ tlb_flush_mmu(tlb);
+
+ /* keep the page table cache within bounds */
+ check_pgt_cache();
+
+ if (tlb->pages != tlb->local)
+ free_pages((unsigned long)tlb->pages, 0);
+}
+
+/*
+ * Memorize the range for the TLB flush.
+ */
+static inline void
+tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, unsigned long addr)
+{
+ tlb_add_flush(tlb, addr);
+}
+
+/*
+ * In the case of tlb vma handling, we can optimise these away in the
+ * case where we're doing a full MM flush. When we're doing a munmap,
+ * the vmas are adjusted to only cover the region to be torn down.
+ */
+static inline void
+tlb_start_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)
+{
+ if (!tlb->fullmm) {
+ tlb->vma = vma;
+ tlb->range_start = TASK_SIZE;
+ tlb->range_end = 0;
+ }
+}
+
+static inline void
+tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)
+{
+ if (!tlb->fullmm)
+ tlb_flush(tlb);
+}
+
+static inline int __tlb_remove_page(struct mmu_gather *tlb, struct page *page)
+{
+ tlb->pages[tlb->nr++] = page;
+ VM_BUG_ON(tlb->nr > tlb->max);
+ return tlb->max - tlb->nr;
+}
+
+static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page)
+{
+ if (!__tlb_remove_page(tlb, page))
+ tlb_flush_mmu(tlb);
+}
+
+static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
+ unsigned long addr)
+{
+ pgtable_page_dtor(pte);
+ tlb_add_flush(tlb, addr);
+ tlb_remove_page(tlb, pte);
+}
+
+#ifndef CONFIG_ARM64_64K_PAGES
+static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp,
+ unsigned long addr)
+{
+ tlb_add_flush(tlb, addr);
+ tlb_remove_page(tlb, virt_to_page(pmdp));
+}
+#endif
+
+#define pte_free_tlb(tlb, ptep, addr) __pte_free_tlb(tlb, ptep, addr)
+#define pmd_free_tlb(tlb, pmdp, addr) __pmd_free_tlb(tlb, pmdp, addr)
+#define pud_free_tlb(tlb, pudp, addr) pud_free((tlb)->mm, pudp)
+
+#define tlb_migrate_finish(mm) do { } while (0)
+
+#endif
diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
new file mode 100644
index 0000000..122d632
--- /dev/null
+++ b/arch/arm64/include/asm/tlbflush.h
@@ -0,0 +1,122 @@
+/*
+ * Based on arch/arm/include/asm/tlbflush.h
+ *
+ * Copyright (C) 1999-2003 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_TLBFLUSH_H
+#define __ASM_TLBFLUSH_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/sched.h>
+#include <asm/cputype.h>
+
+extern void __cpu_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *);
+extern void __cpu_flush_kern_tlb_range(unsigned long, unsigned long);
+
+extern struct cpu_tlb_fns cpu_tlb;
+
+/*
+ * TLB Management
+ * ==============
+ *
+ * The arch/arm64/mm/tlb.S files implement these methods.
+ *
+ * The TLB specific code is expected to perform whatever tests it needs
+ * to determine if it should invalidate the TLB for each call. Start
+ * addresses are inclusive and end addresses are exclusive; it is safe to
+ * round these addresses down.
+ *
+ * flush_tlb_all()
+ *
+ * Invalidate the entire TLB.
+ *
+ * flush_tlb_mm(mm)
+ *
+ * Invalidate all TLB entries in a particular address space.
+ * - mm - mm_struct describing address space
+ *
+ * flush_tlb_range(mm,start,end)
+ *
+ * Invalidate a range of TLB entries in the specified address
+ * space.
+ * - mm - mm_struct describing address space
+ * - start - start address (may not be aligned)
+ * - end - end address (exclusive, may not be aligned)
+ *
+ * flush_tlb_page(vaddr,vma)
+ *
+ * Invalidate the specified page in the specified address range.
+ * - vaddr - virtual address (may not be aligned)
+ * - vma - vma_struct describing address range
+ *
+ * flush_kern_tlb_page(kaddr)
+ *
+ * Invalidate the TLB entry for the specified page. The address
+ * will be in the kernels virtual memory space. Current uses
+ * only require the D-TLB to be invalidated.
+ * - kaddr - Kernel virtual memory address
+ */
+static inline void flush_tlb_all(void)
+{
+ dsb();
+ asm("tlbi vmalle1is");
+ dsb();
+ isb();
+}
+
+static inline void flush_tlb_mm(struct mm_struct *mm)
+{
+ unsigned long asid = (unsigned long)ASID(mm) << 48;
+
+ dsb();
+ asm("tlbi aside1is, %0" : : "r" (asid));
+ dsb();
+}
+
+static inline void flush_tlb_page(struct vm_area_struct *vma,
+ unsigned long uaddr)
+{
+ unsigned long addr = uaddr >> 12 |
+ ((unsigned long)ASID(vma->vm_mm) << 48);
+
+ dsb();
+ asm("tlbi vae1is, %0" : : "r" (addr));
+ dsb();
+}
+
+/*
+ * Convert calls to our calling convention.
+ */
+#define flush_tlb_range(vma,start,end) __cpu_flush_user_tlb_range(start,end,vma)
+#define flush_tlb_kernel_range(s,e) __cpu_flush_kern_tlb_range(s,e)
+
+/*
+ * On AArch64, the cache coherency is handled via the set_pte_at() function.
+ */
+static inline void update_mmu_cache(struct vm_area_struct *vma,
+ unsigned long addr, pte_t *ptep)
+{
+ /*
+ * set_pte() does not have a DSB, so make sure that the page table
+ * write is visible.
+ */
+ dsb();
+}
+
+#endif
+
+#endif
diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h
new file mode 100644
index 0000000..10ca8ff
--- /dev/null
+++ b/arch/arm64/include/asm/traps.h
@@ -0,0 +1,30 @@
+/*
+ * Based on arch/arm/include/asm/traps.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_TRAP_H
+#define __ASM_TRAP_H
+
+static inline int in_exception_text(unsigned long ptr)
+{
+ extern char __exception_text_start[];
+ extern char __exception_text_end[];
+
+ return ptr >= (unsigned long)&__exception_text_start &&
+ ptr < (unsigned long)&__exception_text_end;
+}
+
+#endif
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
new file mode 100644
index 0000000..008f848
--- /dev/null
+++ b/arch/arm64/include/asm/uaccess.h
@@ -0,0 +1,297 @@
+/*
+ * Based on arch/arm/include/asm/uaccess.h
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_UACCESS_H
+#define __ASM_UACCESS_H
+
+/*
+ * User space memory access functions
+ */
+#include <linux/string.h>
+#include <linux/thread_info.h>
+
+#include <asm/ptrace.h>
+#include <asm/errno.h>
+#include <asm/memory.h>
+#include <asm/compiler.h>
+
+#define VERIFY_READ 0
+#define VERIFY_WRITE 1
+
+/*
+ * The exception table consists of pairs of addresses: the first is the
+ * address of an instruction that is allowed to fault, and the second is
+ * the address at which the program should continue. No registers are
+ * modified, so it is entirely up to the continuation code to figure out
+ * what to do.
+ *
+ * All the routines below use bits of fixup code that are out of line
+ * with the main instruction path. This means when everything is well,
+ * we don't even have to jump over them. Further, they do not intrude
+ * on our cache or tlb entries.
+ */
+
+struct exception_table_entry
+{
+ unsigned long insn, fixup;
+};
+
+extern int fixup_exception(struct pt_regs *regs);
+
+#define KERNEL_DS (-1UL)
+#define get_ds() (KERNEL_DS)
+
+#define USER_DS TASK_SIZE_64
+#define get_fs() (current_thread_info()->addr_limit)
+
+static inline void set_fs(mm_segment_t fs)
+{
+ current_thread_info()->addr_limit = fs;
+}
+
+#define segment_eq(a,b) ((a) == (b))
+
+/*
+ * Return 1 if addr < current->addr_limit, 0 otherwise.
+ */
+#define __addr_ok(addr) \
+({ \
+ unsigned long flag; \
+ asm("cmp %1, %0; cset %0, lo" \
+ : "=&r" (flag) \
+ : "r" (addr), "0" (current_thread_info()->addr_limit) \
+ : "cc"); \
+ flag; \
+})
+
+/*
+ * Test whether a block of memory is a valid user space address.
+ * Returns 1 if the range is valid, 0 otherwise.
+ *
+ * This is equivalent to the following test:
+ * (u65)addr + (u65)size < (u65)current->addr_limit
+ *
+ * This needs 65-bit arithmetic.
+ */
+#define __range_ok(addr, size) \
+({ \
+ unsigned long flag, roksum; \
+ __chk_user_ptr(addr); \
+ asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, cc" \
+ : "=&r" (flag), "=&r" (roksum) \
+ : "1" (addr), "Ir" (size), \
+ "r" (current_thread_info()->addr_limit) \
+ : "cc"); \
+ flag; \
+})
+
+#define access_ok(type, addr, size) __range_ok(addr, size)
+
+/*
+ * The "__xxx" versions of the user access functions do not verify the address
+ * space - it must have been done previously with a separate "access_ok()"
+ * call.
+ *
+ * The "__xxx_error" versions set the third argument to -EFAULT if an error
+ * occurs, and leave it unchanged on success.
+ */
+#define __get_user_asm(instr, reg, x, addr, err) \
+ asm volatile( \
+ "1: " instr " " reg "1, [%2]\n" \
+ "2:\n" \
+ " .section .fixup, \"ax\"\n" \
+ " .align 2\n" \
+ "3: mov %w0, %3\n" \
+ " mov %1, #0\n" \
+ " b 2b\n" \
+ " .previous\n" \
+ " .section __ex_table,\"a\"\n" \
+ " .align 3\n" \
+ " .quad 1b, 3b\n" \
+ " .previous" \
+ : "+r" (err), "=&r" (x) \
+ : "r" (addr), "i" (-EFAULT))
+
+#define __get_user_err(x, ptr, err) \
+do { \
+ unsigned long __gu_val; \
+ __chk_user_ptr(ptr); \
+ switch (sizeof(*(ptr))) { \
+ case 1: \
+ __get_user_asm("ldrb", "%w", __gu_val, (ptr), (err)); \
+ break; \
+ case 2: \
+ __get_user_asm("ldrh", "%w", __gu_val, (ptr), (err)); \
+ break; \
+ case 4: \
+ __get_user_asm("ldr", "%w", __gu_val, (ptr), (err)); \
+ break; \
+ case 8: \
+ __get_user_asm("ldr", "%", __gu_val, (ptr), (err)); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+ (x) = (__typeof__(*(ptr)))__gu_val; \
+} while (0)
+
+#define __get_user(x, ptr) \
+({ \
+ int __gu_err = 0; \
+ __get_user_err((x), (ptr), __gu_err); \
+ __gu_err; \
+})
+
+#define __get_user_error(x, ptr, err) \
+({ \
+ __get_user_err((x), (ptr), (err)); \
+ (void)0; \
+})
+
+#define __get_user_unaligned __get_user
+
+#define get_user(x, ptr) \
+({ \
+ might_sleep(); \
+ access_ok(VERIFY_READ, (ptr), sizeof(*(ptr))) ? \
+ __get_user((x), (ptr)) : \
+ ((x) = 0, -EFAULT); \
+})
+
+#define __put_user_asm(instr, reg, x, addr, err) \
+ asm volatile( \
+ "1: " instr " " reg "1, [%2]\n" \
+ "2:\n" \
+ " .section .fixup,\"ax\"\n" \
+ " .align 2\n" \
+ "3: mov %w0, %3\n" \
+ " b 2b\n" \
+ " .previous\n" \
+ " .section __ex_table,\"a\"\n" \
+ " .align 3\n" \
+ " .quad 1b, 3b\n" \
+ " .previous" \
+ : "+r" (err) \
+ : "r" (x), "r" (addr), "i" (-EFAULT))
+
+#define __put_user_err(x, ptr, err) \
+do { \
+ __typeof__(*(ptr)) __pu_val = (x); \
+ __chk_user_ptr(ptr); \
+ switch (sizeof(*(ptr))) { \
+ case 1: \
+ __put_user_asm("strb", "%w", __pu_val, (ptr), (err)); \
+ break; \
+ case 2: \
+ __put_user_asm("strh", "%w", __pu_val, (ptr), (err)); \
+ break; \
+ case 4: \
+ __put_user_asm("str", "%w", __pu_val, (ptr), (err)); \
+ break; \
+ case 8: \
+ __put_user_asm("str", "%", __pu_val, (ptr), (err)); \
+ break; \
+ default: \
+ BUILD_BUG(); \
+ } \
+} while (0)
+
+#define __put_user(x, ptr) \
+({ \
+ int __pu_err = 0; \
+ __put_user_err((x), (ptr), __pu_err); \
+ __pu_err; \
+})
+
+#define __put_user_error(x, ptr, err) \
+({ \
+ __put_user_err((x), (ptr), (err)); \
+ (void)0; \
+})
+
+#define __put_user_unaligned __put_user
+
+#define put_user(x, ptr) \
+({ \
+ might_sleep(); \
+ access_ok(VERIFY_WRITE, (ptr), sizeof(*(ptr))) ? \
+ __put_user((x), (ptr)) : \
+ -EFAULT; \
+})
+
+extern unsigned long __must_check __copy_from_user(void *to, const void __user *from, unsigned long n);
+extern unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n);
+extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n);
+extern unsigned long __must_check __clear_user(void __user *addr, unsigned long n);
+
+extern unsigned long __must_check __strncpy_from_user(char *to, const char __user *from, unsigned long count);
+extern unsigned long __must_check __strnlen_user(const char __user *s, long n);
+
+static inline unsigned long __must_check copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+ if (access_ok(VERIFY_READ, from, n))
+ n = __copy_from_user(to, from, n);
+ else /* security hole - plug it */
+ memset(to, 0, n);
+ return n;
+}
+
+static inline unsigned long __must_check copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+ if (access_ok(VERIFY_WRITE, to, n))
+ n = __copy_to_user(to, from, n);
+ return n;
+}
+
+static inline unsigned long __must_check copy_in_user(void __user *to, const void __user *from, unsigned long n)
+{
+ if (access_ok(VERIFY_READ, from, n) && access_ok(VERIFY_WRITE, to, n))
+ n = __copy_in_user(to, from, n);
+ return n;
+}
+
+#define __copy_to_user_inatomic __copy_to_user
+#define __copy_from_user_inatomic __copy_from_user
+
+static inline unsigned long __must_check clear_user(void __user *to, unsigned long n)
+{
+ if (access_ok(VERIFY_WRITE, to, n))
+ n = __clear_user(to, n);
+ return n;
+}
+
+static inline long __must_check strncpy_from_user(char *dst, const char __user *src, long count)
+{
+ long res = -EFAULT;
+ if (access_ok(VERIFY_READ, src, 1))
+ res = __strncpy_from_user(dst, src, count);
+ return res;
+}
+
+#define strlen_user(s) strnlen_user(s, ~0UL >> 1)
+
+static inline long __must_check strnlen_user(const char __user *s, long n)
+{
+ unsigned long res = 0;
+
+ if (__addr_ok(s))
+ res = __strnlen_user(s, n);
+
+ return res;
+}
+
+#endif /* __ASM_UACCESS_H */
diff --git a/arch/arm64/include/asm/ucontext.h b/arch/arm64/include/asm/ucontext.h
new file mode 100644
index 0000000..bde9607
--- /dev/null
+++ b/arch/arm64/include/asm/ucontext.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_UCONTEXT_H
+#define __ASM_UCONTEXT_H
+
+struct ucontext {
+ unsigned long uc_flags;
+ struct ucontext *uc_link;
+ stack_t uc_stack;
+ sigset_t uc_sigmask;
+ /* glibc uses a 1024-bit sigset_t */
+ __u8 __unused[(1024 - sizeof(sigset_t)) / 8];
+ /* last for future expansion */
+ struct sigcontext uc_mcontext;
+};
+
+#endif /* __ASM_UCONTEXT_H */
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
new file mode 100644
index 0000000..fe18a68
--- /dev/null
+++ b/arch/arm64/include/asm/unistd.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#if !defined(__ASM_UNISTD_H) || defined(__SYSCALL)
+#define __ASM_UNISTD_H
+
+#ifndef __SYSCALL_COMPAT
+#include <asm-generic/unistd.h>
+#endif
+
+#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
+#include <asm/unistd32.h>
+#endif
+
+#endif /* __ASM_UNISTD_H */
diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
new file mode 100644
index 0000000..a50405f
--- /dev/null
+++ b/arch/arm64/include/asm/unistd32.h
@@ -0,0 +1,758 @@
+/*
+ * Based on arch/arm/include/asm/unistd.h
+ *
+ * Copyright (C) 2001-2005 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#if !defined(__ASM_UNISTD32_H) || defined(__SYSCALL)
+#define __ASM_UNISTD32_H
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+/*
+ * This file contains the system call numbers.
+ */
+
+#ifdef __SYSCALL_COMPAT
+
+#define __NR_restart_syscall 0
+__SYSCALL(__NR_restart_syscall, sys_restart_syscall)
+#define __NR_exit 1
+__SYSCALL(__NR_exit, sys_exit)
+#define __NR_fork 2
+__SYSCALL(__NR_fork, sys_fork)
+#define __NR_read 3
+__SYSCALL(__NR_read, sys_read)
+#define __NR_write 4
+__SYSCALL(__NR_write, sys_write)
+#define __NR_open 5
+__SYSCALL(__NR_open, sys_open)
+#define __NR_close 6
+__SYSCALL(__NR_close, sys_close)
+__SYSCALL(7, sys_ni_syscall) /* 7 was sys_waitpid */
+#define __NR_creat 8
+__SYSCALL(__NR_creat, sys_creat)
+#define __NR_link 9
+__SYSCALL(__NR_link, sys_link)
+#define __NR_unlink 10
+__SYSCALL(__NR_unlink, sys_unlink)
+#define __NR_execve 11
+__SYSCALL(__NR_execve, sys_execve)
+#define __NR_chdir 12
+__SYSCALL(__NR_chdir, sys_chdir)
+__SYSCALL(13, sys_ni_syscall) /* 13 was sys_time */
+#define __NR_mknod 14
+__SYSCALL(__NR_mknod, sys_mknod)
+#define __NR_chmod 15
+__SYSCALL(__NR_chmod, sys_chmod)
+#define __NR_lchown 16
+__SYSCALL(__NR_lchown, sys_lchown16)
+__SYSCALL(17, sys_ni_syscall) /* 17 was sys_break */
+__SYSCALL(18, sys_ni_syscall) /* 18 was sys_stat */
+#define __NR_lseek 19
+__SYSCALL(__NR_lseek, sys_lseek)
+#define __NR_getpid 20
+__SYSCALL(__NR_getpid, sys_getpid)
+#define __NR_mount 21
+__SYSCALL(__NR_mount, sys_mount)
+__SYSCALL(22, sys_ni_syscall) /* 22 was sys_umount */
+#define __NR_setuid 23
+__SYSCALL(__NR_setuid, sys_setuid16)
+#define __NR_getuid 24
+__SYSCALL(__NR_getuid, sys_getuid16)
+__SYSCALL(25, sys_ni_syscall) /* 25 was sys_stime */
+#define __NR_ptrace 26
+__SYSCALL(__NR_ptrace, sys_ptrace)
+__SYSCALL(27, sys_ni_syscall) /* 27 was sys_alarm */
+__SYSCALL(28, sys_ni_syscall) /* 28 was sys_fstat */
+#define __NR_pause 29
+__SYSCALL(__NR_pause, sys_pause)
+__SYSCALL(30, sys_ni_syscall) /* 30 was sys_utime */
+__SYSCALL(31, sys_ni_syscall) /* 31 was sys_stty */
+__SYSCALL(32, sys_ni_syscall) /* 32 was sys_gtty */
+#define __NR_access 33
+__SYSCALL(__NR_access, sys_access)
+#define __NR_nice 34
+__SYSCALL(__NR_nice, sys_nice)
+__SYSCALL(35, sys_ni_syscall) /* 35 was sys_ftime */
+#define __NR_sync 36
+__SYSCALL(__NR_sync, sys_sync)
+#define __NR_kill 37
+__SYSCALL(__NR_kill, sys_kill)
+#define __NR_rename 38
+__SYSCALL(__NR_rename, sys_rename)
+#define __NR_mkdir 39
+__SYSCALL(__NR_mkdir, sys_mkdir)
+#define __NR_rmdir 40
+__SYSCALL(__NR_rmdir, sys_rmdir)
+#define __NR_dup 41
+__SYSCALL(__NR_dup, sys_dup)
+#define __NR_pipe 42
+__SYSCALL(__NR_pipe, sys_pipe)
+#define __NR_times 43
+__SYSCALL(__NR_times, sys_times)
+__SYSCALL(44, sys_ni_syscall) /* 44 was sys_prof */
+#define __NR_brk 45
+__SYSCALL(__NR_brk, sys_brk)
+#define __NR_setgid 46
+__SYSCALL(__NR_setgid, sys_setgid16)
+#define __NR_getgid 47
+__SYSCALL(__NR_getgid, sys_getgid16)
+__SYSCALL(48, sys_ni_syscall) /* 48 was sys_signal */
+#define __NR_geteuid 49
+__SYSCALL(__NR_geteuid, sys_geteuid16)
+#define __NR_getegid 50
+__SYSCALL(__NR_getegid, sys_getegid16)
+#define __NR_acct 51
+__SYSCALL(__NR_acct, sys_acct)
+#define __NR_umount2 52
+__SYSCALL(__NR_umount2, sys_umount)
+__SYSCALL(53, sys_ni_syscall) /* 53 was sys_lock */
+#define __NR_ioctl 54
+__SYSCALL(__NR_ioctl, sys_ioctl)
+#define __NR_fcntl 55
+__SYSCALL(__NR_fcntl, sys_fcntl)
+__SYSCALL(56, sys_ni_syscall) /* 56 was sys_mpx */
+#define __NR_setpgid 57
+__SYSCALL(__NR_setpgid, sys_setpgid)
+__SYSCALL(58, sys_ni_syscall) /* 58 was sys_ulimit */
+__SYSCALL(59, sys_ni_syscall) /* 59 was sys_olduname */
+#define __NR_umask 60
+__SYSCALL(__NR_umask, sys_umask)
+#define __NR_chroot 61
+__SYSCALL(__NR_chroot, sys_chroot)
+#define __NR_ustat 62
+__SYSCALL(__NR_ustat, sys_ustat)
+#define __NR_dup2 63
+__SYSCALL(__NR_dup2, sys_dup2)
+#define __NR_getppid 64
+__SYSCALL(__NR_getppid, sys_getppid)
+#define __NR_getpgrp 65
+__SYSCALL(__NR_getpgrp, sys_getpgrp)
+#define __NR_setsid 66
+__SYSCALL(__NR_setsid, sys_setsid)
+#define __NR_sigaction 67
+__SYSCALL(__NR_sigaction, sys_sigaction)
+__SYSCALL(68, sys_ni_syscall) /* 68 was sys_sgetmask */
+__SYSCALL(69, sys_ni_syscall) /* 69 was sys_ssetmask */
+#define __NR_setreuid 70
+__SYSCALL(__NR_setreuid, sys_setreuid16)
+#define __NR_setregid 71
+__SYSCALL(__NR_setregid, sys_setregid16)
+#define __NR_sigsuspend 72
+__SYSCALL(__NR_sigsuspend, sys_sigsuspend)
+#define __NR_sigpending 73
+__SYSCALL(__NR_sigpending, sys_sigpending)
+#define __NR_sethostname 74
+__SYSCALL(__NR_sethostname, sys_sethostname)
+#define __NR_setrlimit 75
+__SYSCALL(__NR_setrlimit, sys_setrlimit)
+__SYSCALL(76, sys_ni_syscall) /* 76 was sys_getrlimit */
+#define __NR_getrusage 77
+__SYSCALL(__NR_getrusage, sys_getrusage)
+#define __NR_gettimeofday 78
+__SYSCALL(__NR_gettimeofday, sys_gettimeofday)
+#define __NR_settimeofday 79
+__SYSCALL(__NR_settimeofday, sys_settimeofday)
+#define __NR_getgroups 80
+__SYSCALL(__NR_getgroups, sys_getgroups16)
+#define __NR_setgroups 81
+__SYSCALL(__NR_setgroups, sys_setgroups16)
+__SYSCALL(82, sys_ni_syscall) /* 82 was sys_select */
+#define __NR_symlink 83
+__SYSCALL(__NR_symlink, sys_symlink)
+__SYSCALL(84, sys_ni_syscall) /* 84 was sys_lstat */
+#define __NR_readlink 85
+__SYSCALL(__NR_readlink, sys_readlink)
+#define __NR_uselib 86
+__SYSCALL(__NR_uselib, sys_uselib)
+#define __NR_swapon 87
+__SYSCALL(__NR_swapon, sys_swapon)
+#define __NR_reboot 88
+__SYSCALL(__NR_reboot, sys_reboot)
+__SYSCALL(89, sys_ni_syscall) /* 89 was sys_readdir */
+__SYSCALL(90, sys_ni_syscall) /* 90 was sys_mmap */
+#define __NR_munmap 91
+__SYSCALL(__NR_munmap, sys_munmap)
+#define __NR_truncate 92
+__SYSCALL(__NR_truncate, sys_truncate)
+#define __NR_ftruncate 93
+__SYSCALL(__NR_ftruncate, sys_ftruncate)
+#define __NR_fchmod 94
+__SYSCALL(__NR_fchmod, sys_fchmod)
+#define __NR_fchown 95
+__SYSCALL(__NR_fchown, sys_fchown16)
+#define __NR_getpriority 96
+__SYSCALL(__NR_getpriority, sys_getpriority)
+#define __NR_setpriority 97
+__SYSCALL(__NR_setpriority, sys_setpriority)
+__SYSCALL(98, sys_ni_syscall) /* 98 was sys_profil */
+#define __NR_statfs 99
+__SYSCALL(__NR_statfs, sys_statfs)
+#define __NR_fstatfs 100
+__SYSCALL(__NR_fstatfs, sys_fstatfs)
+__SYSCALL(101, sys_ni_syscall) /* 101 was sys_ioperm */
+__SYSCALL(102, sys_ni_syscall) /* 102 was sys_socketcall */
+#define __NR_syslog 103
+__SYSCALL(__NR_syslog, sys_syslog)
+#define __NR_setitimer 104
+__SYSCALL(__NR_setitimer, sys_setitimer)
+#define __NR_getitimer 105
+__SYSCALL(__NR_getitimer, sys_getitimer)
+#define __NR_stat 106
+__SYSCALL(__NR_stat, sys_newstat)
+#define __NR_lstat 107
+__SYSCALL(__NR_lstat, sys_newlstat)
+#define __NR_fstat 108
+__SYSCALL(__NR_fstat, sys_newfstat)
+__SYSCALL(109, sys_ni_syscall) /* 109 was sys_uname */
+__SYSCALL(110, sys_ni_syscall) /* 110 was sys_iopl */
+#define __NR_vhangup 111
+__SYSCALL(__NR_vhangup, sys_vhangup)
+__SYSCALL(112, sys_ni_syscall) /* 112 was sys_idle */
+__SYSCALL(113, sys_ni_syscall) /* 113 was sys_syscall */
+#define __NR_wait4 114
+__SYSCALL(__NR_wait4, sys_wait4)
+#define __NR_swapoff 115
+__SYSCALL(__NR_swapoff, sys_swapoff)
+#define __NR_sysinfo 116
+__SYSCALL(__NR_sysinfo, sys_sysinfo)
+__SYSCALL(117, sys_ni_syscall) /* 117 was sys_ipc */
+#define __NR_fsync 118
+__SYSCALL(__NR_fsync, sys_fsync)
+#define __NR_sigreturn 119
+__SYSCALL(__NR_sigreturn, sys_sigreturn)
+#define __NR_clone 120
+__SYSCALL(__NR_clone, sys_clone)
+#define __NR_setdomainname 121
+__SYSCALL(__NR_setdomainname, sys_setdomainname)
+#define __NR_uname 122
+__SYSCALL(__NR_uname, sys_newuname)
+__SYSCALL(123, sys_ni_syscall) /* 123 was sys_modify_ldt */
+#define __NR_adjtimex 124
+__SYSCALL(__NR_adjtimex, sys_adjtimex)
+#define __NR_mprotect 125
+__SYSCALL(__NR_mprotect, sys_mprotect)
+#define __NR_sigprocmask 126
+__SYSCALL(__NR_sigprocmask, sys_sigprocmask)
+__SYSCALL(127, sys_ni_syscall) /* 127 was sys_create_module */
+#define __NR_init_module 128
+__SYSCALL(__NR_init_module, sys_init_module)
+#define __NR_delete_module 129
+__SYSCALL(__NR_delete_module, sys_delete_module)
+__SYSCALL(130, sys_ni_syscall) /* 130 was sys_get_kernel_syms */
+#define __NR_quotactl 131
+__SYSCALL(__NR_quotactl, sys_quotactl)
+#define __NR_getpgid 132
+__SYSCALL(__NR_getpgid, sys_getpgid)
+#define __NR_fchdir 133
+__SYSCALL(__NR_fchdir, sys_fchdir)
+#define __NR_bdflush 134
+__SYSCALL(__NR_bdflush, sys_bdflush)
+#define __NR_sysfs 135
+__SYSCALL(__NR_sysfs, sys_sysfs)
+#define __NR_personality 136
+__SYSCALL(__NR_personality, sys_personality)
+__SYSCALL(137, sys_ni_syscall) /* 137 was sys_afs_syscall */
+#define __NR_setfsuid 138
+__SYSCALL(__NR_setfsuid, sys_setfsuid16)
+#define __NR_setfsgid 139
+__SYSCALL(__NR_setfsgid, sys_setfsgid16)
+#define __NR__llseek 140
+__SYSCALL(__NR__llseek, sys_llseek)
+#define __NR_getdents 141
+__SYSCALL(__NR_getdents, sys_getdents)
+#define __NR__newselect 142
+__SYSCALL(__NR__newselect, sys_select)
+#define __NR_flock 143
+__SYSCALL(__NR_flock, sys_flock)
+#define __NR_msync 144
+__SYSCALL(__NR_msync, sys_msync)
+#define __NR_readv 145
+__SYSCALL(__NR_readv, sys_readv)
+#define __NR_writev 146
+__SYSCALL(__NR_writev, sys_writev)
+#define __NR_getsid 147
+__SYSCALL(__NR_getsid, sys_getsid)
+#define __NR_fdatasync 148
+__SYSCALL(__NR_fdatasync, sys_fdatasync)
+#define __NR__sysctl 149
+__SYSCALL(__NR__sysctl, sys_sysctl)
+#define __NR_mlock 150
+__SYSCALL(__NR_mlock, sys_mlock)
+#define __NR_munlock 151
+__SYSCALL(__NR_munlock, sys_munlock)
+#define __NR_mlockall 152
+__SYSCALL(__NR_mlockall, sys_mlockall)
+#define __NR_munlockall 153
+__SYSCALL(__NR_munlockall, sys_munlockall)
+#define __NR_sched_setparam 154
+__SYSCALL(__NR_sched_setparam, sys_sched_setparam)
+#define __NR_sched_getparam 155
+__SYSCALL(__NR_sched_getparam, sys_sched_getparam)
+#define __NR_sched_setscheduler 156
+__SYSCALL(__NR_sched_setscheduler, sys_sched_setscheduler)
+#define __NR_sched_getscheduler 157
+__SYSCALL(__NR_sched_getscheduler, sys_sched_getscheduler)
+#define __NR_sched_yield 158
+__SYSCALL(__NR_sched_yield, sys_sched_yield)
+#define __NR_sched_get_priority_max 159
+__SYSCALL(__NR_sched_get_priority_max, sys_sched_get_priority_max)
+#define __NR_sched_get_priority_min 160
+__SYSCALL(__NR_sched_get_priority_min, sys_sched_get_priority_min)
+#define __NR_sched_rr_get_interval 161
+__SYSCALL(__NR_sched_rr_get_interval, sys_sched_rr_get_interval)
+#define __NR_nanosleep 162
+__SYSCALL(__NR_nanosleep, sys_nanosleep)
+#define __NR_mremap 163
+__SYSCALL(__NR_mremap, sys_mremap)
+#define __NR_setresuid 164
+__SYSCALL(__NR_setresuid, sys_setresuid16)
+#define __NR_getresuid 165
+__SYSCALL(__NR_getresuid, sys_getresuid16)
+__SYSCALL(166, sys_ni_syscall) /* 166 was sys_vm86 */
+__SYSCALL(167, sys_ni_syscall) /* 167 was sys_query_module */
+#define __NR_poll 168
+__SYSCALL(__NR_poll, sys_poll)
+#define __NR_nfsservctl 169
+__SYSCALL(__NR_nfsservctl, sys_ni_syscall)
+#define __NR_setresgid 170
+__SYSCALL(__NR_setresgid, sys_setresgid16)
+#define __NR_getresgid 171
+__SYSCALL(__NR_getresgid, sys_getresgid16)
+#define __NR_prctl 172
+__SYSCALL(__NR_prctl, sys_prctl)
+#define __NR_rt_sigreturn 173
+__SYSCALL(__NR_rt_sigreturn, sys_rt_sigreturn)
+#define __NR_rt_sigaction 174
+__SYSCALL(__NR_rt_sigaction, sys_rt_sigaction)
+#define __NR_rt_sigprocmask 175
+__SYSCALL(__NR_rt_sigprocmask, sys_rt_sigprocmask)
+#define __NR_rt_sigpending 176
+__SYSCALL(__NR_rt_sigpending, sys_rt_sigpending)
+#define __NR_rt_sigtimedwait 177
+__SYSCALL(__NR_rt_sigtimedwait, sys_rt_sigtimedwait)
+#define __NR_rt_sigqueueinfo 178
+__SYSCALL(__NR_rt_sigqueueinfo, sys_rt_sigqueueinfo)
+#define __NR_rt_sigsuspend 179
+__SYSCALL(__NR_rt_sigsuspend, sys_rt_sigsuspend)
+#define __NR_pread64 180
+__SYSCALL(__NR_pread64, sys_pread64)
+#define __NR_pwrite64 181
+__SYSCALL(__NR_pwrite64, sys_pwrite64)
+#define __NR_chown 182
+__SYSCALL(__NR_chown, sys_chown16)
+#define __NR_getcwd 183
+__SYSCALL(__NR_getcwd, sys_getcwd)
+#define __NR_capget 184
+__SYSCALL(__NR_capget, sys_capget)
+#define __NR_capset 185
+__SYSCALL(__NR_capset, sys_capset)
+#define __NR_sigaltstack 186
+__SYSCALL(__NR_sigaltstack, sys_sigaltstack)
+#define __NR_sendfile 187
+__SYSCALL(__NR_sendfile, sys_sendfile)
+__SYSCALL(188, sys_ni_syscall) /* 188 reserved */
+__SYSCALL(189, sys_ni_syscall) /* 189 reserved */
+#define __NR_vfork 190
+__SYSCALL(__NR_vfork, sys_vfork)
+#define __NR_ugetrlimit 191 /* SuS compliant getrlimit */
+__SYSCALL(__NR_ugetrlimit, sys_getrlimit)
+#define __NR_mmap2 192
+__SYSCALL(__NR_mmap2, sys_mmap2)
+#define __NR_truncate64 193
+__SYSCALL(__NR_truncate64, sys_truncate64)
+#define __NR_ftruncate64 194
+__SYSCALL(__NR_ftruncate64, sys_ftruncate64)
+#define __NR_stat64 195
+__SYSCALL(__NR_stat64, sys_stat64)
+#define __NR_lstat64 196
+__SYSCALL(__NR_lstat64, sys_lstat64)
+#define __NR_fstat64 197
+__SYSCALL(__NR_fstat64, sys_fstat64)
+#define __NR_lchown32 198
+__SYSCALL(__NR_lchown32, sys_lchown)
+#define __NR_getuid32 199
+__SYSCALL(__NR_getuid32, sys_getuid)
+#define __NR_getgid32 200
+__SYSCALL(__NR_getgid32, sys_getgid)
+#define __NR_geteuid32 201
+__SYSCALL(__NR_geteuid32, sys_geteuid)
+#define __NR_getegid32 202
+__SYSCALL(__NR_getegid32, sys_getegid)
+#define __NR_setreuid32 203
+__SYSCALL(__NR_setreuid32, sys_setreuid)
+#define __NR_setregid32 204
+__SYSCALL(__NR_setregid32, sys_setregid)
+#define __NR_getgroups32 205
+__SYSCALL(__NR_getgroups32, sys_getgroups)
+#define __NR_setgroups32 206
+__SYSCALL(__NR_setgroups32, sys_setgroups)
+#define __NR_fchown32 207
+__SYSCALL(__NR_fchown32, sys_fchown)
+#define __NR_setresuid32 208
+__SYSCALL(__NR_setresuid32, sys_setresuid)
+#define __NR_getresuid32 209
+__SYSCALL(__NR_getresuid32, sys_getresuid)
+#define __NR_setresgid32 210
+__SYSCALL(__NR_setresgid32, sys_setresgid)
+#define __NR_getresgid32 211
+__SYSCALL(__NR_getresgid32, sys_getresgid)
+#define __NR_chown32 212
+__SYSCALL(__NR_chown32, sys_chown)
+#define __NR_setuid32 213
+__SYSCALL(__NR_setuid32, sys_setuid)
+#define __NR_setgid32 214
+__SYSCALL(__NR_setgid32, sys_setgid)
+#define __NR_setfsuid32 215
+__SYSCALL(__NR_setfsuid32, sys_setfsuid)
+#define __NR_setfsgid32 216
+__SYSCALL(__NR_setfsgid32, sys_setfsgid)
+#define __NR_getdents64 217
+__SYSCALL(__NR_getdents64, sys_getdents64)
+#define __NR_pivot_root 218
+__SYSCALL(__NR_pivot_root, sys_pivot_root)
+#define __NR_mincore 219
+__SYSCALL(__NR_mincore, sys_mincore)
+#define __NR_madvise 220
+__SYSCALL(__NR_madvise, sys_madvise)
+#define __NR_fcntl64 221
+__SYSCALL(__NR_fcntl64, sys_fcntl64)
+__SYSCALL(222, sys_ni_syscall) /* 222 for tux */
+__SYSCALL(223, sys_ni_syscall) /* 223 is unused */
+#define __NR_gettid 224
+__SYSCALL(__NR_gettid, sys_gettid)
+#define __NR_readahead 225
+__SYSCALL(__NR_readahead, sys_readahead)
+#define __NR_setxattr 226
+__SYSCALL(__NR_setxattr, sys_setxattr)
+#define __NR_lsetxattr 227
+__SYSCALL(__NR_lsetxattr, sys_lsetxattr)
+#define __NR_fsetxattr 228
+__SYSCALL(__NR_fsetxattr, sys_fsetxattr)
+#define __NR_getxattr 229
+__SYSCALL(__NR_getxattr, sys_getxattr)
+#define __NR_lgetxattr 230
+__SYSCALL(__NR_lgetxattr, sys_lgetxattr)
+#define __NR_fgetxattr 231
+__SYSCALL(__NR_fgetxattr, sys_fgetxattr)
+#define __NR_listxattr 232
+__SYSCALL(__NR_listxattr, sys_listxattr)
+#define __NR_llistxattr 233
+__SYSCALL(__NR_llistxattr, sys_llistxattr)
+#define __NR_flistxattr 234
+__SYSCALL(__NR_flistxattr, sys_flistxattr)
+#define __NR_removexattr 235
+__SYSCALL(__NR_removexattr, sys_removexattr)
+#define __NR_lremovexattr 236
+__SYSCALL(__NR_lremovexattr, sys_lremovexattr)
+#define __NR_fremovexattr 237
+__SYSCALL(__NR_fremovexattr, sys_fremovexattr)
+#define __NR_tkill 238
+__SYSCALL(__NR_tkill, sys_tkill)
+#define __NR_sendfile64 239
+__SYSCALL(__NR_sendfile64, sys_sendfile64)
+#define __NR_futex 240
+__SYSCALL(__NR_futex, sys_futex)
+#define __NR_sched_setaffinity 241
+__SYSCALL(__NR_sched_setaffinity, sys_sched_setaffinity)
+#define __NR_sched_getaffinity 242
+__SYSCALL(__NR_sched_getaffinity, sys_sched_getaffinity)
+#define __NR_io_setup 243
+__SYSCALL(__NR_io_setup, sys_io_setup)
+#define __NR_io_destroy 244
+__SYSCALL(__NR_io_destroy, sys_io_destroy)
+#define __NR_io_getevents 245
+__SYSCALL(__NR_io_getevents, sys_io_getevents)
+#define __NR_io_submit 246
+__SYSCALL(__NR_io_submit, sys_io_submit)
+#define __NR_io_cancel 247
+__SYSCALL(__NR_io_cancel, sys_io_cancel)
+#define __NR_exit_group 248
+__SYSCALL(__NR_exit_group, sys_exit_group)
+#define __NR_lookup_dcookie 249
+__SYSCALL(__NR_lookup_dcookie, sys_lookup_dcookie)
+#define __NR_epoll_create 250
+__SYSCALL(__NR_epoll_create, sys_epoll_create)
+#define __NR_epoll_ctl 251
+__SYSCALL(__NR_epoll_ctl, sys_epoll_ctl)
+#define __NR_epoll_wait 252
+__SYSCALL(__NR_epoll_wait, sys_epoll_wait)
+#define __NR_remap_file_pages 253
+__SYSCALL(__NR_remap_file_pages, sys_remap_file_pages)
+__SYSCALL(254, sys_ni_syscall) /* 254 for set_thread_area */
+__SYSCALL(255, sys_ni_syscall) /* 255 for get_thread_area */
+#define __NR_set_tid_address 256
+__SYSCALL(__NR_set_tid_address, sys_set_tid_address)
+#define __NR_timer_create 257
+__SYSCALL(__NR_timer_create, sys_timer_create)
+#define __NR_timer_settime 258
+__SYSCALL(__NR_timer_settime, sys_timer_settime)
+#define __NR_timer_gettime 259
+__SYSCALL(__NR_timer_gettime, sys_timer_gettime)
+#define __NR_timer_getoverrun 260
+__SYSCALL(__NR_timer_getoverrun, sys_timer_getoverrun)
+#define __NR_timer_delete 261
+__SYSCALL(__NR_timer_delete, sys_timer_delete)
+#define __NR_clock_settime 262
+__SYSCALL(__NR_clock_settime, sys_clock_settime)
+#define __NR_clock_gettime 263
+__SYSCALL(__NR_clock_gettime, sys_clock_gettime)
+#define __NR_clock_getres 264
+__SYSCALL(__NR_clock_getres, sys_clock_getres)
+#define __NR_clock_nanosleep 265
+__SYSCALL(__NR_clock_nanosleep, sys_clock_nanosleep)
+#define __NR_statfs64 266
+__SYSCALL(__NR_statfs64, sys_statfs64)
+#define __NR_fstatfs64 267
+__SYSCALL(__NR_fstatfs64, sys_fstatfs64)
+#define __NR_tgkill 268
+__SYSCALL(__NR_tgkill, sys_tgkill)
+#define __NR_utimes 269
+__SYSCALL(__NR_utimes, sys_utimes)
+#define __NR_fadvise64 270
+__SYSCALL(__NR_fadvise64, sys_fadvise64_64)
+#define __NR_pciconfig_iobase 271
+__SYSCALL(__NR_pciconfig_iobase, sys_pciconfig_iobase)
+#define __NR_pciconfig_read 272
+__SYSCALL(__NR_pciconfig_read, sys_pciconfig_read)
+#define __NR_pciconfig_write 273
+__SYSCALL(__NR_pciconfig_write, sys_pciconfig_write)
+#define __NR_mq_open 274
+__SYSCALL(__NR_mq_open, sys_mq_open)
+#define __NR_mq_unlink 275
+__SYSCALL(__NR_mq_unlink, sys_mq_unlink)
+#define __NR_mq_timedsend 276
+__SYSCALL(__NR_mq_timedsend, sys_mq_timedsend)
+#define __NR_mq_timedreceive 277
+__SYSCALL(__NR_mq_timedreceive, sys_mq_timedreceive)
+#define __NR_mq_notify 278
+__SYSCALL(__NR_mq_notify, sys_mq_notify)
+#define __NR_mq_getsetattr 279
+__SYSCALL(__NR_mq_getsetattr, sys_mq_getsetattr)
+#define __NR_waitid 280
+__SYSCALL(__NR_waitid, sys_waitid)
+#define __NR_socket 281
+__SYSCALL(__NR_socket, sys_socket)
+#define __NR_bind 282
+__SYSCALL(__NR_bind, sys_bind)
+#define __NR_connect 283
+__SYSCALL(__NR_connect, sys_connect)
+#define __NR_listen 284
+__SYSCALL(__NR_listen, sys_listen)
+#define __NR_accept 285
+__SYSCALL(__NR_accept, sys_accept)
+#define __NR_getsockname 286
+__SYSCALL(__NR_getsockname, sys_getsockname)
+#define __NR_getpeername 287
+__SYSCALL(__NR_getpeername, sys_getpeername)
+#define __NR_socketpair 288
+__SYSCALL(__NR_socketpair, sys_socketpair)
+#define __NR_send 289
+__SYSCALL(__NR_send, sys_send)
+#define __NR_sendto 290
+__SYSCALL(__NR_sendto, sys_sendto)
+#define __NR_recv 291
+__SYSCALL(__NR_recv, sys_recv)
+#define __NR_recvfrom 292
+__SYSCALL(__NR_recvfrom, sys_recvfrom)
+#define __NR_shutdown 293
+__SYSCALL(__NR_shutdown, sys_shutdown)
+#define __NR_setsockopt 294
+__SYSCALL(__NR_setsockopt, sys_setsockopt)
+#define __NR_getsockopt 295
+__SYSCALL(__NR_getsockopt, sys_getsockopt)
+#define __NR_sendmsg 296
+__SYSCALL(__NR_sendmsg, sys_sendmsg)
+#define __NR_recvmsg 297
+__SYSCALL(__NR_recvmsg, sys_recvmsg)
+#define __NR_semop 298
+__SYSCALL(__NR_semop, sys_semop)
+#define __NR_semget 299
+__SYSCALL(__NR_semget, sys_semget)
+#define __NR_semctl 300
+__SYSCALL(__NR_semctl, sys_semctl)
+#define __NR_msgsnd 301
+__SYSCALL(__NR_msgsnd, sys_msgsnd)
+#define __NR_msgrcv 302
+__SYSCALL(__NR_msgrcv, sys_msgrcv)
+#define __NR_msgget 303
+__SYSCALL(__NR_msgget, sys_msgget)
+#define __NR_msgctl 304
+__SYSCALL(__NR_msgctl, sys_msgctl)
+#define __NR_shmat 305
+__SYSCALL(__NR_shmat, sys_shmat)
+#define __NR_shmdt 306
+__SYSCALL(__NR_shmdt, sys_shmdt)
+#define __NR_shmget 307
+__SYSCALL(__NR_shmget, sys_shmget)
+#define __NR_shmctl 308
+__SYSCALL(__NR_shmctl, sys_shmctl)
+#define __NR_add_key 309
+__SYSCALL(__NR_add_key, sys_add_key)
+#define __NR_request_key 310
+__SYSCALL(__NR_request_key, sys_request_key)
+#define __NR_keyctl 311
+__SYSCALL(__NR_keyctl, sys_keyctl)
+#define __NR_semtimedop 312
+__SYSCALL(__NR_semtimedop, sys_semtimedop)
+#define __NR_vserver 313
+__SYSCALL(__NR_vserver, sys_ni_syscall)
+#define __NR_ioprio_set 314
+__SYSCALL(__NR_ioprio_set, sys_ioprio_set)
+#define __NR_ioprio_get 315
+__SYSCALL(__NR_ioprio_get, sys_ioprio_get)
+#define __NR_inotify_init 316
+__SYSCALL(__NR_inotify_init, sys_inotify_init)
+#define __NR_inotify_add_watch 317
+__SYSCALL(__NR_inotify_add_watch, sys_inotify_add_watch)
+#define __NR_inotify_rm_watch 318
+__SYSCALL(__NR_inotify_rm_watch, sys_inotify_rm_watch)
+#define __NR_mbind 319
+__SYSCALL(__NR_mbind, sys_mbind)
+#define __NR_get_mempolicy 320
+__SYSCALL(__NR_get_mempolicy, sys_get_mempolicy)
+#define __NR_set_mempolicy 321
+__SYSCALL(__NR_set_mempolicy, sys_set_mempolicy)
+#define __NR_openat 322
+__SYSCALL(__NR_openat, sys_openat)
+#define __NR_mkdirat 323
+__SYSCALL(__NR_mkdirat, sys_mkdirat)
+#define __NR_mknodat 324
+__SYSCALL(__NR_mknodat, sys_mknodat)
+#define __NR_fchownat 325
+__SYSCALL(__NR_fchownat, sys_fchownat)
+#define __NR_futimesat 326
+__SYSCALL(__NR_futimesat, sys_futimesat)
+#define __NR_fstatat64 327
+__SYSCALL(__NR_fstatat64, sys_fstatat64)
+#define __NR_unlinkat 328
+__SYSCALL(__NR_unlinkat, sys_unlinkat)
+#define __NR_renameat 329
+__SYSCALL(__NR_renameat, sys_renameat)
+#define __NR_linkat 330
+__SYSCALL(__NR_linkat, sys_linkat)
+#define __NR_symlinkat 331
+__SYSCALL(__NR_symlinkat, sys_symlinkat)
+#define __NR_readlinkat 332
+__SYSCALL(__NR_readlinkat, sys_readlinkat)
+#define __NR_fchmodat 333
+__SYSCALL(__NR_fchmodat, sys_fchmodat)
+#define __NR_faccessat 334
+__SYSCALL(__NR_faccessat, sys_faccessat)
+#define __NR_pselect6 335
+__SYSCALL(__NR_pselect6, sys_pselect6)
+#define __NR_ppoll 336
+__SYSCALL(__NR_ppoll, sys_ppoll)
+#define __NR_unshare 337
+__SYSCALL(__NR_unshare, sys_unshare)
+#define __NR_set_robust_list 338
+__SYSCALL(__NR_set_robust_list, sys_set_robust_list)
+#define __NR_get_robust_list 339
+__SYSCALL(__NR_get_robust_list, sys_get_robust_list)
+#define __NR_splice 340
+__SYSCALL(__NR_splice, sys_splice)
+#define __NR_sync_file_range2 341
+__SYSCALL(__NR_sync_file_range2, sys_sync_file_range2)
+#define __NR_tee 342
+__SYSCALL(__NR_tee, sys_tee)
+#define __NR_vmsplice 343
+__SYSCALL(__NR_vmsplice, sys_vmsplice)
+#define __NR_move_pages 344
+__SYSCALL(__NR_move_pages, sys_move_pages)
+#define __NR_getcpu 345
+__SYSCALL(__NR_getcpu, sys_getcpu)
+#define __NR_epoll_pwait 346
+__SYSCALL(__NR_epoll_pwait, sys_epoll_pwait)
+#define __NR_kexec_load 347
+__SYSCALL(__NR_kexec_load, sys_kexec_load)
+#define __NR_utimensat 348
+__SYSCALL(__NR_utimensat, sys_utimensat)
+#define __NR_signalfd 349
+__SYSCALL(__NR_signalfd, sys_signalfd)
+#define __NR_timerfd_create 350
+__SYSCALL(__NR_timerfd_create, sys_timerfd_create)
+#define __NR_eventfd 351
+__SYSCALL(__NR_eventfd, sys_eventfd)
+#define __NR_fallocate 352
+__SYSCALL(__NR_fallocate, sys_fallocate)
+#define __NR_timerfd_settime 353
+__SYSCALL(__NR_timerfd_settime, sys_timerfd_settime)
+#define __NR_timerfd_gettime 354
+__SYSCALL(__NR_timerfd_gettime, sys_timerfd_gettime)
+#define __NR_signalfd4 355
+__SYSCALL(__NR_signalfd4, sys_signalfd4)
+#define __NR_eventfd2 356
+__SYSCALL(__NR_eventfd2, sys_eventfd2)
+#define __NR_epoll_create1 357
+__SYSCALL(__NR_epoll_create1, sys_epoll_create1)
+#define __NR_dup3 358
+__SYSCALL(__NR_dup3, sys_dup3)
+#define __NR_pipe2 359
+__SYSCALL(__NR_pipe2, sys_pipe2)
+#define __NR_inotify_init1 360
+__SYSCALL(__NR_inotify_init1, sys_inotify_init1)
+#define __NR_preadv 361
+__SYSCALL(__NR_preadv, sys_preadv)
+#define __NR_pwritev 362
+__SYSCALL(__NR_pwritev, sys_pwritev)
+#define __NR_rt_tgsigqueueinfo 363
+__SYSCALL(__NR_rt_tgsigqueueinfo, sys_rt_tgsigqueueinfo)
+#define __NR_perf_event_open 364
+__SYSCALL(__NR_perf_event_open, sys_perf_event_open)
+#define __NR_recvmmsg 365
+__SYSCALL(__NR_recvmmsg, sys_recvmmsg)
+#define __NR_accept4 366
+__SYSCALL(__NR_accept4, sys_accept4)
+#define __NR_fanotify_init 367
+__SYSCALL(__NR_fanotify_init, sys_fanotify_init)
+#define __NR_fanotify_mark 368
+__SYSCALL(__NR_fanotify_mark, sys_fanotify_mark)
+#define __NR_prlimit64 369
+__SYSCALL(__NR_prlimit64, sys_prlimit64)
+#define __NR_name_to_handle_at 370
+__SYSCALL(__NR_name_to_handle_at, sys_name_to_handle_at)
+#define __NR_open_by_handle_at 371
+__SYSCALL(__NR_open_by_handle_at, sys_open_by_handle_at)
+#define __NR_clock_adjtime 372
+__SYSCALL(__NR_clock_adjtime, sys_clock_adjtime)
+#define __NR_syncfs 373
+__SYSCALL(__NR_syncfs, sys_syncfs)
+
+/*
+ * The following SVCs are ARM private.
+ */
+#define __ARM_NR_COMPAT_BASE 0x0f0000
+#define __ARM_NR_compat_cacheflush (__ARM_NR_COMPAT_BASE+2)
+#define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE+5)
+
+#endif /* __SYSCALL_COMPAT */
+
+#define __NR_compat_syscalls 374
+
+#define __ARCH_WANT_COMPAT_IPC_PARSE_VERSION
+#define __ARCH_WANT_COMPAT_STAT64
+#define __ARCH_WANT_SYS_GETHOSTNAME
+#define __ARCH_WANT_SYS_PAUSE
+#define __ARCH_WANT_SYS_GETPGRP
+#define __ARCH_WANT_SYS_LLSEEK
+#define __ARCH_WANT_SYS_NICE
+#define __ARCH_WANT_SYS_SIGPENDING
+#define __ARCH_WANT_SYS_SIGPROCMASK
+#define __ARCH_WANT_COMPAT_SYS_RT_SIGSUSPEND
+
+#endif /* __ASM_UNISTD32_H */
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
new file mode 100644
index 0000000..839ce00
--- /dev/null
+++ b/arch/arm64/include/asm/vdso.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_VDSO_H
+#define __ASM_VDSO_H
+
+#ifdef __KERNEL__
+
+/*
+ * Default link address for the vDSO.
+ * Since we randomise the VDSO mapping, there's little point in trying
+ * to prelink this.
+ */
+#define VDSO_LBASE 0x0
+
+#ifndef __ASSEMBLY__
+
+#include <generated/vdso-offsets.h>
+
+#define VDSO_SYMBOL(base, name) \
+({ \
+ (void *)(vdso_offset_##name - VDSO_LBASE + (unsigned long)(base)); \
+})
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __ASM_VDSO_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
new file mode 100644
index 0000000..de66199
--- /dev/null
+++ b/arch/arm64/include/asm/vdso_datapage.h
@@ -0,0 +1,43 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __ASM_VDSO_DATAPAGE_H
+#define __ASM_VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+struct vdso_data {
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u64 xtime_clock_sec; /* Kernel time */
+ __u64 xtime_clock_nsec;
+ __u64 xtime_coarse_sec; /* Coarse time */
+ __u64 xtime_coarse_nsec;
+ __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u64 wtm_clock_nsec;
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mult; /* Clocksource multiplier */
+ __u32 cs_shift; /* Clocksource shift */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u32 tz_dsttime;
+ __u32 use_syscall;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/.gitignore b/arch/arm64/kernel/.gitignore
new file mode 100644
index 0000000..c5f676c
--- /dev/null
+++ b/arch/arm64/kernel/.gitignore
@@ -0,0 +1 @@
+vmlinux.lds
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
new file mode 100644
index 0000000..e2caff1
--- /dev/null
+++ b/arch/arm64/kernel/Makefile
@@ -0,0 +1,27 @@
+#
+# Makefile for the linux kernel.
+#
+
+CPPFLAGS_vmlinux.lds := -DTEXT_OFFSET=$(TEXT_OFFSET)
+AFLAGS_head.o := -DTEXT_OFFSET=$(TEXT_OFFSET)
+
+# Object file lists.
+arm64-obj-y := cputable.o debug-monitors.o entry.o irq.o fpsimd.o \
+ entry-fpsimd.o process.o ptrace.o setup.o signal.o \
+ sys.o stacktrace.o time.o traps.o io.o vdso.o
+
+arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+ sys_compat.o
+arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
+arm64-obj-$(CONFIG_SMP) += smp.o
+arm64-obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o
+arm64-obj-$(CONFIG_HAVE_HW_BREAKPOINT)+= hw_breakpoint.o
+
+obj-y += $(arm64-obj-y) vdso/
+obj-m += $(arm64-obj-m)
+head-y := head.o
+extra-y := $(head-y) vmlinux.lds
+
+# vDSO - this must be built first to generate the symbol offsets
+$(call objectify,$(arm64-obj-y)): $(obj)/vdso/vdso-offsets.h
+$(obj)/vdso/vdso-offsets.h: $(obj)/vdso
diff --git a/arch/arm64/kernel/arm64ksyms.c b/arch/arm64/kernel/arm64ksyms.c
new file mode 100644
index 0000000..cef3925
--- /dev/null
+++ b/arch/arm64/kernel/arm64ksyms.c
@@ -0,0 +1,46 @@
+/*
+ * Based on arch/arm/kernel/armksyms.c
+ *
+ * Copyright (C) 2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/sched.h>
+#include <linux/string.h>
+#include <linux/cryptohash.h>
+#include <linux/delay.h>
+#include <linux/in6.h>
+#include <linux/syscalls.h>
+#include <linux/uaccess.h>
+#include <linux/io.h>
+
+#include <asm/checksum.h>
+
+ /* user mem (segment) */
+EXPORT_SYMBOL(__strnlen_user);
+EXPORT_SYMBOL(__strncpy_from_user);
+
+EXPORT_SYMBOL(copy_page);
+
+EXPORT_SYMBOL(__copy_from_user);
+EXPORT_SYMBOL(__copy_to_user);
+EXPORT_SYMBOL(__clear_user);
+
+ /* bitops */
+EXPORT_SYMBOL(__atomic_hash);
+
+ /* physical memory */
+EXPORT_SYMBOL(memstart_addr);
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
new file mode 100644
index 0000000..a2a4d81
--- /dev/null
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -0,0 +1,108 @@
+/*
+ * Based on arch/arm/kernel/asm-offsets.c
+ *
+ * Copyright (C) 1995-2003 Russell King
+ * 2001-2002 Keith Owens
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/dma-mapping.h>
+#include <asm/thread_info.h>
+#include <asm/memory.h>
+#include <asm/cputable.h>
+#include <asm/vdso_datapage.h>
+#include <linux/kbuild.h>
+
+int main(void)
+{
+ DEFINE(TSK_ACTIVE_MM, offsetof(struct task_struct, active_mm));
+ BLANK();
+ DEFINE(TI_FLAGS, offsetof(struct thread_info, flags));
+ DEFINE(TI_PREEMPT, offsetof(struct thread_info, preempt_count));
+ DEFINE(TI_ADDR_LIMIT, offsetof(struct thread_info, addr_limit));
+ DEFINE(TI_TASK, offsetof(struct thread_info, task));
+ DEFINE(TI_EXEC_DOMAIN, offsetof(struct thread_info, exec_domain));
+ DEFINE(TI_CPU, offsetof(struct thread_info, cpu));
+ BLANK();
+ DEFINE(THREAD_CPU_CONTEXT, offsetof(struct task_struct, thread.cpu_context));
+ BLANK();
+ DEFINE(S_X0, offsetof(struct pt_regs, regs[0]));
+ DEFINE(S_X1, offsetof(struct pt_regs, regs[1]));
+ DEFINE(S_X2, offsetof(struct pt_regs, regs[2]));
+ DEFINE(S_X3, offsetof(struct pt_regs, regs[3]));
+ DEFINE(S_X4, offsetof(struct pt_regs, regs[4]));
+ DEFINE(S_X5, offsetof(struct pt_regs, regs[5]));
+ DEFINE(S_X6, offsetof(struct pt_regs, regs[6]));
+ DEFINE(S_X7, offsetof(struct pt_regs, regs[7]));
+ DEFINE(S_LR, offsetof(struct pt_regs, regs[30]));
+ DEFINE(S_SP, offsetof(struct pt_regs, sp));
+#ifdef CONFIG_COMPAT
+ DEFINE(S_COMPAT_SP, offsetof(struct pt_regs, compat_sp));
+#endif
+ DEFINE(S_PSTATE, offsetof(struct pt_regs, pstate));
+ DEFINE(S_PC, offsetof(struct pt_regs, pc));
+ DEFINE(S_ORIG_X0, offsetof(struct pt_regs, orig_x0));
+ DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno));
+ DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
+ BLANK();
+ DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id));
+ BLANK();
+ DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
+ DEFINE(VMA_VM_FLAGS, offsetof(struct vm_area_struct, vm_flags));
+ BLANK();
+ DEFINE(VM_EXEC, VM_EXEC);
+ BLANK();
+ DEFINE(PAGE_SZ, PAGE_SIZE);
+ BLANK();
+ DEFINE(CPU_INFO_SZ, sizeof(struct cpu_info));
+ DEFINE(CPU_INFO_SETUP, offsetof(struct cpu_info, cpu_setup));
+ BLANK();
+ DEFINE(DMA_BIDIRECTIONAL, DMA_BIDIRECTIONAL);
+ DEFINE(DMA_TO_DEVICE, DMA_TO_DEVICE);
+ DEFINE(DMA_FROM_DEVICE, DMA_FROM_DEVICE);
+ BLANK();
+ DEFINE(CLOCK_REALTIME, CLOCK_REALTIME);
+ DEFINE(CLOCK_MONOTONIC, CLOCK_MONOTONIC);
+ DEFINE(CLOCK_REALTIME_RES, MONOTONIC_RES_NSEC);
+ DEFINE(CLOCK_REALTIME_COARSE, CLOCK_REALTIME_COARSE);
+ DEFINE(CLOCK_MONOTONIC_COARSE,CLOCK_MONOTONIC_COARSE);
+ DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
+ DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
+ BLANK();
+ DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
+ DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
+ DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
+ DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
+ DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
+ DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
+ DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
+ DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
+ DEFINE(VDSO_CS_MULT, offsetof(struct vdso_data, cs_mult));
+ DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
+ DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
+ DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
+ BLANK();
+ DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
+ DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
+ DEFINE(TSPEC_TV_SEC, offsetof(struct timespec, tv_sec));
+ DEFINE(TSPEC_TV_NSEC, offsetof(struct timespec, tv_nsec));
+ BLANK();
+ DEFINE(TZ_MINWEST, offsetof(struct timezone, tz_minuteswest));
+ DEFINE(TZ_DSTTIME, offsetof(struct timezone, tz_dsttime));
+ return 0;
+}
diff --git a/arch/arm64/kernel/cputable.c b/arch/arm64/kernel/cputable.c
new file mode 100644
index 0000000..63cfc4a
--- /dev/null
+++ b/arch/arm64/kernel/cputable.c
@@ -0,0 +1,33 @@
+/*
+ * arch/arm64/kernel/cputable.c
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/init.h>
+
+#include <asm/cputable.h>
+
+extern unsigned long __cpu_setup(void);
+
+struct cpu_info __initdata cpu_table[] = {
+ {
+ .cpu_id_val = 0x000f0000,
+ .cpu_id_mask = 0x000f0000,
+ .cpu_name = "AArch64 Processor",
+ .cpu_setup = __cpu_setup,
+ },
+ { /* Empty */ },
+};
diff --git a/arch/arm64/kernel/debug-monitors.c b/arch/arm64/kernel/debug-monitors.c
new file mode 100644
index 0000000..0c3ba9f
--- /dev/null
+++ b/arch/arm64/kernel/debug-monitors.c
@@ -0,0 +1,288 @@
+/*
+ * ARMv8 single-step debug support and mdscr context switching.
+ *
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/cpu.h>
+#include <linux/debugfs.h>
+#include <linux/hardirq.h>
+#include <linux/init.h>
+#include <linux/ptrace.h>
+#include <linux/stat.h>
+
+#include <asm/debug-monitors.h>
+#include <asm/local.h>
+#include <asm/cputype.h>
+#include <asm/system_misc.h>
+
+/* Low-level stepping controls. */
+#define DBG_MDSCR_SS (1 << 0)
+#define DBG_SPSR_SS (1 << 21)
+
+/* MDSCR_EL1 enabling bits */
+#define DBG_MDSCR_KDE (1 << 13)
+#define DBG_MDSCR_MDE (1 << 15)
+#define DBG_MDSCR_MASK ~(DBG_MDSCR_KDE | DBG_MDSCR_MDE)
+
+/* Determine debug architecture. */
+u8 debug_monitors_arch(void)
+{
+ return read_cpuid(ID_AA64DFR0_EL1) & 0xf;
+}
+
+/*
+ * MDSCR access routines.
+ */
+static void mdscr_write(u32 mdscr)
+{
+ unsigned long flags;
+ local_dbg_save(flags);
+ asm volatile("msr mdscr_el1, %0" :: "r" (mdscr));
+ local_dbg_restore(flags);
+}
+
+static u32 mdscr_read(void)
+{
+ u32 mdscr;
+ asm volatile("mrs %0, mdscr_el1" : "=r" (mdscr));
+ return mdscr;
+}
+
+/*
+ * Allow root to disable self-hosted debug from userspace.
+ * This is useful if you want to connect an external JTAG debugger.
+ */
+static u32 debug_enabled = 1;
+
+static int create_debug_debugfs_entry(void)
+{
+ debugfs_create_bool("debug_enabled", 0644, NULL, &debug_enabled);
+ return 0;
+}
+fs_initcall(create_debug_debugfs_entry);
+
+static int __init early_debug_disable(char *buf)
+{
+ debug_enabled = 0;
+ return 0;
+}
+
+early_param("nodebugmon", early_debug_disable);
+
+/*
+ * Keep track of debug users on each core.
+ * The ref counts are per-cpu so we use a local_t type.
+ */
+static DEFINE_PER_CPU(local_t, mde_ref_count);
+static DEFINE_PER_CPU(local_t, kde_ref_count);
+
+void enable_debug_monitors(enum debug_el el)
+{
+ u32 mdscr, enable = 0;
+
+ WARN_ON(preemptible());
+
+ if (local_inc_return(&__get_cpu_var(mde_ref_count)) == 1)
+ enable = DBG_MDSCR_MDE;
+
+ if (el == DBG_ACTIVE_EL1 &&
+ local_inc_return(&__get_cpu_var(kde_ref_count)) == 1)
+ enable |= DBG_MDSCR_KDE;
+
+ if (enable && debug_enabled) {
+ mdscr = mdscr_read();
+ mdscr |= enable;
+ mdscr_write(mdscr);
+ }
+}
+
+void disable_debug_monitors(enum debug_el el)
+{
+ u32 mdscr, disable = 0;
+
+ WARN_ON(preemptible());
+
+ if (local_dec_and_test(&__get_cpu_var(mde_ref_count)))
+ disable = ~DBG_MDSCR_MDE;
+
+ if (el == DBG_ACTIVE_EL1 &&
+ local_dec_and_test(&__get_cpu_var(kde_ref_count)))
+ disable &= ~DBG_MDSCR_KDE;
+
+ if (disable) {
+ mdscr = mdscr_read();
+ mdscr &= disable;
+ mdscr_write(mdscr);
+ }
+}
+
+/*
+ * OS lock clearing.
+ */
+static void clear_os_lock(void *unused)
+{
+ asm volatile("msr mdscr_el1, %0" : : "r" (0));
+ isb();
+ asm volatile("msr oslar_el1, %0" : : "r" (0));
+ isb();
+}
+
+static int __cpuinit os_lock_notify(struct notifier_block *self,
+ unsigned long action, void *data)
+{
+ int cpu = (unsigned long)data;
+ if (action == CPU_ONLINE)
+ smp_call_function_single(cpu, clear_os_lock, NULL, 1);
+ return NOTIFY_OK;
+}
+
+static struct notifier_block __cpuinitdata os_lock_nb = {
+ .notifier_call = os_lock_notify,
+};
+
+static int __cpuinit debug_monitors_init(void)
+{
+ /* Clear the OS lock. */
+ smp_call_function(clear_os_lock, NULL, 1);
+ clear_os_lock(NULL);
+
+ /* Register hotplug handler. */
+ register_cpu_notifier(&os_lock_nb);
+ return 0;
+}
+postcore_initcall(debug_monitors_init);
+
+/*
+ * Single step API and exception handling.
+ */
+static void set_regs_spsr_ss(struct pt_regs *regs)
+{
+ unsigned long spsr;
+
+ spsr = regs->pstate;
+ spsr &= ~DBG_SPSR_SS;
+ spsr |= DBG_SPSR_SS;
+ regs->pstate = spsr;
+}
+
+static void clear_regs_spsr_ss(struct pt_regs *regs)
+{
+ unsigned long spsr;
+
+ spsr = regs->pstate;
+ spsr &= ~DBG_SPSR_SS;
+ regs->pstate = spsr;
+}
+
+static int single_step_handler(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ siginfo_t info;
+
+ /*
+ * If we are stepping a pending breakpoint, call the hw_breakpoint
+ * handler first.
+ */
+ if (!reinstall_suspended_bps(regs))
+ return 0;
+
+ if (user_mode(regs)) {
+ info.si_signo = SIGTRAP;
+ info.si_errno = 0;
+ info.si_code = TRAP_HWBKPT;
+ info.si_addr = (void __user *)instruction_pointer(regs);
+ force_sig_info(SIGTRAP, &info, current);
+
+ /*
+ * ptrace will disable single step unless explicitly
+ * asked to re-enable it. For other clients, it makes
+ * sense to leave it enabled (i.e. rewind the controls
+ * to the active-not-pending state).
+ */
+ user_rewind_single_step(current);
+ } else {
+ /* TODO: route to KGDB */
+ pr_warning("Unexpected kernel single-step exception at EL1\n");
+ /*
+ * Re-enable stepping since we know that we will be
+ * returning to regs.
+ */
+ set_regs_spsr_ss(regs);
+ }
+
+ return 0;
+}
+
+static int __init single_step_init(void)
+{
+ hook_debug_fault_code(DBG_ESR_EVT_HWSS, single_step_handler, SIGTRAP,
+ TRAP_HWBKPT, "single-step handler");
+ return 0;
+}
+arch_initcall(single_step_init);
+
+/* Re-enable single step for syscall restarting. */
+void user_rewind_single_step(struct task_struct *task)
+{
+ /*
+ * If single step is active for this thread, then set SPSR.SS
+ * to 1 to avoid returning to the active-pending state.
+ */
+ if (test_ti_thread_flag(task_thread_info(task), TIF_SINGLESTEP))
+ set_regs_spsr_ss(task_pt_regs(task));
+}
+
+void user_fastforward_single_step(struct task_struct *task)
+{
+ if (test_ti_thread_flag(task_thread_info(task), TIF_SINGLESTEP))
+ clear_regs_spsr_ss(task_pt_regs(task));
+}
+
+/* Kernel API */
+void kernel_enable_single_step(struct pt_regs *regs)
+{
+ WARN_ON(!irqs_disabled());
+ set_regs_spsr_ss(regs);
+ mdscr_write(mdscr_read() | DBG_MDSCR_SS);
+ enable_debug_monitors(DBG_ACTIVE_EL1);
+}
+
+void kernel_disable_single_step(void)
+{
+ WARN_ON(!irqs_disabled());
+ mdscr_write(mdscr_read() & ~DBG_MDSCR_SS);
+ disable_debug_monitors(DBG_ACTIVE_EL1);
+}
+
+int kernel_active_single_step(void)
+{
+ WARN_ON(!irqs_disabled());
+ return mdscr_read() & DBG_MDSCR_SS;
+}
+
+/* ptrace API */
+void user_enable_single_step(struct task_struct *task)
+{
+ set_ti_thread_flag(task_thread_info(task), TIF_SINGLESTEP);
+ set_regs_spsr_ss(task_pt_regs(task));
+}
+
+void user_disable_single_step(struct task_struct *task)
+{
+ clear_ti_thread_flag(task_thread_info(task), TIF_SINGLESTEP);
+}
diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
new file mode 100644
index 0000000..17988a6
--- /dev/null
+++ b/arch/arm64/kernel/entry-fpsimd.S
@@ -0,0 +1,80 @@
+/*
+ * FP/SIMD state saving and restoring
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/assembler.h>
+
+/*
+ * Save the FP registers.
+ *
+ * x0 - pointer to struct fpsimd_state
+ */
+ENTRY(fpsimd_save_state)
+ stp q0, q1, [x0, #16 * 0]
+ stp q2, q3, [x0, #16 * 2]
+ stp q4, q5, [x0, #16 * 4]
+ stp q6, q7, [x0, #16 * 6]
+ stp q8, q9, [x0, #16 * 8]
+ stp q10, q11, [x0, #16 * 10]
+ stp q12, q13, [x0, #16 * 12]
+ stp q14, q15, [x0, #16 * 14]
+ stp q16, q17, [x0, #16 * 16]
+ stp q18, q19, [x0, #16 * 18]
+ stp q20, q21, [x0, #16 * 20]
+ stp q22, q23, [x0, #16 * 22]
+ stp q24, q25, [x0, #16 * 24]
+ stp q26, q27, [x0, #16 * 26]
+ stp q28, q29, [x0, #16 * 28]
+ stp q30, q31, [x0, #16 * 30]!
+ mrs x8, fpsr
+ str w8, [x0, #16 * 2]
+ mrs x8, fpcr
+ str w8, [x0, #16 * 2 + 4]
+ ret
+ENDPROC(fpsimd_save_state)
+
+/*
+ * Load the FP registers.
+ *
+ * x0 - pointer to struct fpsimd_state
+ */
+ENTRY(fpsimd_load_state)
+ ldp q0, q1, [x0, #16 * 0]
+ ldp q2, q3, [x0, #16 * 2]
+ ldp q4, q5, [x0, #16 * 4]
+ ldp q6, q7, [x0, #16 * 6]
+ ldp q8, q9, [x0, #16 * 8]
+ ldp q10, q11, [x0, #16 * 10]
+ ldp q12, q13, [x0, #16 * 12]
+ ldp q14, q15, [x0, #16 * 14]
+ ldp q16, q17, [x0, #16 * 16]
+ ldp q18, q19, [x0, #16 * 18]
+ ldp q20, q21, [x0, #16 * 20]
+ ldp q22, q23, [x0, #16 * 22]
+ ldp q24, q25, [x0, #16 * 24]
+ ldp q26, q27, [x0, #16 * 26]
+ ldp q28, q29, [x0, #16 * 28]
+ ldp q30, q31, [x0, #16 * 30]!
+ ldr w8, [x0, #16 * 2]
+ ldr w9, [x0, #16 * 2 + 4]
+ msr fpsr, x8
+ msr fpcr, x9
+ ret
+ENDPROC(fpsimd_load_state)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
new file mode 100644
index 0000000..38cf853
--- /dev/null
+++ b/arch/arm64/kernel/entry.S
@@ -0,0 +1,695 @@
+/*
+ * Low-level exception handling code
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Authors: Catalin Marinas <catalin.marinas@arm.com>
+ * Will Deacon <will.deacon@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+
+#include <asm/assembler.h>
+#include <asm/asm-offsets.h>
+#include <asm/errno.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+
+/*
+ * Bad Abort numbers
+ *-----------------
+ */
+#define BAD_SYNC 0
+#define BAD_IRQ 1
+#define BAD_FIQ 2
+#define BAD_ERROR 3
+
+ .macro kernel_entry, el, regsize = 64
+ sub sp, sp, #S_FRAME_SIZE - S_LR // room for LR, SP, SPSR, ELR
+ .if \regsize == 32
+ mov w0, w0 // zero upper 32 bits of x0
+ .endif
+ push x28, x29
+ push x26, x27
+ push x24, x25
+ push x22, x23
+ push x20, x21
+ push x18, x19
+ push x16, x17
+ push x14, x15
+ push x12, x13
+ push x10, x11
+ push x8, x9
+ push x6, x7
+ push x4, x5
+ push x2, x3
+ push x0, x1
+ .if \el == 0
+ mrs x21, sp_el0
+ .else
+ add x21, sp, #S_FRAME_SIZE
+ .endif
+ mrs x22, elr_el1
+ mrs x23, spsr_el1
+ stp lr, x21, [sp, #S_LR]
+ stp x22, x23, [sp, #S_PC]
+
+ /*
+ * Set syscallno to -1 by default (overridden later if real syscall).
+ */
+ .if \el == 0
+ mvn x21, xzr
+ str x21, [sp, #S_SYSCALLNO]
+ .endif
+
+ /*
+ * Registers that may be useful after this macro is invoked:
+ *
+ * x21 - aborted SP
+ * x22 - aborted PC
+ * x23 - aborted PSTATE
+ */
+ .endm
+
+ .macro kernel_exit, el, ret = 0
+ ldp x21, x22, [sp, #S_PC] // load ELR, SPSR
+ .if \el == 0
+ ldr x23, [sp, #S_SP] // load return stack pointer
+ .endif
+ .if \ret
+ ldr x1, [sp, #S_X1] // preserve x0 (syscall return)
+ add sp, sp, S_X2
+ .else
+ pop x0, x1
+ .endif
+ pop x2, x3 // load the rest of the registers
+ pop x4, x5
+ pop x6, x7
+ pop x8, x9
+ msr elr_el1, x21 // set up the return data
+ msr spsr_el1, x22
+ .if \el == 0
+ msr sp_el0, x23
+ .endif
+ pop x10, x11
+ pop x12, x13
+ pop x14, x15
+ pop x16, x17
+ pop x18, x19
+ pop x20, x21
+ pop x22, x23
+ pop x24, x25
+ pop x26, x27
+ pop x28, x29
+ ldr lr, [sp], #S_FRAME_SIZE - S_LR // load LR and restore SP
+ eret // return to kernel
+ .endm
+
+ .macro get_thread_info, rd
+ mov \rd, sp
+ and \rd, \rd, #~((1 << 13) - 1) // top of 8K stack
+ .endm
+
+/*
+ * These are the registers used in the syscall handler, and allow us to
+ * have in theory up to 7 arguments to a function - x0 to x6.
+ *
+ * x7 is reserved for the system call number in 32-bit mode.
+ */
+sc_nr .req x25 // number of system calls
+scno .req x26 // syscall number
+stbl .req x27 // syscall table pointer
+tsk .req x28 // current thread_info
+
+/*
+ * Interrupt handling.
+ */
+ .macro irq_handler
+ ldr x1, handle_arch_irq
+ mov x0, sp
+ blr x1
+ .endm
+
+ .text
+
+/*
+ * Exception vectors.
+ */
+ .macro ventry label
+ .align 7
+ b \label
+ .endm
+
+ .align 11
+ENTRY(vectors)
+ ventry el1_sync_invalid // Synchronous EL1t
+ ventry el1_irq_invalid // IRQ EL1t
+ ventry el1_fiq_invalid // FIQ EL1t
+ ventry el1_error_invalid // Error EL1t
+
+ ventry el1_sync // Synchronous EL1h
+ ventry el1_irq // IRQ EL1h
+ ventry el1_fiq_invalid // FIQ EL1h
+ ventry el1_error_invalid // Error EL1h
+
+ ventry el0_sync // Synchronous 64-bit EL0
+ ventry el0_irq // IRQ 64-bit EL0
+ ventry el0_fiq_invalid // FIQ 64-bit EL0
+ ventry el0_error_invalid // Error 64-bit EL0
+
+#ifdef CONFIG_COMPAT
+ ventry el0_sync_compat // Synchronous 32-bit EL0
+ ventry el0_irq_compat // IRQ 32-bit EL0
+ ventry el0_fiq_invalid_compat // FIQ 32-bit EL0
+ ventry el0_error_invalid_compat // Error 32-bit EL0
+#else
+ ventry el0_sync_invalid // Synchronous 32-bit EL0
+ ventry el0_irq_invalid // IRQ 32-bit EL0
+ ventry el0_fiq_invalid // FIQ 32-bit EL0
+ ventry el0_error_invalid // Error 32-bit EL0
+#endif
+END(vectors)
+
+/*
+ * Invalid mode handlers
+ */
+ .macro inv_entry, el, reason, regsize = 64
+ kernel_entry el, \regsize
+ mov x0, sp
+ mov x1, #\reason
+ mrs x2, esr_el1
+ b bad_mode
+ .endm
+
+el0_sync_invalid:
+ inv_entry 0, BAD_SYNC
+ENDPROC(el0_sync_invalid)
+
+el0_irq_invalid:
+ inv_entry 0, BAD_IRQ
+ENDPROC(el0_irq_invalid)
+
+el0_fiq_invalid:
+ inv_entry 0, BAD_FIQ
+ENDPROC(el0_fiq_invalid)
+
+el0_error_invalid:
+ inv_entry 0, BAD_ERROR
+ENDPROC(el0_error_invalid)
+
+#ifdef CONFIG_COMPAT
+el0_fiq_invalid_compat:
+ inv_entry 0, BAD_FIQ, 32
+ENDPROC(el0_fiq_invalid_compat)
+
+el0_error_invalid_compat:
+ inv_entry 0, BAD_ERROR, 32
+ENDPROC(el0_error_invalid_compat)
+#endif
+
+el1_sync_invalid:
+ inv_entry 1, BAD_SYNC
+ENDPROC(el1_sync_invalid)
+
+el1_irq_invalid:
+ inv_entry 1, BAD_IRQ
+ENDPROC(el1_irq_invalid)
+
+el1_fiq_invalid:
+ inv_entry 1, BAD_FIQ
+ENDPROC(el1_fiq_invalid)
+
+el1_error_invalid:
+ inv_entry 1, BAD_ERROR
+ENDPROC(el1_error_invalid)
+
+/*
+ * EL1 mode handlers.
+ */
+ .align 6
+el1_sync:
+ kernel_entry 1
+ mrs x1, esr_el1 // read the syndrome register
+ lsr x24, x1, #26 // exception class
+ cmp x24, #0x25 // data abort in EL1
+ b.eq el1_da
+ cmp x24, #0x18 // configurable trap
+ b.eq el1_undef
+ cmp x24, #0x26 // stack alignment exception
+ b.eq el1_sp_pc
+ cmp x24, #0x22 // pc alignment exception
+ b.eq el1_sp_pc
+ cmp x24, #0x00 // unknown exception in EL1
+ b.eq el1_undef
+ cmp x24, #0x30 // debug exception in EL1
+ b.ge el1_dbg
+ b el1_inv
+el1_da:
+ /*
+ * Data abort handling
+ */
+ mrs x0, far_el1
+ enable_dbg_if_not_stepping x2
+ // re-enable interrupts if they were enabled in the aborted context
+ tbnz x23, #7, 1f // PSR_I_BIT
+ enable_irq
+1:
+ mov x2, sp // struct pt_regs
+ bl do_mem_abort
+
+ // disable interrupts before pulling preserved data off the stack
+ disable_irq
+ kernel_exit 1
+el1_sp_pc:
+ /*
+ * Stack or PC alignment exception handling
+ */
+ mrs x0, far_el1
+ mov x1, x25
+ mov x2, sp
+ b do_sp_pc_abort
+el1_undef:
+ /*
+ * Undefined instruction
+ */
+ mov x0, sp
+ b do_undefinstr
+el1_dbg:
+ /*
+ * Debug exception handling
+ */
+ tbz x24, #0, el1_inv // EL1 only
+ mrs x0, far_el1
+ mov x2, sp // struct pt_regs
+ bl do_debug_exception
+
+ kernel_exit 1
+el1_inv:
+ // TODO: add support for undefined instructions in kernel mode
+ mov x0, sp
+ mov x1, #BAD_SYNC
+ mrs x2, esr_el1
+ b bad_mode
+ENDPROC(el1_sync)
+
+ .align 6
+el1_irq:
+ kernel_entry 1
+ enable_dbg_if_not_stepping x0
+#ifdef CONFIG_TRACE_IRQFLAGS
+ bl trace_hardirqs_off
+#endif
+#ifdef CONFIG_PREEMPT
+ get_thread_info tsk
+ ldr x24, [tsk, #TI_PREEMPT] // get preempt count
+ add x0, x24, #1 // increment it
+ str x0, [tsk, #TI_PREEMPT]
+#endif
+ irq_handler
+#ifdef CONFIG_PREEMPT
+ str x24, [tsk, #TI_PREEMPT] // restore preempt count
+ cbnz x24, 1f // preempt count != 0
+ ldr x0, [tsk, #TI_FLAGS] // get flags
+ tbz x0, #TIF_NEED_RESCHED, 1f // needs rescheduling?
+ bl el1_preempt
+1:
+#endif
+#ifdef CONFIG_TRACE_IRQFLAGS
+ bl trace_hardirqs_on
+#endif
+ kernel_exit 1
+ENDPROC(el1_irq)
+
+#ifdef CONFIG_PREEMPT
+el1_preempt:
+ mov x24, lr
+1: enable_dbg
+ bl preempt_schedule_irq // irq en/disable is done inside
+ ldr x0, [tsk, #TI_FLAGS] // get new tasks TI_FLAGS
+ tbnz x0, #TIF_NEED_RESCHED, 1b // needs rescheduling?
+ ret x24
+#endif
+
+/*
+ * EL0 mode handlers.
+ */
+ .align 6
+el0_sync:
+ kernel_entry 0
+ mrs x25, esr_el1 // read the syndrome register
+ lsr x24, x25, #26 // exception class
+ cmp x24, #0x15 // SVC in 64-bit state
+ b.eq el0_svc
+ adr lr, ret_from_exception
+ cmp x24, #0x24 // data abort in EL0
+ b.eq el0_da
+ cmp x24, #0x20 // instruction abort in EL0
+ b.eq el0_ia
+ cmp x24, #0x07 // FP/ASIMD access
+ b.eq el0_fpsimd_acc
+ cmp x24, #0x2c // FP/ASIMD exception
+ b.eq el0_fpsimd_exc
+ cmp x24, #0x18 // configurable trap
+ b.eq el0_undef
+ cmp x24, #0x26 // stack alignment exception
+ b.eq el0_sp_pc
+ cmp x24, #0x22 // pc alignment exception
+ b.eq el0_sp_pc
+ cmp x24, #0x00 // unknown exception in EL0
+ b.eq el0_undef
+ cmp x24, #0x30 // debug exception in EL0
+ b.ge el0_dbg
+ b el0_inv
+
+#ifdef CONFIG_COMPAT
+ .align 6
+el0_sync_compat:
+ kernel_entry 0, 32
+ mrs x25, esr_el1 // read the syndrome register
+ lsr x24, x25, #26 // exception class
+ cmp x24, #0x11 // SVC in 32-bit state
+ b.eq el0_svc_compat
+ adr lr, ret_from_exception
+ cmp x24, #0x24 // data abort in EL0
+ b.eq el0_da
+ cmp x24, #0x20 // instruction abort in EL0
+ b.eq el0_ia
+ cmp x24, #0x07 // FP/ASIMD access
+ b.eq el0_fpsimd_acc
+ cmp x24, #0x28 // FP/ASIMD exception
+ b.eq el0_fpsimd_exc
+ cmp x24, #0x00 // unknown exception in EL0
+ b.eq el0_undef
+ cmp x24, #0x30 // debug exception in EL0
+ b.ge el0_dbg
+ b el0_inv
+el0_svc_compat:
+ /*
+ * AArch32 syscall handling
+ */
+ adr stbl, compat_sys_call_table // load compat syscall table pointer
+ uxtw scno, w7 // syscall number in w7 (r7)
+ mov sc_nr, #__NR_compat_syscalls
+ b el0_svc_naked
+
+ .align 6
+el0_irq_compat:
+ kernel_entry 0, 32
+ b el0_irq_naked
+#endif
+
+el0_da:
+ /*
+ * Data abort handling
+ */
+ mrs x0, far_el1
+ disable_step x1
+ isb
+ enable_dbg
+ // enable interrupts before calling the main handler
+ enable_irq
+ mov x1, x25
+ mov x2, sp
+ b do_mem_abort
+el0_ia:
+ /*
+ * Instruction abort handling
+ */
+ mrs x0, far_el1
+ disable_step x1
+ isb
+ enable_dbg
+ // enable interrupts before calling the main handler
+ enable_irq
+ orr x1, x25, #1 << 24 // use reserved ISS bit for instruction aborts
+ mov x2, sp
+ b do_mem_abort
+el0_fpsimd_acc:
+ /*
+ * Floating Point or Advanced SIMD access
+ */
+ mov x0, x25
+ mov x1, sp
+ b do_fpsimd_acc
+el0_fpsimd_exc:
+ /*
+ * Floating Point or Advanced SIMD exception
+ */
+ mov x0, x25
+ mov x1, sp
+ b do_fpsimd_exc
+el0_sp_pc:
+ /*
+ * Stack or PC alignment exception handling
+ */
+ mrs x0, far_el1
+ disable_step x1
+ isb
+ enable_dbg
+ // enable interrupts before calling the main handler
+ enable_irq
+ mov x1, x25
+ mov x2, sp
+ b do_sp_pc_abort
+el0_undef:
+ /*
+ * Undefined instruction
+ */
+ mov x0, sp
+ b do_undefinstr
+el0_dbg:
+ /*
+ * Debug exception handling
+ */
+ tbnz x24, #0, el0_inv // EL0 only
+ mrs x0, far_el1
+ disable_step x1
+ mov x1, x25
+ mov x2, sp
+ b do_debug_exception
+el0_inv:
+ mov x0, sp
+ mov x1, #BAD_SYNC
+ mrs x2, esr_el1
+ b bad_mode
+ENDPROC(el0_sync)
+
+ .align 6
+el0_irq:
+ kernel_entry 0
+el0_irq_naked:
+ disable_step x1
+ isb
+ enable_dbg
+#ifdef CONFIG_TRACE_IRQFLAGS
+ bl trace_hardirqs_off
+#endif
+ get_thread_info tsk
+#ifdef CONFIG_PREEMPT
+ ldr x24, [tsk, #TI_PREEMPT] // get preempt count
+ add x23, x24, #1 // increment it
+ str x23, [tsk, #TI_PREEMPT]
+#endif
+ irq_handler
+#ifdef CONFIG_PREEMPT
+ ldr x0, [tsk, #TI_PREEMPT]
+ str x24, [tsk, #TI_PREEMPT]
+ cmp x0, x23
+ b.eq 1f
+ mov x1, #0
+ str x1, [x1] // BUG
+1:
+#endif
+#ifdef CONFIG_TRACE_IRQFLAGS
+ bl trace_hardirqs_on
+#endif
+ b ret_to_user
+ENDPROC(el0_irq)
+
+/*
+ * This is the return code to user mode for abort handlers
+ */
+ret_from_exception:
+ get_thread_info tsk
+ b ret_to_user
+ENDPROC(ret_from_exception)
+
+/*
+ * Register switch for AArch64. The callee-saved registers need to be saved
+ * and restored. On entry:
+ * x0 = previous task_struct (must be preserved across the switch)
+ * x1 = next task_struct
+ * Previous and next are guaranteed not to be the same.
+ *
+ */
+ENTRY(cpu_switch_to)
+ add x8, x0, #THREAD_CPU_CONTEXT
+ mov x9, sp
+ stp x19, x20, [x8], #16 // store callee-saved registers
+ stp x21, x22, [x8], #16
+ stp x23, x24, [x8], #16
+ stp x25, x26, [x8], #16
+ stp x27, x28, [x8], #16
+ stp x29, x9, [x8], #16
+ str lr, [x8]
+ add x8, x1, #THREAD_CPU_CONTEXT
+ ldp x19, x20, [x8], #16 // restore callee-saved registers
+ ldp x21, x22, [x8], #16
+ ldp x23, x24, [x8], #16
+ ldp x25, x26, [x8], #16
+ ldp x27, x28, [x8], #16
+ ldp x29, x9, [x8], #16
+ ldr lr, [x8]
+ mov sp, x9
+ ret
+ENDPROC(cpu_switch_to)
+
+/*
+ * This is the fast syscall return path. We do as little as possible here,
+ * and this includes saving x0 back into the kernel stack.
+ */
+ret_fast_syscall:
+ disable_irq // disable interrupts
+ ldr x1, [tsk, #TI_FLAGS]
+ and x2, x1, #_TIF_WORK_MASK
+ cbnz x2, fast_work_pending
+ tbz x1, #TIF_SINGLESTEP, fast_exit
+ disable_dbg
+ enable_step x2
+fast_exit:
+ kernel_exit 0, ret = 1
+
+/*
+ * Ok, we need to do extra processing, enter the slow path.
+ */
+fast_work_pending:
+ str x0, [sp, #S_X0] // returned x0
+work_pending:
+ tbnz x1, #TIF_NEED_RESCHED, work_resched
+ /* TIF_SIGPENDING or TIF_NOTIFY_RESUME case */
+ ldr x2, [sp, #S_PSTATE]
+ mov x0, sp // 'regs'
+ tst x2, #PSR_MODE_MASK // user mode regs?
+ b.ne no_work_pending // returning to kernel
+ bl do_notify_resume
+ b ret_to_user
+work_resched:
+ enable_dbg
+ bl schedule
+
+/*
+ * "slow" syscall return path.
+ */
+ENTRY(ret_to_user)
+ disable_irq // disable interrupts
+ ldr x1, [tsk, #TI_FLAGS]
+ and x2, x1, #_TIF_WORK_MASK
+ cbnz x2, work_pending
+ tbz x1, #TIF_SINGLESTEP, no_work_pending
+ disable_dbg
+ enable_step x2
+no_work_pending:
+ kernel_exit 0, ret = 0
+ENDPROC(ret_to_user)
+
+/*
+ * This is how we return from a fork.
+ */
+ENTRY(ret_from_fork)
+ bl schedule_tail
+ get_thread_info tsk
+ b ret_to_user
+ENDPROC(ret_from_fork)
+
+/*
+ * SVC handler.
+ */
+ .align 6
+el0_svc:
+ adrp stbl, sys_call_table // load syscall table pointer
+ uxtw scno, w8 // syscall number in w8
+ mov sc_nr, #__NR_syscalls
+el0_svc_naked: // compat entry point
+ stp x0, scno, [sp, #S_ORIG_X0] // save the original x0 and syscall number
+ disable_step x16
+ isb
+ enable_dbg
+ enable_irq
+
+ get_thread_info tsk
+ ldr x16, [tsk, #TI_FLAGS] // check for syscall tracing
+ tbnz x16, #TIF_SYSCALL_TRACE, __sys_trace // are we tracing syscalls?
+ adr lr, ret_fast_syscall // return address
+ cmp scno, sc_nr // check upper syscall limit
+ b.hs ni_sys
+ ldr x16, [stbl, scno, lsl #3] // address in the syscall table
+ br x16 // call sys_* routine
+ni_sys:
+ mov x0, sp
+ b do_ni_syscall
+ENDPROC(el0_svc)
+
+ /*
+ * This is the really slow path. We're going to be doing context
+ * switches, and waiting for our parent to respond.
+ */
+__sys_trace:
+ mov x1, sp
+ mov w0, #0 // trace entry
+ bl syscall_trace
+ adr lr, __sys_trace_return // return address
+ uxtw scno, w0 // syscall number (possibly new)
+ mov x1, sp // pointer to regs
+ cmp scno, sc_nr // check upper syscall limit
+ b.hs ni_sys
+ ldp x0, x1, [sp] // restore the syscall args
+ ldp x2, x3, [sp, #S_X2]
+ ldp x4, x5, [sp, #S_X4]
+ ldp x6, x7, [sp, #S_X6]
+ ldr x16, [stbl, scno, lsl #3] // address in the syscall table
+ br x16 // call sys_* routine
+
+__sys_trace_return:
+ str x0, [sp] // save returned x0
+ mov x1, sp
+ mov w0, #1 // trace exit
+ bl syscall_trace
+ b ret_to_user
+
+/*
+ * Special system call wrappers.
+ */
+ENTRY(sys_execve_wrapper)
+ mov x3, sp
+ b sys_execve
+ENDPROC(sys_execve_wrapper)
+
+ENTRY(sys_clone_wrapper)
+ mov x5, sp
+ b sys_clone
+ENDPROC(sys_clone_wrapper)
+
+ENTRY(sys_rt_sigreturn_wrapper)
+ mov x0, sp
+ b sys_rt_sigreturn
+ENDPROC(sys_rt_sigreturn_wrapper)
+
+ENTRY(sys_sigaltstack_wrapper)
+ ldr x2, [sp, #S_SP]
+ b sys_sigaltstack
+ENDPROC(sys_sigaltstack_wrapper)
+
+ENTRY(handle_arch_irq)
+ .quad 0
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
new file mode 100644
index 0000000..e8b8357
--- /dev/null
+++ b/arch/arm64/kernel/fpsimd.c
@@ -0,0 +1,106 @@
+/*
+ * FP/SIMD context switching and fault handling
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/signal.h>
+
+#include <asm/fpsimd.h>
+#include <asm/cputype.h>
+
+#define FPEXC_IOF (1 << 0)
+#define FPEXC_DZF (1 << 1)
+#define FPEXC_OFF (1 << 2)
+#define FPEXC_UFF (1 << 3)
+#define FPEXC_IXF (1 << 4)
+#define FPEXC_IDF (1 << 7)
+
+/*
+ * Trapped FP/ASIMD access.
+ */
+void do_fpsimd_acc(unsigned int esr, struct pt_regs *regs)
+{
+ /* TODO: implement lazy context saving/restoring */
+ WARN_ON(1);
+}
+
+/*
+ * Raise a SIGFPE for the current process.
+ */
+void do_fpsimd_exc(unsigned int esr, struct pt_regs *regs)
+{
+ siginfo_t info;
+ unsigned int si_code = 0;
+
+ if (esr & FPEXC_IOF)
+ si_code = FPE_FLTINV;
+ else if (esr & FPEXC_DZF)
+ si_code = FPE_FLTDIV;
+ else if (esr & FPEXC_OFF)
+ si_code = FPE_FLTOVF;
+ else if (esr & FPEXC_UFF)
+ si_code = FPE_FLTUND;
+ else if (esr & FPEXC_IXF)
+ si_code = FPE_FLTRES;
+
+ memset(&info, 0, sizeof(info));
+ info.si_signo = SIGFPE;
+ info.si_code = si_code;
+ info.si_addr = (void __user *)instruction_pointer(regs);
+
+ send_sig_info(SIGFPE, &info, current);
+}
+
+void fpsimd_thread_switch(struct task_struct *next)
+{
+ /* check if not kernel threads */
+ if (current->mm)
+ fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (next->mm)
+ fpsimd_load_state(&next->thread.fpsimd_state);
+}
+
+void fpsimd_flush_thread(void)
+{
+ memset(¤t->thread.fpsimd_state, 0, sizeof(struct fpsimd_state));
+ fpsimd_load_state(¤t->thread.fpsimd_state);
+}
+
+/*
+ * FP/SIMD support code initialisation.
+ */
+static int __init fpsimd_init(void)
+{
+ u64 pfr = read_cpuid(ID_AA64PFR0_EL1);
+
+ if (pfr & (0xf << 16)) {
+ pr_notice("Floating-point is not implemented\n");
+ return 0;
+ }
+ elf_hwcap |= HWCAP_FP;
+
+ if (pfr & (0xf << 20))
+ pr_notice("Advanced SIMD is not implemented\n");
+ else
+ elf_hwcap |= HWCAP_ASIMD;
+
+ return 0;
+}
+late_initcall(fpsimd_init);
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
new file mode 100644
index 0000000..a2f02b6
--- /dev/null
+++ b/arch/arm64/kernel/head.S
@@ -0,0 +1,510 @@
+/*
+ * Low-level CPU initialisation
+ * Based on arch/arm/kernel/head.S
+ *
+ * Copyright (C) 1994-2002 Russell King
+ * Copyright (C) 2003-2012 ARM Ltd.
+ * Authors: Catalin Marinas <catalin.marinas@arm.com>
+ * Will Deacon <will.deacon@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+
+#include <asm/assembler.h>
+#include <asm/ptrace.h>
+#include <asm/asm-offsets.h>
+#include <asm/memory.h>
+#include <asm/thread_info.h>
+#include <asm/pgtable-hwdef.h>
+#include <asm/pgtable.h>
+#include <asm/page.h>
+
+/*
+ * swapper_pg_dir is the virtual address of the initial page table. We place
+ * the page tables 3 * PAGE_SIZE below KERNEL_RAM_VADDR. The idmap_pg_dir has
+ * 2 pages and is placed below swapper_pg_dir.
+ */
+#define KERNEL_RAM_VADDR (PAGE_OFFSET + TEXT_OFFSET)
+
+#if (KERNEL_RAM_VADDR & 0xfffff) != 0x80000
+#error KERNEL_RAM_VADDR must start at 0xXXX80000
+#endif
+
+#define SWAPPER_DIR_SIZE (3 * PAGE_SIZE)
+#define IDMAP_DIR_SIZE (2 * PAGE_SIZE)
+
+ .globl swapper_pg_dir
+ .equ swapper_pg_dir, KERNEL_RAM_VADDR - SWAPPER_DIR_SIZE
+
+ .globl idmap_pg_dir
+ .equ idmap_pg_dir, swapper_pg_dir - IDMAP_DIR_SIZE
+
+ .macro pgtbl, ttb0, ttb1, phys
+ add \ttb1, \phys, #TEXT_OFFSET - SWAPPER_DIR_SIZE
+ sub \ttb0, \ttb1, #IDMAP_DIR_SIZE
+ .endm
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#define BLOCK_SHIFT PAGE_SHIFT
+#define BLOCK_SIZE PAGE_SIZE
+#else
+#define BLOCK_SHIFT SECTION_SHIFT
+#define BLOCK_SIZE SECTION_SIZE
+#endif
+
+#define KERNEL_START KERNEL_RAM_VADDR
+#define KERNEL_END _end
+
+/*
+ * Initial memory map attributes.
+ */
+#ifndef CONFIG_SMP
+#define PTE_FLAGS PTE_TYPE_PAGE | PTE_AF
+#define PMD_FLAGS PMD_TYPE_SECT | PMD_SECT_AF
+#else
+#define PTE_FLAGS PTE_TYPE_PAGE | PTE_AF | PTE_SHARED
+#define PMD_FLAGS PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S
+#endif
+
+#ifdef CONFIG_ARM64_64K_PAGES
+#define MM_MMUFLAGS PTE_ATTRINDX(MT_NORMAL) | PTE_FLAGS
+#define IO_MMUFLAGS PTE_ATTRINDX(MT_DEVICE_nGnRE) | PTE_XN | PTE_FLAGS
+#else
+#define MM_MMUFLAGS PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS
+#define IO_MMUFLAGS PMD_ATTRINDX(MT_DEVICE_nGnRE) | PMD_SECT_XN | PMD_FLAGS
+#endif
+
+/*
+ * Kernel startup entry point.
+ * ---------------------------
+ *
+ * The requirements are:
+ * MMU = off, D-cache = off, I-cache = on or off,
+ * x0 = physical address to the FDT blob.
+ *
+ * This code is mostly position independent so you call this at
+ * __pa(PAGE_OFFSET + TEXT_OFFSET).
+ *
+ * Note that the callee-saved registers are used for storing variables
+ * that are useful before the MMU is enabled. The allocations are described
+ * in the entry routines.
+ */
+ __HEAD
+
+ /*
+ * DO NOT MODIFY. Image header expected by Linux boot-loaders.
+ */
+ b stext // branch to kernel start, magic
+ .long 0 // reserved
+ .quad TEXT_OFFSET // Image load offset from start of RAM
+ .quad 0 // reserved
+ .quad 0 // reserved
+
+ENTRY(stext)
+ mov x21, x0 // x21=FDT
+ bl el2_setup // Drop to EL1
+ mrs x22, midr_el1 // x22=cpuid
+ mov x0, x22
+ bl lookup_processor_type
+ mov x23, x0 // x23=current cpu_table
+ cbz x23, __error_p // invalid processor (x23=0)?
+ bl __calc_phys_offset // x24=PHYS_OFFSET, x28=PHYS_OFFSET-PAGE_OFFSET
+ bl __vet_fdt
+ bl __create_page_tables // x25=TTBR0, x26=TTBR1
+ /*
+ * The following calls CPU specific code in a position independent
+ * manner. See arch/arm64/mm/proc.S for details. x23 = base of
+ * cpu_info structure selected by lookup_processor_type above.
+ * On return, the CPU will be ready for the MMU to be turned on and
+ * the TCR will have been set.
+ */
+ ldr x27, __switch_data // address to jump to after
+ // MMU has been enabled
+ adr lr, __enable_mmu // return (PIC) address
+ ldr x12, [x23, #CPU_INFO_SETUP]
+ add x12, x12, x28 // __virt_to_phys
+ br x12 // initialise processor
+ENDPROC(stext)
+
+/*
+ * If we're fortunate enough to boot at EL2, ensure that the world is
+ * sane before dropping to EL1.
+ */
+ENTRY(el2_setup)
+ mrs x0, CurrentEL
+ cmp x0, #PSR_MODE_EL2t
+ ccmp x0, #PSR_MODE_EL2h, #0x4, ne
+ b.eq 1f
+ ret
+
+ /* Hyp configuration. */
+1: mov x0, #(1 << 31) // 64-bit EL1
+ msr hcr_el2, x0
+
+ /* Generic timers. */
+ mrs x0, cnthctl_el2
+ orr x0, x0, #3 // Enable EL1 physical timers
+ msr cnthctl_el2, x0
+
+ /* Populate ID registers. */
+ mrs x0, midr_el1
+ mrs x1, mpidr_el1
+ msr vpidr_el2, x0
+ msr vmpidr_el2, x1
+
+ /* sctlr_el1 */
+ mov x0, #0x0800 // Set/clear RES{1,0} bits
+ movk x0, #0x30d0, lsl #16
+ msr sctlr_el1, x0
+
+ /* Coprocessor traps. */
+ mov x0, #0x33ff
+ msr cptr_el2, x0 // Disable copro. traps to EL2
+
+#ifdef CONFIG_COMPAT
+ msr hstr_el2, xzr // Disable CP15 traps to EL2
+#endif
+
+ /* spsr */
+ mov x0, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\
+ PSR_MODE_EL1h)
+ msr spsr_el2, x0
+ msr elr_el2, lr
+ eret
+ENDPROC(el2_setup)
+
+ .align 3
+2: .quad .
+ .quad PAGE_OFFSET
+
+#ifdef CONFIG_SMP
+ .pushsection .smp.pen.text, "ax"
+ .align 3
+1: .quad .
+ .quad secondary_holding_pen_release
+
+ /*
+ * This provides a "holding pen" for platforms to hold all secondary
+ * cores are held until we're ready for them to initialise.
+ */
+ENTRY(secondary_holding_pen)
+ bl el2_setup // Drop to EL1
+ mrs x0, mpidr_el1
+ and x0, x0, #15 // CPU number
+ adr x1, 1b
+ ldp x2, x3, [x1]
+ sub x1, x1, x2
+ add x3, x3, x1
+pen: ldr x4, [x3]
+ cmp x4, x0
+ b.eq secondary_startup
+ wfe
+ b pen
+ENDPROC(secondary_holding_pen)
+ .popsection
+
+ENTRY(secondary_startup)
+ /*
+ * Common entry point for secondary CPUs.
+ */
+ mrs x22, midr_el1 // x22=cpuid
+ mov x0, x22
+ bl lookup_processor_type
+ mov x23, x0 // x23=current cpu_table
+ cbz x23, __error_p // invalid processor (x23=0)?
+
+ bl __calc_phys_offset // x24=phys offset
+ pgtbl x25, x26, x24 // x25=TTBR0, x26=TTBR1
+ ldr x12, [x23, #CPU_INFO_SETUP]
+ add x12, x12, x28 // __virt_to_phys
+ blr x12 // initialise processor
+
+ ldr x21, =secondary_data
+ ldr x27, =__secondary_switched // address to jump to after enabling the MMU
+ b __enable_mmu
+ENDPROC(secondary_startup)
+
+ENTRY(__secondary_switched)
+ ldr x0, [x21] // get secondary_data.stack
+ mov sp, x0
+ mov x29, #0
+ b secondary_start_kernel
+ENDPROC(__secondary_switched)
+#endif /* CONFIG_SMP */
+
+/*
+ * Setup common bits before finally enabling the MMU. Essentially this is just
+ * loading the page table pointer and vector base registers.
+ *
+ * On entry to this code, x0 must contain the SCTLR_EL1 value for turning on
+ * the MMU.
+ */
+__enable_mmu:
+ ldr x5, =vectors
+ msr vbar_el1, x5
+ msr ttbr0_el1, x25 // load TTBR0
+ msr ttbr1_el1, x26 // load TTBR1
+ isb
+ b __turn_mmu_on
+ENDPROC(__enable_mmu)
+
+/*
+ * Enable the MMU. This completely changes the structure of the visible memory
+ * space. You will not be able to trace execution through this.
+ *
+ * x0 = system control register
+ * x27 = *virtual* address to jump to upon completion
+ *
+ * other registers depend on the function called upon completion
+ */
+ .align 6
+__turn_mmu_on:
+ msr sctlr_el1, x0
+ isb
+ br x27
+ENDPROC(__turn_mmu_on)
+
+/*
+ * Calculate the start of physical memory.
+ */
+__calc_phys_offset:
+ adr x0, 1f
+ ldp x1, x2, [x0]
+ sub x28, x0, x1 // x28 = PHYS_OFFSET - PAGE_OFFSET
+ add x24, x2, x28 // x24 = PHYS_OFFSET
+ ret
+ENDPROC(__calc_phys_offset)
+
+ .align 3
+1: .quad .
+ .quad PAGE_OFFSET
+
+/*
+ * Macro to populate the PGD for the corresponding block entry in the next
+ * level (tbl) for the given virtual address.
+ *
+ * Preserves: pgd, tbl, virt
+ * Corrupts: tmp1, tmp2
+ */
+ .macro create_pgd_entry, pgd, tbl, virt, tmp1, tmp2
+ lsr \tmp1, \virt, #PGDIR_SHIFT
+ and \tmp1, \tmp1, #PTRS_PER_PGD - 1 // PGD index
+ orr \tmp2, \tbl, #3 // PGD entry table type
+ str \tmp2, [\pgd, \tmp1, lsl #3]
+ .endm
+
+/*
+ * Macro to populate block entries in the page table for the start..end
+ * virtual range (inclusive).
+ *
+ * Preserves: tbl, flags
+ * Corrupts: phys, start, end, pstate
+ */
+ .macro create_block_map, tbl, flags, phys, start, end, idmap=0
+ lsr \phys, \phys, #BLOCK_SHIFT
+ .if \idmap
+ and \start, \phys, #PTRS_PER_PTE - 1 // table index
+ .else
+ lsr \start, \start, #BLOCK_SHIFT
+ and \start, \start, #PTRS_PER_PTE - 1 // table index
+ .endif
+ orr \phys, \flags, \phys, lsl #BLOCK_SHIFT // table entry
+ .ifnc \start,\end
+ lsr \end, \end, #BLOCK_SHIFT
+ and \end, \end, #PTRS_PER_PTE - 1 // table end index
+ .endif
+9999: str \phys, [\tbl, \start, lsl #3] // store the entry
+ .ifnc \start,\end
+ add \start, \start, #1 // next entry
+ add \phys, \phys, #BLOCK_SIZE // next block
+ cmp \start, \end
+ b.ls 9999b
+ .endif
+ .endm
+
+/*
+ * Setup the initial page tables. We only setup the barest amount which is
+ * required to get the kernel running. The following sections are required:
+ * - identity mapping to enable the MMU (low address, TTBR0)
+ * - first few MB of the kernel linear mapping to jump to once the MMU has
+ * been enabled, including the FDT blob (TTBR1)
+ */
+__create_page_tables:
+ pgtbl x25, x26, x24 // idmap_pg_dir and swapper_pg_dir addresses
+
+ /*
+ * Clear the idmap and swapper page tables.
+ */
+ mov x0, x25
+ add x6, x26, #SWAPPER_DIR_SIZE
+1: stp xzr, xzr, [x0], #16
+ stp xzr, xzr, [x0], #16
+ stp xzr, xzr, [x0], #16
+ stp xzr, xzr, [x0], #16
+ cmp x0, x6
+ b.lo 1b
+
+ ldr x7, =MM_MMUFLAGS
+
+ /*
+ * Create the identity mapping.
+ */
+ add x0, x25, #PAGE_SIZE // section table address
+ adr x3, __turn_mmu_on // virtual/physical address
+ create_pgd_entry x25, x0, x3, x5, x6
+ create_block_map x0, x7, x3, x5, x5, idmap=1
+
+ /*
+ * Map the kernel image (starting with PHYS_OFFSET).
+ */
+ add x0, x26, #PAGE_SIZE // section table address
+ mov x5, #PAGE_OFFSET
+ create_pgd_entry x26, x0, x5, x3, x6
+ ldr x6, =KERNEL_END - 1
+ mov x3, x24 // phys offset
+ create_block_map x0, x7, x3, x5, x6
+
+ /*
+ * Map the FDT blob (maximum 2MB; must be within 512MB of
+ * PHYS_OFFSET).
+ */
+ mov x3, x21 // FDT phys address
+ and x3, x3, #~((1 << 21) - 1) // 2MB aligned
+ mov x6, #PAGE_OFFSET
+ sub x5, x3, x24 // subtract PHYS_OFFSET
+ tst x5, #~((1 << 29) - 1) // within 512MB?
+ csel x21, xzr, x21, ne // zero the FDT pointer
+ b.ne 1f
+ add x5, x5, x6 // __va(FDT blob)
+ add x6, x5, #1 << 21 // 2MB for the FDT blob
+ sub x6, x6, #1 // inclusive range
+ create_block_map x0, x7, x3, x5, x6
+1:
+ ret
+ENDPROC(__create_page_tables)
+ .ltorg
+
+ .align 3
+ .type __switch_data, %object
+__switch_data:
+ .quad __mmap_switched
+ .quad __data_loc // x4
+ .quad _data // x5
+ .quad __bss_start // x6
+ .quad _end // x7
+ .quad processor_id // x4
+ .quad __fdt_pointer // x5
+ .quad memstart_addr // x6
+ .quad init_thread_union + THREAD_START_SP // sp
+
+/*
+ * The following fragment of code is executed with the MMU on in MMU mode, and
+ * uses absolute addresses; this is not position independent.
+ */
+__mmap_switched:
+ adr x3, __switch_data + 8
+
+ ldp x4, x5, [x3], #16
+ ldp x6, x7, [x3], #16
+ cmp x4, x5 // Copy data segment if needed
+1: ccmp x5, x6, #4, ne
+ b.eq 2f
+ ldr x16, [x4], #8
+ str x16, [x5], #8
+ b 1b
+2:
+1: cmp x6, x7
+ b.hs 2f
+ str xzr, [x6], #8 // Clear BSS
+ b 1b
+2:
+ ldp x4, x5, [x3], #16
+ ldr x6, [x3], #8
+ ldr x16, [x3]
+ mov sp, x16
+ str x22, [x4] // Save processor ID
+ str x21, [x5] // Save FDT pointer
+ str x24, [x6] // Save PHYS_OFFSET
+ mov x29, #0
+ b start_kernel
+ENDPROC(__mmap_switched)
+
+/*
+ * Exception handling. Something went wrong and we can't proceed. We ought to
+ * tell the user, but since we don't have any guarantee that we're even
+ * running on the right architecture, we do virtually nothing.
+ */
+__error_p:
+ENDPROC(__error_p)
+
+__error:
+1: nop
+ b 1b
+ENDPROC(__error)
+
+/*
+ * This function gets the processor ID in w0 and searches the cpu_table[] for
+ * a match. It returns a pointer to the struct cpu_info it found. The
+ * cpu_table[] must end with an empty (all zeros) structure.
+ *
+ * This routine can be called via C code and it needs to work with the MMU
+ * both disabled and enabled (the offset is calculated automatically).
+ */
+ENTRY(lookup_processor_type)
+ adr x1, __lookup_processor_type_data
+ ldp x2, x3, [x1]
+ sub x1, x1, x2 // get offset between VA and PA
+ add x3, x3, x1 // convert VA to PA
+1:
+ ldp w5, w6, [x3] // load cpu_id_val and cpu_id_mask
+ cbz w5, 2f // end of list?
+ and w6, w6, w0
+ cmp w5, w6
+ b.eq 3f
+ add x3, x3, #CPU_INFO_SZ
+ b 1b
+2:
+ mov x3, #0 // unknown processor
+3:
+ mov x0, x3
+ ret
+ENDPROC(lookup_processor_type)
+
+ .align 3
+ .type __lookup_processor_type_data, %object
+__lookup_processor_type_data:
+ .quad .
+ .quad cpu_table
+ .size __lookup_processor_type_data, . - __lookup_processor_type_data
+
+/*
+ * Determine validity of the x21 FDT pointer.
+ * The dtb must be 8-byte aligned and live in the first 512M of memory.
+ */
+__vet_fdt:
+ tst x21, #0x7
+ b.ne 1f
+ cmp x21, x24
+ b.lt 1f
+ mov x0, #(1 << 29)
+ add x0, x0, x24
+ cmp x21, x0
+ b.ge 1f
+ ret
+1:
+ mov x21, #0
+ ret
+ENDPROC(__vet_fdt)
diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c
new file mode 100644
index 0000000..5ab825c
--- /dev/null
+++ b/arch/arm64/kernel/hw_breakpoint.c
@@ -0,0 +1,880 @@
+/*
+ * HW_breakpoint: a unified kernel/user-space hardware breakpoint facility,
+ * using the CPU's debug registers.
+ *
+ * Copyright (C) 2012 ARM Limited
+ * Author: Will Deacon <will.deacon@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#define pr_fmt(fmt) "hw-breakpoint: " fmt
+
+#include <linux/errno.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/perf_event.h>
+#include <linux/ptrace.h>
+#include <linux/smp.h>
+
+#include <asm/compat.h>
+#include <asm/current.h>
+#include <asm/debug-monitors.h>
+#include <asm/hw_breakpoint.h>
+#include <asm/kdebug.h>
+#include <asm/traps.h>
+#include <asm/cputype.h>
+#include <asm/system_misc.h>
+
+/* Breakpoint currently in use for each BRP. */
+static DEFINE_PER_CPU(struct perf_event *, bp_on_reg[ARM_MAX_BRP]);
+
+/* Watchpoint currently in use for each WRP. */
+static DEFINE_PER_CPU(struct perf_event *, wp_on_reg[ARM_MAX_WRP]);
+
+/* Currently stepping a per-CPU kernel breakpoint. */
+static DEFINE_PER_CPU(int, stepping_kernel_bp);
+
+/* Number of BRP/WRP registers on this CPU. */
+static int core_num_brps;
+static int core_num_wrps;
+
+/* Determine number of BRP registers available. */
+static int get_num_brps(void)
+{
+ return ((read_cpuid(ID_AA64DFR0_EL1) >> 12) & 0xf) + 1;
+}
+
+/* Determine number of WRP registers available. */
+static int get_num_wrps(void)
+{
+ return ((read_cpuid(ID_AA64DFR0_EL1) >> 20) & 0xf) + 1;
+}
+
+int hw_breakpoint_slots(int type)
+{
+ /*
+ * We can be called early, so don't rely on
+ * our static variables being initialised.
+ */
+ switch (type) {
+ case TYPE_INST:
+ return get_num_brps();
+ case TYPE_DATA:
+ return get_num_wrps();
+ default:
+ pr_warning("unknown slot type: %d\n", type);
+ return 0;
+ }
+}
+
+#define READ_WB_REG_CASE(OFF, N, REG, VAL) \
+ case (OFF + N): \
+ AARCH64_DBG_READ(N, REG, VAL); \
+ break
+
+#define WRITE_WB_REG_CASE(OFF, N, REG, VAL) \
+ case (OFF + N): \
+ AARCH64_DBG_WRITE(N, REG, VAL); \
+ break
+
+#define GEN_READ_WB_REG_CASES(OFF, REG, VAL) \
+ READ_WB_REG_CASE(OFF, 0, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 1, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 2, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 3, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 4, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 5, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 6, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 7, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 8, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 9, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 10, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 11, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 12, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 13, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 14, REG, VAL); \
+ READ_WB_REG_CASE(OFF, 15, REG, VAL)
+
+#define GEN_WRITE_WB_REG_CASES(OFF, REG, VAL) \
+ WRITE_WB_REG_CASE(OFF, 0, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 1, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 2, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 3, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 4, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 5, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 6, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 7, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 8, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 9, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 10, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 11, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 12, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 13, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 14, REG, VAL); \
+ WRITE_WB_REG_CASE(OFF, 15, REG, VAL)
+
+static u64 read_wb_reg(int reg, int n)
+{
+ u64 val = 0;
+
+ switch (reg + n) {
+ GEN_READ_WB_REG_CASES(AARCH64_DBG_REG_BVR, AARCH64_DBG_REG_NAME_BVR, val);
+ GEN_READ_WB_REG_CASES(AARCH64_DBG_REG_BCR, AARCH64_DBG_REG_NAME_BCR, val);
+ GEN_READ_WB_REG_CASES(AARCH64_DBG_REG_WVR, AARCH64_DBG_REG_NAME_WVR, val);
+ GEN_READ_WB_REG_CASES(AARCH64_DBG_REG_WCR, AARCH64_DBG_REG_NAME_WCR, val);
+ default:
+ pr_warning("attempt to read from unknown breakpoint register %d\n", n);
+ }
+
+ return val;
+}
+
+static void write_wb_reg(int reg, int n, u64 val)
+{
+ switch (reg + n) {
+ GEN_WRITE_WB_REG_CASES(AARCH64_DBG_REG_BVR, AARCH64_DBG_REG_NAME_BVR, val);
+ GEN_WRITE_WB_REG_CASES(AARCH64_DBG_REG_BCR, AARCH64_DBG_REG_NAME_BCR, val);
+ GEN_WRITE_WB_REG_CASES(AARCH64_DBG_REG_WVR, AARCH64_DBG_REG_NAME_WVR, val);
+ GEN_WRITE_WB_REG_CASES(AARCH64_DBG_REG_WCR, AARCH64_DBG_REG_NAME_WCR, val);
+ default:
+ pr_warning("attempt to write to unknown breakpoint register %d\n", n);
+ }
+ isb();
+}
+
+/*
+ * Convert a breakpoint privilege level to the corresponding exception
+ * level.
+ */
+static enum debug_el debug_exception_level(int privilege)
+{
+ switch (privilege) {
+ case AARCH64_BREAKPOINT_EL0:
+ return DBG_ACTIVE_EL0;
+ case AARCH64_BREAKPOINT_EL1:
+ return DBG_ACTIVE_EL1;
+ default:
+ pr_warning("invalid breakpoint privilege level %d\n", privilege);
+ return -EINVAL;
+ }
+}
+
+/*
+ * Install a perf counter breakpoint.
+ */
+int arch_install_hw_breakpoint(struct perf_event *bp)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+ struct perf_event **slot, **slots;
+ struct debug_info *debug_info = ¤t->thread.debug;
+ int i, max_slots, ctrl_reg, val_reg, reg_enable;
+ u32 ctrl;
+
+ if (info->ctrl.type == ARM_BREAKPOINT_EXECUTE) {
+ /* Breakpoint */
+ ctrl_reg = AARCH64_DBG_REG_BCR;
+ val_reg = AARCH64_DBG_REG_BVR;
+ slots = __get_cpu_var(bp_on_reg);
+ max_slots = core_num_brps;
+ reg_enable = !debug_info->bps_disabled;
+ } else {
+ /* Watchpoint */
+ ctrl_reg = AARCH64_DBG_REG_WCR;
+ val_reg = AARCH64_DBG_REG_WVR;
+ slots = __get_cpu_var(wp_on_reg);
+ max_slots = core_num_wrps;
+ reg_enable = !debug_info->wps_disabled;
+ }
+
+ for (i = 0; i < max_slots; ++i) {
+ slot = &slots[i];
+
+ if (!*slot) {
+ *slot = bp;
+ break;
+ }
+ }
+
+ if (WARN_ONCE(i == max_slots, "Can't find any breakpoint slot"))
+ return -ENOSPC;
+
+ /* Ensure debug monitors are enabled at the correct exception level. */
+ enable_debug_monitors(debug_exception_level(info->ctrl.privilege));
+
+ /* Setup the address register. */
+ write_wb_reg(val_reg, i, info->address);
+
+ /* Setup the control register. */
+ ctrl = encode_ctrl_reg(info->ctrl);
+ write_wb_reg(ctrl_reg, i, reg_enable ? ctrl | 0x1 : ctrl & ~0x1);
+
+ return 0;
+}
+
+void arch_uninstall_hw_breakpoint(struct perf_event *bp)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+ struct perf_event **slot, **slots;
+ int i, max_slots, base;
+
+ if (info->ctrl.type == ARM_BREAKPOINT_EXECUTE) {
+ /* Breakpoint */
+ base = AARCH64_DBG_REG_BCR;
+ slots = __get_cpu_var(bp_on_reg);
+ max_slots = core_num_brps;
+ } else {
+ /* Watchpoint */
+ base = AARCH64_DBG_REG_WCR;
+ slots = __get_cpu_var(wp_on_reg);
+ max_slots = core_num_wrps;
+ }
+
+ /* Remove the breakpoint. */
+ for (i = 0; i < max_slots; ++i) {
+ slot = &slots[i];
+
+ if (*slot == bp) {
+ *slot = NULL;
+ break;
+ }
+ }
+
+ if (WARN_ONCE(i == max_slots, "Can't find any breakpoint slot"))
+ return;
+
+ /* Reset the control register. */
+ write_wb_reg(base, i, 0);
+
+ /* Release the debug monitors for the correct exception level. */
+ disable_debug_monitors(debug_exception_level(info->ctrl.privilege));
+}
+
+static int get_hbp_len(u8 hbp_len)
+{
+ unsigned int len_in_bytes = 0;
+
+ switch (hbp_len) {
+ case ARM_BREAKPOINT_LEN_1:
+ len_in_bytes = 1;
+ break;
+ case ARM_BREAKPOINT_LEN_2:
+ len_in_bytes = 2;
+ break;
+ case ARM_BREAKPOINT_LEN_4:
+ len_in_bytes = 4;
+ break;
+ case ARM_BREAKPOINT_LEN_8:
+ len_in_bytes = 8;
+ break;
+ }
+
+ return len_in_bytes;
+}
+
+/*
+ * Check whether bp virtual address is in kernel space.
+ */
+int arch_check_bp_in_kernelspace(struct perf_event *bp)
+{
+ unsigned int len;
+ unsigned long va;
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+
+ va = info->address;
+ len = get_hbp_len(info->ctrl.len);
+
+ return (va >= TASK_SIZE) && ((va + len - 1) >= TASK_SIZE);
+}
+
+/*
+ * Extract generic type and length encodings from an arch_hw_breakpoint_ctrl.
+ * Hopefully this will disappear when ptrace can bypass the conversion
+ * to generic breakpoint descriptions.
+ */
+int arch_bp_generic_fields(struct arch_hw_breakpoint_ctrl ctrl,
+ int *gen_len, int *gen_type)
+{
+ /* Type */
+ switch (ctrl.type) {
+ case ARM_BREAKPOINT_EXECUTE:
+ *gen_type = HW_BREAKPOINT_X;
+ break;
+ case ARM_BREAKPOINT_LOAD:
+ *gen_type = HW_BREAKPOINT_R;
+ break;
+ case ARM_BREAKPOINT_STORE:
+ *gen_type = HW_BREAKPOINT_W;
+ break;
+ case ARM_BREAKPOINT_LOAD | ARM_BREAKPOINT_STORE:
+ *gen_type = HW_BREAKPOINT_RW;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /* Len */
+ switch (ctrl.len) {
+ case ARM_BREAKPOINT_LEN_1:
+ *gen_len = HW_BREAKPOINT_LEN_1;
+ break;
+ case ARM_BREAKPOINT_LEN_2:
+ *gen_len = HW_BREAKPOINT_LEN_2;
+ break;
+ case ARM_BREAKPOINT_LEN_4:
+ *gen_len = HW_BREAKPOINT_LEN_4;
+ break;
+ case ARM_BREAKPOINT_LEN_8:
+ *gen_len = HW_BREAKPOINT_LEN_8;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/*
+ * Construct an arch_hw_breakpoint from a perf_event.
+ */
+static int arch_build_bp_info(struct perf_event *bp)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+
+ /* Type */
+ switch (bp->attr.bp_type) {
+ case HW_BREAKPOINT_X:
+ info->ctrl.type = ARM_BREAKPOINT_EXECUTE;
+ break;
+ case HW_BREAKPOINT_R:
+ info->ctrl.type = ARM_BREAKPOINT_LOAD;
+ break;
+ case HW_BREAKPOINT_W:
+ info->ctrl.type = ARM_BREAKPOINT_STORE;
+ break;
+ case HW_BREAKPOINT_RW:
+ info->ctrl.type = ARM_BREAKPOINT_LOAD | ARM_BREAKPOINT_STORE;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /* Len */
+ switch (bp->attr.bp_len) {
+ case HW_BREAKPOINT_LEN_1:
+ info->ctrl.len = ARM_BREAKPOINT_LEN_1;
+ break;
+ case HW_BREAKPOINT_LEN_2:
+ info->ctrl.len = ARM_BREAKPOINT_LEN_2;
+ break;
+ case HW_BREAKPOINT_LEN_4:
+ info->ctrl.len = ARM_BREAKPOINT_LEN_4;
+ break;
+ case HW_BREAKPOINT_LEN_8:
+ info->ctrl.len = ARM_BREAKPOINT_LEN_8;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ /*
+ * On AArch64, we only permit breakpoints of length 4, whereas
+ * AArch32 also requires breakpoints of length 2 for Thumb.
+ * Watchpoints can be of length 1, 2, 4 or 8 bytes.
+ */
+ if (info->ctrl.type == ARM_BREAKPOINT_EXECUTE) {
+ if (is_compat_task()) {
+ if (info->ctrl.len != ARM_BREAKPOINT_LEN_2 &&
+ info->ctrl.len != ARM_BREAKPOINT_LEN_4)
+ return -EINVAL;
+ } else if (info->ctrl.len != ARM_BREAKPOINT_LEN_4) {
+ /*
+ * FIXME: Some tools (I'm looking at you perf) assume
+ * that breakpoints should be sizeof(long). This
+ * is nonsense. For now, we fix up the parameter
+ * but we should probably return -EINVAL instead.
+ */
+ info->ctrl.len = ARM_BREAKPOINT_LEN_4;
+ }
+ }
+
+ /* Address */
+ info->address = bp->attr.bp_addr;
+
+ /*
+ * Privilege
+ * Note that we disallow combined EL0/EL1 breakpoints because
+ * that would complicate the stepping code.
+ */
+ if (arch_check_bp_in_kernelspace(bp))
+ info->ctrl.privilege = AARCH64_BREAKPOINT_EL1;
+ else
+ info->ctrl.privilege = AARCH64_BREAKPOINT_EL0;
+
+ /* Enabled? */
+ info->ctrl.enabled = !bp->attr.disabled;
+
+ return 0;
+}
+
+/*
+ * Validate the arch-specific HW Breakpoint register settings.
+ */
+int arch_validate_hwbkpt_settings(struct perf_event *bp)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+ int ret;
+ u64 alignment_mask, offset;
+
+ /* Build the arch_hw_breakpoint. */
+ ret = arch_build_bp_info(bp);
+ if (ret)
+ return ret;
+
+ /*
+ * Check address alignment.
+ * We don't do any clever alignment correction for watchpoints
+ * because using 64-bit unaligned addresses is deprecated for
+ * AArch64.
+ *
+ * AArch32 tasks expect some simple alignment fixups, so emulate
+ * that here.
+ */
+ if (is_compat_task()) {
+ if (info->ctrl.len == ARM_BREAKPOINT_LEN_8)
+ alignment_mask = 0x7;
+ else
+ alignment_mask = 0x3;
+ offset = info->address & alignment_mask;
+ switch (offset) {
+ case 0:
+ /* Aligned */
+ break;
+ case 1:
+ /* Allow single byte watchpoint. */
+ if (info->ctrl.len == ARM_BREAKPOINT_LEN_1)
+ break;
+ case 2:
+ /* Allow halfword watchpoints and breakpoints. */
+ if (info->ctrl.len == ARM_BREAKPOINT_LEN_2)
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ info->address &= ~alignment_mask;
+ info->ctrl.len <<= offset;
+ } else {
+ if (info->ctrl.type == ARM_BREAKPOINT_EXECUTE)
+ alignment_mask = 0x3;
+ else
+ alignment_mask = 0x7;
+ if (info->address & alignment_mask)
+ return -EINVAL;
+ }
+
+ /*
+ * Disallow per-task kernel breakpoints since these would
+ * complicate the stepping code.
+ */
+ if (info->ctrl.privilege == AARCH64_BREAKPOINT_EL1 && bp->hw.bp_target)
+ return -EINVAL;
+
+ return 0;
+}
+
+/*
+ * Enable/disable all of the breakpoints active at the specified
+ * exception level at the register level.
+ * This is used when single-stepping after a breakpoint exception.
+ */
+static void toggle_bp_registers(int reg, enum debug_el el, int enable)
+{
+ int i, max_slots, privilege;
+ u32 ctrl;
+ struct perf_event **slots;
+
+ switch (reg) {
+ case AARCH64_DBG_REG_BCR:
+ slots = __get_cpu_var(bp_on_reg);
+ max_slots = core_num_brps;
+ break;
+ case AARCH64_DBG_REG_WCR:
+ slots = __get_cpu_var(wp_on_reg);
+ max_slots = core_num_wrps;
+ break;
+ default:
+ return;
+ }
+
+ for (i = 0; i < max_slots; ++i) {
+ if (!slots[i])
+ continue;
+
+ privilege = counter_arch_bp(slots[i])->ctrl.privilege;
+ if (debug_exception_level(privilege) != el)
+ continue;
+
+ ctrl = read_wb_reg(reg, i);
+ if (enable)
+ ctrl |= 0x1;
+ else
+ ctrl &= ~0x1;
+ write_wb_reg(reg, i, ctrl);
+ }
+}
+
+/*
+ * Debug exception handlers.
+ */
+static int breakpoint_handler(unsigned long unused, unsigned int esr,
+ struct pt_regs *regs)
+{
+ int i, step = 0, *kernel_step;
+ u32 ctrl_reg;
+ u64 addr, val;
+ struct perf_event *bp, **slots;
+ struct debug_info *debug_info;
+ struct arch_hw_breakpoint_ctrl ctrl;
+
+ slots = (struct perf_event **)__get_cpu_var(bp_on_reg);
+ addr = instruction_pointer(regs);
+ debug_info = ¤t->thread.debug;
+
+ for (i = 0; i < core_num_brps; ++i) {
+ rcu_read_lock();
+
+ bp = slots[i];
+
+ if (bp == NULL)
+ goto unlock;
+
+ /* Check if the breakpoint value matches. */
+ val = read_wb_reg(AARCH64_DBG_REG_BVR, i);
+ if (val != (addr & ~0x3))
+ goto unlock;
+
+ /* Possible match, check the byte address select to confirm. */
+ ctrl_reg = read_wb_reg(AARCH64_DBG_REG_BCR, i);
+ decode_ctrl_reg(ctrl_reg, &ctrl);
+ if (!((1 << (addr & 0x3)) & ctrl.len))
+ goto unlock;
+
+ counter_arch_bp(bp)->trigger = addr;
+ perf_bp_event(bp, regs);
+
+ /* Do we need to handle the stepping? */
+ if (!bp->overflow_handler)
+ step = 1;
+unlock:
+ rcu_read_unlock();
+ }
+
+ if (!step)
+ return 0;
+
+ if (user_mode(regs)) {
+ debug_info->bps_disabled = 1;
+ toggle_bp_registers(AARCH64_DBG_REG_BCR, DBG_ACTIVE_EL0, 0);
+
+ /* If we're already stepping a watchpoint, just return. */
+ if (debug_info->wps_disabled)
+ return 0;
+
+ if (test_thread_flag(TIF_SINGLESTEP))
+ debug_info->suspended_step = 1;
+ else
+ user_enable_single_step(current);
+ } else {
+ toggle_bp_registers(AARCH64_DBG_REG_BCR, DBG_ACTIVE_EL1, 0);
+ kernel_step = &__get_cpu_var(stepping_kernel_bp);
+
+ if (*kernel_step != ARM_KERNEL_STEP_NONE)
+ return 0;
+
+ if (kernel_active_single_step()) {
+ *kernel_step = ARM_KERNEL_STEP_SUSPEND;
+ } else {
+ *kernel_step = ARM_KERNEL_STEP_ACTIVE;
+ kernel_enable_single_step(regs);
+ }
+ }
+
+ return 0;
+}
+
+static int watchpoint_handler(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ int i, step = 0, *kernel_step, access;
+ u32 ctrl_reg;
+ u64 val, alignment_mask;
+ struct perf_event *wp, **slots;
+ struct debug_info *debug_info;
+ struct arch_hw_breakpoint *info;
+ struct arch_hw_breakpoint_ctrl ctrl;
+
+ slots = (struct perf_event **)__get_cpu_var(wp_on_reg);
+ debug_info = ¤t->thread.debug;
+
+ for (i = 0; i < core_num_wrps; ++i) {
+ rcu_read_lock();
+
+ wp = slots[i];
+
+ if (wp == NULL)
+ goto unlock;
+
+ info = counter_arch_bp(wp);
+ /* AArch32 watchpoints are either 4 or 8 bytes aligned. */
+ if (is_compat_task()) {
+ if (info->ctrl.len == ARM_BREAKPOINT_LEN_8)
+ alignment_mask = 0x7;
+ else
+ alignment_mask = 0x3;
+ } else {
+ alignment_mask = 0x7;
+ }
+
+ /* Check if the watchpoint value matches. */
+ val = read_wb_reg(AARCH64_DBG_REG_WVR, i);
+ if (val != (addr & ~alignment_mask))
+ goto unlock;
+
+ /* Possible match, check the byte address select to confirm. */
+ ctrl_reg = read_wb_reg(AARCH64_DBG_REG_WCR, i);
+ decode_ctrl_reg(ctrl_reg, &ctrl);
+ if (!((1 << (addr & alignment_mask)) & ctrl.len))
+ goto unlock;
+
+ /*
+ * Check that the access type matches.
+ * 0 => load, otherwise => store
+ */
+ access = (esr & AARCH64_ESR_ACCESS_MASK) ? HW_BREAKPOINT_W :
+ HW_BREAKPOINT_R;
+ if (!(access & hw_breakpoint_type(wp)))
+ goto unlock;
+
+ info->trigger = addr;
+ perf_bp_event(wp, regs);
+
+ /* Do we need to handle the stepping? */
+ if (!wp->overflow_handler)
+ step = 1;
+
+unlock:
+ rcu_read_unlock();
+ }
+
+ if (!step)
+ return 0;
+
+ /*
+ * We always disable EL0 watchpoints because the kernel can
+ * cause these to fire via an unprivileged access.
+ */
+ toggle_bp_registers(AARCH64_DBG_REG_WCR, DBG_ACTIVE_EL0, 0);
+
+ if (user_mode(regs)) {
+ debug_info->wps_disabled = 1;
+
+ /* If we're already stepping a breakpoint, just return. */
+ if (debug_info->bps_disabled)
+ return 0;
+
+ if (test_thread_flag(TIF_SINGLESTEP))
+ debug_info->suspended_step = 1;
+ else
+ user_enable_single_step(current);
+ } else {
+ toggle_bp_registers(AARCH64_DBG_REG_WCR, DBG_ACTIVE_EL1, 0);
+ kernel_step = &__get_cpu_var(stepping_kernel_bp);
+
+ if (*kernel_step != ARM_KERNEL_STEP_NONE)
+ return 0;
+
+ if (kernel_active_single_step()) {
+ *kernel_step = ARM_KERNEL_STEP_SUSPEND;
+ } else {
+ *kernel_step = ARM_KERNEL_STEP_ACTIVE;
+ kernel_enable_single_step(regs);
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * Handle single-step exception.
+ */
+int reinstall_suspended_bps(struct pt_regs *regs)
+{
+ struct debug_info *debug_info = ¤t->thread.debug;
+ int handled_exception = 0, *kernel_step;
+
+ kernel_step = &__get_cpu_var(stepping_kernel_bp);
+
+ /*
+ * Called from single-step exception handler.
+ * Return 0 if execution can resume, 1 if a SIGTRAP should be
+ * reported.
+ */
+ if (user_mode(regs)) {
+ if (debug_info->bps_disabled) {
+ debug_info->bps_disabled = 0;
+ toggle_bp_registers(AARCH64_DBG_REG_BCR, DBG_ACTIVE_EL0, 1);
+ handled_exception = 1;
+ }
+
+ if (debug_info->wps_disabled) {
+ debug_info->wps_disabled = 0;
+ toggle_bp_registers(AARCH64_DBG_REG_WCR, DBG_ACTIVE_EL0, 1);
+ handled_exception = 1;
+ }
+
+ if (handled_exception) {
+ if (debug_info->suspended_step) {
+ debug_info->suspended_step = 0;
+ /* Allow exception handling to fall-through. */
+ handled_exception = 0;
+ } else {
+ user_disable_single_step(current);
+ }
+ }
+ } else if (*kernel_step != ARM_KERNEL_STEP_NONE) {
+ toggle_bp_registers(AARCH64_DBG_REG_BCR, DBG_ACTIVE_EL1, 1);
+ toggle_bp_registers(AARCH64_DBG_REG_WCR, DBG_ACTIVE_EL1, 1);
+
+ if (!debug_info->wps_disabled)
+ toggle_bp_registers(AARCH64_DBG_REG_WCR, DBG_ACTIVE_EL0, 1);
+
+ if (*kernel_step != ARM_KERNEL_STEP_SUSPEND) {
+ kernel_disable_single_step();
+ handled_exception = 1;
+ } else {
+ handled_exception = 0;
+ }
+
+ *kernel_step = ARM_KERNEL_STEP_NONE;
+ }
+
+ return !handled_exception;
+}
+
+/*
+ * Context-switcher for restoring suspended breakpoints.
+ */
+void hw_breakpoint_thread_switch(struct task_struct *next)
+{
+ /*
+ * current next
+ * disabled: 0 0 => The usual case, NOTIFY_DONE
+ * 0 1 => Disable the registers
+ * 1 0 => Enable the registers
+ * 1 1 => NOTIFY_DONE. per-task bps will
+ * get taken care of by perf.
+ */
+
+ struct debug_info *current_debug_info, *next_debug_info;
+
+ current_debug_info = ¤t->thread.debug;
+ next_debug_info = &next->thread.debug;
+
+ /* Update breakpoints. */
+ if (current_debug_info->bps_disabled != next_debug_info->bps_disabled)
+ toggle_bp_registers(AARCH64_DBG_REG_BCR,
+ DBG_ACTIVE_EL0,
+ !next_debug_info->bps_disabled);
+
+ /* Update watchpoints. */
+ if (current_debug_info->wps_disabled != next_debug_info->wps_disabled)
+ toggle_bp_registers(AARCH64_DBG_REG_WCR,
+ DBG_ACTIVE_EL0,
+ !next_debug_info->wps_disabled);
+}
+
+/*
+ * CPU initialisation.
+ */
+static void reset_ctrl_regs(void *unused)
+{
+ int i;
+
+ for (i = 0; i < core_num_brps; ++i) {
+ write_wb_reg(AARCH64_DBG_REG_BCR, i, 0UL);
+ write_wb_reg(AARCH64_DBG_REG_BVR, i, 0UL);
+ }
+
+ for (i = 0; i < core_num_wrps; ++i) {
+ write_wb_reg(AARCH64_DBG_REG_WCR, i, 0UL);
+ write_wb_reg(AARCH64_DBG_REG_WVR, i, 0UL);
+ }
+}
+
+static int __cpuinit hw_breakpoint_reset_notify(struct notifier_block *self,
+ unsigned long action,
+ void *hcpu)
+{
+ int cpu = (long)hcpu;
+ if (action == CPU_ONLINE)
+ smp_call_function_single(cpu, reset_ctrl_regs, NULL, 1);
+ return NOTIFY_OK;
+}
+
+static struct notifier_block __cpuinitdata hw_breakpoint_reset_nb = {
+ .notifier_call = hw_breakpoint_reset_notify,
+};
+
+/*
+ * One-time initialisation.
+ */
+static int __init arch_hw_breakpoint_init(void)
+{
+ core_num_brps = get_num_brps();
+ core_num_wrps = get_num_wrps();
+
+ pr_info("found %d breakpoint and %d watchpoint registers.\n",
+ core_num_brps, core_num_wrps);
+
+ /*
+ * Reset the breakpoint resources. We assume that a halting
+ * debugger will leave the world in a nice state for us.
+ */
+ smp_call_function(reset_ctrl_regs, NULL, 1);
+ reset_ctrl_regs(NULL);
+
+ /* Register debug fault handlers. */
+ hook_debug_fault_code(DBG_ESR_EVT_HWBP, breakpoint_handler, SIGTRAP,
+ TRAP_HWBKPT, "hw-breakpoint handler");
+ hook_debug_fault_code(DBG_ESR_EVT_HWWP, watchpoint_handler, SIGTRAP,
+ TRAP_HWBKPT, "hw-watchpoint handler");
+
+ /* Register hotplug notifier. */
+ register_cpu_notifier(&hw_breakpoint_reset_nb);
+
+ return 0;
+}
+arch_initcall(arch_hw_breakpoint_init);
+
+void hw_breakpoint_pmu_read(struct perf_event *bp)
+{
+}
+
+/*
+ * Dummy function to register with die_notifier.
+ */
+int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
+ unsigned long val, void *data)
+{
+ return NOTIFY_DONE;
+}
diff --git a/arch/arm64/kernel/io.c b/arch/arm64/kernel/io.c
new file mode 100644
index 0000000..7d37ead
--- /dev/null
+++ b/arch/arm64/kernel/io.c
@@ -0,0 +1,64 @@
+/*
+ * Based on arch/arm/kernel/io.c
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/types.h>
+#include <linux/io.h>
+
+/*
+ * Copy data from IO memory space to "real" memory space.
+ */
+void __memcpy_fromio(void *to, const volatile void __iomem *from, size_t count)
+{
+ unsigned char *t = to;
+ while (count) {
+ count--;
+ *t = readb(from);
+ t++;
+ from++;
+ }
+}
+EXPORT_SYMBOL(__memcpy_fromio);
+
+/*
+ * Copy data from "real" memory space to IO memory space.
+ */
+void __memcpy_toio(volatile void __iomem *to, const void *from, size_t count)
+{
+ const unsigned char *f = from;
+ while (count) {
+ count--;
+ writeb(*f, to);
+ f++;
+ to++;
+ }
+}
+EXPORT_SYMBOL(__memcpy_toio);
+
+/*
+ * "memset" on IO memory space.
+ */
+void __memset_io(volatile void __iomem *dst, int c, size_t count)
+{
+ while (count) {
+ count--;
+ writeb(c, dst);
+ dst++;
+ }
+}
+EXPORT_SYMBOL(__memset_io);
diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c
new file mode 100644
index 0000000..0373c66
--- /dev/null
+++ b/arch/arm64/kernel/irq.c
@@ -0,0 +1,84 @@
+/*
+ * Based on arch/arm/kernel/irq.c
+ *
+ * Copyright (C) 1992 Linus Torvalds
+ * Modifications for ARM processor Copyright (C) 1995-2000 Russell King.
+ * Support for Dynamic Tick Timer Copyright (C) 2004-2005 Nokia Corporation.
+ * Dynamic Tick Timer written by Tony Lindgren <tony@atomide.com> and
+ * Tuukka Tikkanen <tuukka.tikkanen@elektrobit.com>.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kernel_stat.h>
+#include <linux/irq.h>
+#include <linux/smp.h>
+#include <linux/init.h>
+#include <linux/of_irq.h>
+#include <linux/seq_file.h>
+#include <linux/ratelimit.h>
+
+unsigned long irq_err_count;
+
+int arch_show_interrupts(struct seq_file *p, int prec)
+{
+#ifdef CONFIG_SMP
+ show_ipi_list(p, prec);
+#endif
+ seq_printf(p, "%*s: %10lu\n", prec, "Err", irq_err_count);
+ return 0;
+}
+
+/*
+ * handle_IRQ handles all hardware IRQ's. Decoded IRQs should
+ * not come via this function. Instead, they should provide their
+ * own 'handler'. Used by platform code implementing C-based 1st
+ * level decoding.
+ */
+void handle_IRQ(unsigned int irq, struct pt_regs *regs)
+{
+ struct pt_regs *old_regs = set_irq_regs(regs);
+
+ irq_enter();
+
+ /*
+ * Some hardware gives randomly wrong interrupts. Rather
+ * than crashing, do something sensible.
+ */
+ if (unlikely(irq >= nr_irqs)) {
+ pr_warn_ratelimited("Bad IRQ%u\n", irq);
+ ack_bad_irq(irq);
+ } else {
+ generic_handle_irq(irq);
+ }
+
+ irq_exit();
+ set_irq_regs(old_regs);
+}
+
+/*
+ * Interrupt controllers supported by the kernel.
+ */
+static const struct of_device_id intctrl_of_match[] __initconst = {
+ /* IRQ controllers { .compatible, .data } info to go here */
+ {}
+};
+
+void __init init_IRQ(void)
+{
+ of_irq_init(intctrl_of_match);
+
+ if (!handle_arch_irq)
+ panic("No interrupt controller found.");
+}
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
new file mode 100644
index 0000000..8b69ecb
--- /dev/null
+++ b/arch/arm64/kernel/kuser32.S
@@ -0,0 +1,77 @@
+/*
+ * Low-level user helpers placed in the vectors page for AArch32.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ *
+ * AArch32 user helpers.
+ *
+ * Each segment is 32-byte aligned and will be moved to the top of the high
+ * vector page. New segments (if ever needed) must be added in front of
+ * existing ones. This mechanism should be used only for things that are
+ * really small and justified, and not be abused freely.
+ *
+ * See Documentation/arm/kernel_user_helpers.txt for formal definitions.
+ */
+ .align 5
+ .globl __kuser_helper_start
+__kuser_helper_start:
+
+__kuser_cmpxchg64: // 0xffff0f60
+ .inst 0xe92d00f0 // push {r4, r5, r6, r7}
+ .inst 0xe1c040d0 // ldrd r4, r5, [r0]
+ .inst 0xe1c160d0 // ldrd r6, r7, [r1]
+ .inst 0xf57ff05f // dmb sy
+ .inst 0xe1b20f9f // 1: ldrexd r0, r1, [r2]
+ .inst 0xe0303004 // eors r3, r0, r4
+ .inst 0x00313005 // eoreqs r3, r1, r5
+ .inst 0x01a23f96 // strexdeq r3, r6, [r2]
+ .inst 0x03330001 // teqeq r3, #1
+ .inst 0x0afffff9 // beq 1b
+ .inst 0xf57ff05f // dmb sy
+ .inst 0xe2730000 // rsbs r0, r3, #0
+ .inst 0xe8bd00f0 // pop {r4, r5, r6, r7}
+ .inst 0xe12fff1e // bx lr
+
+ .align 5
+__kuser_memory_barrier: // 0xffff0fa0
+ .inst 0xf57ff05f // dmb sy
+ .inst 0xe12fff1e // bx lr
+
+ .align 5
+__kuser_cmpxchg: // 0xffff0fc0
+ .inst 0xf57ff05f // dmb sy
+ .inst 0xe1923f9f // 1: ldrex r3, [r2]
+ .inst 0xe0533000 // subs r3, r3, r0
+ .inst 0x01823f91 // strexeq r3, r1, [r2]
+ .inst 0x03330001 // teqeq r3, #1
+ .inst 0x0afffffa // beq 1b
+ .inst 0xe2730000 // rsbs r0, r3, #0
+ .inst 0xeaffffef // b <__kuser_memory_barrier>
+
+ .align 5
+__kuser_get_tls: // 0xffff0fe0
+ .inst 0xee1d0f70 // mrc p15, 0, r0, c13, c0, 3
+ .inst 0xe12fff1e // bx lr
+ .rep 5
+ .word 0
+ .endr
+
+__kuser_helper_version: // 0xffff0ffc
+ .word ((__kuser_helper_end - __kuser_helper_start) >> 5)
+ .globl __kuser_helper_end
+__kuser_helper_end:
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
new file mode 100644
index 0000000..ca0e3d5
--- /dev/null
+++ b/arch/arm64/kernel/module.c
@@ -0,0 +1,456 @@
+/*
+ * AArch64 loadable module support.
+ *
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/bitops.h>
+#include <linux/elf.h>
+#include <linux/gfp.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/moduleloader.h>
+#include <linux/vmalloc.h>
+
+void *module_alloc(unsigned long size)
+{
+ return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END,
+ GFP_KERNEL, PAGE_KERNEL_EXEC, -1,
+ __builtin_return_address(0));
+}
+
+enum aarch64_reloc_op {
+ RELOC_OP_NONE,
+ RELOC_OP_ABS,
+ RELOC_OP_PREL,
+ RELOC_OP_PAGE,
+};
+
+static u64 do_reloc(enum aarch64_reloc_op reloc_op, void *place, u64 val)
+{
+ switch (reloc_op) {
+ case RELOC_OP_ABS:
+ return val;
+ case RELOC_OP_PREL:
+ return val - (u64)place;
+ case RELOC_OP_PAGE:
+ return (val & ~0xfff) - ((u64)place & ~0xfff);
+ case RELOC_OP_NONE:
+ return 0;
+ }
+
+ pr_err("do_reloc: unknown relocation operation %d\n", reloc_op);
+ return 0;
+}
+
+static int reloc_data(enum aarch64_reloc_op op, void *place, u64 val, int len)
+{
+ u64 imm_mask = (1 << len) - 1;
+ s64 sval = do_reloc(op, place, val);
+
+ switch (len) {
+ case 16:
+ *(s16 *)place = sval;
+ break;
+ case 32:
+ *(s32 *)place = sval;
+ break;
+ case 64:
+ *(s64 *)place = sval;
+ break;
+ default:
+ pr_err("Invalid length (%d) for data relocation\n", len);
+ return 0;
+ }
+
+ /*
+ * Extract the upper value bits (including the sign bit) and
+ * shift them to bit 0.
+ */
+ sval = (s64)(sval & ~(imm_mask >> 1)) >> (len - 1);
+
+ /*
+ * Overflow has occurred if the value is not representable in
+ * len bits (i.e the bottom len bits are not sign-extended and
+ * the top bits are not all zero).
+ */
+ if ((u64)(sval + 1) > 2)
+ return -ERANGE;
+
+ return 0;
+}
+
+enum aarch64_imm_type {
+ INSN_IMM_MOVNZ,
+ INSN_IMM_MOVK,
+ INSN_IMM_ADR,
+ INSN_IMM_26,
+ INSN_IMM_19,
+ INSN_IMM_16,
+ INSN_IMM_14,
+ INSN_IMM_12,
+ INSN_IMM_9,
+};
+
+static u32 encode_insn_immediate(enum aarch64_imm_type type, u32 insn, u64 imm)
+{
+ u32 immlo, immhi, lomask, himask, mask;
+ int shift;
+
+ switch (type) {
+ case INSN_IMM_MOVNZ:
+ /*
+ * For signed MOVW relocations, we have to manipulate the
+ * instruction encoding depending on whether or not the
+ * immediate is less than zero.
+ */
+ insn &= ~(3 << 29);
+ if ((s64)imm >= 0) {
+ /* >=0: Set the instruction to MOVZ (opcode 10b). */
+ insn |= 2 << 29;
+ } else {
+ /*
+ * <0: Set the instruction to MOVN (opcode 00b).
+ * Since we've masked the opcode already, we
+ * don't need to do anything other than
+ * inverting the new immediate field.
+ */
+ imm = ~imm;
+ }
+ case INSN_IMM_MOVK:
+ mask = BIT(16) - 1;
+ shift = 5;
+ break;
+ case INSN_IMM_ADR:
+ lomask = 0x3;
+ himask = 0x7ffff;
+ immlo = imm & lomask;
+ imm >>= 2;
+ immhi = imm & himask;
+ imm = (immlo << 24) | (immhi);
+ mask = (lomask << 24) | (himask);
+ shift = 5;
+ break;
+ case INSN_IMM_26:
+ mask = BIT(26) - 1;
+ shift = 0;
+ break;
+ case INSN_IMM_19:
+ mask = BIT(19) - 1;
+ shift = 5;
+ break;
+ case INSN_IMM_16:
+ mask = BIT(16) - 1;
+ shift = 5;
+ break;
+ case INSN_IMM_14:
+ mask = BIT(14) - 1;
+ shift = 5;
+ break;
+ case INSN_IMM_12:
+ mask = BIT(12) - 1;
+ shift = 10;
+ break;
+ case INSN_IMM_9:
+ mask = BIT(9) - 1;
+ shift = 12;
+ break;
+ default:
+ pr_err("encode_insn_immediate: unknown immediate encoding %d\n",
+ type);
+ return 0;
+ }
+
+ /* Update the immediate field. */
+ insn &= ~(mask << shift);
+ insn |= (imm & mask) << shift;
+
+ return insn;
+}
+
+static int reloc_insn_movw(enum aarch64_reloc_op op, void *place, u64 val,
+ int lsb, enum aarch64_imm_type imm_type)
+{
+ u64 imm, limit = 0;
+ s64 sval;
+ u32 insn = *(u32 *)place;
+
+ sval = do_reloc(op, place, val);
+ sval >>= lsb;
+ imm = sval & 0xffff;
+
+ /* Update the instruction with the new encoding. */
+ *(u32 *)place = encode_insn_immediate(imm_type, insn, imm);
+
+ /* Shift out the immediate field. */
+ sval >>= 16;
+
+ /*
+ * For unsigned immediates, the overflow check is straightforward.
+ * For signed immediates, the sign bit is actually the bit past the
+ * most significant bit of the field.
+ * The INSN_IMM_16 immediate type is unsigned.
+ */
+ if (imm_type != INSN_IMM_16) {
+ sval++;
+ limit++;
+ }
+
+ /* Check the upper bits depending on the sign of the immediate. */
+ if ((u64)sval > limit)
+ return -ERANGE;
+
+ return 0;
+}
+
+static int reloc_insn_imm(enum aarch64_reloc_op op, void *place, u64 val,
+ int lsb, int len, enum aarch64_imm_type imm_type)
+{
+ u64 imm, imm_mask;
+ s64 sval;
+ u32 insn = *(u32 *)place;
+
+ /* Calculate the relocation value. */
+ sval = do_reloc(op, place, val);
+ sval >>= lsb;
+
+ /* Extract the value bits and shift them to bit 0. */
+ imm_mask = (BIT(lsb + len) - 1) >> lsb;
+ imm = sval & imm_mask;
+
+ /* Update the instruction's immediate field. */
+ *(u32 *)place = encode_insn_immediate(imm_type, insn, imm);
+
+ /*
+ * Extract the upper value bits (including the sign bit) and
+ * shift them to bit 0.
+ */
+ sval = (s64)(sval & ~(imm_mask >> 1)) >> (len - 1);
+
+ /*
+ * Overflow has occurred if the upper bits are not all equal to
+ * the sign bit of the value.
+ */
+ if ((u64)(sval + 1) >= 2)
+ return -ERANGE;
+
+ return 0;
+}
+
+int apply_relocate_add(Elf64_Shdr *sechdrs,
+ const char *strtab,
+ unsigned int symindex,
+ unsigned int relsec,
+ struct module *me)
+{
+ unsigned int i;
+ int ovf;
+ bool overflow_check;
+ Elf64_Sym *sym;
+ void *loc;
+ u64 val;
+ Elf64_Rela *rel = (void *)sechdrs[relsec].sh_addr;
+
+ for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) {
+ /* loc corresponds to P in the AArch64 ELF document. */
+ loc = (void *)sechdrs[sechdrs[relsec].sh_info].sh_addr
+ + rel[i].r_offset;
+
+ /* sym is the ELF symbol we're referring to. */
+ sym = (Elf64_Sym *)sechdrs[symindex].sh_addr
+ + ELF64_R_SYM(rel[i].r_info);
+
+ /* val corresponds to (S + A) in the AArch64 ELF document. */
+ val = sym->st_value + rel[i].r_addend;
+
+ /* Check for overflow by default. */
+ overflow_check = true;
+
+ /* Perform the static relocation. */
+ switch (ELF64_R_TYPE(rel[i].r_info)) {
+ /* Null relocations. */
+ case R_ARM_NONE:
+ case R_AARCH64_NONE:
+ ovf = 0;
+ break;
+
+ /* Data relocations. */
+ case R_AARCH64_ABS64:
+ overflow_check = false;
+ ovf = reloc_data(RELOC_OP_ABS, loc, val, 64);
+ break;
+ case R_AARCH64_ABS32:
+ ovf = reloc_data(RELOC_OP_ABS, loc, val, 32);
+ break;
+ case R_AARCH64_ABS16:
+ ovf = reloc_data(RELOC_OP_ABS, loc, val, 16);
+ break;
+ case R_AARCH64_PREL64:
+ overflow_check = false;
+ ovf = reloc_data(RELOC_OP_PREL, loc, val, 64);
+ break;
+ case R_AARCH64_PREL32:
+ ovf = reloc_data(RELOC_OP_PREL, loc, val, 32);
+ break;
+ case R_AARCH64_PREL16:
+ ovf = reloc_data(RELOC_OP_PREL, loc, val, 16);
+ break;
+
+ /* MOVW instruction relocations. */
+ case R_AARCH64_MOVW_UABS_G0_NC:
+ overflow_check = false;
+ case R_AARCH64_MOVW_UABS_G0:
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 0,
+ INSN_IMM_16);
+ break;
+ case R_AARCH64_MOVW_UABS_G1_NC:
+ overflow_check = false;
+ case R_AARCH64_MOVW_UABS_G1:
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 16,
+ INSN_IMM_16);
+ break;
+ case R_AARCH64_MOVW_UABS_G2_NC:
+ overflow_check = false;
+ case R_AARCH64_MOVW_UABS_G2:
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 32,
+ INSN_IMM_16);
+ break;
+ case R_AARCH64_MOVW_UABS_G3:
+ /* We're using the top bits so we can't overflow. */
+ overflow_check = false;
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 48,
+ INSN_IMM_16);
+ break;
+ case R_AARCH64_MOVW_SABS_G0:
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 0,
+ INSN_IMM_MOVNZ);
+ break;
+ case R_AARCH64_MOVW_SABS_G1:
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 16,
+ INSN_IMM_MOVNZ);
+ break;
+ case R_AARCH64_MOVW_SABS_G2:
+ ovf = reloc_insn_movw(RELOC_OP_ABS, loc, val, 32,
+ INSN_IMM_MOVNZ);
+ break;
+ case R_AARCH64_MOVW_PREL_G0_NC:
+ overflow_check = false;
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 0,
+ INSN_IMM_MOVK);
+ break;
+ case R_AARCH64_MOVW_PREL_G0:
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 0,
+ INSN_IMM_MOVNZ);
+ break;
+ case R_AARCH64_MOVW_PREL_G1_NC:
+ overflow_check = false;
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 16,
+ INSN_IMM_MOVK);
+ break;
+ case R_AARCH64_MOVW_PREL_G1:
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 16,
+ INSN_IMM_MOVNZ);
+ break;
+ case R_AARCH64_MOVW_PREL_G2_NC:
+ overflow_check = false;
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 32,
+ INSN_IMM_MOVK);
+ break;
+ case R_AARCH64_MOVW_PREL_G2:
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 32,
+ INSN_IMM_MOVNZ);
+ break;
+ case R_AARCH64_MOVW_PREL_G3:
+ /* We're using the top bits so we can't overflow. */
+ overflow_check = false;
+ ovf = reloc_insn_movw(RELOC_OP_PREL, loc, val, 48,
+ INSN_IMM_MOVNZ);
+ break;
+
+ /* Immediate instruction relocations. */
+ case R_AARCH64_LD_PREL_LO19:
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 19,
+ INSN_IMM_19);
+ break;
+ case R_AARCH64_ADR_PREL_LO21:
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 0, 21,
+ INSN_IMM_ADR);
+ break;
+ case R_AARCH64_ADR_PREL_PG_HI21_NC:
+ overflow_check = false;
+ case R_AARCH64_ADR_PREL_PG_HI21:
+ ovf = reloc_insn_imm(RELOC_OP_PAGE, loc, val, 12, 21,
+ INSN_IMM_ADR);
+ break;
+ case R_AARCH64_ADD_ABS_LO12_NC:
+ case R_AARCH64_LDST8_ABS_LO12_NC:
+ overflow_check = false;
+ ovf = reloc_insn_imm(RELOC_OP_ABS, loc, val, 0, 12,
+ INSN_IMM_12);
+ break;
+ case R_AARCH64_LDST16_ABS_LO12_NC:
+ overflow_check = false;
+ ovf = reloc_insn_imm(RELOC_OP_ABS, loc, val, 1, 11,
+ INSN_IMM_12);
+ break;
+ case R_AARCH64_LDST32_ABS_LO12_NC:
+ overflow_check = false;
+ ovf = reloc_insn_imm(RELOC_OP_ABS, loc, val, 2, 10,
+ INSN_IMM_12);
+ break;
+ case R_AARCH64_LDST64_ABS_LO12_NC:
+ overflow_check = false;
+ ovf = reloc_insn_imm(RELOC_OP_ABS, loc, val, 3, 9,
+ INSN_IMM_12);
+ break;
+ case R_AARCH64_LDST128_ABS_LO12_NC:
+ overflow_check = false;
+ ovf = reloc_insn_imm(RELOC_OP_ABS, loc, val, 4, 8,
+ INSN_IMM_12);
+ break;
+ case R_AARCH64_TSTBR14:
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 14,
+ INSN_IMM_14);
+ break;
+ case R_AARCH64_CONDBR19:
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 19,
+ INSN_IMM_19);
+ break;
+ case R_AARCH64_JUMP26:
+ case R_AARCH64_CALL26:
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 26,
+ INSN_IMM_26);
+ break;
+
+ default:
+ pr_err("module %s: unsupported RELA relocation: %llu\n",
+ me->name, ELF64_R_TYPE(rel[i].r_info));
+ return -ENOEXEC;
+ }
+
+ if (overflow_check && ovf == -ERANGE)
+ goto overflow;
+
+ }
+
+ return 0;
+
+overflow:
+ pr_err("module %s: overflow in relocation type %d val %Lx\n",
+ me->name, (int)ELF64_R_TYPE(rel[i].r_info), val);
+ return -ENOEXEC;
+}
diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c
new file mode 100644
index 0000000..ecbf2d8
--- /dev/null
+++ b/arch/arm64/kernel/perf_event.c
@@ -0,0 +1,1368 @@
+/*
+ * PMU support
+ *
+ * Copyright (C) 2012 ARM Limited
+ * Author: Will Deacon <will.deacon@arm.com>
+ *
+ * This code is based heavily on the ARMv7 perf event code.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#define pr_fmt(fmt) "hw perfevents: " fmt
+
+#include <linux/bitmap.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/export.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/uaccess.h>
+
+#include <asm/cputype.h>
+#include <asm/irq.h>
+#include <asm/irq_regs.h>
+#include <asm/pmu.h>
+#include <asm/stacktrace.h>
+
+/*
+ * ARMv8 supports a maximum of 32 events.
+ * The cycle counter is included in this total.
+ */
+#define ARMPMU_MAX_HWEVENTS 32
+
+static DEFINE_PER_CPU(struct perf_event * [ARMPMU_MAX_HWEVENTS], hw_events);
+static DEFINE_PER_CPU(unsigned long [BITS_TO_LONGS(ARMPMU_MAX_HWEVENTS)], used_mask);
+static DEFINE_PER_CPU(struct pmu_hw_events, cpu_hw_events);
+
+#define to_arm_pmu(p) (container_of(p, struct arm_pmu, pmu))
+
+/* Set at runtime when we know what CPU type we are. */
+static struct arm_pmu *cpu_pmu;
+
+int
+armpmu_get_max_events(void)
+{
+ int max_events = 0;
+
+ if (cpu_pmu != NULL)
+ max_events = cpu_pmu->num_events;
+
+ return max_events;
+}
+EXPORT_SYMBOL_GPL(armpmu_get_max_events);
+
+int perf_num_counters(void)
+{
+ return armpmu_get_max_events();
+}
+EXPORT_SYMBOL_GPL(perf_num_counters);
+
+#define HW_OP_UNSUPPORTED 0xFFFF
+
+#define C(_x) \
+ PERF_COUNT_HW_CACHE_##_x
+
+#define CACHE_OP_UNSUPPORTED 0xFFFF
+
+static int
+armpmu_map_cache_event(const unsigned (*cache_map)
+ [PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX],
+ u64 config)
+{
+ unsigned int cache_type, cache_op, cache_result, ret;
+
+ cache_type = (config >> 0) & 0xff;
+ if (cache_type >= PERF_COUNT_HW_CACHE_MAX)
+ return -EINVAL;
+
+ cache_op = (config >> 8) & 0xff;
+ if (cache_op >= PERF_COUNT_HW_CACHE_OP_MAX)
+ return -EINVAL;
+
+ cache_result = (config >> 16) & 0xff;
+ if (cache_result >= PERF_COUNT_HW_CACHE_RESULT_MAX)
+ return -EINVAL;
+
+ ret = (int)(*cache_map)[cache_type][cache_op][cache_result];
+
+ if (ret == CACHE_OP_UNSUPPORTED)
+ return -ENOENT;
+
+ return ret;
+}
+
+static int
+armpmu_map_event(const unsigned (*event_map)[PERF_COUNT_HW_MAX], u64 config)
+{
+ int mapping = (*event_map)[config];
+ return mapping == HW_OP_UNSUPPORTED ? -ENOENT : mapping;
+}
+
+static int
+armpmu_map_raw_event(u32 raw_event_mask, u64 config)
+{
+ return (int)(config & raw_event_mask);
+}
+
+static int map_cpu_event(struct perf_event *event,
+ const unsigned (*event_map)[PERF_COUNT_HW_MAX],
+ const unsigned (*cache_map)
+ [PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX],
+ u32 raw_event_mask)
+{
+ u64 config = event->attr.config;
+
+ switch (event->attr.type) {
+ case PERF_TYPE_HARDWARE:
+ return armpmu_map_event(event_map, config);
+ case PERF_TYPE_HW_CACHE:
+ return armpmu_map_cache_event(cache_map, config);
+ case PERF_TYPE_RAW:
+ return armpmu_map_raw_event(raw_event_mask, config);
+ }
+
+ return -ENOENT;
+}
+
+int
+armpmu_event_set_period(struct perf_event *event,
+ struct hw_perf_event *hwc,
+ int idx)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ s64 left = local64_read(&hwc->period_left);
+ s64 period = hwc->sample_period;
+ int ret = 0;
+
+ if (unlikely(left <= -period)) {
+ left = period;
+ local64_set(&hwc->period_left, left);
+ hwc->last_period = period;
+ ret = 1;
+ }
+
+ if (unlikely(left <= 0)) {
+ left += period;
+ local64_set(&hwc->period_left, left);
+ hwc->last_period = period;
+ ret = 1;
+ }
+
+ if (left > (s64)armpmu->max_period)
+ left = armpmu->max_period;
+
+ local64_set(&hwc->prev_count, (u64)-left);
+
+ armpmu->write_counter(idx, (u64)(-left) & 0xffffffff);
+
+ perf_event_update_userpage(event);
+
+ return ret;
+}
+
+u64
+armpmu_event_update(struct perf_event *event,
+ struct hw_perf_event *hwc,
+ int idx)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ u64 delta, prev_raw_count, new_raw_count;
+
+again:
+ prev_raw_count = local64_read(&hwc->prev_count);
+ new_raw_count = armpmu->read_counter(idx);
+
+ if (local64_cmpxchg(&hwc->prev_count, prev_raw_count,
+ new_raw_count) != prev_raw_count)
+ goto again;
+
+ delta = (new_raw_count - prev_raw_count) & armpmu->max_period;
+
+ local64_add(delta, &event->count);
+ local64_sub(delta, &hwc->period_left);
+
+ return new_raw_count;
+}
+
+static void
+armpmu_read(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+
+ /* Don't read disabled counters! */
+ if (hwc->idx < 0)
+ return;
+
+ armpmu_event_update(event, hwc, hwc->idx);
+}
+
+static void
+armpmu_stop(struct perf_event *event, int flags)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ /*
+ * ARM pmu always has to update the counter, so ignore
+ * PERF_EF_UPDATE, see comments in armpmu_start().
+ */
+ if (!(hwc->state & PERF_HES_STOPPED)) {
+ armpmu->disable(hwc, hwc->idx);
+ barrier(); /* why? */
+ armpmu_event_update(event, hwc, hwc->idx);
+ hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
+ }
+}
+
+static void
+armpmu_start(struct perf_event *event, int flags)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+
+ /*
+ * ARM pmu always has to reprogram the period, so ignore
+ * PERF_EF_RELOAD, see the comment below.
+ */
+ if (flags & PERF_EF_RELOAD)
+ WARN_ON_ONCE(!(hwc->state & PERF_HES_UPTODATE));
+
+ hwc->state = 0;
+ /*
+ * Set the period again. Some counters can't be stopped, so when we
+ * were stopped we simply disabled the IRQ source and the counter
+ * may have been left counting. If we don't do this step then we may
+ * get an interrupt too soon or *way* too late if the overflow has
+ * happened since disabling.
+ */
+ armpmu_event_set_period(event, hwc, hwc->idx);
+ armpmu->enable(hwc, hwc->idx);
+}
+
+static void
+armpmu_del(struct perf_event *event, int flags)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ struct pmu_hw_events *hw_events = armpmu->get_hw_events();
+ struct hw_perf_event *hwc = &event->hw;
+ int idx = hwc->idx;
+
+ WARN_ON(idx < 0);
+
+ armpmu_stop(event, PERF_EF_UPDATE);
+ hw_events->events[idx] = NULL;
+ clear_bit(idx, hw_events->used_mask);
+
+ perf_event_update_userpage(event);
+}
+
+static int
+armpmu_add(struct perf_event *event, int flags)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ struct pmu_hw_events *hw_events = armpmu->get_hw_events();
+ struct hw_perf_event *hwc = &event->hw;
+ int idx;
+ int err = 0;
+
+ perf_pmu_disable(event->pmu);
+
+ /* If we don't have a space for the counter then finish early. */
+ idx = armpmu->get_event_idx(hw_events, hwc);
+ if (idx < 0) {
+ err = idx;
+ goto out;
+ }
+
+ /*
+ * If there is an event in the counter we are going to use then make
+ * sure it is disabled.
+ */
+ event->hw.idx = idx;
+ armpmu->disable(hwc, idx);
+ hw_events->events[idx] = event;
+
+ hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE;
+ if (flags & PERF_EF_START)
+ armpmu_start(event, PERF_EF_RELOAD);
+
+ /* Propagate our changes to the userspace mapping. */
+ perf_event_update_userpage(event);
+
+out:
+ perf_pmu_enable(event->pmu);
+ return err;
+}
+
+static int
+validate_event(struct pmu_hw_events *hw_events,
+ struct perf_event *event)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ struct hw_perf_event fake_event = event->hw;
+ struct pmu *leader_pmu = event->group_leader->pmu;
+
+ if (event->pmu != leader_pmu || event->state <= PERF_EVENT_STATE_OFF)
+ return 1;
+
+ return armpmu->get_event_idx(hw_events, &fake_event) >= 0;
+}
+
+static int
+validate_group(struct perf_event *event)
+{
+ struct perf_event *sibling, *leader = event->group_leader;
+ struct pmu_hw_events fake_pmu;
+ DECLARE_BITMAP(fake_used_mask, ARMPMU_MAX_HWEVENTS);
+
+ /*
+ * Initialise the fake PMU. We only need to populate the
+ * used_mask for the purposes of validation.
+ */
+ memset(fake_used_mask, 0, sizeof(fake_used_mask));
+ fake_pmu.used_mask = fake_used_mask;
+
+ if (!validate_event(&fake_pmu, leader))
+ return -EINVAL;
+
+ list_for_each_entry(sibling, &leader->sibling_list, group_entry) {
+ if (!validate_event(&fake_pmu, sibling))
+ return -EINVAL;
+ }
+
+ if (!validate_event(&fake_pmu, event))
+ return -EINVAL;
+
+ return 0;
+}
+
+static void
+armpmu_release_hardware(struct arm_pmu *armpmu)
+{
+ int i, irq, irqs;
+ struct platform_device *pmu_device = armpmu->plat_device;
+
+ irqs = min(pmu_device->num_resources, num_possible_cpus());
+
+ for (i = 0; i < irqs; ++i) {
+ if (!cpumask_test_and_clear_cpu(i, &armpmu->active_irqs))
+ continue;
+ irq = platform_get_irq(pmu_device, i);
+ if (irq >= 0)
+ free_irq(irq, armpmu);
+ }
+}
+
+static int
+armpmu_reserve_hardware(struct arm_pmu *armpmu)
+{
+ int i, err, irq, irqs;
+ struct platform_device *pmu_device = armpmu->plat_device;
+
+ if (!pmu_device) {
+ pr_err("no PMU device registered\n");
+ return -ENODEV;
+ }
+
+ irqs = min(pmu_device->num_resources, num_possible_cpus());
+ if (irqs < 1) {
+ pr_err("no irqs for PMUs defined\n");
+ return -ENODEV;
+ }
+
+ for (i = 0; i < irqs; ++i) {
+ err = 0;
+ irq = platform_get_irq(pmu_device, i);
+ if (irq < 0)
+ continue;
+
+ /*
+ * If we have a single PMU interrupt that we can't shift,
+ * assume that we're running on a uniprocessor machine and
+ * continue. Otherwise, continue without this interrupt.
+ */
+ if (irq_set_affinity(irq, cpumask_of(i)) && irqs > 1) {
+ pr_warning("unable to set irq affinity (irq=%d, cpu=%u)\n",
+ irq, i);
+ continue;
+ }
+
+ err = request_irq(irq, armpmu->handle_irq,
+ IRQF_NOBALANCING,
+ "arm-pmu", armpmu);
+ if (err) {
+ pr_err("unable to request IRQ%d for ARM PMU counters\n",
+ irq);
+ armpmu_release_hardware(armpmu);
+ return err;
+ }
+
+ cpumask_set_cpu(i, &armpmu->active_irqs);
+ }
+
+ return 0;
+}
+
+static void
+hw_perf_event_destroy(struct perf_event *event)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ atomic_t *active_events = &armpmu->active_events;
+ struct mutex *pmu_reserve_mutex = &armpmu->reserve_mutex;
+
+ if (atomic_dec_and_mutex_lock(active_events, pmu_reserve_mutex)) {
+ armpmu_release_hardware(armpmu);
+ mutex_unlock(pmu_reserve_mutex);
+ }
+}
+
+static int
+event_requires_mode_exclusion(struct perf_event_attr *attr)
+{
+ return attr->exclude_idle || attr->exclude_user ||
+ attr->exclude_kernel || attr->exclude_hv;
+}
+
+static int
+__hw_perf_event_init(struct perf_event *event)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ struct hw_perf_event *hwc = &event->hw;
+ int mapping, err;
+
+ mapping = armpmu->map_event(event);
+
+ if (mapping < 0) {
+ pr_debug("event %x:%llx not supported\n", event->attr.type,
+ event->attr.config);
+ return mapping;
+ }
+
+ /*
+ * We don't assign an index until we actually place the event onto
+ * hardware. Use -1 to signify that we haven't decided where to put it
+ * yet. For SMP systems, each core has it's own PMU so we can't do any
+ * clever allocation or constraints checking at this point.
+ */
+ hwc->idx = -1;
+ hwc->config_base = 0;
+ hwc->config = 0;
+ hwc->event_base = 0;
+
+ /*
+ * Check whether we need to exclude the counter from certain modes.
+ */
+ if ((!armpmu->set_event_filter ||
+ armpmu->set_event_filter(hwc, &event->attr)) &&
+ event_requires_mode_exclusion(&event->attr)) {
+ pr_debug("ARM performance counters do not support mode exclusion\n");
+ return -EPERM;
+ }
+
+ /*
+ * Store the event encoding into the config_base field.
+ */
+ hwc->config_base |= (unsigned long)mapping;
+
+ if (!hwc->sample_period) {
+ /*
+ * For non-sampling runs, limit the sample_period to half
+ * of the counter width. That way, the new counter value
+ * is far less likely to overtake the previous one unless
+ * you have some serious IRQ latency issues.
+ */
+ hwc->sample_period = armpmu->max_period >> 1;
+ hwc->last_period = hwc->sample_period;
+ local64_set(&hwc->period_left, hwc->sample_period);
+ }
+
+ err = 0;
+ if (event->group_leader != event) {
+ err = validate_group(event);
+ if (err)
+ return -EINVAL;
+ }
+
+ return err;
+}
+
+static int armpmu_event_init(struct perf_event *event)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(event->pmu);
+ int err = 0;
+ atomic_t *active_events = &armpmu->active_events;
+
+ if (armpmu->map_event(event) == -ENOENT)
+ return -ENOENT;
+
+ event->destroy = hw_perf_event_destroy;
+
+ if (!atomic_inc_not_zero(active_events)) {
+ mutex_lock(&armpmu->reserve_mutex);
+ if (atomic_read(active_events) == 0)
+ err = armpmu_reserve_hardware(armpmu);
+
+ if (!err)
+ atomic_inc(active_events);
+ mutex_unlock(&armpmu->reserve_mutex);
+ }
+
+ if (err)
+ return err;
+
+ err = __hw_perf_event_init(event);
+ if (err)
+ hw_perf_event_destroy(event);
+
+ return err;
+}
+
+static void armpmu_enable(struct pmu *pmu)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(pmu);
+ struct pmu_hw_events *hw_events = armpmu->get_hw_events();
+ int enabled = bitmap_weight(hw_events->used_mask, armpmu->num_events);
+
+ if (enabled)
+ armpmu->start();
+}
+
+static void armpmu_disable(struct pmu *pmu)
+{
+ struct arm_pmu *armpmu = to_arm_pmu(pmu);
+ armpmu->stop();
+}
+
+static void __init armpmu_init(struct arm_pmu *armpmu)
+{
+ atomic_set(&armpmu->active_events, 0);
+ mutex_init(&armpmu->reserve_mutex);
+
+ armpmu->pmu = (struct pmu) {
+ .pmu_enable = armpmu_enable,
+ .pmu_disable = armpmu_disable,
+ .event_init = armpmu_event_init,
+ .add = armpmu_add,
+ .del = armpmu_del,
+ .start = armpmu_start,
+ .stop = armpmu_stop,
+ .read = armpmu_read,
+ };
+}
+
+int __init armpmu_register(struct arm_pmu *armpmu, char *name, int type)
+{
+ armpmu_init(armpmu);
+ return perf_pmu_register(&armpmu->pmu, name, type);
+}
+
+/*
+ * ARMv8 PMUv3 Performance Events handling code.
+ * Common event types.
+ */
+enum armv8_pmuv3_perf_types {
+ /* Required events. */
+ ARMV8_PMUV3_PERFCTR_PMNC_SW_INCR = 0x00,
+ ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL = 0x03,
+ ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS = 0x04,
+ ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED = 0x10,
+ ARMV8_PMUV3_PERFCTR_CLOCK_CYCLES = 0x11,
+ ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED = 0x12,
+
+ /* At least one of the following is required. */
+ ARMV8_PMUV3_PERFCTR_INSTR_EXECUTED = 0x08,
+ ARMV8_PMUV3_PERFCTR_OP_SPEC = 0x1B,
+
+ /* Common architectural events. */
+ ARMV8_PMUV3_PERFCTR_MEM_READ = 0x06,
+ ARMV8_PMUV3_PERFCTR_MEM_WRITE = 0x07,
+ ARMV8_PMUV3_PERFCTR_EXC_TAKEN = 0x09,
+ ARMV8_PMUV3_PERFCTR_EXC_EXECUTED = 0x0A,
+ ARMV8_PMUV3_PERFCTR_CID_WRITE = 0x0B,
+ ARMV8_PMUV3_PERFCTR_PC_WRITE = 0x0C,
+ ARMV8_PMUV3_PERFCTR_PC_IMM_BRANCH = 0x0D,
+ ARMV8_PMUV3_PERFCTR_PC_PROC_RETURN = 0x0E,
+ ARMV8_PMUV3_PERFCTR_MEM_UNALIGNED_ACCESS = 0x0F,
+ ARMV8_PMUV3_PERFCTR_TTBR_WRITE = 0x1C,
+
+ /* Common microarchitectural events. */
+ ARMV8_PMUV3_PERFCTR_L1_ICACHE_REFILL = 0x01,
+ ARMV8_PMUV3_PERFCTR_ITLB_REFILL = 0x02,
+ ARMV8_PMUV3_PERFCTR_DTLB_REFILL = 0x05,
+ ARMV8_PMUV3_PERFCTR_MEM_ACCESS = 0x13,
+ ARMV8_PMUV3_PERFCTR_L1_ICACHE_ACCESS = 0x14,
+ ARMV8_PMUV3_PERFCTR_L1_DCACHE_WB = 0x15,
+ ARMV8_PMUV3_PERFCTR_L2_CACHE_ACCESS = 0x16,
+ ARMV8_PMUV3_PERFCTR_L2_CACHE_REFILL = 0x17,
+ ARMV8_PMUV3_PERFCTR_L2_CACHE_WB = 0x18,
+ ARMV8_PMUV3_PERFCTR_BUS_ACCESS = 0x19,
+ ARMV8_PMUV3_PERFCTR_MEM_ERROR = 0x1A,
+ ARMV8_PMUV3_PERFCTR_BUS_CYCLES = 0x1D,
+
+ /*
+ * This isn't an architected event.
+ * We detect this event number and use the cycle counter instead.
+ */
+ ARMV8_PMUV3_PERFCTR_CPU_CYCLES = 0xFF,
+};
+
+/* PMUv3 HW events mapping. */
+static const unsigned armv8_pmuv3_perf_map[PERF_COUNT_HW_MAX] = {
+ [PERF_COUNT_HW_CPU_CYCLES] = ARMV8_PMUV3_PERFCTR_CPU_CYCLES,
+ [PERF_COUNT_HW_INSTRUCTIONS] = ARMV8_PMUV3_PERFCTR_INSTR_EXECUTED,
+ [PERF_COUNT_HW_CACHE_REFERENCES] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS,
+ [PERF_COUNT_HW_CACHE_MISSES] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL,
+ [PERF_COUNT_HW_BRANCH_INSTRUCTIONS] = HW_OP_UNSUPPORTED,
+ [PERF_COUNT_HW_BRANCH_MISSES] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED,
+ [PERF_COUNT_HW_BUS_CYCLES] = HW_OP_UNSUPPORTED,
+ [PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] = HW_OP_UNSUPPORTED,
+ [PERF_COUNT_HW_STALLED_CYCLES_BACKEND] = HW_OP_UNSUPPORTED,
+};
+
+static const unsigned armv8_pmuv3_perf_cache_map[PERF_COUNT_HW_CACHE_MAX]
+ [PERF_COUNT_HW_CACHE_OP_MAX]
+ [PERF_COUNT_HW_CACHE_RESULT_MAX] = {
+ [C(L1D)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS,
+ [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_ACCESS,
+ [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_L1_DCACHE_REFILL,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+ [C(L1I)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+ [C(LL)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+ [C(DTLB)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+ [C(ITLB)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+ [C(BPU)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED,
+ [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_PRED,
+ [C(RESULT_MISS)] = ARMV8_PMUV3_PERFCTR_PC_BRANCH_MIS_PRED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+ [C(NODE)] = {
+ [C(OP_READ)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_WRITE)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ [C(OP_PREFETCH)] = {
+ [C(RESULT_ACCESS)] = CACHE_OP_UNSUPPORTED,
+ [C(RESULT_MISS)] = CACHE_OP_UNSUPPORTED,
+ },
+ },
+};
+
+/*
+ * Perf Events' indices
+ */
+#define ARMV8_IDX_CYCLE_COUNTER 0
+#define ARMV8_IDX_COUNTER0 1
+#define ARMV8_IDX_COUNTER_LAST (ARMV8_IDX_CYCLE_COUNTER + cpu_pmu->num_events - 1)
+
+#define ARMV8_MAX_COUNTERS 32
+#define ARMV8_COUNTER_MASK (ARMV8_MAX_COUNTERS - 1)
+
+/*
+ * ARMv8 low level PMU access
+ */
+
+/*
+ * Perf Event to low level counters mapping
+ */
+#define ARMV8_IDX_TO_COUNTER(x) \
+ (((x) - ARMV8_IDX_COUNTER0) & ARMV8_COUNTER_MASK)
+
+/*
+ * Per-CPU PMCR: config reg
+ */
+#define ARMV8_PMCR_E (1 << 0) /* Enable all counters */
+#define ARMV8_PMCR_P (1 << 1) /* Reset all counters */
+#define ARMV8_PMCR_C (1 << 2) /* Cycle counter reset */
+#define ARMV8_PMCR_D (1 << 3) /* CCNT counts every 64th cpu cycle */
+#define ARMV8_PMCR_X (1 << 4) /* Export to ETM */
+#define ARMV8_PMCR_DP (1 << 5) /* Disable CCNT if non-invasive debug*/
+#define ARMV8_PMCR_N_SHIFT 11 /* Number of counters supported */
+#define ARMV8_PMCR_N_MASK 0x1f
+#define ARMV8_PMCR_MASK 0x3f /* Mask for writable bits */
+
+/*
+ * PMOVSR: counters overflow flag status reg
+ */
+#define ARMV8_OVSR_MASK 0xffffffff /* Mask for writable bits */
+#define ARMV8_OVERFLOWED_MASK ARMV8_OVSR_MASK
+
+/*
+ * PMXEVTYPER: Event selection reg
+ */
+#define ARMV8_EVTYPE_MASK 0xc00000ff /* Mask for writable bits */
+#define ARMV8_EVTYPE_EVENT 0xff /* Mask for EVENT bits */
+
+/*
+ * Event filters for PMUv3
+ */
+#define ARMV8_EXCLUDE_EL1 (1 << 31)
+#define ARMV8_EXCLUDE_EL0 (1 << 30)
+#define ARMV8_INCLUDE_EL2 (1 << 27)
+
+static inline u32 armv8pmu_pmcr_read(void)
+{
+ u32 val;
+ asm volatile("mrs %0, pmcr_el0" : "=r" (val));
+ return val;
+}
+
+static inline void armv8pmu_pmcr_write(u32 val)
+{
+ val &= ARMV8_PMCR_MASK;
+ isb();
+ asm volatile("msr pmcr_el0, %0" :: "r" (val));
+}
+
+static inline int armv8pmu_has_overflowed(u32 pmovsr)
+{
+ return pmovsr & ARMV8_OVERFLOWED_MASK;
+}
+
+static inline int armv8pmu_counter_valid(int idx)
+{
+ return idx >= ARMV8_IDX_CYCLE_COUNTER && idx <= ARMV8_IDX_COUNTER_LAST;
+}
+
+static inline int armv8pmu_counter_has_overflowed(u32 pmnc, int idx)
+{
+ int ret = 0;
+ u32 counter;
+
+ if (!armv8pmu_counter_valid(idx)) {
+ pr_err("CPU%u checking wrong counter %d overflow status\n",
+ smp_processor_id(), idx);
+ } else {
+ counter = ARMV8_IDX_TO_COUNTER(idx);
+ ret = pmnc & BIT(counter);
+ }
+
+ return ret;
+}
+
+static inline int armv8pmu_select_counter(int idx)
+{
+ u32 counter;
+
+ if (!armv8pmu_counter_valid(idx)) {
+ pr_err("CPU%u selecting wrong PMNC counter %d\n",
+ smp_processor_id(), idx);
+ return -EINVAL;
+ }
+
+ counter = ARMV8_IDX_TO_COUNTER(idx);
+ asm volatile("msr pmselr_el0, %0" :: "r" (counter));
+ isb();
+
+ return idx;
+}
+
+static inline u32 armv8pmu_read_counter(int idx)
+{
+ u32 value = 0;
+
+ if (!armv8pmu_counter_valid(idx))
+ pr_err("CPU%u reading wrong counter %d\n",
+ smp_processor_id(), idx);
+ else if (idx == ARMV8_IDX_CYCLE_COUNTER)
+ asm volatile("mrs %0, pmccntr_el0" : "=r" (value));
+ else if (armv8pmu_select_counter(idx) == idx)
+ asm volatile("mrs %0, pmxevcntr_el0" : "=r" (value));
+
+ return value;
+}
+
+static inline void armv8pmu_write_counter(int idx, u32 value)
+{
+ if (!armv8pmu_counter_valid(idx))
+ pr_err("CPU%u writing wrong counter %d\n",
+ smp_processor_id(), idx);
+ else if (idx == ARMV8_IDX_CYCLE_COUNTER)
+ asm volatile("msr pmccntr_el0, %0" :: "r" (value));
+ else if (armv8pmu_select_counter(idx) == idx)
+ asm volatile("msr pmxevcntr_el0, %0" :: "r" (value));
+}
+
+static inline void armv8pmu_write_evtype(int idx, u32 val)
+{
+ if (armv8pmu_select_counter(idx) == idx) {
+ val &= ARMV8_EVTYPE_MASK;
+ asm volatile("msr pmxevtyper_el0, %0" :: "r" (val));
+ }
+}
+
+static inline int armv8pmu_enable_counter(int idx)
+{
+ u32 counter;
+
+ if (!armv8pmu_counter_valid(idx)) {
+ pr_err("CPU%u enabling wrong PMNC counter %d\n",
+ smp_processor_id(), idx);
+ return -EINVAL;
+ }
+
+ counter = ARMV8_IDX_TO_COUNTER(idx);
+ asm volatile("msr pmcntenset_el0, %0" :: "r" (BIT(counter)));
+ return idx;
+}
+
+static inline int armv8pmu_disable_counter(int idx)
+{
+ u32 counter;
+
+ if (!armv8pmu_counter_valid(idx)) {
+ pr_err("CPU%u disabling wrong PMNC counter %d\n",
+ smp_processor_id(), idx);
+ return -EINVAL;
+ }
+
+ counter = ARMV8_IDX_TO_COUNTER(idx);
+ asm volatile("msr pmcntenclr_el0, %0" :: "r" (BIT(counter)));
+ return idx;
+}
+
+static inline int armv8pmu_enable_intens(int idx)
+{
+ u32 counter;
+
+ if (!armv8pmu_counter_valid(idx)) {
+ pr_err("CPU%u enabling wrong PMNC counter IRQ enable %d\n",
+ smp_processor_id(), idx);
+ return -EINVAL;
+ }
+
+ counter = ARMV8_IDX_TO_COUNTER(idx);
+ asm volatile("msr pmintenset_el1, %0" :: "r" (BIT(counter)));
+ return idx;
+}
+
+static inline int armv8pmu_disable_intens(int idx)
+{
+ u32 counter;
+
+ if (!armv8pmu_counter_valid(idx)) {
+ pr_err("CPU%u disabling wrong PMNC counter IRQ enable %d\n",
+ smp_processor_id(), idx);
+ return -EINVAL;
+ }
+
+ counter = ARMV8_IDX_TO_COUNTER(idx);
+ asm volatile("msr pmintenclr_el1, %0" :: "r" (BIT(counter)));
+ isb();
+ /* Clear the overflow flag in case an interrupt is pending. */
+ asm volatile("msr pmovsclr_el0, %0" :: "r" (BIT(counter)));
+ isb();
+ return idx;
+}
+
+static inline u32 armv8pmu_getreset_flags(void)
+{
+ u32 value;
+
+ /* Read */
+ asm volatile("mrs %0, pmovsclr_el0" : "=r" (value));
+
+ /* Write to clear flags */
+ value &= ARMV8_OVSR_MASK;
+ asm volatile("msr pmovsclr_el0, %0" :: "r" (value));
+
+ return value;
+}
+
+static void armv8pmu_enable_event(struct hw_perf_event *hwc, int idx)
+{
+ unsigned long flags;
+ struct pmu_hw_events *events = cpu_pmu->get_hw_events();
+
+ /*
+ * Enable counter and interrupt, and set the counter to count
+ * the event that we're interested in.
+ */
+ raw_spin_lock_irqsave(&events->pmu_lock, flags);
+
+ /*
+ * Disable counter
+ */
+ armv8pmu_disable_counter(idx);
+
+ /*
+ * Set event (if destined for PMNx counters).
+ */
+ armv8pmu_write_evtype(idx, hwc->config_base);
+
+ /*
+ * Enable interrupt for this counter
+ */
+ armv8pmu_enable_intens(idx);
+
+ /*
+ * Enable counter
+ */
+ armv8pmu_enable_counter(idx);
+
+ raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+}
+
+static void armv8pmu_disable_event(struct hw_perf_event *hwc, int idx)
+{
+ unsigned long flags;
+ struct pmu_hw_events *events = cpu_pmu->get_hw_events();
+
+ /*
+ * Disable counter and interrupt
+ */
+ raw_spin_lock_irqsave(&events->pmu_lock, flags);
+
+ /*
+ * Disable counter
+ */
+ armv8pmu_disable_counter(idx);
+
+ /*
+ * Disable interrupt for this counter
+ */
+ armv8pmu_disable_intens(idx);
+
+ raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+}
+
+static irqreturn_t armv8pmu_handle_irq(int irq_num, void *dev)
+{
+ u32 pmovsr;
+ struct perf_sample_data data;
+ struct pmu_hw_events *cpuc;
+ struct pt_regs *regs;
+ int idx;
+
+ /*
+ * Get and reset the IRQ flags
+ */
+ pmovsr = armv8pmu_getreset_flags();
+
+ /*
+ * Did an overflow occur?
+ */
+ if (!armv8pmu_has_overflowed(pmovsr))
+ return IRQ_NONE;
+
+ /*
+ * Handle the counter(s) overflow(s)
+ */
+ regs = get_irq_regs();
+
+ cpuc = &__get_cpu_var(cpu_hw_events);
+ for (idx = 0; idx < cpu_pmu->num_events; ++idx) {
+ struct perf_event *event = cpuc->events[idx];
+ struct hw_perf_event *hwc;
+
+ /* Ignore if we don't have an event. */
+ if (!event)
+ continue;
+
+ /*
+ * We have a single interrupt for all counters. Check that
+ * each counter has overflowed before we process it.
+ */
+ if (!armv8pmu_counter_has_overflowed(pmovsr, idx))
+ continue;
+
+ hwc = &event->hw;
+ armpmu_event_update(event, hwc, idx);
+ perf_sample_data_init(&data, 0, hwc->last_period);
+ if (!armpmu_event_set_period(event, hwc, idx))
+ continue;
+
+ if (perf_event_overflow(event, &data, regs))
+ cpu_pmu->disable(hwc, idx);
+ }
+
+ /*
+ * Handle the pending perf events.
+ *
+ * Note: this call *must* be run with interrupts disabled. For
+ * platforms that can have the PMU interrupts raised as an NMI, this
+ * will not work.
+ */
+ irq_work_run();
+
+ return IRQ_HANDLED;
+}
+
+static void armv8pmu_start(void)
+{
+ unsigned long flags;
+ struct pmu_hw_events *events = cpu_pmu->get_hw_events();
+
+ raw_spin_lock_irqsave(&events->pmu_lock, flags);
+ /* Enable all counters */
+ armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMCR_E);
+ raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+}
+
+static void armv8pmu_stop(void)
+{
+ unsigned long flags;
+ struct pmu_hw_events *events = cpu_pmu->get_hw_events();
+
+ raw_spin_lock_irqsave(&events->pmu_lock, flags);
+ /* Disable all counters */
+ armv8pmu_pmcr_write(armv8pmu_pmcr_read() & ~ARMV8_PMCR_E);
+ raw_spin_unlock_irqrestore(&events->pmu_lock, flags);
+}
+
+static int armv8pmu_get_event_idx(struct pmu_hw_events *cpuc,
+ struct hw_perf_event *event)
+{
+ int idx;
+ unsigned long evtype = event->config_base & ARMV8_EVTYPE_EVENT;
+
+ /* Always place a cycle counter into the cycle counter. */
+ if (evtype == ARMV8_PMUV3_PERFCTR_CPU_CYCLES) {
+ if (test_and_set_bit(ARMV8_IDX_CYCLE_COUNTER, cpuc->used_mask))
+ return -EAGAIN;
+
+ return ARMV8_IDX_CYCLE_COUNTER;
+ }
+
+ /*
+ * For anything other than a cycle counter, try and use
+ * the events counters
+ */
+ for (idx = ARMV8_IDX_COUNTER0; idx < cpu_pmu->num_events; ++idx) {
+ if (!test_and_set_bit(idx, cpuc->used_mask))
+ return idx;
+ }
+
+ /* The counters are all in use. */
+ return -EAGAIN;
+}
+
+/*
+ * Add an event filter to a given event. This will only work for PMUv2 PMUs.
+ */
+static int armv8pmu_set_event_filter(struct hw_perf_event *event,
+ struct perf_event_attr *attr)
+{
+ unsigned long config_base = 0;
+
+ if (attr->exclude_idle)
+ return -EPERM;
+ if (attr->exclude_user)
+ config_base |= ARMV8_EXCLUDE_EL0;
+ if (attr->exclude_kernel)
+ config_base |= ARMV8_EXCLUDE_EL1;
+ if (!attr->exclude_hv)
+ config_base |= ARMV8_INCLUDE_EL2;
+
+ /*
+ * Install the filter into config_base as this is used to
+ * construct the event type.
+ */
+ event->config_base = config_base;
+
+ return 0;
+}
+
+static void armv8pmu_reset(void *info)
+{
+ u32 idx, nb_cnt = cpu_pmu->num_events;
+
+ /* The counter and interrupt enable registers are unknown at reset. */
+ for (idx = ARMV8_IDX_CYCLE_COUNTER; idx < nb_cnt; ++idx)
+ armv8pmu_disable_event(NULL, idx);
+
+ /* Initialize & Reset PMNC: C and P bits. */
+ armv8pmu_pmcr_write(ARMV8_PMCR_P | ARMV8_PMCR_C);
+
+ /* Disable access from userspace. */
+ asm volatile("msr pmuserenr_el0, %0" :: "r" (0));
+}
+
+static int armv8_pmuv3_map_event(struct perf_event *event)
+{
+ return map_cpu_event(event, &armv8_pmuv3_perf_map,
+ &armv8_pmuv3_perf_cache_map, 0xFF);
+}
+
+static struct arm_pmu armv8pmu = {
+ .handle_irq = armv8pmu_handle_irq,
+ .enable = armv8pmu_enable_event,
+ .disable = armv8pmu_disable_event,
+ .read_counter = armv8pmu_read_counter,
+ .write_counter = armv8pmu_write_counter,
+ .get_event_idx = armv8pmu_get_event_idx,
+ .start = armv8pmu_start,
+ .stop = armv8pmu_stop,
+ .reset = armv8pmu_reset,
+ .max_period = (1LLU << 32) - 1,
+};
+
+static u32 __init armv8pmu_read_num_pmnc_events(void)
+{
+ u32 nb_cnt;
+
+ /* Read the nb of CNTx counters supported from PMNC */
+ nb_cnt = (armv8pmu_pmcr_read() >> ARMV8_PMCR_N_SHIFT) & ARMV8_PMCR_N_MASK;
+
+ /* Add the CPU cycles counter and return */
+ return nb_cnt + 1;
+}
+
+static struct arm_pmu *__init armv8_pmuv3_pmu_init(void)
+{
+ armv8pmu.name = "arm/armv8-pmuv3";
+ armv8pmu.map_event = armv8_pmuv3_map_event;
+ armv8pmu.num_events = armv8pmu_read_num_pmnc_events();
+ armv8pmu.set_event_filter = armv8pmu_set_event_filter;
+ return &armv8pmu;
+}
+
+/*
+ * Ensure the PMU has sane values out of reset.
+ * This requires SMP to be available, so exists as a separate initcall.
+ */
+static int __init
+cpu_pmu_reset(void)
+{
+ if (cpu_pmu && cpu_pmu->reset)
+ return on_each_cpu(cpu_pmu->reset, NULL, 1);
+ return 0;
+}
+arch_initcall(cpu_pmu_reset);
+
+/*
+ * PMU platform driver and devicetree bindings.
+ */
+static struct of_device_id armpmu_of_device_ids[] = {
+ {.compatible = "arm,armv8-pmuv3"},
+ {},
+};
+
+static int __devinit armpmu_device_probe(struct platform_device *pdev)
+{
+ if (!cpu_pmu)
+ return -ENODEV;
+
+ cpu_pmu->plat_device = pdev;
+ return 0;
+}
+
+static struct platform_driver armpmu_driver = {
+ .driver = {
+ .name = "arm-pmu",
+ .of_match_table = armpmu_of_device_ids,
+ },
+ .probe = armpmu_device_probe,
+};
+
+static int __init register_pmu_driver(void)
+{
+ return platform_driver_register(&armpmu_driver);
+}
+device_initcall(register_pmu_driver);
+
+static struct pmu_hw_events *armpmu_get_cpu_events(void)
+{
+ return &__get_cpu_var(cpu_hw_events);
+}
+
+static void __init cpu_pmu_init(struct arm_pmu *armpmu)
+{
+ int cpu;
+ for_each_possible_cpu(cpu) {
+ struct pmu_hw_events *events = &per_cpu(cpu_hw_events, cpu);
+ events->events = per_cpu(hw_events, cpu);
+ events->used_mask = per_cpu(used_mask, cpu);
+ raw_spin_lock_init(&events->pmu_lock);
+ }
+ armpmu->get_hw_events = armpmu_get_cpu_events;
+}
+
+static int __init init_hw_perf_events(void)
+{
+ u64 dfr = read_cpuid(ID_AA64DFR0_EL1);
+
+ switch ((dfr >> 8) & 0xf) {
+ case 0x1: /* PMUv3 */
+ cpu_pmu = armv8_pmuv3_pmu_init();
+ break;
+ }
+
+ if (cpu_pmu) {
+ pr_info("enabled with %s PMU driver, %d counters available\n",
+ cpu_pmu->name, cpu_pmu->num_events);
+ cpu_pmu_init(cpu_pmu);
+ armpmu_register(cpu_pmu, "cpu", PERF_TYPE_RAW);
+ } else {
+ pr_info("no hardware support available\n");
+ }
+
+ return 0;
+}
+early_initcall(init_hw_perf_events);
+
+/*
+ * Callchain handling code.
+ */
+struct frame_tail {
+ struct frame_tail __user *fp;
+ unsigned long lr;
+} __attribute__((packed));
+
+/*
+ * Get the return address for a single stackframe and return a pointer to the
+ * next frame tail.
+ */
+static struct frame_tail __user *
+user_backtrace(struct frame_tail __user *tail,
+ struct perf_callchain_entry *entry)
+{
+ struct frame_tail buftail;
+ unsigned long err;
+
+ /* Also check accessibility of one struct frame_tail beyond */
+ if (!access_ok(VERIFY_READ, tail, sizeof(buftail)))
+ return NULL;
+
+ pagefault_disable();
+ err = __copy_from_user_inatomic(&buftail, tail, sizeof(buftail));
+ pagefault_enable();
+
+ if (err)
+ return NULL;
+
+ perf_callchain_store(entry, buftail.lr);
+
+ /*
+ * Frame pointers should strictly progress back up the stack
+ * (towards higher addresses).
+ */
+ if (tail >= buftail.fp)
+ return NULL;
+
+ return buftail.fp;
+}
+
+void perf_callchain_user(struct perf_callchain_entry *entry,
+ struct pt_regs *regs)
+{
+ struct frame_tail __user *tail;
+
+ tail = (struct frame_tail __user *)regs->regs[29];
+
+ while (entry->nr < PERF_MAX_STACK_DEPTH &&
+ tail && !((unsigned long)tail & 0xf))
+ tail = user_backtrace(tail, entry);
+}
+
+/*
+ * Gets called by walk_stackframe() for every stackframe. This will be called
+ * whist unwinding the stackframe and is like a subroutine return so we use
+ * the PC.
+ */
+static int callchain_trace(struct stackframe *frame, void *data)
+{
+ struct perf_callchain_entry *entry = data;
+ perf_callchain_store(entry, frame->pc);
+ return 0;
+}
+
+void perf_callchain_kernel(struct perf_callchain_entry *entry,
+ struct pt_regs *regs)
+{
+ struct stackframe frame;
+
+ frame.fp = regs->regs[29];
+ frame.sp = regs->sp;
+ frame.pc = regs->pc;
+ walk_stackframe(&frame, callchain_trace, entry);
+}
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
new file mode 100644
index 0000000..f22965e
--- /dev/null
+++ b/arch/arm64/kernel/process.c
@@ -0,0 +1,408 @@
+/*
+ * Based on arch/arm/kernel/process.c
+ *
+ * Original Copyright (C) 1995 Linus Torvalds
+ * Copyright (C) 1996-2000 Russell King - Converted to ARM.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <stdarg.h>
+
+#include <linux/export.h>
+#include <linux/sched.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/stddef.h>
+#include <linux/unistd.h>
+#include <linux/user.h>
+#include <linux/delay.h>
+#include <linux/reboot.h>
+#include <linux/interrupt.h>
+#include <linux/kallsyms.h>
+#include <linux/init.h>
+#include <linux/cpu.h>
+#include <linux/elfcore.h>
+#include <linux/pm.h>
+#include <linux/tick.h>
+#include <linux/utsname.h>
+#include <linux/uaccess.h>
+#include <linux/random.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/personality.h>
+#include <linux/notifier.h>
+
+#include <asm/compat.h>
+#include <asm/cacheflush.h>
+#include <asm/processor.h>
+#include <asm/stacktrace.h>
+#include <asm/fpsimd.h>
+
+static void setup_restart(void)
+{
+ /*
+ * Tell the mm system that we are going to reboot -
+ * we may need it to insert some 1:1 mappings so that
+ * soft boot works.
+ */
+ setup_mm_for_reboot();
+
+ /* Clean and invalidate caches */
+ flush_cache_all();
+
+ /* Turn D-cache off */
+ cpu_cache_off();
+
+ /* Push out any further dirty data, and ensure cache is empty */
+ flush_cache_all();
+}
+
+void soft_restart(unsigned long addr)
+{
+ setup_restart();
+ cpu_reset(addr);
+}
+
+/*
+ * Function pointers to optional machine specific functions
+ */
+void (*pm_power_off)(void);
+EXPORT_SYMBOL_GPL(pm_power_off);
+
+void (*pm_restart)(const char *cmd);
+EXPORT_SYMBOL_GPL(pm_restart);
+
+
+/*
+ * This is our default idle handler.
+ */
+static void default_idle(void)
+{
+ /*
+ * This should do all the clock switching and wait for interrupt
+ * tricks
+ */
+ cpu_do_idle();
+ local_irq_enable();
+}
+
+void (*pm_idle)(void) = default_idle;
+EXPORT_SYMBOL_GPL(pm_idle);
+
+/*
+ * The idle thread, has rather strange semantics for calling pm_idle,
+ * but this is what x86 does and we need to do the same, so that
+ * things like cpuidle get called in the same way. The only difference
+ * is that we always respect 'hlt_counter' to prevent low power idle.
+ */
+void cpu_idle(void)
+{
+ local_fiq_enable();
+
+ /* endless idle loop with no priority at all */
+ while (1) {
+ tick_nohz_idle_enter();
+ rcu_idle_enter();
+ while (!need_resched()) {
+ /*
+ * We need to disable interrupts here to ensure
+ * we don't miss a wakeup call.
+ */
+ local_irq_disable();
+ if (!need_resched()) {
+ stop_critical_timings();
+ pm_idle();
+ start_critical_timings();
+ /*
+ * pm_idle functions should always return
+ * with IRQs enabled.
+ */
+ WARN_ON(irqs_disabled());
+ } else {
+ local_irq_enable();
+ }
+ }
+ rcu_idle_exit();
+ tick_nohz_idle_exit();
+ schedule_preempt_disabled();
+ }
+}
+
+void machine_shutdown(void)
+{
+#ifdef CONFIG_SMP
+ smp_send_stop();
+#endif
+}
+
+void machine_halt(void)
+{
+ machine_shutdown();
+ while (1);
+}
+
+void machine_power_off(void)
+{
+ machine_shutdown();
+ if (pm_power_off)
+ pm_power_off();
+}
+
+void machine_restart(char *cmd)
+{
+ machine_shutdown();
+
+ /* Disable interrupts first */
+ local_irq_disable();
+ local_fiq_disable();
+
+ /* Now call the architecture specific reboot code. */
+ if (pm_restart)
+ pm_restart(cmd);
+
+ /*
+ * Whoops - the architecture was unable to reboot.
+ */
+ printk("Reboot failed -- System halted\n");
+ while (1);
+}
+
+void __show_regs(struct pt_regs *regs)
+{
+ int i;
+
+ printk("CPU: %d %s (%s %.*s)\n",
+ raw_smp_processor_id(), print_tainted(),
+ init_utsname()->release,
+ (int)strcspn(init_utsname()->version, " "),
+ init_utsname()->version);
+ print_symbol("PC is at %s\n", instruction_pointer(regs));
+ print_symbol("LR is at %s\n", regs->regs[30]);
+ printk("pc : [<%016llx>] lr : [<%016llx>] pstate: %08llx\n",
+ regs->pc, regs->regs[30], regs->pstate);
+ printk("sp : %016llx\n", regs->sp);
+ for (i = 29; i >= 0; i--) {
+ printk("x%-2d: %016llx ", i, regs->regs[i]);
+ if (i % 2 == 0)
+ printk("\n");
+ }
+ printk("\n");
+}
+
+void show_regs(struct pt_regs * regs)
+{
+ printk("\n");
+ printk("Pid: %d, comm: %20s\n", task_pid_nr(current), current->comm);
+ __show_regs(regs);
+}
+
+/*
+ * Free current thread data structures etc..
+ */
+void exit_thread(void)
+{
+}
+
+void flush_thread(void)
+{
+ fpsimd_flush_thread();
+ flush_ptrace_hw_breakpoint(current);
+}
+
+void release_thread(struct task_struct *dead_task)
+{
+}
+
+int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
+{
+ fpsimd_save_state(¤t->thread.fpsimd_state);
+ *dst = *src;
+ return 0;
+}
+
+asmlinkage void ret_from_fork(void) asm("ret_from_fork");
+
+int copy_thread(unsigned long clone_flags, unsigned long stack_start,
+ unsigned long stk_sz, struct task_struct *p,
+ struct pt_regs *regs)
+{
+ struct pt_regs *childregs = task_pt_regs(p);
+ unsigned long tls = p->thread.tp_value;
+
+ *childregs = *regs;
+ childregs->regs[0] = 0;
+
+ if (is_compat_thread(task_thread_info(p)))
+ childregs->compat_sp = stack_start;
+ else {
+ /*
+ * Read the current TLS pointer from tpidr_el0 as it may be
+ * out-of-sync with the saved value.
+ */
+ asm("mrs %0, tpidr_el0" : "=r" (tls));
+ childregs->sp = stack_start;
+ }
+
+ memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context));
+ p->thread.cpu_context.sp = (unsigned long)childregs;
+ p->thread.cpu_context.pc = (unsigned long)ret_from_fork;
+
+ /* If a TLS pointer was passed to clone, use that for the new thread. */
+ if (clone_flags & CLONE_SETTLS)
+ tls = regs->regs[3];
+ p->thread.tp_value = tls;
+
+ ptrace_hw_copy_thread(p);
+
+ return 0;
+}
+
+static void tls_thread_switch(struct task_struct *next)
+{
+ unsigned long tpidr, tpidrro;
+
+ if (!is_compat_task()) {
+ asm("mrs %0, tpidr_el0" : "=r" (tpidr));
+ current->thread.tp_value = tpidr;
+ }
+
+ if (is_compat_thread(task_thread_info(next))) {
+ tpidr = 0;
+ tpidrro = next->thread.tp_value;
+ } else {
+ tpidr = next->thread.tp_value;
+ tpidrro = 0;
+ }
+
+ asm(
+ " msr tpidr_el0, %0\n"
+ " msr tpidrro_el0, %1"
+ : : "r" (tpidr), "r" (tpidrro));
+}
+
+/*
+ * Thread switching.
+ */
+struct task_struct *__switch_to(struct task_struct *prev,
+ struct task_struct *next)
+{
+ struct task_struct *last;
+
+ fpsimd_thread_switch(next);
+ tls_thread_switch(next);
+ hw_breakpoint_thread_switch(next);
+
+ /* the actual thread switch */
+ last = cpu_switch_to(prev, next);
+
+ return last;
+}
+
+/*
+ * Fill in the task's elfregs structure for a core dump.
+ */
+int dump_task_regs(struct task_struct *t, elf_gregset_t *elfregs)
+{
+ elf_core_copy_regs(elfregs, task_pt_regs(t));
+ return 1;
+}
+
+/*
+ * fill in the fpe structure for a core dump...
+ */
+int dump_fpu (struct pt_regs *regs, struct user_fp *fp)
+{
+ return 0;
+}
+EXPORT_SYMBOL(dump_fpu);
+
+/*
+ * Shuffle the argument into the correct register before calling the
+ * thread function. x1 is the thread argument, x2 is the pointer to
+ * the thread function, and x3 points to the exit function.
+ */
+extern void kernel_thread_helper(void);
+asm( ".section .text\n"
+" .align\n"
+" .type kernel_thread_helper, #function\n"
+"kernel_thread_helper:\n"
+" mov x0, x1\n"
+" mov x30, x3\n"
+" br x2\n"
+" .size kernel_thread_helper, . - kernel_thread_helper\n"
+" .previous");
+
+#define kernel_thread_exit do_exit
+
+/*
+ * Create a kernel thread.
+ */
+pid_t kernel_thread(int (*fn)(void *), void *arg, unsigned long flags)
+{
+ struct pt_regs regs;
+
+ memset(®s, 0, sizeof(regs));
+
+ regs.regs[1] = (unsigned long)arg;
+ regs.regs[2] = (unsigned long)fn;
+ regs.regs[3] = (unsigned long)kernel_thread_exit;
+ regs.pc = (unsigned long)kernel_thread_helper;
+ regs.pstate = PSR_MODE_EL1h;
+
+ return do_fork(flags|CLONE_VM|CLONE_UNTRACED, 0, ®s, 0, NULL, NULL);
+}
+EXPORT_SYMBOL(kernel_thread);
+
+unsigned long get_wchan(struct task_struct *p)
+{
+ struct stackframe frame;
+ int count = 0;
+ if (!p || p == current || p->state == TASK_RUNNING)
+ return 0;
+
+ frame.fp = thread_saved_fp(p);
+ frame.sp = thread_saved_sp(p);
+ frame.pc = thread_saved_pc(p);
+ do {
+ int ret = unwind_frame(&frame);
+ if (ret < 0)
+ return 0;
+ if (!in_sched_functions(frame.pc))
+ return frame.pc;
+ } while (count ++ < 16);
+ return 0;
+}
+
+unsigned long arch_align_stack(unsigned long sp)
+{
+ if (!(current->personality & ADDR_NO_RANDOMIZE) && randomize_va_space)
+ sp -= get_random_int() & ~PAGE_MASK;
+ return sp & ~0xf;
+}
+
+static unsigned long randomize_base(unsigned long base)
+{
+ unsigned long range_end = base + (STACK_RND_MASK << PAGE_SHIFT) + 1;
+ return randomize_range(base, range_end, 0) ? : base;
+}
+
+unsigned long arch_randomize_brk(struct mm_struct *mm)
+{
+ return randomize_base(mm->brk);
+}
+
+unsigned long randomize_et_dyn(unsigned long base)
+{
+ return randomize_base(base);
+}
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
new file mode 100644
index 0000000..ac3550e
--- /dev/null
+++ b/arch/arm64/kernel/ptrace.c
@@ -0,0 +1,1126 @@
+/*
+ * Based on arch/arm/kernel/ptrace.c
+ *
+ * By Ross Biro 1/23/92
+ * edited by Linus Torvalds
+ * ARM modifications Copyright (C) 2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/smp.h>
+#include <linux/ptrace.h>
+#include <linux/user.h>
+#include <linux/security.h>
+#include <linux/init.h>
+#include <linux/signal.h>
+#include <linux/uaccess.h>
+#include <linux/perf_event.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/regset.h>
+#include <linux/tracehook.h>
+#include <linux/elf.h>
+
+#include <asm/compat.h>
+#include <asm/debug-monitors.h>
+#include <asm/pgtable.h>
+#include <asm/traps.h>
+#include <asm/system_misc.h>
+
+/*
+ * TODO: does not yet catch signals sent when the child dies.
+ * in exit.c or in signal.c.
+ */
+
+/*
+ * Called by kernel/ptrace.c when detaching..
+ */
+void ptrace_disable(struct task_struct *child)
+{
+}
+
+/*
+ * Handle hitting a breakpoint.
+ */
+static int ptrace_break(struct pt_regs *regs)
+{
+ siginfo_t info = {
+ .si_signo = SIGTRAP,
+ .si_errno = 0,
+ .si_code = TRAP_BRKPT,
+ .si_addr = (void __user *)instruction_pointer(regs),
+ };
+
+ force_sig_info(SIGTRAP, &info, current);
+ return 0;
+}
+
+static int arm64_break_trap(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ return ptrace_break(regs);
+}
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+/*
+ * Handle hitting a HW-breakpoint.
+ */
+static void ptrace_hbptriggered(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ struct arch_hw_breakpoint *bkpt = counter_arch_bp(bp);
+ siginfo_t info = {
+ .si_signo = SIGTRAP,
+ .si_errno = 0,
+ .si_code = TRAP_HWBKPT,
+ .si_addr = (void __user *)(bkpt->trigger),
+ };
+
+#ifdef CONFIG_COMPAT
+ int i;
+
+ if (!is_compat_task())
+ goto send_sig;
+
+ for (i = 0; i < ARM_MAX_BRP; ++i) {
+ if (current->thread.debug.hbp_break[i] == bp) {
+ info.si_errno = (i << 1) + 1;
+ break;
+ }
+ }
+ for (i = ARM_MAX_BRP; i < ARM_MAX_HBP_SLOTS && !bp; ++i) {
+ if (current->thread.debug.hbp_watch[i] == bp) {
+ info.si_errno = -((i << 1) + 1);
+ break;
+ }
+ }
+
+send_sig:
+#endif
+ force_sig_info(SIGTRAP, &info, current);
+}
+
+/*
+ * Unregister breakpoints from this task and reset the pointers in
+ * the thread_struct.
+ */
+void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
+{
+ int i;
+ struct thread_struct *t = &tsk->thread;
+
+ for (i = 0; i < ARM_MAX_BRP; i++) {
+ if (t->debug.hbp_break[i]) {
+ unregister_hw_breakpoint(t->debug.hbp_break[i]);
+ t->debug.hbp_break[i] = NULL;
+ }
+ }
+
+ for (i = 0; i < ARM_MAX_WRP; i++) {
+ if (t->debug.hbp_watch[i]) {
+ unregister_hw_breakpoint(t->debug.hbp_watch[i]);
+ t->debug.hbp_watch[i] = NULL;
+ }
+ }
+}
+
+void ptrace_hw_copy_thread(struct task_struct *tsk)
+{
+ memset(&tsk->thread.debug, 0, sizeof(struct debug_info));
+}
+
+static struct perf_event *ptrace_hbp_get_event(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx)
+{
+ struct perf_event *bp = ERR_PTR(-EINVAL);
+
+ switch (note_type) {
+ case NT_ARM_HW_BREAK:
+ if (idx < ARM_MAX_BRP)
+ bp = tsk->thread.debug.hbp_break[idx];
+ break;
+ case NT_ARM_HW_WATCH:
+ if (idx < ARM_MAX_WRP)
+ bp = tsk->thread.debug.hbp_watch[idx];
+ break;
+ }
+
+ return bp;
+}
+
+static int ptrace_hbp_set_event(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx,
+ struct perf_event *bp)
+{
+ int err = -EINVAL;
+
+ switch (note_type) {
+ case NT_ARM_HW_BREAK:
+ if (idx < ARM_MAX_BRP) {
+ tsk->thread.debug.hbp_break[idx] = bp;
+ err = 0;
+ }
+ break;
+ case NT_ARM_HW_WATCH:
+ if (idx < ARM_MAX_WRP) {
+ tsk->thread.debug.hbp_watch[idx] = bp;
+ err = 0;
+ }
+ break;
+ }
+
+ return err;
+}
+
+static struct perf_event *ptrace_hbp_create(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx)
+{
+ struct perf_event *bp;
+ struct perf_event_attr attr;
+ int err, type;
+
+ switch (note_type) {
+ case NT_ARM_HW_BREAK:
+ type = HW_BREAKPOINT_X;
+ break;
+ case NT_ARM_HW_WATCH:
+ type = HW_BREAKPOINT_RW;
+ break;
+ default:
+ return ERR_PTR(-EINVAL);
+ }
+
+ ptrace_breakpoint_init(&attr);
+
+ /*
+ * Initialise fields to sane defaults
+ * (i.e. values that will pass validation).
+ */
+ attr.bp_addr = 0;
+ attr.bp_len = HW_BREAKPOINT_LEN_4;
+ attr.bp_type = type;
+ attr.disabled = 1;
+
+ bp = register_user_hw_breakpoint(&attr, ptrace_hbptriggered, NULL, tsk);
+ if (IS_ERR(bp))
+ return bp;
+
+ err = ptrace_hbp_set_event(note_type, tsk, idx, bp);
+ if (err)
+ return ERR_PTR(err);
+
+ return bp;
+}
+
+static int ptrace_hbp_fill_attr_ctrl(unsigned int note_type,
+ struct arch_hw_breakpoint_ctrl ctrl,
+ struct perf_event_attr *attr)
+{
+ int err, len, type;
+
+ err = arch_bp_generic_fields(ctrl, &len, &type);
+ if (err)
+ return err;
+
+ switch (note_type) {
+ case NT_ARM_HW_BREAK:
+ if ((type & HW_BREAKPOINT_X) != type)
+ return -EINVAL;
+ break;
+ case NT_ARM_HW_WATCH:
+ if ((type & HW_BREAKPOINT_RW) != type)
+ return -EINVAL;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ attr->bp_len = len;
+ attr->bp_type = type;
+ attr->disabled = !ctrl.enabled;
+
+ return 0;
+}
+
+static int ptrace_hbp_get_resource_info(unsigned int note_type, u32 *info)
+{
+ u8 num;
+ u32 reg = 0;
+
+ switch (note_type) {
+ case NT_ARM_HW_BREAK:
+ num = hw_breakpoint_slots(TYPE_INST);
+ break;
+ case NT_ARM_HW_WATCH:
+ num = hw_breakpoint_slots(TYPE_DATA);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ reg |= debug_monitors_arch();
+ reg <<= 8;
+ reg |= num;
+
+ *info = reg;
+ return 0;
+}
+
+static int ptrace_hbp_get_ctrl(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx,
+ u32 *ctrl)
+{
+ struct perf_event *bp = ptrace_hbp_get_event(note_type, tsk, idx);
+
+ if (IS_ERR(bp))
+ return PTR_ERR(bp);
+
+ *ctrl = bp ? encode_ctrl_reg(counter_arch_bp(bp)->ctrl) : 0;
+ return 0;
+}
+
+static int ptrace_hbp_get_addr(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx,
+ u64 *addr)
+{
+ struct perf_event *bp = ptrace_hbp_get_event(note_type, tsk, idx);
+
+ if (IS_ERR(bp))
+ return PTR_ERR(bp);
+
+ *addr = bp ? bp->attr.bp_addr : 0;
+ return 0;
+}
+
+static struct perf_event *ptrace_hbp_get_initialised_bp(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx)
+{
+ struct perf_event *bp = ptrace_hbp_get_event(note_type, tsk, idx);
+
+ if (!bp)
+ bp = ptrace_hbp_create(note_type, tsk, idx);
+
+ return bp;
+}
+
+static int ptrace_hbp_set_ctrl(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx,
+ u32 uctrl)
+{
+ int err;
+ struct perf_event *bp;
+ struct perf_event_attr attr;
+ struct arch_hw_breakpoint_ctrl ctrl;
+
+ bp = ptrace_hbp_get_initialised_bp(note_type, tsk, idx);
+ if (IS_ERR(bp)) {
+ err = PTR_ERR(bp);
+ return err;
+ }
+
+ attr = bp->attr;
+ decode_ctrl_reg(uctrl, &ctrl);
+ err = ptrace_hbp_fill_attr_ctrl(note_type, ctrl, &attr);
+ if (err)
+ return err;
+
+ return modify_user_hw_breakpoint(bp, &attr);
+}
+
+static int ptrace_hbp_set_addr(unsigned int note_type,
+ struct task_struct *tsk,
+ unsigned long idx,
+ u64 addr)
+{
+ int err;
+ struct perf_event *bp;
+ struct perf_event_attr attr;
+
+ bp = ptrace_hbp_get_initialised_bp(note_type, tsk, idx);
+ if (IS_ERR(bp)) {
+ err = PTR_ERR(bp);
+ return err;
+ }
+
+ attr = bp->attr;
+ attr.bp_addr = addr;
+ err = modify_user_hw_breakpoint(bp, &attr);
+ return err;
+}
+
+#define PTRACE_HBP_ADDR_SZ sizeof(u64)
+#define PTRACE_HBP_CTRL_SZ sizeof(u32)
+#define PTRACE_HBP_REG_OFF sizeof(u32)
+
+static int hw_break_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ unsigned int note_type = regset->core_note_type;
+ int ret, idx = 0, offset = PTRACE_HBP_REG_OFF, limit;
+ u32 info, ctrl;
+ u64 addr;
+
+ /* Resource info */
+ ret = ptrace_hbp_get_resource_info(note_type, &info);
+ if (ret)
+ return ret;
+
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &info, 0, 4);
+ if (ret)
+ return ret;
+
+ /* (address, ctrl) registers */
+ limit = regset->n * regset->size;
+ while (count && offset < limit) {
+ ret = ptrace_hbp_get_addr(note_type, target, idx, &addr);
+ if (ret)
+ return ret;
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &addr,
+ offset, offset + PTRACE_HBP_ADDR_SZ);
+ if (ret)
+ return ret;
+ offset += PTRACE_HBP_ADDR_SZ;
+
+ ret = ptrace_hbp_get_ctrl(note_type, target, idx, &ctrl);
+ if (ret)
+ return ret;
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &ctrl,
+ offset, offset + PTRACE_HBP_CTRL_SZ);
+ if (ret)
+ return ret;
+ offset += PTRACE_HBP_CTRL_SZ;
+ idx++;
+ }
+
+ return 0;
+}
+
+static int hw_break_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ unsigned int note_type = regset->core_note_type;
+ int ret, idx = 0, offset = PTRACE_HBP_REG_OFF, limit;
+ u32 ctrl;
+ u64 addr;
+
+ /* Resource info */
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf, 0, 4);
+ if (ret)
+ return ret;
+
+ /* (address, ctrl) registers */
+ limit = regset->n * regset->size;
+ while (count && offset < limit) {
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &addr,
+ offset, offset + PTRACE_HBP_ADDR_SZ);
+ if (ret)
+ return ret;
+ ret = ptrace_hbp_set_addr(note_type, target, idx, addr);
+ if (ret)
+ return ret;
+ offset += PTRACE_HBP_ADDR_SZ;
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &ctrl,
+ offset, offset + PTRACE_HBP_CTRL_SZ);
+ if (ret)
+ return ret;
+ ret = ptrace_hbp_set_ctrl(note_type, target, idx, ctrl);
+ if (ret)
+ return ret;
+ offset += PTRACE_HBP_CTRL_SZ;
+ idx++;
+ }
+
+ return 0;
+}
+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
+
+static int gpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ struct user_pt_regs *uregs = &task_pt_regs(target)->user_regs;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
+}
+
+static int gpr_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ struct user_pt_regs newregs;
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newregs, 0, -1);
+ if (ret)
+ return ret;
+
+ if (!valid_user_regs(&newregs))
+ return -EINVAL;
+
+ task_pt_regs(target)->user_regs = newregs;
+ return 0;
+}
+
+/*
+ * TODO: update fp accessors for lazy context switching (sync/flush hwstate)
+ */
+static int fpr_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ struct user_fpsimd_state *uregs;
+ uregs = &target->thread.fpsimd_state.user_fpsimd;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
+}
+
+static int fpr_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ struct user_fpsimd_state newstate;
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
+ if (ret)
+ return ret;
+
+ target->thread.fpsimd_state.user_fpsimd = newstate;
+ return ret;
+}
+
+static int tls_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ unsigned long *tls = &target->thread.tp_value;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, tls, 0, -1);
+}
+
+static int tls_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ unsigned long tls;
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &tls, 0, -1);
+ if (ret)
+ return ret;
+
+ target->thread.tp_value = tls;
+ return ret;
+}
+
+enum aarch64_regset {
+ REGSET_GPR,
+ REGSET_FPR,
+ REGSET_TLS,
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ REGSET_HW_BREAK,
+ REGSET_HW_WATCH,
+#endif
+};
+
+static const struct user_regset aarch64_regsets[] = {
+ [REGSET_GPR] = {
+ .core_note_type = NT_PRSTATUS,
+ .n = sizeof(struct user_pt_regs) / sizeof(u64),
+ .size = sizeof(u64),
+ .align = sizeof(u64),
+ .get = gpr_get,
+ .set = gpr_set
+ },
+ [REGSET_FPR] = {
+ .core_note_type = NT_PRFPREG,
+ .n = sizeof(struct user_fpsimd_state) / sizeof(u32),
+ /*
+ * We pretend we have 32-bit registers because the fpsr and
+ * fpcr are 32-bits wide.
+ */
+ .size = sizeof(u32),
+ .align = sizeof(u32),
+ .get = fpr_get,
+ .set = fpr_set
+ },
+ [REGSET_TLS] = {
+ .core_note_type = NT_ARM_TLS,
+ .n = 1,
+ .size = sizeof(void *),
+ .align = sizeof(void *),
+ .get = tls_get,
+ .set = tls_set,
+ },
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ [REGSET_HW_BREAK] = {
+ .core_note_type = NT_ARM_HW_BREAK,
+ .n = sizeof(struct user_hwdebug_state) / sizeof(u32),
+ .size = sizeof(u32),
+ .align = sizeof(u32),
+ .get = hw_break_get,
+ .set = hw_break_set,
+ },
+ [REGSET_HW_WATCH] = {
+ .core_note_type = NT_ARM_HW_WATCH,
+ .n = sizeof(struct user_hwdebug_state) / sizeof(u32),
+ .size = sizeof(u32),
+ .align = sizeof(u32),
+ .get = hw_break_get,
+ .set = hw_break_set,
+ },
+#endif
+};
+
+static const struct user_regset_view user_aarch64_view = {
+ .name = "aarch64", .e_machine = EM_AARCH64,
+ .regsets = aarch64_regsets, .n = ARRAY_SIZE(aarch64_regsets)
+};
+
+#ifdef CONFIG_COMPAT
+#include <linux/compat.h>
+
+enum compat_regset {
+ REGSET_COMPAT_GPR,
+ REGSET_COMPAT_VFP,
+};
+
+static int compat_gpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ int ret = 0;
+ unsigned int i, start, num_regs;
+
+ /* Calculate the number of AArch32 registers contained in count */
+ num_regs = count / regset->size;
+
+ /* Convert pos into an register number */
+ start = pos / regset->size;
+
+ if (start + num_regs > regset->n)
+ return -EIO;
+
+ for (i = 0; i < num_regs; ++i) {
+ unsigned int idx = start + i;
+ void *reg;
+
+ switch (idx) {
+ case 15:
+ reg = (void *)&task_pt_regs(target)->pc;
+ break;
+ case 16:
+ reg = (void *)&task_pt_regs(target)->pstate;
+ break;
+ case 17:
+ reg = (void *)&task_pt_regs(target)->orig_x0;
+ break;
+ default:
+ reg = (void *)&task_pt_regs(target)->regs[idx];
+ }
+
+ ret = copy_to_user(ubuf, reg, sizeof(compat_ulong_t));
+
+ if (ret)
+ break;
+ else
+ ubuf += sizeof(compat_ulong_t);
+ }
+
+ return ret;
+}
+
+static int compat_gpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct pt_regs newregs;
+ int ret = 0;
+ unsigned int i, start, num_regs;
+
+ /* Calculate the number of AArch32 registers contained in count */
+ num_regs = count / regset->size;
+
+ /* Convert pos into an register number */
+ start = pos / regset->size;
+
+ if (start + num_regs > regset->n)
+ return -EIO;
+
+ newregs = *task_pt_regs(target);
+
+ for (i = 0; i < num_regs; ++i) {
+ unsigned int idx = start + i;
+ void *reg;
+
+ switch (idx) {
+ case 15:
+ reg = (void *)&newregs.pc;
+ break;
+ case 16:
+ reg = (void *)&newregs.pstate;
+ break;
+ case 17:
+ reg = (void *)&newregs.orig_x0;
+ break;
+ default:
+ reg = (void *)&newregs.regs[idx];
+ }
+
+ ret = copy_from_user(reg, ubuf, sizeof(compat_ulong_t));
+
+ if (ret)
+ goto out;
+ else
+ ubuf += sizeof(compat_ulong_t);
+ }
+
+ if (valid_user_regs(&newregs.user_regs))
+ *task_pt_regs(target) = newregs;
+ else
+ ret = -EINVAL;
+
+out:
+ return ret;
+}
+
+static int compat_vfp_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ struct user_fpsimd_state *uregs;
+ compat_ulong_t fpscr;
+ int ret;
+
+ uregs = &target->thread.fpsimd_state.user_fpsimd;
+
+ /*
+ * The VFP registers are packed into the fpsimd_state, so they all sit
+ * nicely together for us. We just need to create the fpscr separately.
+ */
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0,
+ VFP_STATE_SIZE - sizeof(compat_ulong_t));
+
+ if (count && !ret) {
+ fpscr = (uregs->fpsr & VFP_FPSCR_STAT_MASK) |
+ (uregs->fpcr & VFP_FPSCR_CTRL_MASK);
+ ret = put_user(fpscr, (compat_ulong_t *)ubuf);
+ }
+
+ return ret;
+}
+
+static int compat_vfp_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ struct user_fpsimd_state *uregs;
+ compat_ulong_t fpscr;
+ int ret;
+
+ if (pos + count > VFP_STATE_SIZE)
+ return -EIO;
+
+ uregs = &target->thread.fpsimd_state.user_fpsimd;
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, uregs, 0,
+ VFP_STATE_SIZE - sizeof(compat_ulong_t));
+
+ if (count && !ret) {
+ ret = get_user(fpscr, (compat_ulong_t *)ubuf);
+ uregs->fpsr = fpscr & VFP_FPSCR_STAT_MASK;
+ uregs->fpcr = fpscr & VFP_FPSCR_CTRL_MASK;
+ }
+
+ return ret;
+}
+
+static const struct user_regset aarch32_regsets[] = {
+ [REGSET_COMPAT_GPR] = {
+ .core_note_type = NT_PRSTATUS,
+ .n = COMPAT_ELF_NGREG,
+ .size = sizeof(compat_elf_greg_t),
+ .align = sizeof(compat_elf_greg_t),
+ .get = compat_gpr_get,
+ .set = compat_gpr_set
+ },
+ [REGSET_COMPAT_VFP] = {
+ .core_note_type = NT_ARM_VFP,
+ .n = VFP_STATE_SIZE / sizeof(compat_ulong_t),
+ .size = sizeof(compat_ulong_t),
+ .align = sizeof(compat_ulong_t),
+ .get = compat_vfp_get,
+ .set = compat_vfp_set
+ },
+};
+
+static const struct user_regset_view user_aarch32_view = {
+ .name = "aarch32", .e_machine = EM_ARM,
+ .regsets = aarch32_regsets, .n = ARRAY_SIZE(aarch32_regsets)
+};
+
+int aarch32_break_trap(struct pt_regs *regs)
+{
+ unsigned int instr;
+ bool bp = false;
+ void __user *pc = (void __user *)instruction_pointer(regs);
+
+ if (compat_thumb_mode(regs)) {
+ /* get 16-bit Thumb instruction */
+ get_user(instr, (u16 __user *)pc);
+ if (instr == AARCH32_BREAK_THUMB2_LO) {
+ /* get second half of 32-bit Thumb-2 instruction */
+ get_user(instr, (u16 __user *)(pc + 2));
+ bp = instr == AARCH32_BREAK_THUMB2_HI;
+ } else {
+ bp = instr == AARCH32_BREAK_THUMB;
+ }
+ } else {
+ /* 32-bit ARM instruction */
+ get_user(instr, (u32 __user *)pc);
+ bp = (instr & ~0xf0000000) == AARCH32_BREAK_ARM;
+ }
+
+ if (bp)
+ return ptrace_break(regs);
+ return 1;
+}
+
+static int compat_ptrace_read_user(struct task_struct *tsk, compat_ulong_t off,
+ compat_ulong_t __user *ret)
+{
+ compat_ulong_t tmp;
+
+ if (off & 3)
+ return -EIO;
+
+ if (off == PT_TEXT_ADDR)
+ tmp = tsk->mm->start_code;
+ else if (off == PT_DATA_ADDR)
+ tmp = tsk->mm->start_data;
+ else if (off == PT_TEXT_END_ADDR)
+ tmp = tsk->mm->end_code;
+ else if (off < sizeof(compat_elf_gregset_t))
+ return copy_regset_to_user(tsk, &user_aarch32_view,
+ REGSET_COMPAT_GPR, off,
+ sizeof(compat_ulong_t), ret);
+ else if (off >= COMPAT_USER_SZ)
+ return -EIO;
+ else
+ tmp = 0;
+
+ return put_user(tmp, ret);
+}
+
+static int compat_ptrace_write_user(struct task_struct *tsk, compat_ulong_t off,
+ compat_ulong_t val)
+{
+ int ret;
+
+ if (off & 3 || off >= COMPAT_USER_SZ)
+ return -EIO;
+
+ if (off >= sizeof(compat_elf_gregset_t))
+ return 0;
+
+ ret = copy_regset_from_user(tsk, &user_aarch32_view,
+ REGSET_COMPAT_GPR, off,
+ sizeof(compat_ulong_t),
+ &val);
+ return ret;
+}
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+
+/*
+ * Convert a virtual register number into an index for a thread_info
+ * breakpoint array. Breakpoints are identified using positive numbers
+ * whilst watchpoints are negative. The registers are laid out as pairs
+ * of (address, control), each pair mapping to a unique hw_breakpoint struct.
+ * Register 0 is reserved for describing resource information.
+ */
+static int compat_ptrace_hbp_num_to_idx(compat_long_t num)
+{
+ return (abs(num) - 1) >> 1;
+}
+
+static int compat_ptrace_hbp_get_resource_info(u32 *kdata)
+{
+ u8 num_brps, num_wrps, debug_arch, wp_len;
+ u32 reg = 0;
+
+ num_brps = hw_breakpoint_slots(TYPE_INST);
+ num_wrps = hw_breakpoint_slots(TYPE_DATA);
+
+ debug_arch = debug_monitors_arch();
+ wp_len = 8;
+ reg |= debug_arch;
+ reg <<= 8;
+ reg |= wp_len;
+ reg <<= 8;
+ reg |= num_wrps;
+ reg <<= 8;
+ reg |= num_brps;
+
+ *kdata = reg;
+ return 0;
+}
+
+static int compat_ptrace_hbp_get(unsigned int note_type,
+ struct task_struct *tsk,
+ compat_long_t num,
+ u32 *kdata)
+{
+ u64 addr = 0;
+ u32 ctrl = 0;
+
+ int err, idx = compat_ptrace_hbp_num_to_idx(num);;
+
+ if (num & 1) {
+ err = ptrace_hbp_get_addr(note_type, tsk, idx, &addr);
+ *kdata = (u32)addr;
+ } else {
+ err = ptrace_hbp_get_ctrl(note_type, tsk, idx, &ctrl);
+ *kdata = ctrl;
+ }
+
+ return err;
+}
+
+static int compat_ptrace_hbp_set(unsigned int note_type,
+ struct task_struct *tsk,
+ compat_long_t num,
+ u32 *kdata)
+{
+ u64 addr;
+ u32 ctrl;
+
+ int err, idx = compat_ptrace_hbp_num_to_idx(num);
+
+ if (num & 1) {
+ addr = *kdata;
+ err = ptrace_hbp_set_addr(note_type, tsk, idx, addr);
+ } else {
+ ctrl = *kdata;
+ err = ptrace_hbp_set_ctrl(note_type, tsk, idx, ctrl);
+ }
+
+ return err;
+}
+
+static int compat_ptrace_gethbpregs(struct task_struct *tsk, compat_long_t num,
+ compat_ulong_t __user *data)
+{
+ int ret;
+ u32 kdata;
+ mm_segment_t old_fs = get_fs();
+
+ set_fs(KERNEL_DS);
+ /* Watchpoint */
+ if (num < 0) {
+ ret = compat_ptrace_hbp_get(NT_ARM_HW_WATCH, tsk, num, &kdata);
+ /* Resource info */
+ } else if (num == 0) {
+ ret = compat_ptrace_hbp_get_resource_info(&kdata);
+ /* Breakpoint */
+ } else {
+ ret = compat_ptrace_hbp_get(NT_ARM_HW_BREAK, tsk, num, &kdata);
+ }
+ set_fs(old_fs);
+
+ if (!ret)
+ ret = put_user(kdata, data);
+
+ return ret;
+}
+
+static int compat_ptrace_sethbpregs(struct task_struct *tsk, compat_long_t num,
+ compat_ulong_t __user *data)
+{
+ int ret;
+ u32 kdata = 0;
+ mm_segment_t old_fs = get_fs();
+
+ if (num == 0)
+ return 0;
+
+ ret = get_user(kdata, data);
+ if (ret)
+ return ret;
+
+ set_fs(KERNEL_DS);
+ if (num < 0)
+ ret = compat_ptrace_hbp_set(NT_ARM_HW_WATCH, tsk, num, &kdata);
+ else
+ ret = compat_ptrace_hbp_set(NT_ARM_HW_BREAK, tsk, num, &kdata);
+ set_fs(old_fs);
+
+ return ret;
+}
+#endif /* CONFIG_HAVE_HW_BREAKPOINT */
+
+long compat_arch_ptrace(struct task_struct *child, compat_long_t request,
+ compat_ulong_t caddr, compat_ulong_t cdata)
+{
+ unsigned long addr = caddr;
+ unsigned long data = cdata;
+ void __user *datap = compat_ptr(data);
+ int ret;
+
+ switch (request) {
+ case PTRACE_PEEKUSR:
+ ret = compat_ptrace_read_user(child, addr, datap);
+ break;
+
+ case PTRACE_POKEUSR:
+ ret = compat_ptrace_write_user(child, addr, data);
+ break;
+
+ case COMPAT_PTRACE_GETREGS:
+ ret = copy_regset_to_user(child,
+ &user_aarch32_view,
+ REGSET_COMPAT_GPR,
+ 0, sizeof(compat_elf_gregset_t),
+ datap);
+ break;
+
+ case COMPAT_PTRACE_SETREGS:
+ ret = copy_regset_from_user(child,
+ &user_aarch32_view,
+ REGSET_COMPAT_GPR,
+ 0, sizeof(compat_elf_gregset_t),
+ datap);
+ break;
+
+ case COMPAT_PTRACE_GET_THREAD_AREA:
+ ret = put_user((compat_ulong_t)child->thread.tp_value,
+ (compat_ulong_t __user *)datap);
+ break;
+
+ case COMPAT_PTRACE_SET_SYSCALL:
+ task_pt_regs(child)->syscallno = data;
+ ret = 0;
+ break;
+
+ case COMPAT_PTRACE_GETVFPREGS:
+ ret = copy_regset_to_user(child,
+ &user_aarch32_view,
+ REGSET_COMPAT_VFP,
+ 0, VFP_STATE_SIZE,
+ datap);
+ break;
+
+ case COMPAT_PTRACE_SETVFPREGS:
+ ret = copy_regset_from_user(child,
+ &user_aarch32_view,
+ REGSET_COMPAT_VFP,
+ 0, VFP_STATE_SIZE,
+ datap);
+ break;
+
+#ifdef CONFIG_HAVE_HW_BREAKPOINT
+ case COMPAT_PTRACE_GETHBPREGS:
+ ret = compat_ptrace_gethbpregs(child, addr, datap);
+ break;
+
+ case COMPAT_PTRACE_SETHBPREGS:
+ ret = compat_ptrace_sethbpregs(child, addr, datap);
+ break;
+#endif
+
+ default:
+ ret = compat_ptrace_request(child, request, addr,
+ data);
+ break;
+ }
+
+ return ret;
+}
+#endif /* CONFIG_COMPAT */
+
+const struct user_regset_view *task_user_regset_view(struct task_struct *task)
+{
+#ifdef CONFIG_COMPAT
+ if (is_compat_thread(task_thread_info(task)))
+ return &user_aarch32_view;
+#endif
+ return &user_aarch64_view;
+}
+
+long arch_ptrace(struct task_struct *child, long request,
+ unsigned long addr, unsigned long data)
+{
+ return ptrace_request(child, request, addr, data);
+}
+
+
+static int __init ptrace_break_init(void)
+{
+ hook_debug_fault_code(DBG_ESR_EVT_BRK, arm64_break_trap, SIGTRAP,
+ TRAP_BRKPT, "ptrace BRK handler");
+ return 0;
+}
+core_initcall(ptrace_break_init);
+
+
+asmlinkage int syscall_trace(int dir, struct pt_regs *regs)
+{
+ unsigned long saved_reg;
+
+ if (!test_thread_flag(TIF_SYSCALL_TRACE))
+ return regs->syscallno;
+
+ if (is_compat_task()) {
+ /* AArch32 uses ip (r12) for scratch */
+ saved_reg = regs->regs[12];
+ regs->regs[12] = dir;
+ } else {
+ /*
+ * Save X7. X7 is used to denote syscall entry/exit:
+ * X7 = 0 -> entry, = 1 -> exit
+ */
+ saved_reg = regs->regs[7];
+ regs->regs[7] = dir;
+ }
+
+ if (dir)
+ tracehook_report_syscall_exit(regs, 0);
+ else if (tracehook_report_syscall_entry(regs))
+ regs->syscallno = ~0UL;
+
+ if (is_compat_task())
+ regs->regs[12] = saved_reg;
+ else
+ regs->regs[7] = saved_reg;
+
+ return regs->syscallno;
+}
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
new file mode 100644
index 0000000..48ffb9f
--- /dev/null
+++ b/arch/arm64/kernel/setup.c
@@ -0,0 +1,347 @@
+/*
+ * Based on arch/arm/kernel/setup.c
+ *
+ * Copyright (C) 1995-2001 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/kernel.h>
+#include <linux/stddef.h>
+#include <linux/ioport.h>
+#include <linux/delay.h>
+#include <linux/utsname.h>
+#include <linux/initrd.h>
+#include <linux/console.h>
+#include <linux/bootmem.h>
+#include <linux/seq_file.h>
+#include <linux/screen_info.h>
+#include <linux/init.h>
+#include <linux/kexec.h>
+#include <linux/crash_dump.h>
+#include <linux/root_dev.h>
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/smp.h>
+#include <linux/fs.h>
+#include <linux/proc_fs.h>
+#include <linux/memblock.h>
+#include <linux/of_fdt.h>
+
+#include <asm/cputype.h>
+#include <asm/elf.h>
+#include <asm/cputable.h>
+#include <asm/sections.h>
+#include <asm/setup.h>
+#include <asm/cacheflush.h>
+#include <asm/tlbflush.h>
+#include <asm/traps.h>
+#include <asm/memblock.h>
+
+unsigned int processor_id;
+EXPORT_SYMBOL(processor_id);
+
+unsigned int elf_hwcap __read_mostly;
+EXPORT_SYMBOL_GPL(elf_hwcap);
+
+static const char *cpu_name;
+static const char *machine_name;
+phys_addr_t __fdt_pointer __initdata;
+
+/*
+ * Standard memory resources
+ */
+static struct resource mem_res[] = {
+ {
+ .name = "Kernel code",
+ .start = 0,
+ .end = 0,
+ .flags = IORESOURCE_MEM
+ },
+ {
+ .name = "Kernel data",
+ .start = 0,
+ .end = 0,
+ .flags = IORESOURCE_MEM
+ }
+};
+
+#define kernel_code mem_res[0]
+#define kernel_data mem_res[1]
+
+void __init early_print(const char *str, ...)
+{
+ char buf[256];
+ va_list ap;
+
+ va_start(ap, str);
+ vsnprintf(buf, sizeof(buf), str, ap);
+ va_end(ap);
+
+ printk("%s", buf);
+}
+
+static void __init setup_processor(void)
+{
+ struct cpu_info *cpu_info;
+
+ /*
+ * locate processor in the list of supported processor
+ * types. The linker builds this table for us from the
+ * entries in arch/arm/mm/proc.S
+ */
+ cpu_info = lookup_processor_type(read_cpuid_id());
+ if (!cpu_info) {
+ printk("CPU configuration botched (ID %08x), unable to continue.\n",
+ read_cpuid_id());
+ while (1);
+ }
+
+ cpu_name = cpu_info->cpu_name;
+
+ printk("CPU: %s [%08x] revision %d\n",
+ cpu_name, read_cpuid_id(), read_cpuid_id() & 15);
+
+ sprintf(init_utsname()->machine, "aarch64");
+ elf_hwcap = 0;
+}
+
+static void __init setup_machine_fdt(phys_addr_t dt_phys)
+{
+ struct boot_param_header *devtree;
+ unsigned long dt_root;
+
+ /* Check we have a non-NULL DT pointer */
+ if (!dt_phys) {
+ early_print("\n"
+ "Error: NULL or invalid device tree blob\n"
+ "The dtb must be 8-byte aligned and passed in the first 512MB of memory\n"
+ "\nPlease check your bootloader.\n");
+
+ while (true)
+ cpu_relax();
+
+ }
+
+ devtree = phys_to_virt(dt_phys);
+
+ /* Check device tree validity */
+ if (be32_to_cpu(devtree->magic) != OF_DT_HEADER) {
+ early_print("\n"
+ "Error: invalid device tree blob at physical address 0x%p (virtual address 0x%p)\n"
+ "Expected 0x%x, found 0x%x\n"
+ "\nPlease check your bootloader.\n",
+ dt_phys, devtree, OF_DT_HEADER,
+ be32_to_cpu(devtree->magic));
+
+ while (true)
+ cpu_relax();
+ }
+
+ initial_boot_params = devtree;
+ dt_root = of_get_flat_dt_root();
+
+ machine_name = of_get_flat_dt_prop(dt_root, "model", NULL);
+ if (!machine_name)
+ machine_name = of_get_flat_dt_prop(dt_root, "compatible", NULL);
+ if (!machine_name)
+ machine_name = "<unknown>";
+ pr_info("Machine: %s\n", machine_name);
+
+ /* Retrieve various information from the /chosen node */
+ of_scan_flat_dt(early_init_dt_scan_chosen, boot_command_line);
+ /* Initialize {size,address}-cells info */
+ of_scan_flat_dt(early_init_dt_scan_root, NULL);
+ /* Setup memory, calling early_init_dt_add_memory_arch */
+ of_scan_flat_dt(early_init_dt_scan_memory, NULL);
+}
+
+void __init early_init_dt_add_memory_arch(u64 base, u64 size)
+{
+ size &= PAGE_MASK;
+ memblock_add(base, size);
+}
+
+void * __init early_init_dt_alloc_memory_arch(u64 size, u64 align)
+{
+ return __va(memblock_alloc(size, align));
+}
+
+/*
+ * Limit the memory size that was specified via FDT.
+ */
+static int __init early_mem(char *p)
+{
+ phys_addr_t limit;
+
+ if (!p)
+ return 1;
+
+ limit = memparse(p, &p) & PAGE_MASK;
+ pr_notice("Memory limited to %lldMB\n", limit >> 20);
+
+ memblock_enforce_memory_limit(limit);
+
+ return 0;
+}
+early_param("mem", early_mem);
+
+static void __init request_standard_resources(void)
+{
+ struct memblock_region *region;
+ struct resource *res;
+
+ kernel_code.start = virt_to_phys(_text);
+ kernel_code.end = virt_to_phys(_etext - 1);
+ kernel_data.start = virt_to_phys(_sdata);
+ kernel_data.end = virt_to_phys(_end - 1);
+
+ for_each_memblock(memory, region) {
+ res = alloc_bootmem_low(sizeof(*res));
+ res->name = "System RAM";
+ res->start = __pfn_to_phys(memblock_region_memory_base_pfn(region));
+ res->end = __pfn_to_phys(memblock_region_memory_end_pfn(region)) - 1;
+ res->flags = IORESOURCE_MEM | IORESOURCE_BUSY;
+
+ request_resource(&iomem_resource, res);
+
+ if (kernel_code.start >= res->start &&
+ kernel_code.end <= res->end)
+ request_resource(res, &kernel_code);
+ if (kernel_data.start >= res->start &&
+ kernel_data.end <= res->end)
+ request_resource(res, &kernel_data);
+ }
+}
+
+void __init setup_arch(char **cmdline_p)
+{
+ setup_processor();
+
+ setup_machine_fdt(__fdt_pointer);
+
+ init_mm.start_code = (unsigned long) _text;
+ init_mm.end_code = (unsigned long) _etext;
+ init_mm.end_data = (unsigned long) _edata;
+ init_mm.brk = (unsigned long) _end;
+
+ *cmdline_p = boot_command_line;
+
+ parse_early_param();
+
+ arm64_memblock_init();
+
+ paging_init();
+ request_standard_resources();
+
+ unflatten_device_tree();
+
+#ifdef CONFIG_SMP
+ smp_init_cpus();
+#endif
+
+#ifdef CONFIG_VT
+#if defined(CONFIG_VGA_CONSOLE)
+ conswitchp = &vga_con;
+#elif defined(CONFIG_DUMMY_CONSOLE)
+ conswitchp = &dummy_con;
+#endif
+#endif
+}
+
+static DEFINE_PER_CPU(struct cpu, cpu_data);
+
+static int __init topology_init(void)
+{
+ int i;
+
+ for_each_possible_cpu(i) {
+ struct cpu *cpu = &per_cpu(cpu_data, i);
+ cpu->hotpluggable = 1;
+ register_cpu(cpu, i);
+ }
+
+ return 0;
+}
+subsys_initcall(topology_init);
+
+static const char *hwcap_str[] = {
+ "fp",
+ "asimd",
+ NULL
+};
+
+static int c_show(struct seq_file *m, void *v)
+{
+ int i;
+
+ seq_printf(m, "Processor\t: %s rev %d (%s)\n",
+ cpu_name, read_cpuid_id() & 15, ELF_PLATFORM);
+
+ for_each_online_cpu(i) {
+ /*
+ * glibc reads /proc/cpuinfo to determine the number of
+ * online processors, looking for lines beginning with
+ * "processor". Give glibc what it expects.
+ */
+#ifdef CONFIG_SMP
+ seq_printf(m, "processor\t: %d\n", i);
+#endif
+ seq_printf(m, "BogoMIPS\t: %lu.%02lu\n\n",
+ loops_per_jiffy / (500000UL/HZ),
+ loops_per_jiffy / (5000UL/HZ) % 100);
+ }
+
+ /* dump out the processor features */
+ seq_puts(m, "Features\t: ");
+
+ for (i = 0; hwcap_str[i]; i++)
+ if (elf_hwcap & (1 << i))
+ seq_printf(m, "%s ", hwcap_str[i]);
+
+ seq_printf(m, "\nCPU implementer\t: 0x%02x\n", read_cpuid_id() >> 24);
+ seq_printf(m, "CPU architecture: AArch64\n");
+ seq_printf(m, "CPU variant\t: 0x%x\n", (read_cpuid_id() >> 20) & 15);
+ seq_printf(m, "CPU part\t: 0x%03x\n", (read_cpuid_id() >> 4) & 0xfff);
+ seq_printf(m, "CPU revision\t: %d\n", read_cpuid_id() & 15);
+
+ seq_puts(m, "\n");
+
+ seq_printf(m, "Hardware\t: %s\n", machine_name);
+
+ return 0;
+}
+
+static void *c_start(struct seq_file *m, loff_t *pos)
+{
+ return *pos < 1 ? (void *)1 : NULL;
+}
+
+static void *c_next(struct seq_file *m, void *v, loff_t *pos)
+{
+ ++*pos;
+ return NULL;
+}
+
+static void c_stop(struct seq_file *m, void *v)
+{
+}
+
+const struct seq_operations cpuinfo_op = {
+ .start = c_start,
+ .next = c_next,
+ .stop = c_stop,
+ .show = c_show
+};
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
new file mode 100644
index 0000000..8807ba2
--- /dev/null
+++ b/arch/arm64/kernel/signal.c
@@ -0,0 +1,437 @@
+/*
+ * Based on arch/arm/kernel/signal.c
+ *
+ * Copyright (C) 1995-2009 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/errno.h>
+#include <linux/signal.h>
+#include <linux/personality.h>
+#include <linux/freezer.h>
+#include <linux/uaccess.h>
+#include <linux/tracehook.h>
+#include <linux/ratelimit.h>
+
+#include <asm/compat.h>
+#include <asm/debug-monitors.h>
+#include <asm/elf.h>
+#include <asm/cacheflush.h>
+#include <asm/ucontext.h>
+#include <asm/unistd.h>
+#include <asm/fpsimd.h>
+#include <asm/signal32.h>
+#include <asm/vdso.h>
+
+/*
+ * Do a signal return; undo the signal stack. These are aligned to 128-bit.
+ */
+struct rt_sigframe {
+ struct siginfo info;
+ struct ucontext uc;
+};
+
+static int preserve_fpsimd_context(struct fpsimd_context __user *ctx)
+{
+ struct fpsimd_state *fpsimd = ¤t->thread.fpsimd_state;
+ int err;
+
+ /* dump the hardware registers to the fpsimd_state structure */
+ fpsimd_save_state(fpsimd);
+
+ /* copy the FP and status/control registers */
+ err = __copy_to_user(ctx->vregs, fpsimd->vregs, sizeof(fpsimd->vregs));
+ __put_user_error(fpsimd->fpsr, &ctx->fpsr, err);
+ __put_user_error(fpsimd->fpcr, &ctx->fpcr, err);
+
+ /* copy the magic/size information */
+ __put_user_error(FPSIMD_MAGIC, &ctx->head.magic, err);
+ __put_user_error(sizeof(struct fpsimd_context), &ctx->head.size, err);
+
+ return err ? -EFAULT : 0;
+}
+
+static int restore_fpsimd_context(struct fpsimd_context __user *ctx)
+{
+ struct fpsimd_state fpsimd;
+ __u32 magic, size;
+ int err = 0;
+
+ /* check the magic/size information */
+ __get_user_error(magic, &ctx->head.magic, err);
+ __get_user_error(size, &ctx->head.size, err);
+ if (err)
+ return -EFAULT;
+ if (magic != FPSIMD_MAGIC || size != sizeof(struct fpsimd_context))
+ return -EINVAL;
+
+ /* copy the FP and status/control registers */
+ err = __copy_from_user(fpsimd.vregs, ctx->vregs,
+ sizeof(fpsimd.vregs));
+ __get_user_error(fpsimd.fpsr, &ctx->fpsr, err);
+ __get_user_error(fpsimd.fpcr, &ctx->fpcr, err);
+
+ /* load the hardware registers from the fpsimd_state structure */
+ if (!err) {
+ preempt_disable();
+ fpsimd_load_state(&fpsimd);
+ preempt_enable();
+ }
+
+ return err ? -EFAULT : 0;
+}
+
+static int restore_sigframe(struct pt_regs *regs,
+ struct rt_sigframe __user *sf)
+{
+ sigset_t set;
+ int i, err;
+ struct aux_context __user *aux =
+ (struct aux_context __user *)sf->uc.uc_mcontext.__reserved;
+
+ err = __copy_from_user(&set, &sf->uc.uc_sigmask, sizeof(set));
+ if (err == 0)
+ set_current_blocked(&set);
+
+ for (i = 0; i < 31; i++)
+ __get_user_error(regs->regs[i], &sf->uc.uc_mcontext.regs[i],
+ err);
+ __get_user_error(regs->sp, &sf->uc.uc_mcontext.sp, err);
+ __get_user_error(regs->pc, &sf->uc.uc_mcontext.pc, err);
+ __get_user_error(regs->pstate, &sf->uc.uc_mcontext.pstate, err);
+
+ /*
+ * Avoid sys_rt_sigreturn() restarting.
+ */
+ regs->syscallno = ~0UL;
+
+ err |= !valid_user_regs(®s->user_regs);
+
+ if (err == 0)
+ err |= restore_fpsimd_context(&aux->fpsimd);
+
+ return err;
+}
+
+asmlinkage long sys_rt_sigreturn(struct pt_regs *regs)
+{
+ struct rt_sigframe __user *frame;
+
+ /* Always make any pending restarted system calls return -EINTR */
+ current_thread_info()->restart_block.fn = do_no_restart_syscall;
+
+ /*
+ * Since we stacked the signal on a 128-bit boundary, then 'sp' should
+ * be word aligned here.
+ */
+ if (regs->sp & 15)
+ goto badframe;
+
+ frame = (struct rt_sigframe __user *)regs->sp;
+
+ if (!access_ok(VERIFY_READ, frame, sizeof (*frame)))
+ goto badframe;
+
+ if (restore_sigframe(regs, frame))
+ goto badframe;
+
+ if (do_sigaltstack(&frame->uc.uc_stack,
+ NULL, regs->sp) == -EFAULT)
+ goto badframe;
+
+ return regs->regs[0];
+
+badframe:
+ if (show_unhandled_signals)
+ pr_info_ratelimited("%s[%d]: bad frame in %s: pc=%08llx sp=%08llx\n",
+ current->comm, task_pid_nr(current), __func__,
+ regs->pc, regs->sp);
+ force_sig(SIGSEGV, current);
+ return 0;
+}
+
+asmlinkage long sys_sigaltstack(const stack_t __user *uss, stack_t __user *uoss,
+ unsigned long sp)
+{
+ return do_sigaltstack(uss, uoss, sp);
+}
+
+static int setup_sigframe(struct rt_sigframe __user *sf,
+ struct pt_regs *regs, sigset_t *set)
+{
+ int i, err = 0;
+ struct aux_context __user *aux =
+ (struct aux_context __user *)sf->uc.uc_mcontext.__reserved;
+
+ for (i = 0; i < 31; i++)
+ __put_user_error(regs->regs[i], &sf->uc.uc_mcontext.regs[i],
+ err);
+ __put_user_error(regs->sp, &sf->uc.uc_mcontext.sp, err);
+ __put_user_error(regs->pc, &sf->uc.uc_mcontext.pc, err);
+ __put_user_error(regs->pstate, &sf->uc.uc_mcontext.pstate, err);
+
+ __put_user_error(current->thread.fault_address, &sf->uc.uc_mcontext.fault_address, err);
+
+ err |= __copy_to_user(&sf->uc.uc_sigmask, set, sizeof(*set));
+
+ if (err == 0)
+ err |= preserve_fpsimd_context(&aux->fpsimd);
+
+ /* set the "end" magic */
+ __put_user_error(0, &aux->end.magic, err);
+ __put_user_error(0, &aux->end.size, err);
+
+ return err;
+}
+
+static void __user *get_sigframe(struct k_sigaction *ka, struct pt_regs *regs,
+ int framesize)
+{
+ unsigned long sp, sp_top;
+ void __user *frame;
+
+ sp = sp_top = regs->sp;
+
+ /*
+ * This is the X/Open sanctioned signal stack switching.
+ */
+ if ((ka->sa.sa_flags & SA_ONSTACK) && !sas_ss_flags(sp))
+ sp = sp_top = current->sas_ss_sp + current->sas_ss_size;
+
+ /* room for stack frame (FP, LR) */
+ sp -= 16;
+
+ sp = (sp - framesize) & ~15;
+ frame = (void __user *)sp;
+
+ /*
+ * Check that we can actually write to the signal frame.
+ */
+ if (!access_ok(VERIFY_WRITE, frame, sp_top - sp))
+ frame = NULL;
+
+ return frame;
+}
+
+static int setup_return(struct pt_regs *regs, struct k_sigaction *ka,
+ void __user *frame, int usig)
+{
+ int err = 0;
+ __sigrestore_t sigtramp;
+ unsigned long __user *sp = (unsigned long __user *)regs->sp;
+
+ /* set up the stack frame */
+ __put_user_error(regs->regs[29], sp - 2, err);
+ __put_user_error(regs->regs[30], sp - 1, err);
+
+ regs->regs[0] = usig;
+ regs->regs[29] = regs->sp - 16;
+ regs->sp = (unsigned long)frame;
+ regs->pc = (unsigned long)ka->sa.sa_handler;
+
+ if (ka->sa.sa_flags & SA_RESTORER)
+ sigtramp = ka->sa.sa_restorer;
+ else
+ sigtramp = VDSO_SYMBOL(current->mm->context.vdso, sigtramp);
+
+ regs->regs[30] = (unsigned long)sigtramp;
+
+ return err;
+}
+
+static int setup_rt_frame(int usig, struct k_sigaction *ka, siginfo_t *info,
+ sigset_t *set, struct pt_regs *regs)
+{
+ struct rt_sigframe __user *frame;
+ stack_t stack;
+ int err = 0;
+
+ frame = get_sigframe(ka, regs, sizeof(*frame));
+ if (!frame)
+ return 1;
+
+ __put_user_error(0, &frame->uc.uc_flags, err);
+ __put_user_error(NULL, &frame->uc.uc_link, err);
+
+ memset(&stack, 0, sizeof(stack));
+ stack.ss_sp = (void __user *)current->sas_ss_sp;
+ stack.ss_flags = sas_ss_flags(regs->sp);
+ stack.ss_size = current->sas_ss_size;
+ err |= __copy_to_user(&frame->uc.uc_stack, &stack, sizeof(stack));
+
+ err |= setup_sigframe(frame, regs, set);
+ if (err == 0)
+ err = setup_return(regs, ka, frame, usig);
+
+ if (err == 0 && ka->sa.sa_flags & SA_SIGINFO) {
+ err |= copy_siginfo_to_user(&frame->info, info);
+ regs->regs[1] = (unsigned long)&frame->info;
+ regs->regs[2] = (unsigned long)&frame->uc;
+ }
+
+ return err;
+}
+
+static void setup_restart_syscall(struct pt_regs *regs)
+{
+ if (is_compat_task())
+ compat_setup_restart_syscall(regs);
+ else
+ regs->regs[8] = __NR_restart_syscall;
+}
+
+/*
+ * OK, we're invoking a handler
+ */
+static void handle_signal(unsigned long sig, struct k_sigaction *ka,
+ siginfo_t *info, struct pt_regs *regs)
+{
+ struct thread_info *thread = current_thread_info();
+ struct task_struct *tsk = current;
+ sigset_t *oldset = sigmask_to_save();
+ int usig = sig;
+ int ret;
+
+ /*
+ * translate the signal
+ */
+ if (usig < 32 && thread->exec_domain && thread->exec_domain->signal_invmap)
+ usig = thread->exec_domain->signal_invmap[usig];
+
+ /*
+ * Set up the stack frame
+ */
+ if (is_compat_task()) {
+ if (ka->sa.sa_flags & SA_SIGINFO)
+ ret = compat_setup_rt_frame(usig, ka, info, oldset,
+ regs);
+ else
+ ret = compat_setup_frame(usig, ka, oldset, regs);
+ } else {
+ ret = setup_rt_frame(usig, ka, info, oldset, regs);
+ }
+
+ /*
+ * Check that the resulting registers are actually sane.
+ */
+ ret |= !valid_user_regs(®s->user_regs);
+
+ if (ret != 0) {
+ force_sigsegv(sig, tsk);
+ return;
+ }
+
+ /*
+ * Fast forward the stepping logic so we step into the signal
+ * handler.
+ */
+ user_fastforward_single_step(tsk);
+
+ signal_delivered(sig, info, ka, regs, 0);
+}
+
+/*
+ * Note that 'init' is a special process: it doesn't get signals it doesn't
+ * want to handle. Thus you cannot kill init even with a SIGKILL even by
+ * mistake.
+ *
+ * Note that we go through the signals twice: once to check the signals that
+ * the kernel can handle, and then we build all the user-level signal handling
+ * stack-frames in one go after that.
+ */
+static void do_signal(struct pt_regs *regs)
+{
+ unsigned long continue_addr = 0, restart_addr = 0;
+ struct k_sigaction ka;
+ siginfo_t info;
+ int signr, retval = 0;
+ int syscall = (int)regs->syscallno;
+
+ /*
+ * If we were from a system call, check for system call restarting...
+ */
+ if (syscall >= 0) {
+ continue_addr = regs->pc;
+ restart_addr = continue_addr - (compat_thumb_mode(regs) ? 2 : 4);
+ retval = regs->regs[0];
+
+ /*
+ * Avoid additional syscall restarting via ret_to_user.
+ */
+ regs->syscallno = ~0UL;
+
+ /*
+ * Prepare for system call restart. We do this here so that a
+ * debugger will see the already changed PC.
+ */
+ switch (retval) {
+ case -ERESTARTNOHAND:
+ case -ERESTARTSYS:
+ case -ERESTARTNOINTR:
+ case -ERESTART_RESTARTBLOCK:
+ regs->regs[0] = regs->orig_x0;
+ regs->pc = restart_addr;
+ break;
+ }
+ }
+
+ /*
+ * Get the signal to deliver. When running under ptrace, at this point
+ * the debugger may change all of our registers.
+ */
+ signr = get_signal_to_deliver(&info, &ka, regs, NULL);
+ if (signr > 0) {
+ /*
+ * Depending on the signal settings, we may need to revert the
+ * decision to restart the system call, but skip this if a
+ * debugger has chosen to restart at a different PC.
+ */
+ if (regs->pc == restart_addr &&
+ (retval == -ERESTARTNOHAND ||
+ retval == -ERESTART_RESTARTBLOCK ||
+ (retval == -ERESTARTSYS &&
+ !(ka.sa.sa_flags & SA_RESTART)))) {
+ regs->regs[0] = -EINTR;
+ regs->pc = continue_addr;
+ }
+
+ handle_signal(signr, &ka, &info, regs);
+ return;
+ }
+
+ /*
+ * Handle restarting a different system call. As above, if a debugger
+ * has chosen to restart at a different PC, ignore the restart.
+ */
+ if (syscall >= 0 && regs->pc == restart_addr) {
+ if (retval == -ERESTART_RESTARTBLOCK)
+ setup_restart_syscall(regs);
+ user_rewind_single_step(current);
+ }
+
+ restore_saved_sigmask();
+}
+
+asmlinkage void do_notify_resume(struct pt_regs *regs,
+ unsigned int thread_flags)
+{
+ if (thread_flags & _TIF_SIGPENDING)
+ do_signal(regs);
+
+ if (thread_flags & _TIF_NOTIFY_RESUME) {
+ clear_thread_flag(TIF_NOTIFY_RESUME);
+ tracehook_notify_resume(regs);
+ }
+}
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
new file mode 100644
index 0000000..ac74c2f
--- /dev/null
+++ b/arch/arm64/kernel/signal32.c
@@ -0,0 +1,876 @@
+/*
+ * Based on arch/arm/kernel/signal.c
+ *
+ * Copyright (C) 1995-2009 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ * Modified by Will Deacon <will.deacon@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#define __SYSCALL_COMPAT
+
+#include <linux/compat.h>
+#include <linux/signal.h>
+#include <linux/syscalls.h>
+#include <linux/ratelimit.h>
+
+#include <asm/fpsimd.h>
+#include <asm/signal32.h>
+#include <asm/uaccess.h>
+#include <asm/unistd.h>
+
+typedef struct compat_siginfo {
+ int si_signo;
+ int si_errno;
+ int si_code;
+
+ union {
+ /* The padding is the same size as AArch64. */
+ int _pad[SI_PAD_SIZE];
+
+ /* kill() */
+ struct {
+ compat_pid_t _pid; /* sender's pid */
+ __compat_uid32_t _uid; /* sender's uid */
+ } _kill;
+
+ /* POSIX.1b timers */
+ struct {
+ compat_timer_t _tid; /* timer id */
+ int _overrun; /* overrun count */
+ compat_sigval_t _sigval; /* same as below */
+ int _sys_private; /* not to be passed to user */
+ } _timer;
+
+ /* POSIX.1b signals */
+ struct {
+ compat_pid_t _pid; /* sender's pid */
+ __compat_uid32_t _uid; /* sender's uid */
+ compat_sigval_t _sigval;
+ } _rt;
+
+ /* SIGCHLD */
+ struct {
+ compat_pid_t _pid; /* which child */
+ __compat_uid32_t _uid; /* sender's uid */
+ int _status; /* exit code */
+ compat_clock_t _utime;
+ compat_clock_t _stime;
+ } _sigchld;
+
+ /* SIGILL, SIGFPE, SIGSEGV, SIGBUS */
+ struct {
+ compat_uptr_t _addr; /* faulting insn/memory ref. */
+ short _addr_lsb; /* LSB of the reported address */
+ } _sigfault;
+
+ /* SIGPOLL */
+ struct {
+ compat_long_t _band; /* POLL_IN, POLL_OUT, POLL_MSG */
+ int _fd;
+ } _sigpoll;
+ } _sifields;
+} compat_siginfo_t;
+
+struct compat_sigaction {
+ compat_uptr_t sa_handler;
+ compat_ulong_t sa_flags;
+ compat_uptr_t sa_restorer;
+ compat_sigset_t sa_mask;
+};
+
+struct compat_old_sigaction {
+ compat_uptr_t sa_handler;
+ compat_old_sigset_t sa_mask;
+ compat_ulong_t sa_flags;
+ compat_uptr_t sa_restorer;
+};
+
+typedef struct compat_sigaltstack {
+ compat_uptr_t ss_sp;
+ int ss_flags;
+ compat_size_t ss_size;
+} compat_stack_t;
+
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ struct compat_ucontext *uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_vfp_sigframe {
+ compat_ulong_t magic;
+ compat_ulong_t size;
+ struct compat_user_vfp {
+ compat_u64 fpregs[32];
+ compat_ulong_t fpscr;
+ } ufp;
+ struct compat_user_vfp_exc {
+ compat_ulong_t fpexc;
+ compat_ulong_t fpinst;
+ compat_ulong_t fpinst2;
+ } ufp_exc;
+} __attribute__((__aligned__(8)));
+
+#define VFP_MAGIC 0x56465001
+#define VFP_STORAGE_SIZE sizeof(struct compat_vfp_sigframe)
+
+struct compat_aux_sigframe {
+ struct compat_vfp_sigframe vfp;
+
+ /* Something that isn't a valid magic number for any coprocessor. */
+ unsigned long end_magic;
+} __attribute__((__aligned__(8)));
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
+#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
+
+/*
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ */
+#define MOV_R7_NR_SIGRETURN (0xe3a07000 | __NR_sigreturn)
+#define SVC_SYS_SIGRETURN (0xef000000 | __NR_sigreturn)
+#define MOV_R7_NR_RT_SIGRETURN (0xe3a07000 | __NR_rt_sigreturn)
+#define SVC_SYS_RT_SIGRETURN (0xef000000 | __NR_rt_sigreturn)
+
+/*
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+#define SVC_THUMB_SIGRETURN (((0xdf00 | __NR_sigreturn) << 16) | \
+ 0x2700 | __NR_sigreturn)
+#define SVC_THUMB_RT_SIGRETURN (((0xdf00 | __NR_rt_sigreturn) << 16) | \
+ 0x2700 | __NR_rt_sigreturn)
+
+const compat_ulong_t aarch32_sigret_code[6] = {
+ /*
+ * AArch32 sigreturn code.
+ * We don't construct an OABI SWI - instead we just set the imm24 field
+ * to the EABI syscall number so that we create a sane disassembly.
+ */
+ MOV_R7_NR_SIGRETURN, SVC_SYS_SIGRETURN, SVC_THUMB_SIGRETURN,
+ MOV_R7_NR_RT_SIGRETURN, SVC_SYS_RT_SIGRETURN, SVC_THUMB_RT_SIGRETURN,
+};
+
+static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
+{
+ compat_sigset_t cset;
+
+ cset.sig[0] = set->sig[0] & 0xffffffffull;
+ cset.sig[1] = set->sig[0] >> 32;
+
+ return copy_to_user(uset, &cset, sizeof(*uset));
+}
+
+static inline int get_sigset_t(sigset_t *set,
+ const compat_sigset_t __user *uset)
+{
+ compat_sigset_t s32;
+
+ if (copy_from_user(&s32, uset, sizeof(*uset)))
+ return -EFAULT;
+
+ set->sig[0] = s32.sig[0] | (((long)s32.sig[1]) << 32);
+ return 0;
+}
+
+int copy_siginfo_to_user32(compat_siginfo_t __user *to, siginfo_t *from)
+{
+ int err;
+
+ if (!access_ok(VERIFY_WRITE, to, sizeof(*to)))
+ return -EFAULT;
+
+ /* If you change siginfo_t structure, please be sure
+ * this code is fixed accordingly.
+ * It should never copy any pad contained in the structure
+ * to avoid security leaks, but must copy the generic
+ * 3 ints plus the relevant union member.
+ * This routine must convert siginfo from 64bit to 32bit as well
+ * at the same time.
+ */
+ err = __put_user(from->si_signo, &to->si_signo);
+ err |= __put_user(from->si_errno, &to->si_errno);
+ err |= __put_user((short)from->si_code, &to->si_code);
+ if (from->si_code < 0)
+ err |= __copy_to_user(&to->_sifields._pad, &from->_sifields._pad,
+ SI_PAD_SIZE);
+ else switch (from->si_code & __SI_MASK) {
+ case __SI_KILL:
+ err |= __put_user(from->si_pid, &to->si_pid);
+ err |= __put_user(from->si_uid, &to->si_uid);
+ break;
+ case __SI_TIMER:
+ err |= __put_user(from->si_tid, &to->si_tid);
+ err |= __put_user(from->si_overrun, &to->si_overrun);
+ err |= __put_user((compat_uptr_t)(unsigned long)from->si_ptr,
+ &to->si_ptr);
+ break;
+ case __SI_POLL:
+ err |= __put_user(from->si_band, &to->si_band);
+ err |= __put_user(from->si_fd, &to->si_fd);
+ break;
+ case __SI_FAULT:
+ err |= __put_user((compat_uptr_t)(unsigned long)from->si_addr,
+ &to->si_addr);
+#ifdef BUS_MCEERR_AO
+ /*
+ * Other callers might not initialize the si_lsb field,
+ * so check explicitely for the right codes here.
+ */
+ if (from->si_code == BUS_MCEERR_AR || from->si_code == BUS_MCEERR_AO)
+ err |= __put_user(from->si_addr_lsb, &to->si_addr_lsb);
+#endif
+ break;
+ case __SI_CHLD:
+ err |= __put_user(from->si_pid, &to->si_pid);
+ err |= __put_user(from->si_uid, &to->si_uid);
+ err |= __put_user(from->si_status, &to->si_status);
+ err |= __put_user(from->si_utime, &to->si_utime);
+ err |= __put_user(from->si_stime, &to->si_stime);
+ break;
+ case __SI_RT: /* This is not generated by the kernel as of now. */
+ case __SI_MESGQ: /* But this is */
+ err |= __put_user(from->si_pid, &to->si_pid);
+ err |= __put_user(from->si_uid, &to->si_uid);
+ err |= __put_user((compat_uptr_t)(unsigned long)from->si_ptr, &to->si_ptr);
+ break;
+ default: /* this is just in case for now ... */
+ err |= __put_user(from->si_pid, &to->si_pid);
+ err |= __put_user(from->si_uid, &to->si_uid);
+ break;
+ }
+ return err;
+}
+
+int copy_siginfo_from_user32(siginfo_t *to, compat_siginfo_t __user *from)
+{
+ memset(to, 0, sizeof *to);
+
+ if (copy_from_user(to, from, __ARCH_SI_PREAMBLE_SIZE) ||
+ copy_from_user(to->_sifields._pad,
+ from->_sifields._pad, SI_PAD_SIZE))
+ return -EFAULT;
+
+ return 0;
+}
+
+/*
+ * VFP save/restore code.
+ */
+static int compat_preserve_vfp_context(struct compat_vfp_sigframe __user *frame)
+{
+ struct fpsimd_state *fpsimd = ¤t->thread.fpsimd_state;
+ compat_ulong_t magic = VFP_MAGIC;
+ compat_ulong_t size = VFP_STORAGE_SIZE;
+ compat_ulong_t fpscr, fpexc;
+ int err = 0;
+
+ /*
+ * Save the hardware registers to the fpsimd_state structure.
+ * Note that this also saves V16-31, which aren't visible
+ * in AArch32.
+ */
+ fpsimd_save_state(fpsimd);
+
+ /* Place structure header on the stack */
+ __put_user_error(magic, &frame->magic, err);
+ __put_user_error(size, &frame->size, err);
+
+ /*
+ * Now copy the FP registers. Since the registers are packed,
+ * we can copy the prefix we want (V0-V15) as it is.
+ * FIXME: Won't work if big endian.
+ */
+ err |= __copy_to_user(&frame->ufp.fpregs, fpsimd->vregs,
+ sizeof(frame->ufp.fpregs));
+
+ /* Create an AArch32 fpscr from the fpsr and the fpcr. */
+ fpscr = (fpsimd->fpsr & VFP_FPSCR_STAT_MASK) |
+ (fpsimd->fpcr & VFP_FPSCR_CTRL_MASK);
+ __put_user_error(fpscr, &frame->ufp.fpscr, err);
+
+ /*
+ * The exception register aren't available so we fake up a
+ * basic FPEXC and zero everything else.
+ */
+ fpexc = (1 << 30);
+ __put_user_error(fpexc, &frame->ufp_exc.fpexc, err);
+ __put_user_error(0, &frame->ufp_exc.fpinst, err);
+ __put_user_error(0, &frame->ufp_exc.fpinst2, err);
+
+ return err ? -EFAULT : 0;
+}
+
+static int compat_restore_vfp_context(struct compat_vfp_sigframe __user *frame)
+{
+ struct fpsimd_state fpsimd;
+ compat_ulong_t magic = VFP_MAGIC;
+ compat_ulong_t size = VFP_STORAGE_SIZE;
+ compat_ulong_t fpscr;
+ int err = 0;
+
+ __get_user_error(magic, &frame->magic, err);
+ __get_user_error(size, &frame->size, err);
+
+ if (err)
+ return -EFAULT;
+ if (magic != VFP_MAGIC || size != VFP_STORAGE_SIZE)
+ return -EINVAL;
+
+ /*
+ * Copy the FP registers into the start of the fpsimd_state.
+ * FIXME: Won't work if big endian.
+ */
+ err |= __copy_from_user(fpsimd.vregs, frame->ufp.fpregs,
+ sizeof(frame->ufp.fpregs));
+
+ /* Extract the fpsr and the fpcr from the fpscr */
+ __get_user_error(fpscr, &frame->ufp.fpscr, err);
+ fpsimd.fpsr = fpscr & VFP_FPSCR_STAT_MASK;
+ fpsimd.fpcr = fpscr & VFP_FPSCR_CTRL_MASK;
+
+ /*
+ * We don't need to touch the exception register, so
+ * reload the hardware state.
+ */
+ if (!err) {
+ preempt_disable();
+ fpsimd_load_state(&fpsimd);
+ preempt_enable();
+ }
+
+ return err ? -EFAULT : 0;
+}
+
+/*
+ * atomically swap in the new signal mask, and wait for a signal.
+ */
+asmlinkage int compat_sys_sigsuspend(int restart, compat_ulong_t oldmask,
+ compat_old_sigset_t mask)
+{
+ sigset_t blocked;
+
+ siginitset(¤t->blocked, mask);
+ return sigsuspend(&blocked);
+}
+
+asmlinkage int compat_sys_sigaction(int sig,
+ const struct compat_old_sigaction __user *act,
+ struct compat_old_sigaction __user *oact)
+{
+ struct k_sigaction new_ka, old_ka;
+ int ret;
+ compat_old_sigset_t mask;
+ compat_uptr_t handler, restorer;
+
+ if (act) {
+ if (!access_ok(VERIFY_READ, act, sizeof(*act)) ||
+ __get_user(handler, &act->sa_handler) ||
+ __get_user(restorer, &act->sa_restorer) ||
+ __get_user(new_ka.sa.sa_flags, &act->sa_flags) ||
+ __get_user(mask, &act->sa_mask))
+ return -EFAULT;
+
+ new_ka.sa.sa_handler = compat_ptr(handler);
+ new_ka.sa.sa_restorer = compat_ptr(restorer);
+ siginitset(&new_ka.sa.sa_mask, mask);
+ }
+
+ ret = do_sigaction(sig, act ? &new_ka : NULL, oact ? &old_ka : NULL);
+
+ if (!ret && oact) {
+ if (!access_ok(VERIFY_WRITE, oact, sizeof(*oact)) ||
+ __put_user(ptr_to_compat(old_ka.sa.sa_handler),
+ &oact->sa_handler) ||
+ __put_user(ptr_to_compat(old_ka.sa.sa_restorer),
+ &oact->sa_restorer) ||
+ __put_user(old_ka.sa.sa_flags, &oact->sa_flags) ||
+ __put_user(old_ka.sa.sa_mask.sig[0], &oact->sa_mask))
+ return -EFAULT;
+ }
+
+ return ret;
+}
+
+asmlinkage int compat_sys_rt_sigaction(int sig,
+ const struct compat_sigaction __user *act,
+ struct compat_sigaction __user *oact,
+ compat_size_t sigsetsize)
+{
+ struct k_sigaction new_ka, old_ka;
+ int ret;
+
+ /* XXX: Don't preclude handling different sized sigset_t's. */
+ if (sigsetsize != sizeof(compat_sigset_t))
+ return -EINVAL;
+
+ if (act) {
+ compat_uptr_t handler, restorer;
+
+ ret = get_user(handler, &act->sa_handler);
+ new_ka.sa.sa_handler = compat_ptr(handler);
+ ret |= get_user(restorer, &act->sa_restorer);
+ new_ka.sa.sa_restorer = compat_ptr(restorer);
+ ret |= get_sigset_t(&new_ka.sa.sa_mask, &act->sa_mask);
+ ret |= __get_user(new_ka.sa.sa_flags, &act->sa_flags);
+ if (ret)
+ return -EFAULT;
+ }
+
+ ret = do_sigaction(sig, act ? &new_ka : NULL, oact ? &old_ka : NULL);
+ if (!ret && oact) {
+ ret = put_user(ptr_to_compat(old_ka.sa.sa_handler), &oact->sa_handler);
+ ret |= put_sigset_t(&oact->sa_mask, &old_ka.sa.sa_mask);
+ ret |= __put_user(old_ka.sa.sa_flags, &oact->sa_flags);
+ }
+ return ret;
+}
+
+int compat_do_sigaltstack(compat_uptr_t compat_uss, compat_uptr_t compat_uoss,
+ compat_ulong_t sp)
+{
+ compat_stack_t __user *newstack = compat_ptr(compat_uss);
+ compat_stack_t __user *oldstack = compat_ptr(compat_uoss);
+ compat_uptr_t ss_sp;
+ int ret;
+ mm_segment_t old_fs;
+ stack_t uss, uoss;
+
+ /* Marshall the compat new stack into a stack_t */
+ if (newstack) {
+ if (get_user(ss_sp, &newstack->ss_sp) ||
+ __get_user(uss.ss_flags, &newstack->ss_flags) ||
+ __get_user(uss.ss_size, &newstack->ss_size))
+ return -EFAULT;
+ uss.ss_sp = compat_ptr(ss_sp);
+ }
+
+ old_fs = get_fs();
+ set_fs(KERNEL_DS);
+ /* The __user pointer casts are valid because of the set_fs() */
+ ret = do_sigaltstack(
+ newstack ? (stack_t __user *) &uss : NULL,
+ oldstack ? (stack_t __user *) &uoss : NULL,
+ (unsigned long)sp);
+ set_fs(old_fs);
+
+ /* Convert the old stack_t into a compat stack. */
+ if (!ret && oldstack &&
+ (put_user(ptr_to_compat(uoss.ss_sp), &oldstack->ss_sp) ||
+ __put_user(uoss.ss_flags, &oldstack->ss_flags) ||
+ __put_user(uoss.ss_size, &oldstack->ss_size)))
+ return -EFAULT;
+ return ret;
+}
+
+static int compat_restore_sigframe(struct pt_regs *regs,
+ struct compat_sigframe __user *sf)
+{
+ int err;
+ sigset_t set;
+ struct compat_aux_sigframe __user *aux;
+
+ err = get_sigset_t(&set, &sf->uc.uc_sigmask);
+ if (err == 0) {
+ sigdelsetmask(&set, ~_BLOCKABLE);
+ set_current_blocked(&set);
+ }
+
+ __get_user_error(regs->regs[0], &sf->uc.uc_mcontext.arm_r0, err);
+ __get_user_error(regs->regs[1], &sf->uc.uc_mcontext.arm_r1, err);
+ __get_user_error(regs->regs[2], &sf->uc.uc_mcontext.arm_r2, err);
+ __get_user_error(regs->regs[3], &sf->uc.uc_mcontext.arm_r3, err);
+ __get_user_error(regs->regs[4], &sf->uc.uc_mcontext.arm_r4, err);
+ __get_user_error(regs->regs[5], &sf->uc.uc_mcontext.arm_r5, err);
+ __get_user_error(regs->regs[6], &sf->uc.uc_mcontext.arm_r6, err);
+ __get_user_error(regs->regs[7], &sf->uc.uc_mcontext.arm_r7, err);
+ __get_user_error(regs->regs[8], &sf->uc.uc_mcontext.arm_r8, err);
+ __get_user_error(regs->regs[9], &sf->uc.uc_mcontext.arm_r9, err);
+ __get_user_error(regs->regs[10], &sf->uc.uc_mcontext.arm_r10, err);
+ __get_user_error(regs->regs[11], &sf->uc.uc_mcontext.arm_fp, err);
+ __get_user_error(regs->regs[12], &sf->uc.uc_mcontext.arm_ip, err);
+ __get_user_error(regs->compat_sp, &sf->uc.uc_mcontext.arm_sp, err);
+ __get_user_error(regs->compat_lr, &sf->uc.uc_mcontext.arm_lr, err);
+ __get_user_error(regs->pc, &sf->uc.uc_mcontext.arm_pc, err);
+ __get_user_error(regs->pstate, &sf->uc.uc_mcontext.arm_cpsr, err);
+
+ /*
+ * Avoid compat_sys_sigreturn() restarting.
+ */
+ regs->syscallno = ~0UL;
+
+ err |= !valid_user_regs(®s->user_regs);
+
+ aux = (struct compat_aux_sigframe __user *) sf->uc.uc_regspace;
+ if (err == 0)
+ err |= compat_restore_vfp_context(&aux->vfp);
+
+ return err;
+}
+
+asmlinkage int compat_sys_sigreturn(struct pt_regs *regs)
+{
+ struct compat_sigframe __user *frame;
+
+ /* Always make any pending restarted system calls return -EINTR */
+ current_thread_info()->restart_block.fn = do_no_restart_syscall;
+
+ /*
+ * Since we stacked the signal on a 64-bit boundary,
+ * then 'sp' should be word aligned here. If it's
+ * not, then the user is trying to mess with us.
+ */
+ if (regs->compat_sp & 7)
+ goto badframe;
+
+ frame = (struct compat_sigframe __user *)regs->compat_sp;
+
+ if (!access_ok(VERIFY_READ, frame, sizeof (*frame)))
+ goto badframe;
+
+ if (compat_restore_sigframe(regs, frame))
+ goto badframe;
+
+ return regs->regs[0];
+
+badframe:
+ if (show_unhandled_signals)
+ pr_info_ratelimited("%s[%d]: bad frame in %s: pc=%08llx sp=%08llx\n",
+ current->comm, task_pid_nr(current), __func__,
+ regs->pc, regs->sp);
+ force_sig(SIGSEGV, current);
+ return 0;
+}
+
+asmlinkage int compat_sys_rt_sigreturn(struct pt_regs *regs)
+{
+ struct compat_rt_sigframe __user *frame;
+
+ /* Always make any pending restarted system calls return -EINTR */
+ current_thread_info()->restart_block.fn = do_no_restart_syscall;
+
+ /*
+ * Since we stacked the signal on a 64-bit boundary,
+ * then 'sp' should be word aligned here. If it's
+ * not, then the user is trying to mess with us.
+ */
+ if (regs->compat_sp & 7)
+ goto badframe;
+
+ frame = (struct compat_rt_sigframe __user *)regs->compat_sp;
+
+ if (!access_ok(VERIFY_READ, frame, sizeof (*frame)))
+ goto badframe;
+
+ if (compat_restore_sigframe(regs, &frame->sig))
+ goto badframe;
+
+ if (compat_do_sigaltstack(ptr_to_compat(&frame->sig.uc.uc_stack),
+ ptr_to_compat((void __user *)NULL),
+ regs->compat_sp) == -EFAULT)
+ goto badframe;
+
+ return regs->regs[0];
+
+badframe:
+ if (show_unhandled_signals)
+ pr_info_ratelimited("%s[%d]: bad frame in %s: pc=%08llx sp=%08llx\n",
+ current->comm, task_pid_nr(current), __func__,
+ regs->pc, regs->sp);
+ force_sig(SIGSEGV, current);
+ return 0;
+}
+
+static inline void __user *compat_get_sigframe(struct k_sigaction *ka,
+ struct pt_regs *regs,
+ int framesize)
+{
+ compat_ulong_t sp = regs->compat_sp;
+ void __user *frame;
+
+ /*
+ * This is the X/Open sanctioned signal stack switching.
+ */
+ if ((ka->sa.sa_flags & SA_ONSTACK) && !sas_ss_flags(sp))
+ sp = current->sas_ss_sp + current->sas_ss_size;
+
+ /*
+ * ATPCS B01 mandates 8-byte alignment
+ */
+ frame = compat_ptr((compat_uptr_t)((sp - framesize) & ~7));
+
+ /*
+ * Check that we can actually write to the signal frame.
+ */
+ if (!access_ok(VERIFY_WRITE, frame, framesize))
+ frame = NULL;
+
+ return frame;
+}
+
+static int compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
+ compat_ulong_t __user *rc, void __user *frame,
+ int usig)
+{
+ compat_ulong_t handler = ptr_to_compat(ka->sa.sa_handler);
+ compat_ulong_t retcode;
+ compat_ulong_t spsr = regs->pstate & ~PSR_f;
+ int thumb;
+
+ /* Check if the handler is written for ARM or Thumb */
+ thumb = handler & 1;
+
+ if (thumb) {
+ spsr |= COMPAT_PSR_T_BIT;
+ spsr &= ~COMPAT_PSR_IT_MASK;
+ } else {
+ spsr &= ~COMPAT_PSR_T_BIT;
+ }
+
+ if (ka->sa.sa_flags & SA_RESTORER) {
+ retcode = ptr_to_compat(ka->sa.sa_restorer);
+ } else {
+ /* Set up sigreturn pointer */
+ unsigned int idx = thumb << 1;
+
+ if (ka->sa.sa_flags & SA_SIGINFO)
+ idx += 3;
+
+ retcode = AARCH32_VECTORS_BASE +
+ AARCH32_KERN_SIGRET_CODE_OFFSET +
+ (idx << 2) + thumb;
+ }
+
+ regs->regs[0] = usig;
+ regs->compat_sp = ptr_to_compat(frame);
+ regs->compat_lr = retcode;
+ regs->pc = handler;
+ regs->pstate = spsr;
+
+ return 0;
+}
+
+static int compat_setup_sigframe(struct compat_sigframe __user *sf,
+ struct pt_regs *regs, sigset_t *set)
+{
+ struct compat_aux_sigframe __user *aux;
+ int err = 0;
+
+ __put_user_error(regs->regs[0], &sf->uc.uc_mcontext.arm_r0, err);
+ __put_user_error(regs->regs[1], &sf->uc.uc_mcontext.arm_r1, err);
+ __put_user_error(regs->regs[2], &sf->uc.uc_mcontext.arm_r2, err);
+ __put_user_error(regs->regs[3], &sf->uc.uc_mcontext.arm_r3, err);
+ __put_user_error(regs->regs[4], &sf->uc.uc_mcontext.arm_r4, err);
+ __put_user_error(regs->regs[5], &sf->uc.uc_mcontext.arm_r5, err);
+ __put_user_error(regs->regs[6], &sf->uc.uc_mcontext.arm_r6, err);
+ __put_user_error(regs->regs[7], &sf->uc.uc_mcontext.arm_r7, err);
+ __put_user_error(regs->regs[8], &sf->uc.uc_mcontext.arm_r8, err);
+ __put_user_error(regs->regs[9], &sf->uc.uc_mcontext.arm_r9, err);
+ __put_user_error(regs->regs[10], &sf->uc.uc_mcontext.arm_r10, err);
+ __put_user_error(regs->regs[11], &sf->uc.uc_mcontext.arm_fp, err);
+ __put_user_error(regs->regs[12], &sf->uc.uc_mcontext.arm_ip, err);
+ __put_user_error(regs->compat_sp, &sf->uc.uc_mcontext.arm_sp, err);
+ __put_user_error(regs->compat_lr, &sf->uc.uc_mcontext.arm_lr, err);
+ __put_user_error(regs->pc, &sf->uc.uc_mcontext.arm_pc, err);
+ __put_user_error(regs->pstate, &sf->uc.uc_mcontext.arm_cpsr, err);
+
+ __put_user_error((compat_ulong_t)0, &sf->uc.uc_mcontext.trap_no, err);
+ __put_user_error((compat_ulong_t)0, &sf->uc.uc_mcontext.error_code, err);
+ __put_user_error(current->thread.fault_address, &sf->uc.uc_mcontext.fault_address, err);
+ __put_user_error(set->sig[0], &sf->uc.uc_mcontext.oldmask, err);
+
+ err |= put_sigset_t(&sf->uc.uc_sigmask, set);
+
+ aux = (struct compat_aux_sigframe __user *) sf->uc.uc_regspace;
+
+ if (err == 0)
+ err |= compat_preserve_vfp_context(&aux->vfp);
+ __put_user_error(0, &aux->end_magic, err);
+
+ return err;
+}
+
+/*
+ * 32-bit signal handling routines called from signal.c
+ */
+int compat_setup_rt_frame(int usig, struct k_sigaction *ka, siginfo_t *info,
+ sigset_t *set, struct pt_regs *regs)
+{
+ struct compat_rt_sigframe __user *frame;
+ compat_stack_t stack;
+ int err = 0;
+
+ frame = compat_get_sigframe(ka, regs, sizeof(*frame));
+
+ if (!frame)
+ return 1;
+
+ err |= copy_siginfo_to_user32(&frame->info, info);
+
+ __put_user_error(0, &frame->sig.uc.uc_flags, err);
+ __put_user_error(NULL, &frame->sig.uc.uc_link, err);
+
+ memset(&stack, 0, sizeof(stack));
+ stack.ss_sp = (compat_uptr_t)current->sas_ss_sp;
+ stack.ss_flags = sas_ss_flags(regs->compat_sp);
+ stack.ss_size = current->sas_ss_size;
+ err |= __copy_to_user(&frame->sig.uc.uc_stack, &stack, sizeof(stack));
+
+ err |= compat_setup_sigframe(&frame->sig, regs, set);
+ if (err == 0)
+ err = compat_setup_return(regs, ka, frame->sig.retcode, frame,
+ usig);
+
+ if (err == 0) {
+ regs->regs[1] = (compat_ulong_t)(unsigned long)&frame->info;
+ regs->regs[2] = (compat_ulong_t)(unsigned long)&frame->sig.uc;
+ }
+
+ return err;
+}
+
+int compat_setup_frame(int usig, struct k_sigaction *ka, sigset_t *set,
+ struct pt_regs *regs)
+{
+ struct compat_sigframe __user *frame;
+ int err = 0;
+
+ frame = compat_get_sigframe(ka, regs, sizeof(*frame));
+
+ if (!frame)
+ return 1;
+
+ __put_user_error(0x5ac3c35a, &frame->uc.uc_flags, err);
+
+ err |= compat_setup_sigframe(frame, regs, set);
+ if (err == 0)
+ err = compat_setup_return(regs, ka, frame->retcode, frame, usig);
+
+ return err;
+}
+
+/*
+ * RT signals don't have generic compat wrappers.
+ * See arch/powerpc/kernel/signal_32.c
+ */
+asmlinkage int compat_sys_rt_sigprocmask(int how, compat_sigset_t __user *set,
+ compat_sigset_t __user *oset,
+ compat_size_t sigsetsize)
+{
+ sigset_t s;
+ sigset_t __user *up;
+ int ret;
+ mm_segment_t old_fs = get_fs();
+
+ if (set) {
+ if (get_sigset_t(&s, set))
+ return -EFAULT;
+ }
+
+ set_fs(KERNEL_DS);
+ /* This is valid because of the set_fs() */
+ up = (sigset_t __user *) &s;
+ ret = sys_rt_sigprocmask(how, set ? up : NULL, oset ? up : NULL,
+ sigsetsize);
+ set_fs(old_fs);
+ if (ret)
+ return ret;
+ if (oset) {
+ if (put_sigset_t(oset, &s))
+ return -EFAULT;
+ }
+ return 0;
+}
+
+asmlinkage int compat_sys_rt_sigpending(compat_sigset_t __user *set,
+ compat_size_t sigsetsize)
+{
+ sigset_t s;
+ int ret;
+ mm_segment_t old_fs = get_fs();
+
+ set_fs(KERNEL_DS);
+ /* The __user pointer cast is valid because of the set_fs() */
+ ret = sys_rt_sigpending((sigset_t __user *) &s, sigsetsize);
+ set_fs(old_fs);
+ if (!ret) {
+ if (put_sigset_t(set, &s))
+ return -EFAULT;
+ }
+ return ret;
+}
+
+asmlinkage int compat_sys_rt_sigqueueinfo(int pid, int sig,
+ compat_siginfo_t __user *uinfo)
+{
+ siginfo_t info;
+ int ret;
+ mm_segment_t old_fs = get_fs();
+
+ ret = copy_siginfo_from_user32(&info, uinfo);
+ if (unlikely(ret))
+ return ret;
+
+ set_fs (KERNEL_DS);
+ /* The __user pointer cast is valid because of the set_fs() */
+ ret = sys_rt_sigqueueinfo(pid, sig, (siginfo_t __user *) &info);
+ set_fs (old_fs);
+ return ret;
+}
+
+void compat_setup_restart_syscall(struct pt_regs *regs)
+{
+ regs->regs[7] = __NR_restart_syscall;
+}
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
new file mode 100644
index 0000000..b711525
--- /dev/null
+++ b/arch/arm64/kernel/smp.c
@@ -0,0 +1,469 @@
+/*
+ * SMP initialisation and IPI support
+ * Based on arch/arm/kernel/smp.c
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/spinlock.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/cache.h>
+#include <linux/profile.h>
+#include <linux/errno.h>
+#include <linux/mm.h>
+#include <linux/err.h>
+#include <linux/cpu.h>
+#include <linux/smp.h>
+#include <linux/seq_file.h>
+#include <linux/irq.h>
+#include <linux/percpu.h>
+#include <linux/clockchips.h>
+#include <linux/completion.h>
+#include <linux/of.h>
+
+#include <asm/atomic.h>
+#include <asm/cacheflush.h>
+#include <asm/cputype.h>
+#include <asm/mmu_context.h>
+#include <asm/pgtable.h>
+#include <asm/pgalloc.h>
+#include <asm/processor.h>
+#include <asm/sections.h>
+#include <asm/tlbflush.h>
+#include <asm/ptrace.h>
+#include <asm/mmu_context.h>
+
+/*
+ * as from 2.5, kernels no longer have an init_tasks structure
+ * so we need some other way of telling a new secondary core
+ * where to place its SVC stack
+ */
+struct secondary_data secondary_data;
+volatile unsigned long secondary_holding_pen_release = -1;
+
+enum ipi_msg_type {
+ IPI_RESCHEDULE,
+ IPI_CALL_FUNC,
+ IPI_CALL_FUNC_SINGLE,
+ IPI_CPU_STOP,
+};
+
+static DEFINE_RAW_SPINLOCK(boot_lock);
+
+/*
+ * Write secondary_holding_pen_release in a way that is guaranteed to be
+ * visible to all observers, irrespective of whether they're taking part
+ * in coherency or not. This is necessary for the hotplug code to work
+ * reliably.
+ */
+static void __cpuinit write_pen_release(int val)
+{
+ void *start = (void *)&secondary_holding_pen_release;
+ unsigned long size = sizeof(secondary_holding_pen_release);
+
+ secondary_holding_pen_release = val;
+ __flush_dcache_area(start, size);
+}
+
+/*
+ * Boot a secondary CPU, and assign it the specified idle task.
+ * This also gives us the initial stack to use for this CPU.
+ */
+static int __cpuinit boot_secondary(unsigned int cpu, struct task_struct *idle)
+{
+ unsigned long timeout;
+
+ /*
+ * Set synchronisation state between this boot processor
+ * and the secondary one
+ */
+ raw_spin_lock(&boot_lock);
+
+ /*
+ * Update the pen release flag.
+ */
+ write_pen_release(cpu);
+
+ /*
+ * Send an event, causing the secondaries to read pen_release.
+ */
+ sev();
+
+ timeout = jiffies + (1 * HZ);
+ while (time_before(jiffies, timeout)) {
+ if (secondary_holding_pen_release == -1UL)
+ break;
+ udelay(10);
+ }
+
+ /*
+ * Now the secondary core is starting up let it run its
+ * calibrations, then wait for it to finish
+ */
+ raw_spin_unlock(&boot_lock);
+
+ return secondary_holding_pen_release != -1 ? -ENOSYS : 0;
+}
+
+static DECLARE_COMPLETION(cpu_running);
+
+int __cpuinit __cpu_up(unsigned int cpu, struct task_struct *idle)
+{
+ int ret;
+
+ /*
+ * We need to tell the secondary core where to find its stack and the
+ * page tables.
+ */
+ secondary_data.stack = task_stack_page(idle) + THREAD_START_SP;
+ __flush_dcache_area(&secondary_data, sizeof(secondary_data));
+
+ /*
+ * Now bring the CPU into our world.
+ */
+ ret = boot_secondary(cpu, idle);
+ if (ret == 0) {
+ /*
+ * CPU was successfully started, wait for it to come online or
+ * time out.
+ */
+ wait_for_completion_timeout(&cpu_running,
+ msecs_to_jiffies(1000));
+
+ if (!cpu_online(cpu)) {
+ pr_crit("CPU%u: failed to come online\n", cpu);
+ ret = -EIO;
+ }
+ } else {
+ pr_err("CPU%u: failed to boot: %d\n", cpu, ret);
+ }
+
+ secondary_data.stack = NULL;
+
+ return ret;
+}
+
+/*
+ * This is the secondary CPU boot entry. We're using this CPUs
+ * idle thread stack, but a set of temporary page tables.
+ */
+asmlinkage void __cpuinit secondary_start_kernel(void)
+{
+ struct mm_struct *mm = &init_mm;
+ unsigned int cpu = smp_processor_id();
+
+ printk("CPU%u: Booted secondary processor\n", cpu);
+
+ /*
+ * All kernel threads share the same mm context; grab a
+ * reference and switch to it.
+ */
+ atomic_inc(&mm->mm_count);
+ current->active_mm = mm;
+ cpumask_set_cpu(cpu, mm_cpumask(mm));
+
+ /*
+ * TTBR0 is only used for the identity mapping at this stage. Make it
+ * point to zero page to avoid speculatively fetching new entries.
+ */
+ cpu_set_reserved_ttbr0();
+ flush_tlb_all();
+
+ preempt_disable();
+ trace_hardirqs_off();
+
+ /*
+ * Let the primary processor know we're out of the
+ * pen, then head off into the C entry point
+ */
+ write_pen_release(-1);
+
+ /*
+ * Synchronise with the boot thread.
+ */
+ raw_spin_lock(&boot_lock);
+ raw_spin_unlock(&boot_lock);
+
+ /*
+ * Enable local interrupts.
+ */
+ notify_cpu_starting(cpu);
+ local_irq_enable();
+ local_fiq_enable();
+
+ /*
+ * OK, now it's safe to let the boot CPU continue. Wait for
+ * the CPU migration code to notice that the CPU is online
+ * before we continue.
+ */
+ set_cpu_online(cpu, true);
+ while (!cpu_active(cpu))
+ cpu_relax();
+
+ /*
+ * OK, it's off to the idle thread for us
+ */
+ cpu_idle();
+}
+
+void __init smp_cpus_done(unsigned int max_cpus)
+{
+ unsigned long bogosum = loops_per_jiffy * num_online_cpus();
+
+ pr_info("SMP: Total of %d processors activated (%lu.%02lu BogoMIPS).\n",
+ num_online_cpus(), bogosum / (500000/HZ),
+ (bogosum / (5000/HZ)) % 100);
+}
+
+void __init smp_prepare_boot_cpu(void)
+{
+}
+
+static void (*smp_cross_call)(const struct cpumask *, unsigned int);
+static phys_addr_t cpu_release_addr[NR_CPUS];
+
+/*
+ * Enumerate the possible CPU set from the device tree.
+ */
+void __init smp_init_cpus(void)
+{
+ const char *enable_method;
+ struct device_node *dn = NULL;
+ int cpu = 0;
+
+ while ((dn = of_find_node_by_type(dn, "cpu"))) {
+ if (cpu >= NR_CPUS)
+ goto next;
+
+ /*
+ * We currently support only the "spin-table" enable-method.
+ */
+ enable_method = of_get_property(dn, "enable-method", NULL);
+ if (!enable_method || strcmp(enable_method, "spin-table")) {
+ pr_err("CPU %d: missing or invalid enable-method property: %s\n",
+ cpu, enable_method);
+ goto next;
+ }
+
+ /*
+ * Determine the address from which the CPU is polling.
+ */
+ if (of_property_read_u64(dn, "cpu-release-addr",
+ &cpu_release_addr[cpu])) {
+ pr_err("CPU %d: missing or invalid cpu-release-addr property\n",
+ cpu);
+ goto next;
+ }
+
+ set_cpu_possible(cpu, true);
+next:
+ cpu++;
+ }
+
+ /* sanity check */
+ if (cpu > NR_CPUS)
+ pr_warning("no. of cores (%d) greater than configured maximum of %d - clipping\n",
+ cpu, NR_CPUS);
+}
+
+void __init smp_prepare_cpus(unsigned int max_cpus)
+{
+ int cpu;
+ void **release_addr;
+ unsigned int ncores = num_possible_cpus();
+
+ /*
+ * are we trying to boot more cores than exist?
+ */
+ if (max_cpus > ncores)
+ max_cpus = ncores;
+
+ /*
+ * Initialise the present map (which describes the set of CPUs
+ * actually populated at the present time) and release the
+ * secondaries from the bootloader.
+ */
+ for_each_possible_cpu(cpu) {
+ if (max_cpus == 0)
+ break;
+
+ if (!cpu_release_addr[cpu])
+ continue;
+
+ release_addr = __va(cpu_release_addr[cpu]);
+ release_addr[0] = (void *)__pa(secondary_holding_pen);
+ __flush_dcache_area(release_addr, sizeof(release_addr[0]));
+
+ set_cpu_present(cpu, true);
+ max_cpus--;
+ }
+
+ /*
+ * Send an event to wake up the secondaries.
+ */
+ sev();
+}
+
+
+void __init set_smp_cross_call(void (*fn)(const struct cpumask *, unsigned int))
+{
+ smp_cross_call = fn;
+}
+
+void arch_send_call_function_ipi_mask(const struct cpumask *mask)
+{
+ smp_cross_call(mask, IPI_CALL_FUNC);
+}
+
+void arch_send_call_function_single_ipi(int cpu)
+{
+ smp_cross_call(cpumask_of(cpu), IPI_CALL_FUNC_SINGLE);
+}
+
+static const char *ipi_types[NR_IPI] = {
+#define S(x,s) [x - IPI_RESCHEDULE] = s
+ S(IPI_RESCHEDULE, "Rescheduling interrupts"),
+ S(IPI_CALL_FUNC, "Function call interrupts"),
+ S(IPI_CALL_FUNC_SINGLE, "Single function call interrupts"),
+ S(IPI_CPU_STOP, "CPU stop interrupts"),
+};
+
+void show_ipi_list(struct seq_file *p, int prec)
+{
+ unsigned int cpu, i;
+
+ for (i = 0; i < NR_IPI; i++) {
+ seq_printf(p, "%*s%u:%s", prec - 1, "IPI", i + IPI_RESCHEDULE,
+ prec >= 4 ? " " : "");
+ for_each_present_cpu(cpu)
+ seq_printf(p, "%10u ",
+ __get_irq_stat(cpu, ipi_irqs[i]));
+ seq_printf(p, " %s\n", ipi_types[i]);
+ }
+}
+
+u64 smp_irq_stat_cpu(unsigned int cpu)
+{
+ u64 sum = 0;
+ int i;
+
+ for (i = 0; i < NR_IPI; i++)
+ sum += __get_irq_stat(cpu, ipi_irqs[i]);
+
+ return sum;
+}
+
+static DEFINE_RAW_SPINLOCK(stop_lock);
+
+/*
+ * ipi_cpu_stop - handle IPI from smp_send_stop()
+ */
+static void ipi_cpu_stop(unsigned int cpu)
+{
+ if (system_state == SYSTEM_BOOTING ||
+ system_state == SYSTEM_RUNNING) {
+ raw_spin_lock(&stop_lock);
+ pr_crit("CPU%u: stopping\n", cpu);
+ dump_stack();
+ raw_spin_unlock(&stop_lock);
+ }
+
+ set_cpu_online(cpu, false);
+
+ local_fiq_disable();
+ local_irq_disable();
+
+ while (1)
+ cpu_relax();
+}
+
+/*
+ * Main handler for inter-processor interrupts
+ */
+void handle_IPI(int ipinr, struct pt_regs *regs)
+{
+ unsigned int cpu = smp_processor_id();
+ struct pt_regs *old_regs = set_irq_regs(regs);
+
+ if (ipinr >= IPI_RESCHEDULE && ipinr < IPI_RESCHEDULE + NR_IPI)
+ __inc_irq_stat(cpu, ipi_irqs[ipinr - IPI_RESCHEDULE]);
+
+ switch (ipinr) {
+ case IPI_RESCHEDULE:
+ scheduler_ipi();
+ break;
+
+ case IPI_CALL_FUNC:
+ irq_enter();
+ generic_smp_call_function_interrupt();
+ irq_exit();
+ break;
+
+ case IPI_CALL_FUNC_SINGLE:
+ irq_enter();
+ generic_smp_call_function_single_interrupt();
+ irq_exit();
+ break;
+
+ case IPI_CPU_STOP:
+ irq_enter();
+ ipi_cpu_stop(cpu);
+ irq_exit();
+ break;
+
+ default:
+ pr_crit("CPU%u: Unknown IPI message 0x%x\n", cpu, ipinr);
+ break;
+ }
+ set_irq_regs(old_regs);
+}
+
+void smp_send_reschedule(int cpu)
+{
+ smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE);
+}
+
+void smp_send_stop(void)
+{
+ unsigned long timeout;
+
+ if (num_online_cpus() > 1) {
+ cpumask_t mask;
+
+ cpumask_copy(&mask, cpu_online_mask);
+ cpu_clear(smp_processor_id(), mask);
+
+ smp_cross_call(&mask, IPI_CPU_STOP);
+ }
+
+ /* Wait up to one second for other CPUs to stop */
+ timeout = USEC_PER_SEC;
+ while (num_online_cpus() > 1 && timeout--)
+ udelay(1);
+
+ if (num_online_cpus() > 1)
+ pr_warning("SMP: failed to stop secondary CPUs\n");
+}
+
+/*
+ * not supported here
+ */
+int setup_profiling_timer(unsigned int multiplier)
+{
+ return -EINVAL;
+}
diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
new file mode 100644
index 0000000..d25459f
--- /dev/null
+++ b/arch/arm64/kernel/stacktrace.c
@@ -0,0 +1,127 @@
+/*
+ * Stack tracing support
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/kernel.h>
+#include <linux/export.h>
+#include <linux/sched.h>
+#include <linux/stacktrace.h>
+
+#include <asm/stacktrace.h>
+
+/*
+ * AArch64 PCS assigns the frame pointer to x29.
+ *
+ * A simple function prologue looks like this:
+ * sub sp, sp, #0x10
+ * stp x29, x30, [sp]
+ * mov x29, sp
+ *
+ * A simple function epilogue looks like this:
+ * mov sp, x29
+ * ldp x29, x30, [sp]
+ * add sp, sp, #0x10
+ */
+int unwind_frame(struct stackframe *frame)
+{
+ unsigned long high, low;
+ unsigned long fp = frame->fp;
+
+ low = frame->sp;
+ high = ALIGN(low, THREAD_SIZE);
+
+ if (fp < low || fp > high || fp & 0xf)
+ return -EINVAL;
+
+ frame->sp = fp + 0x10;
+ frame->fp = *(unsigned long *)(fp);
+ frame->pc = *(unsigned long *)(fp + 8);
+
+ return 0;
+}
+
+void notrace walk_stackframe(struct stackframe *frame,
+ int (*fn)(struct stackframe *, void *), void *data)
+{
+ while (1) {
+ int ret;
+
+ if (fn(frame, data))
+ break;
+ ret = unwind_frame(frame);
+ if (ret < 0)
+ break;
+ }
+}
+EXPORT_SYMBOL(walk_stackframe);
+
+#ifdef CONFIG_STACKTRACE
+struct stack_trace_data {
+ struct stack_trace *trace;
+ unsigned int no_sched_functions;
+ unsigned int skip;
+};
+
+static int save_trace(struct stackframe *frame, void *d)
+{
+ struct stack_trace_data *data = d;
+ struct stack_trace *trace = data->trace;
+ unsigned long addr = frame->pc;
+
+ if (data->no_sched_functions && in_sched_functions(addr))
+ return 0;
+ if (data->skip) {
+ data->skip--;
+ return 0;
+ }
+
+ trace->entries[trace->nr_entries++] = addr;
+
+ return trace->nr_entries >= trace->max_entries;
+}
+
+void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace)
+{
+ struct stack_trace_data data;
+ struct stackframe frame;
+
+ data.trace = trace;
+ data.skip = trace->skip;
+
+ if (tsk != current) {
+ data.no_sched_functions = 1;
+ frame.fp = thread_saved_fp(tsk);
+ frame.sp = thread_saved_sp(tsk);
+ frame.pc = thread_saved_pc(tsk);
+ } else {
+ register unsigned long current_sp asm("sp");
+ data.no_sched_functions = 0;
+ frame.fp = (unsigned long)__builtin_frame_address(0);
+ frame.sp = current_sp;
+ frame.pc = (unsigned long)save_stack_trace_tsk;
+ }
+
+ walk_stackframe(&frame, save_trace, &data);
+ if (trace->nr_entries < trace->max_entries)
+ trace->entries[trace->nr_entries++] = ULONG_MAX;
+}
+
+void save_stack_trace(struct stack_trace *trace)
+{
+ save_stack_trace_tsk(current, trace);
+}
+EXPORT_SYMBOL_GPL(save_stack_trace);
+#endif
diff --git a/arch/arm64/kernel/sys.c b/arch/arm64/kernel/sys.c
new file mode 100644
index 0000000..905fcfb
--- /dev/null
+++ b/arch/arm64/kernel/sys.c
@@ -0,0 +1,138 @@
+/*
+ * AArch64-specific system calls implementation
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/compiler.h>
+#include <linux/errno.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/export.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/syscalls.h>
+
+/*
+ * Clone a task - this clones the calling program thread.
+ */
+asmlinkage long sys_clone(unsigned long clone_flags, unsigned long newsp,
+ int __user *parent_tidptr, unsigned long tls_val,
+ int __user *child_tidptr, struct pt_regs *regs)
+{
+ if (!newsp)
+ newsp = regs->sp;
+ /* 16-byte aligned stack mandatory on AArch64 */
+ if (newsp & 15)
+ return -EINVAL;
+ return do_fork(clone_flags, newsp, regs, 0, parent_tidptr, child_tidptr);
+}
+
+/*
+ * sys_execve() executes a new program.
+ */
+asmlinkage long sys_execve(const char __user *filenamei,
+ const char __user *const __user *argv,
+ const char __user *const __user *envp,
+ struct pt_regs *regs)
+{
+ long error;
+ char * filename;
+
+ filename = getname(filenamei);
+ error = PTR_ERR(filename);
+ if (IS_ERR(filename))
+ goto out;
+ error = do_execve(filename, argv, envp, regs);
+ putname(filename);
+out:
+ return error;
+}
+
+int kernel_execve(const char *filename,
+ const char *const argv[],
+ const char *const envp[])
+{
+ struct pt_regs regs;
+ int ret;
+
+ memset(®s, 0, sizeof(struct pt_regs));
+ ret = do_execve(filename,
+ (const char __user *const __user *)argv,
+ (const char __user *const __user *)envp, ®s);
+ if (ret < 0)
+ goto out;
+
+ /*
+ * Save argc to the register structure for userspace.
+ */
+ regs.regs[0] = ret;
+
+ /*
+ * We were successful. We won't be returning to our caller, but
+ * instead to user space by manipulating the kernel stack.
+ */
+ asm( "add x0, %0, %1\n\t"
+ "mov x1, %2\n\t"
+ "mov x2, %3\n\t"
+ "bl memmove\n\t" /* copy regs to top of stack */
+ "mov x27, #0\n\t" /* not a syscall */
+ "mov x28, %0\n\t" /* thread structure */
+ "mov sp, x0\n\t" /* reposition stack pointer */
+ "b ret_to_user"
+ :
+ : "r" (current_thread_info()),
+ "Ir" (THREAD_START_SP - sizeof(regs)),
+ "r" (®s),
+ "Ir" (sizeof(regs))
+ : "x0", "x1", "x2", "x27", "x28", "x30", "memory");
+
+ out:
+ return ret;
+}
+EXPORT_SYMBOL(kernel_execve);
+
+asmlinkage long sys_mmap(unsigned long addr, unsigned long len,
+ unsigned long prot, unsigned long flags,
+ unsigned long fd, off_t off)
+{
+ if (offset_in_page(off) != 0)
+ return -EINVAL;
+
+ return sys_mmap_pgoff(addr, len, prot, flags, fd, off >> PAGE_SHIFT);
+}
+
+/*
+ * Wrappers to pass the pt_regs argument.
+ */
+#define sys_execve sys_execve_wrapper
+#define sys_clone sys_clone_wrapper
+#define sys_rt_sigreturn sys_rt_sigreturn_wrapper
+#define sys_sigaltstack sys_sigaltstack_wrapper
+
+#include <asm/syscalls.h>
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym) [nr] = sym,
+
+/*
+ * The sys_call_table array must be 4K aligned to be accessible from
+ * kernel/entry.S.
+ */
+void *sys_call_table[__NR_syscalls] __aligned(4096) = {
+ [0 ... __NR_syscalls - 1] = sys_ni_syscall,
+#include <asm/unistd.h>
+};
diff --git a/arch/arm64/kernel/sys32.S b/arch/arm64/kernel/sys32.S
new file mode 100644
index 0000000..5e4dc93
--- /dev/null
+++ b/arch/arm64/kernel/sys32.S
@@ -0,0 +1,282 @@
+/*
+ * Compat system call wrappers
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Authors: Will Deacon <will.deacon@arm.com>
+ * Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/assembler.h>
+#include <asm/asm-offsets.h>
+
+/*
+ * System call wrappers for the AArch32 compatibility layer.
+ */
+compat_sys_fork_wrapper:
+ mov x0, sp
+ b compat_sys_fork
+ENDPROC(compat_sys_fork_wrapper)
+
+compat_sys_vfork_wrapper:
+ mov x0, sp
+ b compat_sys_vfork
+ENDPROC(compat_sys_vfork_wrapper)
+
+compat_sys_execve_wrapper:
+ mov x3, sp
+ b compat_sys_execve
+ENDPROC(compat_sys_execve_wrapper)
+
+compat_sys_clone_wrapper:
+ mov x5, sp
+ b compat_sys_clone
+ENDPROC(compat_sys_clone_wrapper)
+
+compat_sys_sigreturn_wrapper:
+ mov x0, sp
+ mov x27, #0 // prevent syscall restart handling (why)
+ b compat_sys_sigreturn
+ENDPROC(compat_sys_sigreturn_wrapper)
+
+compat_sys_rt_sigreturn_wrapper:
+ mov x0, sp
+ mov x27, #0 // prevent syscall restart handling (why)
+ b compat_sys_rt_sigreturn
+ENDPROC(compat_sys_rt_sigreturn_wrapper)
+
+compat_sys_sigaltstack_wrapper:
+ ldr x2, [sp, #S_COMPAT_SP]
+ b compat_do_sigaltstack
+ENDPROC(compat_sys_sigaltstack_wrapper)
+
+compat_sys_statfs64_wrapper:
+ mov w3, #84
+ cmp w1, #88
+ csel w1, w3, w1, eq
+ b compat_sys_statfs64
+ENDPROC(compat_sys_statfs64_wrapper)
+
+compat_sys_fstatfs64_wrapper:
+ mov w3, #84
+ cmp w1, #88
+ csel w1, w3, w1, eq
+ b compat_sys_fstatfs64
+ENDPROC(compat_sys_fstatfs64_wrapper)
+
+/*
+ * Wrappers for AArch32 syscalls that either take 64-bit parameters
+ * in registers or that take 32-bit parameters which require sign
+ * extension.
+ */
+compat_sys_lseek_wrapper:
+ sxtw x1, w1
+ b sys_lseek
+ENDPROC(compat_sys_lseek_wrapper)
+
+compat_sys_pread64_wrapper:
+ orr x3, x4, x5, lsl #32
+ b sys_pread64
+ENDPROC(compat_sys_pread64_wrapper)
+
+compat_sys_pwrite64_wrapper:
+ orr x3, x4, x5, lsl #32
+ b sys_pwrite64
+ENDPROC(compat_sys_pwrite64_wrapper)
+
+compat_sys_truncate64_wrapper:
+ orr x1, x2, x3, lsl #32
+ b sys_truncate
+ENDPROC(compat_sys_truncate64_wrapper)
+
+compat_sys_ftruncate64_wrapper:
+ orr x1, x2, x3, lsl #32
+ b sys_ftruncate
+ENDPROC(compat_sys_ftruncate64_wrapper)
+
+compat_sys_readahead_wrapper:
+ orr x1, x2, x3, lsl #32
+ mov w2, w4
+ b sys_readahead
+ENDPROC(compat_sys_readahead_wrapper)
+
+compat_sys_lookup_dcookie:
+ orr x0, x0, x1, lsl #32
+ mov w1, w2
+ mov w2, w3
+ b sys_lookup_dcookie
+ENDPROC(compat_sys_lookup_dcookie)
+
+compat_sys_fadvise64_64_wrapper:
+ mov w6, w1
+ orr x1, x2, x3, lsl #32
+ orr x2, x4, x5, lsl #32
+ mov w3, w6
+ b sys_fadvise64_64
+ENDPROC(compat_sys_fadvise64_64_wrapper)
+
+compat_sys_sync_file_range2_wrapper:
+ orr x2, x2, x3, lsl #32
+ orr x3, x4, x5, lsl #32
+ b sys_sync_file_range2
+ENDPROC(compat_sys_sync_file_range2_wrapper)
+
+compat_sys_fallocate_wrapper:
+ orr x2, x2, x3, lsl #32
+ orr x3, x4, x5, lsl #32
+ b sys_fallocate
+ENDPROC(compat_sys_fallocate_wrapper)
+
+compat_sys_fanotify_mark_wrapper:
+ orr x2, x2, x3, lsl #32
+ mov w3, w4
+ mov w4, w5
+ b sys_fanotify_mark
+ENDPROC(compat_sys_fanotify_mark_wrapper)
+
+/*
+ * Use the compat system call wrappers.
+ */
+#define sys_fork compat_sys_fork_wrapper
+#define sys_open compat_sys_open
+#define sys_execve compat_sys_execve_wrapper
+#define sys_lseek compat_sys_lseek_wrapper
+#define sys_mount compat_sys_mount
+#define sys_ptrace compat_sys_ptrace
+#define sys_times compat_sys_times
+#define sys_ioctl compat_sys_ioctl
+#define sys_fcntl compat_sys_fcntl
+#define sys_ustat compat_sys_ustat
+#define sys_sigaction compat_sys_sigaction
+#define sys_sigsuspend compat_sys_sigsuspend
+#define sys_sigpending compat_sys_sigpending
+#define sys_setrlimit compat_sys_setrlimit
+#define sys_getrusage compat_sys_getrusage
+#define sys_gettimeofday compat_sys_gettimeofday
+#define sys_settimeofday compat_sys_settimeofday
+#define sys_statfs compat_sys_statfs
+#define sys_fstatfs compat_sys_fstatfs
+#define sys_setitimer compat_sys_setitimer
+#define sys_getitimer compat_sys_getitimer
+#define sys_newstat compat_sys_newstat
+#define sys_newlstat compat_sys_newlstat
+#define sys_newfstat compat_sys_newfstat
+#define sys_wait4 compat_sys_wait4
+#define sys_sysinfo compat_sys_sysinfo
+#define sys_sigreturn compat_sys_sigreturn_wrapper
+#define sys_clone compat_sys_clone_wrapper
+#define sys_adjtimex compat_sys_adjtimex
+#define sys_sigprocmask compat_sys_sigprocmask
+#define sys_getdents compat_sys_getdents
+#define sys_select compat_sys_select
+#define sys_readv compat_sys_readv
+#define sys_writev compat_sys_writev
+#define sys_sysctl compat_sys_sysctl
+#define sys_sched_rr_get_interval compat_sys_sched_rr_get_interval
+#define sys_nanosleep compat_sys_nanosleep
+#define sys_rt_sigreturn compat_sys_rt_sigreturn_wrapper
+#define sys_rt_sigaction compat_sys_rt_sigaction
+#define sys_rt_sigprocmask compat_sys_rt_sigprocmask
+#define sys_rt_sigpending compat_sys_rt_sigpending
+#define sys_rt_sigtimedwait compat_sys_rt_sigtimedwait
+#define sys_rt_sigqueueinfo compat_sys_rt_sigqueueinfo
+#define sys_rt_sigsuspend compat_sys_rt_sigsuspend
+#define sys_pread64 compat_sys_pread64_wrapper
+#define sys_pwrite64 compat_sys_pwrite64_wrapper
+#define sys_sigaltstack compat_sys_sigaltstack_wrapper
+#define sys_sendfile compat_sys_sendfile
+#define sys_vfork compat_sys_vfork_wrapper
+#define sys_getrlimit compat_sys_getrlimit
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_truncate64 compat_sys_truncate64_wrapper
+#define sys_ftruncate64 compat_sys_ftruncate64_wrapper
+#define sys_getdents64 compat_sys_getdents64
+#define sys_fcntl64 compat_sys_fcntl64
+#define sys_readahead compat_sys_readahead_wrapper
+#define sys_futex compat_sys_futex
+#define sys_sched_setaffinity compat_sys_sched_setaffinity
+#define sys_sched_getaffinity compat_sys_sched_getaffinity
+#define sys_io_setup compat_sys_io_setup
+#define sys_io_getevents compat_sys_io_getevents
+#define sys_io_submit compat_sys_io_submit
+#define sys_lookup_dcookie compat_sys_lookup_dcookie
+#define sys_timer_create compat_sys_timer_create
+#define sys_timer_settime compat_sys_timer_settime
+#define sys_timer_gettime compat_sys_timer_gettime
+#define sys_clock_settime compat_sys_clock_settime
+#define sys_clock_gettime compat_sys_clock_gettime
+#define sys_clock_getres compat_sys_clock_getres
+#define sys_clock_nanosleep compat_sys_clock_nanosleep
+#define sys_statfs64 compat_sys_statfs64_wrapper
+#define sys_fstatfs64 compat_sys_fstatfs64_wrapper
+#define sys_utimes compat_sys_utimes
+#define sys_fadvise64_64 compat_sys_fadvise64_64_wrapper
+#define sys_mq_open compat_sys_mq_open
+#define sys_mq_timedsend compat_sys_mq_timedsend
+#define sys_mq_timedreceive compat_sys_mq_timedreceive
+#define sys_mq_notify compat_sys_mq_notify
+#define sys_mq_getsetattr compat_sys_mq_getsetattr
+#define sys_waitid compat_sys_waitid
+#define sys_recv compat_sys_recv
+#define sys_recvfrom compat_sys_recvfrom
+#define sys_setsockopt compat_sys_setsockopt
+#define sys_getsockopt compat_sys_getsockopt
+#define sys_sendmsg compat_sys_sendmsg
+#define sys_recvmsg compat_sys_recvmsg
+#define sys_semctl compat_sys_semctl
+#define sys_msgsnd compat_sys_msgsnd
+#define sys_msgrcv compat_sys_msgrcv
+#define sys_msgctl compat_sys_msgctl
+#define sys_shmat compat_sys_shmat
+#define sys_shmctl compat_sys_shmctl
+#define sys_keyctl compat_sys_keyctl
+#define sys_semtimedop compat_sys_semtimedop
+#define sys_mbind compat_sys_mbind
+#define sys_get_mempolicy compat_sys_get_mempolicy
+#define sys_set_mempolicy compat_sys_set_mempolicy
+#define sys_openat compat_sys_openat
+#define sys_futimesat compat_sys_futimesat
+#define sys_pselect6 compat_sys_pselect6
+#define sys_ppoll compat_sys_ppoll
+#define sys_set_robust_list compat_sys_set_robust_list
+#define sys_get_robust_list compat_sys_get_robust_list
+#define sys_sync_file_range2 compat_sys_sync_file_range2_wrapper
+#define sys_vmsplice compat_sys_vmsplice
+#define sys_move_pages compat_sys_move_pages
+#define sys_epoll_pwait compat_sys_epoll_pwait
+#define sys_kexec_load compat_sys_kexec_load
+#define sys_utimensat compat_sys_utimensat
+#define sys_signalfd compat_sys_signalfd
+#define sys_fallocate compat_sys_fallocate_wrapper
+#define sys_timerfd_settime compat_sys_timerfd_settime
+#define sys_timerfd_gettime compat_sys_timerfd_gettime
+#define sys_signalfd4 compat_sys_signalfd4
+#define sys_preadv compat_sys_preadv
+#define sys_pwritev compat_sys_pwritev
+#define sys_rt_tgsigqueueinfo compat_sys_rt_tgsigqueueinfo
+#define sys_recvmmsg compat_sys_recvmmsg
+#define sys_fanotify_mark compat_sys_fanotify_mark_wrapper
+
+#undef __SYSCALL
+#define __SYSCALL(x, y) .quad y // x
+#define __SYSCALL_COMPAT
+
+/*
+ * The system calls table must be 4KB aligned.
+ */
+ .align 12
+ENTRY(compat_sys_call_table)
+#include <asm/unistd.h>
diff --git a/arch/arm64/kernel/sys_compat.c b/arch/arm64/kernel/sys_compat.c
new file mode 100644
index 0000000..967e92f
--- /dev/null
+++ b/arch/arm64/kernel/sys_compat.c
@@ -0,0 +1,164 @@
+/*
+ * Based on arch/arm/kernel/sys_arm.c
+ *
+ * Copyright (C) People who wrote linux/arch/i386/kernel/sys_i386.c
+ * Copyright (C) 1995, 1996 Russell King.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#define __SYSCALL_COMPAT
+
+#include <linux/compat.h>
+#include <linux/personality.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/syscalls.h>
+#include <linux/uaccess.h>
+
+#include <asm/cacheflush.h>
+#include <asm/unistd.h>
+
+asmlinkage int compat_sys_fork(struct pt_regs *regs)
+{
+ return do_fork(SIGCHLD, regs->compat_sp, regs, 0, NULL, NULL);
+}
+
+asmlinkage int compat_sys_clone(unsigned long clone_flags, unsigned long newsp,
+ int __user *parent_tidptr, int tls_val,
+ int __user *child_tidptr, struct pt_regs *regs)
+{
+ if (!newsp)
+ newsp = regs->compat_sp;
+
+ return do_fork(clone_flags, newsp, regs, 0, parent_tidptr, child_tidptr);
+}
+
+asmlinkage int compat_sys_vfork(struct pt_regs *regs)
+{
+ return do_fork(CLONE_VFORK | CLONE_VM | SIGCHLD, regs->compat_sp,
+ regs, 0, NULL, NULL);
+}
+
+asmlinkage int compat_sys_execve(const char __user *filenamei,
+ compat_uptr_t argv, compat_uptr_t envp,
+ struct pt_regs *regs)
+{
+ int error;
+ char * filename;
+
+ filename = getname(filenamei);
+ error = PTR_ERR(filename);
+ if (IS_ERR(filename))
+ goto out;
+ error = compat_do_execve(filename, compat_ptr(argv), compat_ptr(envp),
+ regs);
+ putname(filename);
+out:
+ return error;
+}
+
+asmlinkage int compat_sys_sched_rr_get_interval(compat_pid_t pid,
+ struct compat_timespec __user *interval)
+{
+ struct timespec t;
+ int ret;
+ mm_segment_t old_fs = get_fs();
+
+ set_fs(KERNEL_DS);
+ ret = sys_sched_rr_get_interval(pid, (struct timespec __user *)&t);
+ set_fs(old_fs);
+ if (put_compat_timespec(&t, interval))
+ return -EFAULT;
+ return ret;
+}
+
+asmlinkage int compat_sys_sendfile(int out_fd, int in_fd,
+ compat_off_t __user *offset, s32 count)
+{
+ mm_segment_t old_fs = get_fs();
+ int ret;
+ off_t of;
+
+ if (offset && get_user(of, offset))
+ return -EFAULT;
+
+ set_fs(KERNEL_DS);
+ ret = sys_sendfile(out_fd, in_fd, offset ? (off_t __user *)&of : NULL,
+ count);
+ set_fs(old_fs);
+
+ if (offset && put_user(of, offset))
+ return -EFAULT;
+ return ret;
+}
+
+static inline void
+do_compat_cache_op(unsigned long start, unsigned long end, int flags)
+{
+ struct mm_struct *mm = current->active_mm;
+ struct vm_area_struct *vma;
+
+ if (end < start || flags)
+ return;
+
+ down_read(&mm->mmap_sem);
+ vma = find_vma(mm, start);
+ if (vma && vma->vm_start < end) {
+ if (start < vma->vm_start)
+ start = vma->vm_start;
+ if (end > vma->vm_end)
+ end = vma->vm_end;
+ up_read(&mm->mmap_sem);
+ __flush_cache_user_range(start & PAGE_MASK, PAGE_ALIGN(end));
+ return;
+ }
+ up_read(&mm->mmap_sem);
+}
+
+/*
+ * Handle all unrecognised system calls.
+ */
+long compat_arm_syscall(struct pt_regs *regs)
+{
+ unsigned int no = regs->regs[7];
+
+ switch (no) {
+ /*
+ * Flush a region from virtual address 'r0' to virtual address 'r1'
+ * _exclusive_. There is no alignment requirement on either address;
+ * user space does not need to know the hardware cache layout.
+ *
+ * r2 contains flags. It should ALWAYS be passed as ZERO until it
+ * is defined to be something else. For now we ignore it, but may
+ * the fires of hell burn in your belly if you break this rule. ;)
+ *
+ * (at a later date, we may want to allow this call to not flush
+ * various aspects of the cache. Passing '0' will guarantee that
+ * everything necessary gets flushed to maintain consistency in
+ * the specified region).
+ */
+ case __ARM_NR_compat_cacheflush:
+ do_compat_cache_op(regs->regs[0], regs->regs[1], regs->regs[2]);
+ return 0;
+
+ case __ARM_NR_compat_set_tls:
+ current->thread.tp_value = regs->regs[0];
+ asm ("msr tpidrro_el0, %0" : : "r" (regs->regs[0]));
+ return 0;
+
+ default:
+ return -ENOSYS;
+ }
+}
diff --git a/arch/arm64/kernel/time.c b/arch/arm64/kernel/time.c
new file mode 100644
index 0000000..3b4b725
--- /dev/null
+++ b/arch/arm64/kernel/time.c
@@ -0,0 +1,65 @@
+/*
+ * Based on arch/arm/kernel/time.c
+ *
+ * Copyright (C) 1991, 1992, 1995 Linus Torvalds
+ * Modifications for ARM (C) 1994-2001 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/kernel.h>
+#include <linux/interrupt.h>
+#include <linux/time.h>
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/smp.h>
+#include <linux/timex.h>
+#include <linux/errno.h>
+#include <linux/profile.h>
+#include <linux/syscore_ops.h>
+#include <linux/timer.h>
+#include <linux/irq.h>
+
+#include <clocksource/arm_generic.h>
+
+#include <asm/thread_info.h>
+#include <asm/stacktrace.h>
+
+#ifdef CONFIG_SMP
+unsigned long profile_pc(struct pt_regs *regs)
+{
+ struct stackframe frame;
+
+ if (!in_lock_functions(regs->pc))
+ return regs->pc;
+
+ frame.fp = regs->regs[29];
+ frame.sp = regs->sp;
+ frame.pc = regs->pc;
+ do {
+ int ret = unwind_frame(&frame);
+ if (ret < 0)
+ return 0;
+ } while (in_lock_functions(frame.pc));
+
+ return frame.pc;
+}
+EXPORT_SYMBOL(profile_pc);
+#endif
+
+void __init time_init(void)
+{
+ arm_generic_timer_init();
+}
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
new file mode 100644
index 0000000..3883f84
--- /dev/null
+++ b/arch/arm64/kernel/traps.c
@@ -0,0 +1,348 @@
+/*
+ * Based on arch/arm/kernel/traps.c
+ *
+ * Copyright (C) 1995-2009 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/signal.h>
+#include <linux/personality.h>
+#include <linux/kallsyms.h>
+#include <linux/spinlock.h>
+#include <linux/uaccess.h>
+#include <linux/hardirq.h>
+#include <linux/kdebug.h>
+#include <linux/module.h>
+#include <linux/kexec.h>
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/syscalls.h>
+
+#include <asm/atomic.h>
+#include <asm/traps.h>
+#include <asm/stacktrace.h>
+#include <asm/exception.h>
+#include <asm/system_misc.h>
+
+static const char *handler[]= {
+ "Synchronous Abort",
+ "IRQ",
+ "FIQ",
+ "Error"
+};
+
+int show_unhandled_signals = 1;
+
+/*
+ * Dump out the contents of some memory nicely...
+ */
+static void dump_mem(const char *lvl, const char *str, unsigned long bottom,
+ unsigned long top)
+{
+ unsigned long first;
+ mm_segment_t fs;
+ int i;
+
+ /*
+ * We need to switch to kernel mode so that we can use __get_user
+ * to safely read from kernel space. Note that we now dump the
+ * code first, just in case the backtrace kills us.
+ */
+ fs = get_fs();
+ set_fs(KERNEL_DS);
+
+ printk("%s%s(0x%016lx to 0x%016lx)\n", lvl, str, bottom, top);
+
+ for (first = bottom & ~31; first < top; first += 32) {
+ unsigned long p;
+ char str[sizeof(" 12345678") * 8 + 1];
+
+ memset(str, ' ', sizeof(str));
+ str[sizeof(str) - 1] = '\0';
+
+ for (p = first, i = 0; i < 8 && p < top; i++, p += 4) {
+ if (p >= bottom && p < top) {
+ unsigned int val;
+ if (__get_user(val, (unsigned int *)p) == 0)
+ sprintf(str + i * 9, " %08x", val);
+ else
+ sprintf(str + i * 9, " ????????");
+ }
+ }
+ printk("%s%04lx:%s\n", lvl, first & 0xffff, str);
+ }
+
+ set_fs(fs);
+}
+
+static void dump_backtrace_entry(unsigned long where, unsigned long stack)
+{
+ print_ip_sym(where);
+ if (in_exception_text(where))
+ dump_mem("", "Exception stack", stack,
+ stack + sizeof(struct pt_regs));
+}
+
+static void dump_instr(const char *lvl, struct pt_regs *regs)
+{
+ unsigned long addr = instruction_pointer(regs);
+ mm_segment_t fs;
+ char str[sizeof("00000000 ") * 5 + 2 + 1], *p = str;
+ int i;
+
+ /*
+ * We need to switch to kernel mode so that we can use __get_user
+ * to safely read from kernel space. Note that we now dump the
+ * code first, just in case the backtrace kills us.
+ */
+ fs = get_fs();
+ set_fs(KERNEL_DS);
+
+ for (i = -4; i < 1; i++) {
+ unsigned int val, bad;
+
+ bad = __get_user(val, &((u32 *)addr)[i]);
+
+ if (!bad)
+ p += sprintf(p, i == 0 ? "(%08x) " : "%08x ", val);
+ else {
+ p += sprintf(p, "bad PC value");
+ break;
+ }
+ }
+ printk("%sCode: %s\n", lvl, str);
+
+ set_fs(fs);
+}
+
+static void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk)
+{
+ struct stackframe frame;
+ const register unsigned long current_sp asm ("sp");
+
+ pr_debug("%s(regs = %p tsk = %p)\n", __func__, regs, tsk);
+
+ if (!tsk)
+ tsk = current;
+
+ if (regs) {
+ frame.fp = regs->regs[29];
+ frame.sp = regs->sp;
+ frame.pc = regs->pc;
+ } else if (tsk == current) {
+ frame.fp = (unsigned long)__builtin_frame_address(0);
+ frame.sp = current_sp;
+ frame.pc = (unsigned long)dump_backtrace;
+ } else {
+ /*
+ * task blocked in __switch_to
+ */
+ frame.fp = thread_saved_fp(tsk);
+ frame.sp = thread_saved_sp(tsk);
+ frame.pc = thread_saved_pc(tsk);
+ }
+
+ printk("Call trace:\n");
+ while (1) {
+ unsigned long where = frame.pc;
+ int ret;
+
+ ret = unwind_frame(&frame);
+ if (ret < 0)
+ break;
+ dump_backtrace_entry(where, frame.sp);
+ }
+}
+
+void dump_stack(void)
+{
+ dump_backtrace(NULL, NULL);
+}
+
+EXPORT_SYMBOL(dump_stack);
+
+void show_stack(struct task_struct *tsk, unsigned long *sp)
+{
+ dump_backtrace(NULL, tsk);
+ barrier();
+}
+
+#ifdef CONFIG_PREEMPT
+#define S_PREEMPT " PREEMPT"
+#else
+#define S_PREEMPT ""
+#endif
+#ifdef CONFIG_SMP
+#define S_SMP " SMP"
+#else
+#define S_SMP ""
+#endif
+
+static int __die(const char *str, int err, struct thread_info *thread,
+ struct pt_regs *regs)
+{
+ struct task_struct *tsk = thread->task;
+ static int die_counter;
+ int ret;
+
+ pr_emerg("Internal error: %s: %x [#%d]" S_PREEMPT S_SMP "\n",
+ str, err, ++die_counter);
+
+ /* trap and error numbers are mostly meaningless on ARM */
+ ret = notify_die(DIE_OOPS, str, regs, err, 0, SIGSEGV);
+ if (ret == NOTIFY_STOP)
+ return ret;
+
+ print_modules();
+ __show_regs(regs);
+ pr_emerg("Process %.*s (pid: %d, stack limit = 0x%p)\n",
+ TASK_COMM_LEN, tsk->comm, task_pid_nr(tsk), thread + 1);
+
+ if (!user_mode(regs) || in_interrupt()) {
+ dump_mem(KERN_EMERG, "Stack: ", regs->sp,
+ THREAD_SIZE + (unsigned long)task_stack_page(tsk));
+ dump_backtrace(regs, tsk);
+ dump_instr(KERN_EMERG, regs);
+ }
+
+ return ret;
+}
+
+static DEFINE_RAW_SPINLOCK(die_lock);
+
+/*
+ * This function is protected against re-entrancy.
+ */
+void die(const char *str, struct pt_regs *regs, int err)
+{
+ struct thread_info *thread = current_thread_info();
+ int ret;
+
+ oops_enter();
+
+ raw_spin_lock_irq(&die_lock);
+ console_verbose();
+ bust_spinlocks(1);
+ ret = __die(str, err, thread, regs);
+
+ if (regs && kexec_should_crash(thread->task))
+ crash_kexec(regs);
+
+ bust_spinlocks(0);
+ add_taint(TAINT_DIE);
+ raw_spin_unlock_irq(&die_lock);
+ oops_exit();
+
+ if (in_interrupt())
+ panic("Fatal exception in interrupt");
+ if (panic_on_oops)
+ panic("Fatal exception");
+ if (ret != NOTIFY_STOP)
+ do_exit(SIGSEGV);
+}
+
+void arm64_notify_die(const char *str, struct pt_regs *regs,
+ struct siginfo *info, int err)
+{
+ if (user_mode(regs))
+ force_sig_info(info->si_signo, info, current);
+ else
+ die(str, regs, err);
+}
+
+asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
+{
+ siginfo_t info;
+ void __user *pc = (void __user *)instruction_pointer(regs);
+
+#ifdef CONFIG_COMPAT
+ /* check for AArch32 breakpoint instructions */
+ if (compat_user_mode(regs) && aarch32_break_trap(regs) == 0)
+ return;
+#endif
+
+ if (show_unhandled_signals) {
+ pr_info("%s[%d]: undefined instruction: pc=%p\n",
+ current->comm, task_pid_nr(current), pc);
+ dump_instr(KERN_INFO, regs);
+ }
+
+ info.si_signo = SIGILL;
+ info.si_errno = 0;
+ info.si_code = ILL_ILLOPC;
+ info.si_addr = pc;
+
+ arm64_notify_die("Oops - undefined instruction", regs, &info, 0);
+}
+
+long compat_arm_syscall(struct pt_regs *regs);
+
+asmlinkage long do_ni_syscall(struct pt_regs *regs)
+{
+#ifdef CONFIG_COMPAT
+ long ret;
+ if (is_compat_task()) {
+ ret = compat_arm_syscall(regs);
+ if (ret != -ENOSYS)
+ return ret;
+ }
+#endif
+
+ if (show_unhandled_signals) {
+ pr_info("%s[%d]: syscall %d\n", current->comm,
+ task_pid_nr(current), (int)regs->syscallno);
+ dump_instr("", regs);
+ if (user_mode(regs))
+ __show_regs(regs);
+ }
+
+ return sys_ni_syscall();
+}
+
+/*
+ * bad_mode handles the impossible case in the exception vector.
+ */
+asmlinkage void bad_mode(struct pt_regs *regs, int reason, unsigned int esr)
+{
+ console_verbose();
+
+ pr_crit("Bad mode in %s handler detected, code 0x%08x\n",
+ handler[reason], esr);
+
+ die("Oops - bad mode", regs, 0);
+ local_irq_disable();
+ panic("bad mode");
+}
+
+void __pte_error(const char *file, int line, unsigned long val)
+{
+ printk("%s:%d: bad pte %016lx.\n", file, line, val);
+}
+
+void __pmd_error(const char *file, int line, unsigned long val)
+{
+ printk("%s:%d: bad pmd %016lx.\n", file, line, val);
+}
+
+void __pgd_error(const char *file, int line, unsigned long val)
+{
+ printk("%s:%d: bad pgd %016lx.\n", file, line, val);
+}
+
+void __init trap_init(void)
+{
+ return;
+}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
new file mode 100644
index 0000000..17948fc
--- /dev/null
+++ b/arch/arm64/kernel/vdso.c
@@ -0,0 +1,261 @@
+/*
+ * VDSO implementation for AArch64 and vector page setup for AArch32.
+ *
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/clocksource.h>
+#include <linux/elf.h>
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/gfp.h>
+#include <linux/mm.h>
+#include <linux/sched.h>
+#include <linux/signal.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#include <asm/cacheflush.h>
+#include <asm/signal32.h>
+#include <asm/vdso.h>
+#include <asm/vdso_datapage.h>
+
+extern char vdso_start, vdso_end;
+static unsigned long vdso_pages;
+static struct page **vdso_pagelist;
+
+/*
+ * The vDSO data page.
+ */
+static union {
+ struct vdso_data data;
+ u8 page[PAGE_SIZE];
+} vdso_data_store __page_aligned_data;
+struct vdso_data *vdso_data = &vdso_data_store.data;
+
+#ifdef CONFIG_COMPAT
+/*
+ * Create and map the vectors page for AArch32 tasks.
+ */
+static struct page *vectors_page[1];
+
+static int alloc_vectors_page(void)
+{
+ extern char __kuser_helper_start[], __kuser_helper_end[];
+ int kuser_sz = __kuser_helper_end - __kuser_helper_start;
+ unsigned long vpage;
+
+ vpage = get_zeroed_page(GFP_ATOMIC);
+
+ if (!vpage)
+ return -ENOMEM;
+
+ /* kuser helpers */
+ memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
+ kuser_sz);
+
+ /* sigreturn code */
+ memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
+ aarch32_sigret_code, sizeof(aarch32_sigret_code));
+
+ flush_icache_range(vpage, vpage + PAGE_SIZE);
+ vectors_page[0] = virt_to_page(vpage);
+
+ return 0;
+}
+arch_initcall(alloc_vectors_page);
+
+int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ unsigned long addr = AARCH32_VECTORS_BASE;
+ int ret;
+
+ down_write(&mm->mmap_sem);
+ current->mm->context.vdso = (void *)addr;
+
+ /* Map vectors page at the high address. */
+ ret = install_special_mapping(mm, addr, PAGE_SIZE,
+ VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
+ vectors_page);
+
+ up_write(&mm->mmap_sem);
+
+ return ret;
+}
+#endif /* CONFIG_COMPAT */
+
+static int __init vdso_init(void)
+{
+ struct page *pg;
+ char *vbase;
+ int i, ret = 0;
+
+ vdso_pages = (&vdso_end - &vdso_start) >> PAGE_SHIFT;
+ pr_info("vdso: %ld pages (%ld code, %ld data) at base %p\n",
+ vdso_pages + 1, vdso_pages, 1L, &vdso_start);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kzalloc(sizeof(struct page *) * (vdso_pages + 1),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL) {
+ pr_err("Failed to allocate vDSO pagelist!\n");
+ return -ENOMEM;
+ }
+
+ /* Grab the vDSO code pages. */
+ for (i = 0; i < vdso_pages; i++) {
+ pg = virt_to_page(&vdso_start + i*PAGE_SIZE);
+ ClearPageReserved(pg);
+ get_page(pg);
+ vdso_pagelist[i] = pg;
+ }
+
+ /* Sanity check the shared object header. */
+ vbase = vmap(vdso_pagelist, 1, 0, PAGE_KERNEL);
+ if (vbase == NULL) {
+ pr_err("Failed to map vDSO pagelist!\n");
+ return -ENOMEM;
+ } else if (memcmp(vbase, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ ret = -EINVAL;
+ goto unmap;
+ }
+
+ /* Grab the vDSO data page. */
+ pg = virt_to_page(vdso_data);
+ get_page(pg);
+ vdso_pagelist[i] = pg;
+
+unmap:
+ vunmap(vbase);
+ return ret;
+}
+arch_initcall(vdso_init);
+
+int arch_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ unsigned long vdso_base, vdso_mapping_len;
+ int ret;
+
+ /* Be sure to map the data page */
+ vdso_mapping_len = (vdso_pages + 1) << PAGE_SHIFT;
+
+ down_write(&mm->mmap_sem);
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = vdso_base;
+ goto up_fail;
+ }
+ mm->context.vdso = (void *)vdso_base;
+
+ ret = install_special_mapping(mm, vdso_base, vdso_mapping_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_pagelist);
+ if (ret) {
+ mm->context.vdso = NULL;
+ goto up_fail;
+ }
+
+up_fail:
+ up_write(&mm->mmap_sem);
+
+ return ret;
+}
+
+const char *arch_vma_name(struct vm_area_struct *vma)
+{
+ /*
+ * We can re-use the vdso pointer in mm_context_t for identifying
+ * the vectors page for compat applications. The vDSO will always
+ * sit above TASK_UNMAPPED_BASE and so we don't need to worry about
+ * it conflicting with the vectors base.
+ */
+ if (vma->vm_mm && vma->vm_start == (long)vma->vm_mm->context.vdso) {
+#ifdef CONFIG_COMPAT
+ if (vma->vm_start == AARCH32_VECTORS_BASE)
+ return "[vectors]";
+#endif
+ return "[vdso]";
+ }
+
+ return NULL;
+}
+
+/*
+ * We define AT_SYSINFO_EHDR, so we need these function stubs to keep
+ * Linux happy.
+ */
+int in_gate_area_no_mm(unsigned long addr)
+{
+ return 0;
+}
+
+int in_gate_area(struct mm_struct *mm, unsigned long addr)
+{
+ return 0;
+}
+
+struct vm_area_struct *get_gate_vma(struct mm_struct *mm)
+{
+ return NULL;
+}
+
+/*
+ * Update the vDSO data page to keep in sync with kernel timekeeping.
+ */
+void update_vsyscall(struct timespec *ts, struct timespec *wtm,
+ struct clocksource *clock, u32 mult)
+{
+ struct timespec xtime_coarse;
+ u32 use_syscall = strcmp(clock->name, "arch_sys_counter");
+
+ ++vdso_data->tb_seq_count;
+ smp_wmb();
+
+ xtime_coarse = __current_kernel_time();
+ vdso_data->use_syscall = use_syscall;
+ vdso_data->xtime_coarse_sec = xtime_coarse.tv_sec;
+ vdso_data->xtime_coarse_nsec = xtime_coarse.tv_nsec;
+
+ if (!use_syscall) {
+ vdso_data->cs_cycle_last = clock->cycle_last;
+ vdso_data->xtime_clock_sec = ts->tv_sec;
+ vdso_data->xtime_clock_nsec = ts->tv_nsec;
+ vdso_data->cs_mult = mult;
+ vdso_data->cs_shift = clock->shift;
+ vdso_data->wtm_clock_sec = wtm->tv_sec;
+ vdso_data->wtm_clock_nsec = wtm->tv_nsec;
+ }
+
+ smp_wmb();
+ ++vdso_data->tb_seq_count;
+}
+
+void update_vsyscall_tz(void)
+{
+ ++vdso_data->tb_seq_count;
+ smp_wmb();
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
+ smp_wmb();
+ ++vdso_data->tb_seq_count;
+}
diff --git a/arch/arm64/kernel/vdso/.gitignore b/arch/arm64/kernel/vdso/.gitignore
new file mode 100644
index 0000000..b8cc94e
--- /dev/null
+++ b/arch/arm64/kernel/vdso/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso-offsets.h
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
new file mode 100644
index 0000000..d8064af
--- /dev/null
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -0,0 +1,63 @@
+#
+# Building a vDSO image for AArch64.
+#
+# Author: Will Deacon <will.deacon@arm.com>
+# Heavily based on the vDSO Makefiles for other archs.
+#
+
+obj-vdso := gettimeofday.o note.o sigreturn.o
+
+# Build rules
+targets := $(obj-vdso) vdso.so vdso.so.dbg
+obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
+
+ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
+ $(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+
+obj-y += vdso.o
+extra-y += vdso.lds vdso-offsets.h
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (incbin is bad)
+$(obj)/vdso.o : $(obj)/vdso.so
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
+ $(call if_changed,vdsold)
+
+# Strip rule for the .so file
+$(obj)/%.so: OBJCOPYFLAGS := -S
+$(obj)/%.so: $(obj)/%.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+# Generate VDSO offsets using helper script
+gen-vdsosym := $(srctree)/$(src)/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+define cmd_vdsosym
+ $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@ && \
+ cp $@ include/generated/
+endef
+
+$(obj)/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Assembly rules for the .S files
+$(obj-vdso): %.o: %.S
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC) $(c_flags) -Wl,-T $^ -o $@
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso/gen_vdso_offsets.sh b/arch/arm64/kernel/vdso/gen_vdso_offsets.sh
new file mode 100755
index 0000000..01924ff
--- /dev/null
+++ b/arch/arm64/kernel/vdso/gen_vdso_offsets.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+#
+# Match symbols in the DSO that look like VDSO_*; produce a header file
+# of constant offsets into the shared object.
+#
+# Doing this inside the Makefile will break the $(filter-out) function,
+# causing Kbuild to rebuild the vdso-offsets header file every time.
+#
+# Author: Will Deacon <will.deacon@arm.com
+#
+
+LC_ALL=C
+sed -n -e 's/^00*/0/' -e \
+'s/^\([0-9a-fA-F]*\) . VDSO_\([a-zA-Z0-9_]*\)$/\#define vdso_offset_\2\t0x\1/p'
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
new file mode 100644
index 0000000..dcb8c20
--- /dev/null
+++ b/arch/arm64/kernel/vdso/gettimeofday.S
@@ -0,0 +1,242 @@
+/*
+ * Userspace implementations of gettimeofday() and friends.
+ *
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define NSEC_PER_SEC_LO16 0xca00
+#define NSEC_PER_SEC_HI16 0x3b9a
+
+vdso_data .req x6
+use_syscall .req w7
+seqcnt .req w8
+
+ .macro seqcnt_acquire
+9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
+ tbnz seqcnt, #0, 9999b
+ dmb ishld
+ ldr use_syscall, [vdso_data, #VDSO_USE_SYSCALL]
+ .endm
+
+ .macro seqcnt_read, cnt
+ dmb ishld
+ ldr \cnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
+ .endm
+
+ .macro seqcnt_check, cnt, fail
+ cmp \cnt, seqcnt
+ b.ne \fail
+ .endm
+
+ .text
+
+/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
+ENTRY(__kernel_gettimeofday)
+ .cfi_startproc
+ mov x2, x30
+ .cfi_register x30, x2
+
+ /* Acquire the sequence counter and get the timespec. */
+ adr vdso_data, _vdso_data
+1: seqcnt_acquire
+ cbnz use_syscall, 4f
+
+ /* If tv is NULL, skip to the timezone code. */
+ cbz x0, 2f
+ bl __do_get_tspec
+ seqcnt_check w13, 1b
+
+ /* Convert ns to us. */
+ mov x11, #1000
+ udiv x10, x10, x11
+ stp x9, x10, [x0, #TVAL_TV_SEC]
+2:
+ /* If tz is NULL, return 0. */
+ cbz x1, 3f
+ ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
+ seqcnt_read w13
+ seqcnt_check w13, 1b
+ stp w4, w5, [x1, #TZ_MINWEST]
+3:
+ mov x0, xzr
+ ret x2
+4:
+ /* Syscall fallback. */
+ mov x8, #__NR_gettimeofday
+ svc #0
+ ret x2
+ .cfi_endproc
+ENDPROC(__kernel_gettimeofday)
+
+/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
+ENTRY(__kernel_clock_gettime)
+ .cfi_startproc
+ cmp w0, #CLOCK_REALTIME
+ ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
+ b.ne 2f
+
+ mov x2, x30
+ .cfi_register x30, x2
+
+ /* Get kernel timespec. */
+ adr vdso_data, _vdso_data
+1: seqcnt_acquire
+ cbnz use_syscall, 7f
+
+ bl __do_get_tspec
+ seqcnt_check w13, 1b
+
+ cmp w0, #CLOCK_MONOTONIC
+ b.ne 6f
+
+ /* Get wtm timespec. */
+ ldp x14, x15, [vdso_data, #VDSO_WTM_CLK_SEC]
+
+ /* Check the sequence counter. */
+ seqcnt_read w13
+ seqcnt_check w13, 1b
+ b 4f
+2:
+ cmp w0, #CLOCK_REALTIME_COARSE
+ ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
+ b.ne 8f
+
+ /* Get coarse timespec. */
+ adr vdso_data, _vdso_data
+3: seqcnt_acquire
+ ldp x9, x10, [vdso_data, #VDSO_XTIME_CRS_SEC]
+
+ cmp w0, #CLOCK_MONOTONIC_COARSE
+ b.ne 6f
+
+ /* Get wtm timespec. */
+ ldp x14, x15, [vdso_data, #VDSO_WTM_CLK_SEC]
+
+ /* Check the sequence counter. */
+ seqcnt_read w13
+ seqcnt_check w13, 3b
+4:
+ /* Add on wtm timespec. */
+ add x9, x9, x14
+ add x10, x10, x15
+
+ /* Normalise the new timespec. */
+ mov x14, #NSEC_PER_SEC_LO16
+ movk x14, #NSEC_PER_SEC_HI16, lsl #16
+ cmp x10, x14
+ b.lt 5f
+ sub x10, x10, x14
+ add x9, x9, #1
+5:
+ cmp x10, #0
+ b.ge 6f
+ add x10, x10, x14
+ sub x9, x9, #1
+
+6: /* Store to the user timespec. */
+ stp x9, x10, [x1, #TSPEC_TV_SEC]
+ mov x0, xzr
+ ret x2
+7:
+ mov x30, x2
+8: /* Syscall fallback. */
+ mov x8, #__NR_clock_gettime
+ svc #0
+ ret
+ .cfi_endproc
+ENDPROC(__kernel_clock_gettime)
+
+/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
+ENTRY(__kernel_clock_getres)
+ .cfi_startproc
+ cbz w1, 3f
+
+ cmp w0, #CLOCK_REALTIME
+ ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
+ b.ne 1f
+
+ ldr x2, 5f
+ b 2f
+1:
+ cmp w0, #CLOCK_REALTIME_COARSE
+ ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
+ b.ne 4f
+ ldr x2, 6f
+2:
+ stp xzr, x2, [x1]
+
+3: /* res == NULL. */
+ mov w0, wzr
+ ret
+
+4: /* Syscall fallback. */
+ mov x8, #__NR_clock_getres
+ svc #0
+ ret
+5:
+ .quad CLOCK_REALTIME_RES
+6:
+ .quad CLOCK_COARSE_RES
+ .cfi_endproc
+ENDPROC(__kernel_clock_getres)
+
+/*
+ * Read the current time from the architected counter.
+ * Expects vdso_data to be initialised.
+ * Clobbers the temporary registers (x9 - x15).
+ * Returns:
+ * - (x9, x10) = (ts->tv_sec, ts->tv_nsec)
+ * - (x11, x12) = (xtime->tv_sec, xtime->tv_nsec)
+ * - w13 = vDSO sequence counter
+ */
+ENTRY(__do_get_tspec)
+ .cfi_startproc
+
+ /* Read from the vDSO data page. */
+ ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
+ ldp x11, x12, [vdso_data, #VDSO_XTIME_CLK_SEC]
+ ldp w14, w15, [vdso_data, #VDSO_CS_MULT]
+ seqcnt_read w13
+
+ /* Read the physical counter. */
+ isb
+ mrs x9, cntpct_el0
+
+ /* Calculate cycle delta and convert to ns. */
+ sub x10, x9, x10
+ /* We can only guarantee 56 bits of precision. */
+ movn x9, #0xff0, lsl #48
+ and x10, x9, x10
+ mul x10, x10, x14
+ lsr x10, x10, x15
+
+ /* Use the kernel time to calculate the new timespec. */
+ add x10, x12, x10
+ mov x14, #NSEC_PER_SEC_LO16
+ movk x14, #NSEC_PER_SEC_HI16, lsl #16
+ udiv x15, x10, x14
+ add x9, x15, x11
+ mul x14, x14, x15
+ sub x10, x10, x14
+
+ ret
+ .cfi_endproc
+ENDPROC(__do_get_tspec)
diff --git a/arch/arm64/kernel/vdso/note.S b/arch/arm64/kernel/vdso/note.S
new file mode 100644
index 0000000..b82c85e
--- /dev/null
+++ b/arch/arm64/kernel/vdso/note.S
@@ -0,0 +1,28 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+
+ELFNOTE_START(Linux, 0, "a")
+ .long LINUX_VERSION_CODE
+ELFNOTE_END
diff --git a/arch/arm64/kernel/vdso/sigreturn.S b/arch/arm64/kernel/vdso/sigreturn.S
new file mode 100644
index 0000000..20d98ef
--- /dev/null
+++ b/arch/arm64/kernel/vdso/sigreturn.S
@@ -0,0 +1,37 @@
+/*
+ * Sigreturn trampoline for returning from a signal when the SA_RESTORER
+ * flag is not set.
+ *
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/linkage.h>
+#include <asm/unistd.h>
+
+ .text
+
+ nop
+ENTRY(__kernel_rt_sigreturn)
+ .cfi_startproc
+ .cfi_signal_frame
+ .cfi_def_cfa x29, 0
+ .cfi_offset x29, 0 * 8
+ .cfi_offset x30, 1 * 8
+ mov x8, #__NR_rt_sigreturn
+ svc #0
+ .cfi_endproc
+ENDPROC(__kernel_rt_sigreturn)
diff --git a/arch/arm64/kernel/vdso/vdso.S b/arch/arm64/kernel/vdso/vdso.S
new file mode 100644
index 0000000..60c1db5
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vdso.S
@@ -0,0 +1,33 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ __PAGE_ALIGNED_DATA
+
+ .globl vdso_start, vdso_end
+ .balign PAGE_SIZE
+vdso_start:
+ .incbin "arch/arm64/kernel/vdso/vdso.so"
+ .balign PAGE_SIZE
+vdso_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
new file mode 100644
index 0000000..8154b8d
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vdso.lds.S
@@ -0,0 +1,100 @@
+/*
+ * GNU linker script for the VDSO library.
+*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ * Heavily based on the vDSO linker scripts for other archs.
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf64-littleaarch64", "elf64-bigaarch64", "elf64-littleaarch64")
+OUTPUT_ARCH(aarch64)
+
+SECTIONS
+{
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ . = ALIGN(16);
+
+ .text : { *(.text*) } :text =0xd503201f
+ PROVIDE (__etext = .);
+ PROVIDE (_etext = .);
+ PROVIDE (etext = .);
+
+ .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr
+ .eh_frame : { KEEP (*(.eh_frame)) } :text
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ _end = .;
+ PROVIDE(end = .);
+
+ . = ALIGN(PAGE_SIZE);
+ PROVIDE(_vdso_data = .);
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+ eh_frame_hdr PT_GNU_EH_FRAME;
+}
+
+/*
+ * This controls what symbols we export from the DSO.
+ */
+VERSION
+{
+ LINUX_2.6.39 {
+ global:
+ __kernel_rt_sigreturn;
+ __kernel_gettimeofday;
+ __kernel_clock_gettime;
+ __kernel_clock_getres;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_sigtramp = __kernel_rt_sigreturn;
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
new file mode 100644
index 0000000..3fae2be
--- /dev/null
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -0,0 +1,126 @@
+/*
+ * ld script to make ARM Linux kernel
+ * taken from the i386 version by Russell King
+ * Written by Martin Mares <mj@atrey.karlin.mff.cuni.cz>
+ */
+
+#include <asm-generic/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/memory.h>
+#include <asm/page.h>
+
+#define ARM_EXIT_KEEP(x)
+#define ARM_EXIT_DISCARD(x) x
+
+OUTPUT_ARCH(aarch64)
+ENTRY(stext)
+
+jiffies = jiffies_64;
+
+SECTIONS
+{
+ /*
+ * XXX: The linker does not define how output sections are
+ * assigned to input sections when there are multiple statements
+ * matching the same input section name. There is no documented
+ * order of matching.
+ */
+ /DISCARD/ : {
+ ARM_EXIT_DISCARD(EXIT_TEXT)
+ ARM_EXIT_DISCARD(EXIT_DATA)
+ EXIT_CALL
+ *(.discard)
+ *(.discard.*)
+ }
+
+ . = PAGE_OFFSET + TEXT_OFFSET;
+
+ .head.text : {
+ _text = .;
+ HEAD_TEXT
+ }
+ .text : { /* Real text segment */
+ _stext = .; /* Text and read-only data */
+ *(.smp.pen.text)
+ __exception_text_start = .;
+ *(.exception.text)
+ __exception_text_end = .;
+ IRQENTRY_TEXT
+ TEXT_TEXT
+ SCHED_TEXT
+ LOCK_TEXT
+ *(.fixup)
+ *(.gnu.warning)
+ . = ALIGN(16);
+ *(.got) /* Global offset table */
+ }
+
+ RO_DATA(PAGE_SIZE)
+
+ _etext = .; /* End of text and rodata section */
+
+ . = ALIGN(PAGE_SIZE);
+ __init_begin = .;
+
+ INIT_TEXT_SECTION(8)
+ .exit.text : {
+ ARM_EXIT_KEEP(EXIT_TEXT)
+ }
+ . = ALIGN(16);
+ .init.data : {
+ INIT_DATA
+ INIT_SETUP(16)
+ INIT_CALLS
+ CON_INITCALL
+ SECURITY_INITCALL
+ INIT_RAM_FS
+ }
+ .exit.data : {
+ ARM_EXIT_KEEP(EXIT_DATA)
+ }
+
+ PERCPU_SECTION(64)
+
+ __init_end = .;
+ . = ALIGN(THREAD_SIZE);
+ __data_loc = .;
+
+ .data : AT(__data_loc) {
+ _data = .; /* address in memory */
+ _sdata = .;
+
+ /*
+ * first, the init task union, aligned
+ * to an 8192 byte boundary.
+ */
+ INIT_TASK_DATA(THREAD_SIZE)
+ NOSAVE_DATA
+ CACHELINE_ALIGNED_DATA(64)
+ READ_MOSTLY_DATA(64)
+
+ /*
+ * The exception fixup table (might need resorting at runtime)
+ */
+ . = ALIGN(32);
+ __start___ex_table = .;
+ *(__ex_table)
+ __stop___ex_table = .;
+
+ /*
+ * and the usual data section
+ */
+ DATA_DATA
+ CONSTRUCTORS
+
+ _edata = .;
+ }
+ _edata_loc = __data_loc + SIZEOF(.data);
+
+ NOTES
+
+ BSS_SECTION(0, 0, 0)
+ _end = .;
+
+ STABS_DEBUG
+ .comment 0 : { *(.comment) }
+}
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
new file mode 100644
index 0000000..2fb7f60
--- /dev/null
+++ b/arch/arm64/lib/Makefile
@@ -0,0 +1,4 @@
+lib-y := bitops.o delay.o \
+ strncpy_from_user.o strnlen_user.o clear_user.o \
+ copy_from_user.o copy_to_user.o copy_in_user.o \
+ copy_page.o clear_page.o
diff --git a/arch/arm64/lib/bitops.c b/arch/arm64/lib/bitops.c
new file mode 100644
index 0000000..aa4965e
--- /dev/null
+++ b/arch/arm64/lib/bitops.c
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/atomic.h>
+
+#ifdef CONFIG_SMP
+arch_spinlock_t __atomic_hash[ATOMIC_HASH_SIZE] __lock_aligned = {
+ [0 ... (ATOMIC_HASH_SIZE-1)] = __ARCH_SPIN_LOCK_UNLOCKED
+};
+#endif
diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S
new file mode 100644
index 0000000..ef08e90
--- /dev/null
+++ b/arch/arm64/lib/clear_page.S
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/assembler.h>
+#include <asm/page.h>
+
+/*
+ * Clear page @dest
+ *
+ * Parameters:
+ * x0 - dest
+ */
+ENTRY(clear_page)
+ mrs x1, dczid_el0
+ and w1, w1, #0xf
+ mov x2, #4
+ lsl x1, x2, x1
+
+1: dc zva, x0
+ add x0, x0, x1
+ tst x0, #(PAGE_SIZE - 1)
+ b.ne 1b
+ ret
+ENDPROC(clear_page)
diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S
new file mode 100644
index 0000000..6e0ed93
--- /dev/null
+++ b/arch/arm64/lib/clear_user.S
@@ -0,0 +1,58 @@
+/*
+ * Based on arch/arm/lib/clear_user.S
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+ .text
+
+/* Prototype: int __clear_user(void *addr, size_t sz)
+ * Purpose : clear some user memory
+ * Params : addr - user memory address to clear
+ * : sz - number of bytes to clear
+ * Returns : number of bytes NOT cleared
+ *
+ * Alignment fixed up by hardware.
+ */
+ENTRY(__clear_user)
+ mov x2, x1 // save the size for fixup return
+ subs x1, x1, #8
+ b.mi 2f
+1:
+USER(9f, str xzr, [x0], #8 )
+ subs x1, x1, #8
+ b.pl 1b
+2: adds x1, x1, #4
+ b.mi 3f
+USER(9f, str wzr, [x0], #4 )
+ sub x1, x1, #4
+3: adds x1, x1, #2
+ b.mi 4f
+USER(9f, strh wzr, [x0], #2 )
+ sub x1, x1, #2
+4: adds x1, x1, #1
+ b.mi 5f
+ strb wzr, [x0]
+5: mov x0, #0
+ ret
+ENDPROC(__clear_user)
+
+ .section .fixup,"ax"
+ .align 2
+9: mov x0, x2 // return the original size
+ ret
+ .previous
diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S
new file mode 100644
index 0000000..5e27add
--- /dev/null
+++ b/arch/arm64/lib/copy_from_user.S
@@ -0,0 +1,66 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * Copy from user space to a kernel buffer (alignment handled by the hardware)
+ *
+ * Parameters:
+ * x0 - to
+ * x1 - from
+ * x2 - n
+ * Returns:
+ * x0 - bytes not copied
+ */
+ENTRY(__copy_from_user)
+ add x4, x1, x2 // upper user buffer boundary
+ subs x2, x2, #8
+ b.mi 2f
+1:
+USER(9f, ldr x3, [x1], #8 )
+ subs x2, x2, #8
+ str x3, [x0], #8
+ b.pl 1b
+2: adds x2, x2, #4
+ b.mi 3f
+USER(9f, ldr w3, [x1], #4 )
+ sub x2, x2, #4
+ str w3, [x0], #4
+3: adds x2, x2, #2
+ b.mi 4f
+USER(9f, ldrh w3, [x1], #2 )
+ sub x2, x2, #2
+ strh w3, [x0], #2
+4: adds x2, x2, #1
+ b.mi 5f
+USER(9f, ldrb w3, [x1] )
+ strb w3, [x0]
+5: mov x0, #0
+ ret
+ENDPROC(__copy_from_user)
+
+ .section .fixup,"ax"
+ .align 2
+9: sub x2, x4, x1
+ mov x3, x2
+10: strb wzr, [x0], #1 // zero remaining buffer space
+ subs x3, x3, #1
+ b.ne 10b
+ mov x0, x2 // bytes not copied
+ ret
+ .previous
diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S
new file mode 100644
index 0000000..84b6c9b
--- /dev/null
+++ b/arch/arm64/lib/copy_in_user.S
@@ -0,0 +1,63 @@
+/*
+ * Copy from user space to user space
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * Copy from user space to user space (alignment handled by the hardware)
+ *
+ * Parameters:
+ * x0 - to
+ * x1 - from
+ * x2 - n
+ * Returns:
+ * x0 - bytes not copied
+ */
+ENTRY(__copy_in_user)
+ add x4, x0, x2 // upper user buffer boundary
+ subs x2, x2, #8
+ b.mi 2f
+1:
+USER(9f, ldr x3, [x1], #8 )
+ subs x2, x2, #8
+USER(9f, str x3, [x0], #8 )
+ b.pl 1b
+2: adds x2, x2, #4
+ b.mi 3f
+USER(9f, ldr w3, [x1], #4 )
+ sub x2, x2, #4
+USER(9f, str w3, [x0], #4 )
+3: adds x2, x2, #2
+ b.mi 4f
+USER(9f, ldrh w3, [x1], #2 )
+ sub x2, x2, #2
+USER(9f, strh w3, [x0], #2 )
+4: adds x2, x2, #1
+ b.mi 5f
+USER(9f, ldrb w3, [x1] )
+USER(9f, strb w3, [x0] )
+5: mov x0, #0
+ ret
+ENDPROC(__copy_in_user)
+
+ .section .fixup,"ax"
+ .align 2
+9: sub x0, x4, x0 // bytes not copied
+ ret
+ .previous
diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S
new file mode 100644
index 0000000..512b9a7
--- /dev/null
+++ b/arch/arm64/lib/copy_page.S
@@ -0,0 +1,46 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/assembler.h>
+#include <asm/page.h>
+
+/*
+ * Copy a page from src to dest (both are page aligned)
+ *
+ * Parameters:
+ * x0 - dest
+ * x1 - src
+ */
+ENTRY(copy_page)
+ /* Assume cache line size is 64 bytes. */
+ prfm pldl1strm, [x1, #64]
+1: ldp x2, x3, [x1]
+ ldp x4, x5, [x1, #16]
+ ldp x6, x7, [x1, #32]
+ ldp x8, x9, [x1, #48]
+ add x1, x1, #64
+ prfm pldl1strm, [x1, #64]
+ stnp x2, x3, [x0]
+ stnp x4, x5, [x0, #16]
+ stnp x6, x7, [x0, #32]
+ stnp x8, x9, [x0, #48]
+ add x0, x0, #64
+ tst x1, #(PAGE_SIZE - 1)
+ b.ne 1b
+ ret
+ENDPROC(copy_page)
diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S
new file mode 100644
index 0000000..a0aeeb9
--- /dev/null
+++ b/arch/arm64/lib/copy_to_user.S
@@ -0,0 +1,61 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+
+/*
+ * Copy to user space from a kernel buffer (alignment handled by the hardware)
+ *
+ * Parameters:
+ * x0 - to
+ * x1 - from
+ * x2 - n
+ * Returns:
+ * x0 - bytes not copied
+ */
+ENTRY(__copy_to_user)
+ add x4, x0, x2 // upper user buffer boundary
+ subs x2, x2, #8
+ b.mi 2f
+1:
+ ldr x3, [x1], #8
+ subs x2, x2, #8
+USER(9f, str x3, [x0], #8 )
+ b.pl 1b
+2: adds x2, x2, #4
+ b.mi 3f
+ ldr w3, [x1], #4
+ sub x2, x2, #4
+USER(9f, str w3, [x0], #4 )
+3: adds x2, x2, #2
+ b.mi 4f
+ ldrh w3, [x1], #2
+ sub x2, x2, #2
+USER(9f, strh w3, [x0], #2 )
+4: adds x2, x2, #1
+ b.mi 5f
+ ldrb w3, [x1]
+USER(9f, strb w3, [x0] )
+5: mov x0, #0
+ ret
+ENDPROC(__copy_to_user)
+
+ .section .fixup,"ax"
+ .align 2
+9: sub x0, x4, x0 // bytes not copied
+ ret
+ .previous
diff --git a/arch/arm64/lib/delay.c b/arch/arm64/lib/delay.c
new file mode 100644
index 0000000..dad4ec9
--- /dev/null
+++ b/arch/arm64/lib/delay.c
@@ -0,0 +1,55 @@
+/*
+ * Delay loops based on the OpenRISC implementation.
+ *
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/timex.h>
+
+void __delay(unsigned long cycles)
+{
+ cycles_t start = get_cycles();
+
+ while ((get_cycles() - start) < cycles)
+ cpu_relax();
+}
+EXPORT_SYMBOL(__delay);
+
+inline void __const_udelay(unsigned long xloops)
+{
+ unsigned long loops;
+
+ loops = xloops * loops_per_jiffy * HZ;
+ __delay(loops >> 32);
+}
+EXPORT_SYMBOL(__const_udelay);
+
+void __udelay(unsigned long usecs)
+{
+ __const_udelay(usecs * 0x10C7UL); /* 2**32 / 1000000 (rounded up) */
+}
+EXPORT_SYMBOL(__udelay);
+
+void __ndelay(unsigned long nsecs)
+{
+ __const_udelay(nsecs * 0x5UL); /* 2**32 / 1000000000 (rounded up) */
+}
+EXPORT_SYMBOL(__ndelay);
diff --git a/arch/arm64/lib/strncpy_from_user.S b/arch/arm64/lib/strncpy_from_user.S
new file mode 100644
index 0000000..56e448a
--- /dev/null
+++ b/arch/arm64/lib/strncpy_from_user.S
@@ -0,0 +1,50 @@
+/*
+ * Based on arch/arm/lib/strncpy_from_user.S
+ *
+ * Copyright (C) 1995-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+#include <asm/errno.h>
+
+ .text
+ .align 5
+
+/*
+ * Copy a string from user space to kernel space.
+ * x0 = dst, x1 = src, x2 = byte length
+ * returns the number of characters copied (strlen of copied string),
+ * -EFAULT on exception, or "len" if we fill the whole buffer
+ */
+ENTRY(__strncpy_from_user)
+ mov x4, x1
+1: subs x2, x2, #1
+ bmi 2f
+USER(9f, ldrb w3, [x1], #1 )
+ strb w3, [x0], #1
+ cbnz w3, 1b
+ sub x1, x1, #1 // take NUL character out of count
+2: sub x0, x1, x4
+ ret
+ENDPROC(__strncpy_from_user)
+
+ .section .fixup,"ax"
+ .align 0
+9: strb wzr, [x0] // null terminate
+ mov x0, #-EFAULT
+ ret
+ .previous
diff --git a/arch/arm64/lib/strnlen_user.S b/arch/arm64/lib/strnlen_user.S
new file mode 100644
index 0000000..7f7b176
--- /dev/null
+++ b/arch/arm64/lib/strnlen_user.S
@@ -0,0 +1,47 @@
+/*
+ * Based on arch/arm/lib/strnlen_user.S
+ *
+ * Copyright (C) 1995-2000 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+#include <asm/errno.h>
+
+ .text
+ .align 5
+
+/* Prototype: unsigned long __strnlen_user(const char *str, long n)
+ * Purpose : get length of a string in user memory
+ * Params : str - address of string in user memory
+ * Returns : length of string *including terminator*
+ * or zero on exception, or n if too long
+ */
+ENTRY(__strnlen_user)
+ mov x2, x0
+1: subs x1, x1, #1
+ b.mi 2f
+USER(9f, ldrb w3, [x0], #1 )
+ cbnz w3, 1b
+2: sub x0, x0, x2
+ ret
+ENDPROC(__strnlen_user)
+
+ .section .fixup,"ax"
+ .align 0
+9: mov x0, #0
+ ret
+ .previous
diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile
new file mode 100644
index 0000000..3140a2a
--- /dev/null
+++ b/arch/arm64/mm/Makefile
@@ -0,0 +1,4 @@
+obj-y := dma-mapping.o extable.o fault.o init.o \
+ cache.o copypage.o flush.o \
+ ioremap.o mmap.o pgd.o mmu.o \
+ context.o tlb.o proc.o
diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
new file mode 100644
index 0000000..abe69b8
--- /dev/null
+++ b/arch/arm64/mm/cache.S
@@ -0,0 +1,168 @@
+/*
+ * Cache maintenance
+ *
+ * Copyright (C) 2001 Deep Blue Solutions Ltd.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+#include <linux/init.h>
+#include <asm/assembler.h>
+
+#include "proc-macros.S"
+
+/*
+ * __flush_dcache_all()
+ *
+ * Flush the whole D-cache.
+ *
+ * Corrupted registers: x0-x7, x9-x11
+ */
+ENTRY(__flush_dcache_all)
+ dsb sy // ensure ordering with previous memory accesses
+ mrs x0, clidr_el1 // read clidr
+ and x3, x0, #0x7000000 // extract loc from clidr
+ lsr x3, x3, #23 // left align loc bit field
+ cbz x3, finished // if loc is 0, then no need to clean
+ mov x10, #0 // start clean at cache level 0
+loop1:
+ add x2, x10, x10, lsr #1 // work out 3x current cache level
+ lsr x1, x0, x2 // extract cache type bits from clidr
+ and x1, x1, #7 // mask of the bits for current cache only
+ cmp x1, #2 // see what cache we have at this level
+ b.lt skip // skip if no cache, or just i-cache
+ save_and_disable_irqs x9 // make CSSELR and CCSIDR access atomic
+ msr csselr_el1, x10 // select current cache level in csselr
+ isb // isb to sych the new cssr&csidr
+ mrs x1, ccsidr_el1 // read the new ccsidr
+ restore_irqs x9
+ and x2, x1, #7 // extract the length of the cache lines
+ add x2, x2, #4 // add 4 (line length offset)
+ mov x4, #0x3ff
+ and x4, x4, x1, lsr #3 // find maximum number on the way size
+ clz x5, x4 // find bit position of way size increment
+ mov x7, #0x7fff
+ and x7, x7, x1, lsr #13 // extract max number of the index size
+loop2:
+ mov x9, x4 // create working copy of max way size
+loop3:
+ lsl x6, x9, x5
+ orr x11, x10, x6 // factor way and cache number into x11
+ lsl x6, x7, x2
+ orr x11, x11, x6 // factor index number into x11
+ dc cisw, x11 // clean & invalidate by set/way
+ subs x9, x9, #1 // decrement the way
+ b.ge loop3
+ subs x7, x7, #1 // decrement the index
+ b.ge loop2
+skip:
+ add x10, x10, #2 // increment cache number
+ cmp x3, x10
+ b.gt loop1
+finished:
+ mov x10, #0 // swith back to cache level 0
+ msr csselr_el1, x10 // select current cache level in csselr
+ dsb sy
+ isb
+ ret
+ENDPROC(__flush_dcache_all)
+
+/*
+ * flush_cache_all()
+ *
+ * Flush the entire cache system. The data cache flush is now achieved
+ * using atomic clean / invalidates working outwards from L1 cache. This
+ * is done using Set/Way based cache maintainance instructions. The
+ * instruction cache can still be invalidated back to the point of
+ * unification in a single instruction.
+ */
+ENTRY(flush_cache_all)
+ mov x12, lr
+ bl __flush_dcache_all
+ mov x0, #0
+ ic ialluis // I+BTB cache invalidate
+ ret x12
+ENDPROC(flush_cache_all)
+
+/*
+ * flush_icache_range(start,end)
+ *
+ * Ensure that the I and D caches are coherent within specified region.
+ * This is typically used when code has been written to a memory region,
+ * and will be executed.
+ *
+ * - start - virtual start address of region
+ * - end - virtual end address of region
+ */
+ENTRY(flush_icache_range)
+ /* FALLTHROUGH */
+
+/*
+ * __flush_cache_user_range(start,end)
+ *
+ * Ensure that the I and D caches are coherent within specified region.
+ * This is typically used when code has been written to a memory region,
+ * and will be executed.
+ *
+ * - start - virtual start address of region
+ * - end - virtual end address of region
+ */
+ENTRY(__flush_cache_user_range)
+ dcache_line_size x2, x3
+ sub x3, x2, #1
+ bic x4, x0, x3
+1:
+USER(9f, dc cvau, x4 ) // clean D line to PoU
+ add x4, x4, x2
+ cmp x4, x1
+ b.lo 1b
+ dsb sy
+
+ icache_line_size x2, x3
+ sub x3, x2, #1
+ bic x4, x0, x3
+1:
+USER(9f, ic ivau, x4 ) // invalidate I line PoU
+ add x4, x4, x2
+ cmp x4, x1
+ b.lo 1b
+9: // ignore any faulting cache operation
+ dsb sy
+ isb
+ ret
+ENDPROC(flush_icache_range)
+ENDPROC(__flush_cache_user_range)
+
+/*
+ * __flush_kern_dcache_page(kaddr)
+ *
+ * Ensure that the data held in the page kaddr is written back to the
+ * page in question.
+ *
+ * - kaddr - kernel address
+ * - size - size in question
+ */
+ENTRY(__flush_dcache_area)
+ dcache_line_size x2, x3
+ add x1, x0, x1
+ sub x3, x2, #1
+ bic x0, x0, x3
+1: dc civac, x0 // clean & invalidate D line / unified line
+ add x0, x0, x2
+ cmp x0, x1
+ b.lo 1b
+ dsb sy
+ ret
+ENDPROC(__flush_dcache_area)
diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c
new file mode 100644
index 0000000..baa758d
--- /dev/null
+++ b/arch/arm64/mm/context.c
@@ -0,0 +1,159 @@
+/*
+ * Based on arch/arm/mm/context.c
+ *
+ * Copyright (C) 2002-2003 Deep Blue Solutions Ltd, all rights reserved.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/smp.h>
+#include <linux/percpu.h>
+
+#include <asm/mmu_context.h>
+#include <asm/tlbflush.h>
+#include <asm/cachetype.h>
+
+#define asid_bits(reg) \
+ (((read_cpuid(ID_AA64MMFR0_EL1) & 0xf0) >> 2) + 8)
+
+#define ASID_FIRST_VERSION (1 << MAX_ASID_BITS)
+
+static DEFINE_RAW_SPINLOCK(cpu_asid_lock);
+unsigned int cpu_last_asid = ASID_FIRST_VERSION;
+
+/*
+ * We fork()ed a process, and we need a new context for the child to run in.
+ */
+void __init_new_context(struct task_struct *tsk, struct mm_struct *mm)
+{
+ mm->context.id = 0;
+ raw_spin_lock_init(&mm->context.id_lock);
+}
+
+static void flush_context(void)
+{
+ /* set the reserved TTBR0 before flushing the TLB */
+ cpu_set_reserved_ttbr0();
+ flush_tlb_all();
+ if (icache_is_aivivt())
+ __flush_icache_all();
+}
+
+#ifdef CONFIG_SMP
+
+static void set_mm_context(struct mm_struct *mm, unsigned int asid)
+{
+ unsigned long flags;
+
+ /*
+ * Locking needed for multi-threaded applications where the same
+ * mm->context.id could be set from different CPUs during the
+ * broadcast. This function is also called via IPI so the
+ * mm->context.id_lock has to be IRQ-safe.
+ */
+ raw_spin_lock_irqsave(&mm->context.id_lock, flags);
+ if (likely((mm->context.id ^ cpu_last_asid) >> MAX_ASID_BITS)) {
+ /*
+ * Old version of ASID found. Set the new one and reset
+ * mm_cpumask(mm).
+ */
+ mm->context.id = asid;
+ cpumask_clear(mm_cpumask(mm));
+ }
+ raw_spin_unlock_irqrestore(&mm->context.id_lock, flags);
+
+ /*
+ * Set the mm_cpumask(mm) bit for the current CPU.
+ */
+ cpumask_set_cpu(smp_processor_id(), mm_cpumask(mm));
+}
+
+/*
+ * Reset the ASID on the current CPU. This function call is broadcast from the
+ * CPU handling the ASID rollover and holding cpu_asid_lock.
+ */
+static void reset_context(void *info)
+{
+ unsigned int asid;
+ unsigned int cpu = smp_processor_id();
+ struct mm_struct *mm = current->active_mm;
+
+ smp_rmb();
+ asid = cpu_last_asid + cpu;
+
+ flush_context();
+ set_mm_context(mm, asid);
+
+ /* set the new ASID */
+ cpu_switch_mm(mm->pgd, mm);
+}
+
+#else
+
+static inline void set_mm_context(struct mm_struct *mm, unsigned int asid)
+{
+ mm->context.id = asid;
+ cpumask_copy(mm_cpumask(mm), cpumask_of(smp_processor_id()));
+}
+
+#endif
+
+void __new_context(struct mm_struct *mm)
+{
+ unsigned int asid;
+ unsigned int bits = asid_bits();
+
+ raw_spin_lock(&cpu_asid_lock);
+#ifdef CONFIG_SMP
+ /*
+ * Check the ASID again, in case the change was broadcast from another
+ * CPU before we acquired the lock.
+ */
+ if (!unlikely((mm->context.id ^ cpu_last_asid) >> MAX_ASID_BITS)) {
+ cpumask_set_cpu(smp_processor_id(), mm_cpumask(mm));
+ raw_spin_unlock(&cpu_asid_lock);
+ return;
+ }
+#endif
+ /*
+ * At this point, it is guaranteed that the current mm (with an old
+ * ASID) isn't active on any other CPU since the ASIDs are changed
+ * simultaneously via IPI.
+ */
+ asid = ++cpu_last_asid;
+
+ /*
+ * If we've used up all our ASIDs, we need to start a new version and
+ * flush the TLB.
+ */
+ if (unlikely((asid & ((1 << bits) - 1)) == 0)) {
+ /* increment the ASID version */
+ cpu_last_asid += (1 << MAX_ASID_BITS) - (1 << bits);
+ if (cpu_last_asid == 0)
+ cpu_last_asid = ASID_FIRST_VERSION;
+ asid = cpu_last_asid + smp_processor_id();
+ flush_context();
+#ifdef CONFIG_SMP
+ smp_wmb();
+ smp_call_function(reset_context, NULL, 1);
+#endif
+ cpu_last_asid += NR_CPUS - 1;
+ }
+
+ set_mm_context(mm, asid);
+ raw_spin_unlock(&cpu_asid_lock);
+}
diff --git a/arch/arm64/mm/copypage.c b/arch/arm64/mm/copypage.c
new file mode 100644
index 0000000..9aecbac
--- /dev/null
+++ b/arch/arm64/mm/copypage.c
@@ -0,0 +1,34 @@
+/*
+ * Based on arch/arm/mm/copypage.c
+ *
+ * Copyright (C) 2002 Deep Blue Solutions Ltd, All Rights Reserved.
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/mm.h>
+
+#include <asm/page.h>
+#include <asm/cacheflush.h>
+
+void __cpu_copy_user_page(void *kto, const void *kfrom, unsigned long vaddr)
+{
+ copy_page(kto, kfrom);
+ __flush_dcache_area(kto, PAGE_SIZE);
+}
+
+void __cpu_clear_user_page(void *kaddr, unsigned long vaddr)
+{
+ clear_page(kaddr);
+}
diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c
new file mode 100644
index 0000000..5eb2444
--- /dev/null
+++ b/arch/arm64/mm/dma-mapping.c
@@ -0,0 +1,79 @@
+/*
+ * SWIOTLB-based DMA API implementation
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/gfp.h>
+#include <linux/export.h>
+#include <linux/slab.h>
+#include <linux/dma-mapping.h>
+#include <linux/vmalloc.h>
+#include <linux/swiotlb.h>
+
+#include <asm/cacheflush.h>
+
+struct dma_map_ops *dma_ops;
+EXPORT_SYMBOL(dma_ops);
+
+static void *arm64_swiotlb_alloc_coherent(struct device *dev, size_t size,
+ dma_addr_t *dma_handle, gfp_t flags,
+ struct dma_attrs *attrs)
+{
+ if (IS_ENABLED(CONFIG_ZONE_DMA32) &&
+ dev->coherent_dma_mask <= DMA_BIT_MASK(32))
+ flags |= GFP_DMA32;
+ return swiotlb_alloc_coherent(dev, size, dma_handle, flags);
+}
+
+static void arm64_swiotlb_free_coherent(struct device *dev, size_t size,
+ void *vaddr, dma_addr_t dma_handle,
+ struct dma_attrs *attrs)
+{
+ swiotlb_free_coherent(dev, size, vaddr, dma_handle);
+}
+
+static struct dma_map_ops arm64_swiotlb_dma_ops = {
+ .alloc = arm64_swiotlb_alloc_coherent,
+ .free = arm64_swiotlb_free_coherent,
+ .map_page = swiotlb_map_page,
+ .unmap_page = swiotlb_unmap_page,
+ .map_sg = swiotlb_map_sg_attrs,
+ .unmap_sg = swiotlb_unmap_sg_attrs,
+ .sync_single_for_cpu = swiotlb_sync_single_for_cpu,
+ .sync_single_for_device = swiotlb_sync_single_for_device,
+ .sync_sg_for_cpu = swiotlb_sync_sg_for_cpu,
+ .sync_sg_for_device = swiotlb_sync_sg_for_device,
+ .dma_supported = swiotlb_dma_supported,
+ .mapping_error = swiotlb_dma_mapping_error,
+};
+
+void __init swiotlb_init_with_default_size(size_t default_size, int verbose);
+
+void __init arm64_swiotlb_init(size_t max_size)
+{
+ dma_ops = &arm64_swiotlb_dma_ops;
+ swiotlb_init_with_default_size(min((size_t)SZ_64M, max_size), 1);
+}
+
+#define PREALLOC_DMA_DEBUG_ENTRIES 4096
+
+static int __init dma_debug_do_init(void)
+{
+ dma_debug_init(PREALLOC_DMA_DEBUG_ENTRIES);
+ return 0;
+}
+fs_initcall(dma_debug_do_init);
diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c
new file mode 100644
index 0000000..79444279
--- /dev/null
+++ b/arch/arm64/mm/extable.c
@@ -0,0 +1,17 @@
+/*
+ * Based on arch/arm/mm/extable.c
+ */
+
+#include <linux/module.h>
+#include <linux/uaccess.h>
+
+int fixup_exception(struct pt_regs *regs)
+{
+ const struct exception_table_entry *fixup;
+
+ fixup = search_exception_tables(instruction_pointer(regs));
+ if (fixup)
+ regs->pc = fixup->fixup;
+
+ return fixup != NULL;
+}
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
new file mode 100644
index 0000000..1909a69
--- /dev/null
+++ b/arch/arm64/mm/fault.c
@@ -0,0 +1,534 @@
+/*
+ * Based on arch/arm/mm/fault.c
+ *
+ * Copyright (C) 1995 Linus Torvalds
+ * Copyright (C) 1995-2004 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/module.h>
+#include <linux/signal.h>
+#include <linux/mm.h>
+#include <linux/hardirq.h>
+#include <linux/init.h>
+#include <linux/kprobes.h>
+#include <linux/uaccess.h>
+#include <linux/page-flags.h>
+#include <linux/sched.h>
+#include <linux/highmem.h>
+#include <linux/perf_event.h>
+
+#include <asm/exception.h>
+#include <asm/debug-monitors.h>
+#include <asm/system_misc.h>
+#include <asm/pgtable.h>
+#include <asm/tlbflush.h>
+
+/*
+ * Dump out the page tables associated with 'addr' in mm 'mm'.
+ */
+void show_pte(struct mm_struct *mm, unsigned long addr)
+{
+ pgd_t *pgd;
+
+ if (!mm)
+ mm = &init_mm;
+
+ pr_alert("pgd = %p\n", mm->pgd);
+ pgd = pgd_offset(mm, addr);
+ pr_alert("[%08lx] *pgd=%016llx", addr, pgd_val(*pgd));
+
+ do {
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ if (pgd_none_or_clear_bad(pgd))
+ break;
+
+ pud = pud_offset(pgd, addr);
+ if (pud_none_or_clear_bad(pud))
+ break;
+
+ pmd = pmd_offset(pud, addr);
+ printk(", *pmd=%016llx", pmd_val(*pmd));
+ if (pmd_none_or_clear_bad(pmd))
+ break;
+
+ pte = pte_offset_map(pmd, addr);
+ printk(", *pte=%016llx", pte_val(*pte));
+ pte_unmap(pte);
+ } while(0);
+
+ printk("\n");
+}
+
+/*
+ * The kernel tried to access some page that wasn't present.
+ */
+static void __do_kernel_fault(struct mm_struct *mm, unsigned long addr,
+ unsigned int esr, struct pt_regs *regs)
+{
+ /*
+ * Are we prepared to handle this kernel fault?
+ */
+ if (fixup_exception(regs))
+ return;
+
+ /*
+ * No handler, we'll have to terminate things with extreme prejudice.
+ */
+ bust_spinlocks(1);
+ pr_alert("Unable to handle kernel %s at virtual address %08lx\n",
+ (addr < PAGE_SIZE) ? "NULL pointer dereference" :
+ "paging request", addr);
+
+ show_pte(mm, addr);
+ die("Oops", regs, esr);
+ bust_spinlocks(0);
+ do_exit(SIGKILL);
+}
+
+/*
+ * Something tried to access memory that isn't in our memory map. User mode
+ * accesses just cause a SIGSEGV
+ */
+static void __do_user_fault(struct task_struct *tsk, unsigned long addr,
+ unsigned int esr, unsigned int sig, int code,
+ struct pt_regs *regs)
+{
+ struct siginfo si;
+
+ if (show_unhandled_signals) {
+ pr_info("%s[%d]: unhandled page fault (%d) at 0x%08lx, code 0x%03x\n",
+ tsk->comm, task_pid_nr(tsk), sig, addr, esr);
+ show_pte(tsk->mm, addr);
+ show_regs(regs);
+ }
+
+ tsk->thread.fault_address = addr;
+ si.si_signo = sig;
+ si.si_errno = 0;
+ si.si_code = code;
+ si.si_addr = (void __user *)addr;
+ force_sig_info(sig, &si, tsk);
+}
+
+void do_bad_area(unsigned long addr, unsigned int esr, struct pt_regs *regs)
+{
+ struct task_struct *tsk = current;
+ struct mm_struct *mm = tsk->active_mm;
+
+ /*
+ * If we are in kernel mode at this point, we have no context to
+ * handle this fault with.
+ */
+ if (user_mode(regs))
+ __do_user_fault(tsk, addr, esr, SIGSEGV, SEGV_MAPERR, regs);
+ else
+ __do_kernel_fault(mm, addr, esr, regs);
+}
+
+#define VM_FAULT_BADMAP 0x010000
+#define VM_FAULT_BADACCESS 0x020000
+
+#define ESR_WRITE (1 << 6)
+#define ESR_LNX_EXEC (1 << 24)
+
+/*
+ * Check that the permissions on the VMA allow for the fault which occurred.
+ * If we encountered a write fault, we must have write permission, otherwise
+ * we allow any permission.
+ */
+static inline bool access_error(unsigned int esr, struct vm_area_struct *vma)
+{
+ unsigned int mask = VM_READ | VM_WRITE | VM_EXEC;
+
+ if (esr & ESR_WRITE)
+ mask = VM_WRITE;
+ if (esr & ESR_LNX_EXEC)
+ mask = VM_EXEC;
+
+ return vma->vm_flags & mask ? false : true;
+}
+
+static int __do_page_fault(struct mm_struct *mm, unsigned long addr,
+ unsigned int esr, unsigned int flags,
+ struct task_struct *tsk)
+{
+ struct vm_area_struct *vma;
+ int fault;
+
+ vma = find_vma(mm, addr);
+ fault = VM_FAULT_BADMAP;
+ if (unlikely(!vma))
+ goto out;
+ if (unlikely(vma->vm_start > addr))
+ goto check_stack;
+
+ /*
+ * Ok, we have a good vm_area for this memory access, so we can handle
+ * it.
+ */
+good_area:
+ if (access_error(esr, vma)) {
+ fault = VM_FAULT_BADACCESS;
+ goto out;
+ }
+
+ return handle_mm_fault(mm, vma, addr & PAGE_MASK, flags);
+
+check_stack:
+ if (vma->vm_flags & VM_GROWSDOWN && !expand_stack(vma, addr))
+ goto good_area;
+out:
+ return fault;
+}
+
+static int __kprobes do_page_fault(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ struct task_struct *tsk;
+ struct mm_struct *mm;
+ int fault, sig, code;
+ int write = esr & ESR_WRITE;
+ unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE |
+ (write ? FAULT_FLAG_WRITE : 0);
+
+ tsk = current;
+ mm = tsk->mm;
+
+ /* Enable interrupts if they were enabled in the parent context. */
+ if (interrupts_enabled(regs))
+ local_irq_enable();
+
+ /*
+ * If we're in an interrupt or have no user context, we must not take
+ * the fault.
+ */
+ if (in_atomic() || !mm)
+ goto no_context;
+
+ /*
+ * As per x86, we may deadlock here. However, since the kernel only
+ * validly references user space from well defined areas of the code,
+ * we can bug out early if this is from code which shouldn't.
+ */
+ if (!down_read_trylock(&mm->mmap_sem)) {
+ if (!user_mode(regs) && !search_exception_tables(regs->pc))
+ goto no_context;
+retry:
+ down_read(&mm->mmap_sem);
+ } else {
+ /*
+ * The above down_read_trylock() might have succeeded in which
+ * case, we'll have missed the might_sleep() from down_read().
+ */
+ might_sleep();
+#ifdef CONFIG_DEBUG_VM
+ if (!user_mode(regs) && !search_exception_tables(regs->pc))
+ goto no_context;
+#endif
+ }
+
+ fault = __do_page_fault(mm, addr, esr, flags, tsk);
+
+ /*
+ * If we need to retry but a fatal signal is pending, handle the
+ * signal first. We do not need to release the mmap_sem because it
+ * would already be released in __lock_page_or_retry in mm/filemap.c.
+ */
+ if ((fault & VM_FAULT_RETRY) && fatal_signal_pending(current))
+ return 0;
+
+ /*
+ * Major/minor page fault accounting is only done on the initial
+ * attempt. If we go through a retry, it is extremely likely that the
+ * page will be found in page cache at that point.
+ */
+
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr);
+ if (flags & FAULT_FLAG_ALLOW_RETRY) {
+ if (fault & VM_FAULT_MAJOR) {
+ tsk->maj_flt++;
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1, regs,
+ addr);
+ } else {
+ tsk->min_flt++;
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1, regs,
+ addr);
+ }
+ if (fault & VM_FAULT_RETRY) {
+ /*
+ * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk of
+ * starvation.
+ */
+ flags &= ~FAULT_FLAG_ALLOW_RETRY;
+ goto retry;
+ }
+ }
+
+ up_read(&mm->mmap_sem);
+
+ /*
+ * Handle the "normal" case first - VM_FAULT_MAJOR / VM_FAULT_MINOR
+ */
+ if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP |
+ VM_FAULT_BADACCESS))))
+ return 0;
+
+ if (fault & VM_FAULT_OOM) {
+ /*
+ * We ran out of memory, call the OOM killer, and return to
+ * userspace (which will retry the fault, or kill us if we got
+ * oom-killed).
+ */
+ pagefault_out_of_memory();
+ return 0;
+ }
+
+ /*
+ * If we are in kernel mode at this point, we have no context to
+ * handle this fault with.
+ */
+ if (!user_mode(regs))
+ goto no_context;
+
+ if (fault & VM_FAULT_SIGBUS) {
+ /*
+ * We had some memory, but were unable to successfully fix up
+ * this page fault.
+ */
+ sig = SIGBUS;
+ code = BUS_ADRERR;
+ } else {
+ /*
+ * Something tried to access memory that isn't in our memory
+ * map.
+ */
+ sig = SIGSEGV;
+ code = fault == VM_FAULT_BADACCESS ?
+ SEGV_ACCERR : SEGV_MAPERR;
+ }
+
+ __do_user_fault(tsk, addr, esr, sig, code, regs);
+ return 0;
+
+no_context:
+ __do_kernel_fault(mm, addr, esr, regs);
+ return 0;
+}
+
+/*
+ * First Level Translation Fault Handler
+ *
+ * We enter here because the first level page table doesn't contain a valid
+ * entry for the address.
+ *
+ * If the address is in kernel space (>= TASK_SIZE), then we are probably
+ * faulting in the vmalloc() area.
+ *
+ * If the init_task's first level page tables contains the relevant entry, we
+ * copy the it to this task. If not, we send the process a signal, fixup the
+ * exception, or oops the kernel.
+ *
+ * NOTE! We MUST NOT take any locks for this case. We may be in an interrupt
+ * or a critical region, and should only copy the information from the master
+ * page table, nothing more.
+ */
+static int __kprobes do_translation_fault(unsigned long addr,
+ unsigned int esr,
+ struct pt_regs *regs)
+{
+ if (addr < TASK_SIZE)
+ return do_page_fault(addr, esr, regs);
+
+ do_bad_area(addr, esr, regs);
+ return 0;
+}
+
+/*
+ * Some section permission faults need to be handled gracefully. They can
+ * happen due to a __{get,put}_user during an oops.
+ */
+static int do_sect_fault(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ do_bad_area(addr, esr, regs);
+ return 0;
+}
+
+/*
+ * This abort handler always returns "fault".
+ */
+static int do_bad(unsigned long addr, unsigned int esr, struct pt_regs *regs)
+{
+ return 1;
+}
+
+static struct fault_info {
+ int (*fn)(unsigned long addr, unsigned int esr, struct pt_regs *regs);
+ int sig;
+ int code;
+ const char *name;
+} fault_info[] = {
+ { do_bad, SIGBUS, 0, "ttbr address size fault" },
+ { do_bad, SIGBUS, 0, "level 1 address size fault" },
+ { do_bad, SIGBUS, 0, "level 2 address size fault" },
+ { do_bad, SIGBUS, 0, "level 3 address size fault" },
+ { do_translation_fault, SIGSEGV, SEGV_MAPERR, "input address range fault" },
+ { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 1 translation fault" },
+ { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 2 translation fault" },
+ { do_page_fault, SIGSEGV, SEGV_MAPERR, "level 3 translation fault" },
+ { do_bad, SIGBUS, 0, "reserved access flag fault" },
+ { do_bad, SIGSEGV, SEGV_ACCERR, "level 1 access flag fault" },
+ { do_bad, SIGSEGV, SEGV_ACCERR, "level 2 access flag fault" },
+ { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 access flag fault" },
+ { do_bad, SIGBUS, 0, "reserved permission fault" },
+ { do_bad, SIGSEGV, SEGV_ACCERR, "level 1 permission fault" },
+ { do_sect_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" },
+ { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" },
+ { do_bad, SIGBUS, 0, "synchronous external abort" },
+ { do_bad, SIGBUS, 0, "asynchronous external abort" },
+ { do_bad, SIGBUS, 0, "unknown 18" },
+ { do_bad, SIGBUS, 0, "unknown 19" },
+ { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" },
+ { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" },
+ { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" },
+ { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" },
+ { do_bad, SIGBUS, 0, "synchronous parity error" },
+ { do_bad, SIGBUS, 0, "asynchronous parity error" },
+ { do_bad, SIGBUS, 0, "unknown 26" },
+ { do_bad, SIGBUS, 0, "unknown 27" },
+ { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" },
+ { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" },
+ { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" },
+ { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" },
+ { do_bad, SIGBUS, 0, "unknown 32" },
+ { do_bad, SIGBUS, BUS_ADRALN, "alignment fault" },
+ { do_bad, SIGBUS, 0, "debug event" },
+ { do_bad, SIGBUS, 0, "unknown 35" },
+ { do_bad, SIGBUS, 0, "unknown 36" },
+ { do_bad, SIGBUS, 0, "unknown 37" },
+ { do_bad, SIGBUS, 0, "unknown 38" },
+ { do_bad, SIGBUS, 0, "unknown 39" },
+ { do_bad, SIGBUS, 0, "unknown 40" },
+ { do_bad, SIGBUS, 0, "unknown 41" },
+ { do_bad, SIGBUS, 0, "unknown 42" },
+ { do_bad, SIGBUS, 0, "unknown 43" },
+ { do_bad, SIGBUS, 0, "unknown 44" },
+ { do_bad, SIGBUS, 0, "unknown 45" },
+ { do_bad, SIGBUS, 0, "unknown 46" },
+ { do_bad, SIGBUS, 0, "unknown 47" },
+ { do_bad, SIGBUS, 0, "unknown 48" },
+ { do_bad, SIGBUS, 0, "unknown 49" },
+ { do_bad, SIGBUS, 0, "unknown 50" },
+ { do_bad, SIGBUS, 0, "unknown 51" },
+ { do_bad, SIGBUS, 0, "implementation fault (lockdown abort)" },
+ { do_bad, SIGBUS, 0, "unknown 53" },
+ { do_bad, SIGBUS, 0, "unknown 54" },
+ { do_bad, SIGBUS, 0, "unknown 55" },
+ { do_bad, SIGBUS, 0, "unknown 56" },
+ { do_bad, SIGBUS, 0, "unknown 57" },
+ { do_bad, SIGBUS, 0, "implementation fault (coprocessor abort)" },
+ { do_bad, SIGBUS, 0, "unknown 59" },
+ { do_bad, SIGBUS, 0, "unknown 60" },
+ { do_bad, SIGBUS, 0, "unknown 61" },
+ { do_bad, SIGBUS, 0, "unknown 62" },
+ { do_bad, SIGBUS, 0, "unknown 63" },
+};
+
+/*
+ * Dispatch a data abort to the relevant handler.
+ */
+asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr,
+ struct pt_regs *regs)
+{
+ const struct fault_info *inf = fault_info + (esr & 63);
+ struct siginfo info;
+
+ if (!inf->fn(addr, esr, regs))
+ return;
+
+ pr_alert("Unhandled fault: %s (0x%08x) at 0x%016lx\n",
+ inf->name, esr, addr);
+
+ info.si_signo = inf->sig;
+ info.si_errno = 0;
+ info.si_code = inf->code;
+ info.si_addr = (void __user *)addr;
+ arm64_notify_die("", regs, &info, esr);
+}
+
+/*
+ * Handle stack alignment exceptions.
+ */
+asmlinkage void __exception do_sp_pc_abort(unsigned long addr,
+ unsigned int esr,
+ struct pt_regs *regs)
+{
+ struct siginfo info;
+
+ info.si_signo = SIGBUS;
+ info.si_errno = 0;
+ info.si_code = BUS_ADRALN;
+ info.si_addr = (void __user *)addr;
+ arm64_notify_die("", regs, &info, esr);
+}
+
+static struct fault_info debug_fault_info[] = {
+ { do_bad, SIGTRAP, TRAP_HWBKPT, "hardware breakpoint" },
+ { do_bad, SIGTRAP, TRAP_HWBKPT, "hardware single-step" },
+ { do_bad, SIGTRAP, TRAP_HWBKPT, "hardware watchpoint" },
+ { do_bad, SIGBUS, 0, "unknown 3" },
+ { do_bad, SIGTRAP, TRAP_BRKPT, "aarch32 BKPT" },
+ { do_bad, SIGTRAP, 0, "aarch32 vector catch" },
+ { do_bad, SIGTRAP, TRAP_BRKPT, "aarch64 BRK" },
+ { do_bad, SIGBUS, 0, "unknown 7" },
+};
+
+void __init hook_debug_fault_code(int nr,
+ int (*fn)(unsigned long, unsigned int, struct pt_regs *),
+ int sig, int code, const char *name)
+{
+ BUG_ON(nr < 0 || nr >= ARRAY_SIZE(debug_fault_info));
+
+ debug_fault_info[nr].fn = fn;
+ debug_fault_info[nr].sig = sig;
+ debug_fault_info[nr].code = code;
+ debug_fault_info[nr].name = name;
+}
+
+asmlinkage int __exception do_debug_exception(unsigned long addr,
+ unsigned int esr,
+ struct pt_regs *regs)
+{
+ const struct fault_info *inf = debug_fault_info + DBG_ESR_EVT(esr);
+ struct siginfo info;
+
+ if (!inf->fn(addr, esr, regs))
+ return 1;
+
+ pr_alert("Unhandled debug exception: %s (0x%08x) at 0x%016lx\n",
+ inf->name, esr, addr);
+
+ info.si_signo = inf->sig;
+ info.si_errno = 0;
+ info.si_code = inf->code;
+ info.si_addr = (void __user *)addr;
+ arm64_notify_die("", regs, &info, esr);
+
+ return 0;
+}
diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c
new file mode 100644
index 0000000..c144adb
--- /dev/null
+++ b/arch/arm64/mm/flush.c
@@ -0,0 +1,135 @@
+/*
+ * Based on arch/arm/mm/flush.c
+ *
+ * Copyright (C) 1995-2002 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <linux/pagemap.h>
+
+#include <asm/cacheflush.h>
+#include <asm/cachetype.h>
+#include <asm/tlbflush.h>
+
+#include "mm.h"
+
+void flush_cache_mm(struct mm_struct *mm)
+{
+}
+
+void flush_cache_range(struct vm_area_struct *vma, unsigned long start,
+ unsigned long end)
+{
+ if (vma->vm_flags & VM_EXEC)
+ __flush_icache_all();
+}
+
+void flush_cache_page(struct vm_area_struct *vma, unsigned long user_addr,
+ unsigned long pfn)
+{
+}
+
+static void flush_ptrace_access(struct vm_area_struct *vma, struct page *page,
+ unsigned long uaddr, void *kaddr,
+ unsigned long len)
+{
+ if (vma->vm_flags & VM_EXEC) {
+ unsigned long addr = (unsigned long)kaddr;
+ if (icache_is_aliasing()) {
+ __flush_dcache_area(kaddr, len);
+ __flush_icache_all();
+ } else {
+ flush_icache_range(addr, addr + len);
+ }
+ }
+}
+
+/*
+ * Copy user data from/to a page which is mapped into a different processes
+ * address space. Really, we want to allow our "user space" model to handle
+ * this.
+ *
+ * Note that this code needs to run on the current CPU.
+ */
+void copy_to_user_page(struct vm_area_struct *vma, struct page *page,
+ unsigned long uaddr, void *dst, const void *src,
+ unsigned long len)
+{
+#ifdef CONFIG_SMP
+ preempt_disable();
+#endif
+ memcpy(dst, src, len);
+ flush_ptrace_access(vma, page, uaddr, dst, len);
+#ifdef CONFIG_SMP
+ preempt_enable();
+#endif
+}
+
+void __flush_dcache_page(struct page *page)
+{
+ __flush_dcache_area(page_address(page), PAGE_SIZE);
+}
+
+void __sync_icache_dcache(pte_t pte, unsigned long addr)
+{
+ unsigned long pfn;
+ struct page *page;
+
+ pfn = pte_pfn(pte);
+ if (!pfn_valid(pfn))
+ return;
+
+ page = pfn_to_page(pfn);
+ if (!test_and_set_bit(PG_dcache_clean, &page->flags)) {
+ __flush_dcache_page(page);
+ __flush_icache_all();
+ } else if (icache_is_aivivt()) {
+ __flush_icache_all();
+ }
+}
+
+/*
+ * Ensure cache coherency between kernel mapping and userspace mapping of this
+ * page.
+ */
+void flush_dcache_page(struct page *page)
+{
+ struct address_space *mapping;
+
+ /*
+ * The zero page is never written to, so never has any dirty cache
+ * lines, and therefore never needs to be flushed.
+ */
+ if (page == ZERO_PAGE(0))
+ return;
+
+ mapping = page_mapping(page);
+ if (mapping && mapping_mapped(mapping)) {
+ __flush_dcache_page(page);
+ __flush_icache_all();
+ set_bit(PG_dcache_clean, &page->flags);
+ } else {
+ clear_bit(PG_dcache_clean, &page->flags);
+ }
+}
+EXPORT_SYMBOL(flush_dcache_page);
+
+/*
+ * Additional functions defined in assembly.
+ */
+EXPORT_SYMBOL(flush_cache_all);
+EXPORT_SYMBOL(flush_icache_range);
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
new file mode 100644
index 0000000..5f719ba
--- /dev/null
+++ b/arch/arm64/mm/init.c
@@ -0,0 +1,437 @@
+/*
+ * Based on arch/arm/mm/init.c
+ *
+ * Copyright (C) 1995-2005 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/kernel.h>
+#include <linux/export.h>
+#include <linux/errno.h>
+#include <linux/swap.h>
+#include <linux/init.h>
+#include <linux/bootmem.h>
+#include <linux/mman.h>
+#include <linux/nodemask.h>
+#include <linux/initrd.h>
+#include <linux/gfp.h>
+#include <linux/memblock.h>
+#include <linux/sort.h>
+#include <linux/of_fdt.h>
+
+#include <asm/prom.h>
+#include <asm/sections.h>
+#include <asm/setup.h>
+#include <asm/sizes.h>
+#include <asm/tlb.h>
+
+#include "mm.h"
+
+static unsigned long phys_initrd_start __initdata = 0;
+static unsigned long phys_initrd_size __initdata = 0;
+
+phys_addr_t memstart_addr __read_mostly = 0;
+
+void __init early_init_dt_setup_initrd_arch(unsigned long start,
+ unsigned long end)
+{
+ phys_initrd_start = start;
+ phys_initrd_size = end - start;
+}
+
+static int __init early_initrd(char *p)
+{
+ unsigned long start, size;
+ char *endp;
+
+ start = memparse(p, &endp);
+ if (*endp == ',') {
+ size = memparse(endp + 1, NULL);
+
+ phys_initrd_start = start;
+ phys_initrd_size = size;
+ }
+ return 0;
+}
+early_param("initrd", early_initrd);
+
+#define MAX_DMA32_PFN ((4UL * 1024 * 1024 * 1024) >> PAGE_SHIFT)
+
+static void __init zone_sizes_init(unsigned long min, unsigned long max)
+{
+ struct memblock_region *reg;
+ unsigned long zone_size[MAX_NR_ZONES], zhole_size[MAX_NR_ZONES];
+ unsigned long max_dma32 = min;
+
+ memset(zone_size, 0, sizeof(zone_size));
+
+#ifdef CONFIG_ZONE_DMA32
+ /* 4GB maximum for 32-bit only capable devices */
+ max_dma32 = min(max, MAX_DMA32_PFN);
+ zone_size[ZONE_DMA32] = max_dma32 - min;
+#endif
+ zone_size[ZONE_NORMAL] = max - max_dma32;
+
+ memcpy(zhole_size, zone_size, sizeof(zhole_size));
+
+ for_each_memblock(memory, reg) {
+ unsigned long start = memblock_region_memory_base_pfn(reg);
+ unsigned long end = memblock_region_memory_end_pfn(reg);
+
+ if (start >= max)
+ continue;
+#ifdef CONFIG_ZONE_DMA32
+ if (start < max_dma32) {
+ unsigned long dma_end = min(end, max_dma32);
+ zhole_size[ZONE_DMA32] -= dma_end - start;
+ }
+#endif
+ if (end > max_dma32) {
+ unsigned long normal_end = min(end, max);
+ unsigned long normal_start = max(start, max_dma32);
+ zhole_size[ZONE_NORMAL] -= normal_end - normal_start;
+ }
+ }
+
+ free_area_init_node(0, zone_size, min, zhole_size);
+}
+
+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
+int pfn_valid(unsigned long pfn)
+{
+ return memblock_is_memory(pfn << PAGE_SHIFT);
+}
+EXPORT_SYMBOL(pfn_valid);
+#endif
+
+#ifndef CONFIG_SPARSEMEM
+static void arm64_memory_present(void)
+{
+}
+#else
+static void arm64_memory_present(void)
+{
+ struct memblock_region *reg;
+
+ for_each_memblock(memory, reg)
+ memory_present(0, memblock_region_memory_base_pfn(reg),
+ memblock_region_memory_end_pfn(reg));
+}
+#endif
+
+void __init arm64_memblock_init(void)
+{
+ u64 *reserve_map, base, size;
+
+ /* Register the kernel text, kernel data and initrd with memblock */
+ memblock_reserve(__pa(_text), _end - _text);
+#ifdef CONFIG_BLK_DEV_INITRD
+ if (phys_initrd_size) {
+ memblock_reserve(phys_initrd_start, phys_initrd_size);
+
+ /* Now convert initrd to virtual addresses */
+ initrd_start = __phys_to_virt(phys_initrd_start);
+ initrd_end = initrd_start + phys_initrd_size;
+ }
+#endif
+
+ /*
+ * Reserve the page tables. These are already in use,
+ * and can only be in node 0.
+ */
+ memblock_reserve(__pa(swapper_pg_dir), SWAPPER_DIR_SIZE);
+ memblock_reserve(__pa(idmap_pg_dir), IDMAP_DIR_SIZE);
+
+ /* Reserve the dtb region */
+ memblock_reserve(virt_to_phys(initial_boot_params),
+ be32_to_cpu(initial_boot_params->totalsize));
+
+ /*
+ * Process the reserve map. This will probably overlap the initrd
+ * and dtb locations which are already reserved, but overlapping
+ * doesn't hurt anything
+ */
+ reserve_map = ((void*)initial_boot_params) +
+ be32_to_cpu(initial_boot_params->off_mem_rsvmap);
+ while (1) {
+ base = be64_to_cpup(reserve_map++);
+ size = be64_to_cpup(reserve_map++);
+ if (!size)
+ break;
+ memblock_reserve(base, size);
+ }
+
+ memblock_allow_resize();
+ memblock_dump_all();
+}
+
+void __init bootmem_init(void)
+{
+ unsigned long min, max;
+
+ min = PFN_UP(memblock_start_of_DRAM());
+ max = PFN_DOWN(memblock_end_of_DRAM());
+
+ /*
+ * Sparsemem tries to allocate bootmem in memory_present(), so must be
+ * done after the fixed reservations.
+ */
+ arm64_memory_present();
+
+ sparse_init();
+ zone_sizes_init(min, max);
+
+ high_memory = __va((max << PAGE_SHIFT) - 1) + 1;
+ max_pfn = max_low_pfn = max;
+}
+
+static inline int free_area(unsigned long pfn, unsigned long end, char *s)
+{
+ unsigned int pages = 0, size = (end - pfn) << (PAGE_SHIFT - 10);
+
+ for (; pfn < end; pfn++) {
+ struct page *page = pfn_to_page(pfn);
+ ClearPageReserved(page);
+ init_page_count(page);
+ __free_page(page);
+ pages++;
+ }
+
+ if (size && s)
+ pr_info("Freeing %s memory: %dK\n", s, size);
+
+ return pages;
+}
+
+/*
+ * Poison init memory with an undefined instruction (0x0).
+ */
+static inline void poison_init_mem(void *s, size_t count)
+{
+ memset(s, 0, count);
+}
+
+#ifndef CONFIG_SPARSEMEM_VMEMMAP
+static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
+{
+ struct page *start_pg, *end_pg;
+ unsigned long pg, pgend;
+
+ /*
+ * Convert start_pfn/end_pfn to a struct page pointer.
+ */
+ start_pg = pfn_to_page(start_pfn - 1) + 1;
+ end_pg = pfn_to_page(end_pfn - 1) + 1;
+
+ /*
+ * Convert to physical addresses, and round start upwards and end
+ * downwards.
+ */
+ pg = (unsigned long)PAGE_ALIGN(__pa(start_pg));
+ pgend = (unsigned long)__pa(end_pg) & PAGE_MASK;
+
+ /*
+ * If there are free pages between these, free the section of the
+ * memmap array.
+ */
+ if (pg < pgend)
+ free_bootmem(pg, pgend - pg);
+}
+
+/*
+ * The mem_map array can get very big. Free the unused area of the memory map.
+ */
+static void __init free_unused_memmap(void)
+{
+ unsigned long start, prev_end = 0;
+ struct memblock_region *reg;
+
+ for_each_memblock(memory, reg) {
+ start = __phys_to_pfn(reg->base);
+
+#ifdef CONFIG_SPARSEMEM
+ /*
+ * Take care not to free memmap entries that don't exist due
+ * to SPARSEMEM sections which aren't present.
+ */
+ start = min(start, ALIGN(prev_end, PAGES_PER_SECTION));
+#endif
+ /*
+ * If we had a previous bank, and there is a space between the
+ * current bank and the previous, free it.
+ */
+ if (prev_end && prev_end < start)
+ free_memmap(prev_end, start);
+
+ /*
+ * Align up here since the VM subsystem insists that the
+ * memmap entries are valid from the bank end aligned to
+ * MAX_ORDER_NR_PAGES.
+ */
+ prev_end = ALIGN(start + __phys_to_pfn(reg->size),
+ MAX_ORDER_NR_PAGES);
+ }
+
+#ifdef CONFIG_SPARSEMEM
+ if (!IS_ALIGNED(prev_end, PAGES_PER_SECTION))
+ free_memmap(prev_end, ALIGN(prev_end, PAGES_PER_SECTION));
+#endif
+}
+#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
+
+/*
+ * mem_init() marks the free areas in the mem_map and tells us how much memory
+ * is free. This is done after various parts of the system have claimed their
+ * memory after the kernel image.
+ */
+void __init mem_init(void)
+{
+ unsigned long reserved_pages, free_pages;
+ struct memblock_region *reg;
+
+#if CONFIG_SWIOTLB
+ extern void __init arm64_swiotlb_init(size_t max_size);
+ arm64_swiotlb_init(max_pfn << (PAGE_SHIFT - 1));
+#endif
+
+ max_mapnr = pfn_to_page(max_pfn + PHYS_PFN_OFFSET) - mem_map;
+
+#ifndef CONFIG_SPARSEMEM_VMEMMAP
+ /* this will put all unused low memory onto the freelists */
+ free_unused_memmap();
+#endif
+
+ totalram_pages += free_all_bootmem();
+
+ reserved_pages = free_pages = 0;
+
+ for_each_memblock(memory, reg) {
+ unsigned int pfn1, pfn2;
+ struct page *page, *end;
+
+ pfn1 = __phys_to_pfn(reg->base);
+ pfn2 = pfn1 + __phys_to_pfn(reg->size);
+
+ page = pfn_to_page(pfn1);
+ end = pfn_to_page(pfn2 - 1) + 1;
+
+ do {
+ if (PageReserved(page))
+ reserved_pages++;
+ else if (!page_count(page))
+ free_pages++;
+ page++;
+ } while (page < end);
+ }
+
+ /*
+ * Since our memory may not be contiguous, calculate the real number
+ * of pages we have in this system.
+ */
+ pr_info("Memory:");
+ num_physpages = 0;
+ for_each_memblock(memory, reg) {
+ unsigned long pages = memblock_region_memory_end_pfn(reg) -
+ memblock_region_memory_base_pfn(reg);
+ num_physpages += pages;
+ printk(" %ldMB", pages >> (20 - PAGE_SHIFT));
+ }
+ printk(" = %luMB total\n", num_physpages >> (20 - PAGE_SHIFT));
+
+ pr_notice("Memory: %luk/%luk available, %luk reserved\n",
+ nr_free_pages() << (PAGE_SHIFT-10),
+ free_pages << (PAGE_SHIFT-10),
+ reserved_pages << (PAGE_SHIFT-10));
+
+#define MLK(b, t) b, t, ((t) - (b)) >> 10
+#define MLM(b, t) b, t, ((t) - (b)) >> 20
+#define MLK_ROUNDUP(b, t) b, t, DIV_ROUND_UP(((t) - (b)), SZ_1K)
+
+ pr_notice("Virtual kernel memory layout:\n"
+ " vmalloc : 0x%16lx - 0x%16lx (%6ld MB)\n"
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+ " vmemmap : 0x%16lx - 0x%16lx (%6ld MB)\n"
+#endif
+ " modules : 0x%16lx - 0x%16lx (%6ld MB)\n"
+ " memory : 0x%16lx - 0x%16lx (%6ld MB)\n"
+ " .init : 0x%p" " - 0x%p" " (%6ld kB)\n"
+ " .text : 0x%p" " - 0x%p" " (%6ld kB)\n"
+ " .data : 0x%p" " - 0x%p" " (%6ld kB)\n",
+ MLM(VMALLOC_START, VMALLOC_END),
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+ MLM((unsigned long)virt_to_page(PAGE_OFFSET),
+ (unsigned long)virt_to_page(high_memory)),
+#endif
+ MLM(MODULES_VADDR, MODULES_END),
+ MLM(PAGE_OFFSET, (unsigned long)high_memory),
+
+ MLK_ROUNDUP(__init_begin, __init_end),
+ MLK_ROUNDUP(_text, _etext),
+ MLK_ROUNDUP(_sdata, _edata));
+
+#undef MLK
+#undef MLM
+#undef MLK_ROUNDUP
+
+ /*
+ * Check boundaries twice: Some fundamental inconsistencies can be
+ * detected at build time already.
+ */
+#ifdef CONFIG_COMPAT
+ BUILD_BUG_ON(TASK_SIZE_32 > TASK_SIZE_64);
+#endif
+ BUILD_BUG_ON(TASK_SIZE_64 > MODULES_VADDR);
+ BUG_ON(TASK_SIZE_64 > MODULES_VADDR);
+
+ if (PAGE_SIZE >= 16384 && num_physpages <= 128) {
+ extern int sysctl_overcommit_memory;
+ /*
+ * On a machine this small we won't get anywhere without
+ * overcommit, so turn it on by default.
+ */
+ sysctl_overcommit_memory = OVERCOMMIT_ALWAYS;
+ }
+}
+
+void free_initmem(void)
+{
+ poison_init_mem(__init_begin, __init_end - __init_begin);
+ totalram_pages += free_area(__phys_to_pfn(__pa(__init_begin)),
+ __phys_to_pfn(__pa(__init_end)),
+ "init");
+}
+
+#ifdef CONFIG_BLK_DEV_INITRD
+
+static int keep_initrd;
+
+void free_initrd_mem(unsigned long start, unsigned long end)
+{
+ if (!keep_initrd) {
+ poison_init_mem((void *)start, PAGE_ALIGN(end) - start);
+ totalram_pages += free_area(__phys_to_pfn(__pa(start)),
+ __phys_to_pfn(__pa(end)),
+ "initrd");
+ }
+}
+
+static int __init keepinitrd_setup(char *__unused)
+{
+ keep_initrd = 1;
+ return 1;
+}
+
+__setup("keepinitrd", keepinitrd_setup);
+#endif
diff --git a/arch/arm64/mm/ioremap.c b/arch/arm64/mm/ioremap.c
new file mode 100644
index 0000000..1725cd6
--- /dev/null
+++ b/arch/arm64/mm/ioremap.c
@@ -0,0 +1,84 @@
+/*
+ * Based on arch/arm/mm/ioremap.c
+ *
+ * (C) Copyright 1995 1996 Linus Torvalds
+ * Hacked for ARM by Phil Blundell <philb@gnu.org>
+ * Hacked to allow all architectures to build, and various cleanups
+ * by Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/io.h>
+
+static void __iomem *__ioremap_caller(phys_addr_t phys_addr, size_t size,
+ pgprot_t prot, void *caller)
+{
+ unsigned long last_addr;
+ unsigned long offset = phys_addr & ~PAGE_MASK;
+ int err;
+ unsigned long addr;
+ struct vm_struct *area;
+
+ /*
+ * Page align the mapping address and size, taking account of any
+ * offset.
+ */
+ phys_addr &= PAGE_MASK;
+ size = PAGE_ALIGN(size + offset);
+
+ /*
+ * Don't allow wraparound, zero size or outside PHYS_MASK.
+ */
+ last_addr = phys_addr + size - 1;
+ if (!size || last_addr < phys_addr || (last_addr & ~PHYS_MASK))
+ return NULL;
+
+ /*
+ * Don't allow RAM to be mapped.
+ */
+ if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr))))
+ return NULL;
+
+ area = get_vm_area_caller(size, VM_IOREMAP, caller);
+ if (!area)
+ return NULL;
+ addr = (unsigned long)area->addr;
+
+ err = ioremap_page_range(addr, addr + size, phys_addr, prot);
+ if (err) {
+ vunmap((void *)addr);
+ return NULL;
+ }
+
+ return (void __iomem *)(offset + addr);
+}
+
+void __iomem *__ioremap(phys_addr_t phys_addr, size_t size, pgprot_t prot)
+{
+ return __ioremap_caller(phys_addr, size, prot,
+ __builtin_return_address(0));
+}
+EXPORT_SYMBOL(__ioremap);
+
+void __iounmap(volatile void __iomem *io_addr)
+{
+ void *addr = (void *)(PAGE_MASK & (unsigned long)io_addr);
+
+ vunmap(addr);
+}
+EXPORT_SYMBOL(__iounmap);
diff --git a/arch/arm64/mm/mm.h b/arch/arm64/mm/mm.h
new file mode 100644
index 0000000..d8d6e78
--- /dev/null
+++ b/arch/arm64/mm/mm.h
@@ -0,0 +1,2 @@
+extern void __flush_dcache_page(struct page *page);
+extern void __init bootmem_init(void);
diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c
new file mode 100644
index 0000000..7c7be78
--- /dev/null
+++ b/arch/arm64/mm/mmap.c
@@ -0,0 +1,144 @@
+/*
+ * Based on arch/arm/mm/mmap.c
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/elf.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/export.h>
+#include <linux/shm.h>
+#include <linux/sched.h>
+#include <linux/io.h>
+#include <linux/personality.h>
+#include <linux/random.h>
+
+#include <asm/cputype.h>
+
+/*
+ * Leave enough space between the mmap area and the stack to honour ulimit in
+ * the face of randomisation.
+ */
+#define MIN_GAP (SZ_128M + ((STACK_RND_MASK << PAGE_SHIFT) + 1))
+#define MAX_GAP (STACK_TOP/6*5)
+
+static int mmap_is_legacy(void)
+{
+ if (current->personality & ADDR_COMPAT_LAYOUT)
+ return 1;
+
+ if (rlimit(RLIMIT_STACK) == RLIM_INFINITY)
+ return 1;
+
+ return sysctl_legacy_va_layout;
+}
+
+/*
+ * Since get_random_int() returns the same value within a 1 jiffy window, we
+ * will almost always get the same randomisation for the stack and mmap
+ * region. This will mean the relative distance between stack and mmap will be
+ * the same.
+ *
+ * To avoid this we can shift the randomness by 1 bit.
+ */
+static unsigned long mmap_rnd(void)
+{
+ unsigned long rnd = 0;
+
+ if (current->flags & PF_RANDOMIZE)
+ rnd = (long)get_random_int() & (STACK_RND_MASK >> 1);
+
+ return rnd << (PAGE_SHIFT + 1);
+}
+
+static unsigned long mmap_base(void)
+{
+ unsigned long gap = rlimit(RLIMIT_STACK);
+
+ if (gap < MIN_GAP)
+ gap = MIN_GAP;
+ else if (gap > MAX_GAP)
+ gap = MAX_GAP;
+
+ return PAGE_ALIGN(STACK_TOP - gap - mmap_rnd());
+}
+
+/*
+ * This function, called very early during the creation of a new process VM
+ * image, sets up which VM layout function to use:
+ */
+void arch_pick_mmap_layout(struct mm_struct *mm)
+{
+ /*
+ * Fall back to the standard layout if the personality bit is set, or
+ * if the expected stack growth is unlimited:
+ */
+ if (mmap_is_legacy()) {
+ mm->mmap_base = TASK_UNMAPPED_BASE;
+ mm->get_unmapped_area = arch_get_unmapped_area;
+ mm->unmap_area = arch_unmap_area;
+ } else {
+ mm->mmap_base = mmap_base();
+ mm->get_unmapped_area = arch_get_unmapped_area_topdown;
+ mm->unmap_area = arch_unmap_area_topdown;
+ }
+}
+EXPORT_SYMBOL_GPL(arch_pick_mmap_layout);
+
+
+/*
+ * You really shouldn't be using read() or write() on /dev/mem. This might go
+ * away in the future.
+ */
+int valid_phys_addr_range(unsigned long addr, size_t size)
+{
+ if (addr < PHYS_OFFSET)
+ return 0;
+ if (addr + size > __pa(high_memory - 1) + 1)
+ return 0;
+
+ return 1;
+}
+
+/*
+ * Do not allow /dev/mem mappings beyond the supported physical range.
+ */
+int valid_mmap_phys_addr_range(unsigned long pfn, size_t size)
+{
+ return !(((pfn << PAGE_SHIFT) + size) & ~PHYS_MASK);
+}
+
+#ifdef CONFIG_STRICT_DEVMEM
+
+#include <linux/ioport.h>
+
+/*
+ * devmem_is_allowed() checks to see if /dev/mem access to a certain address
+ * is valid. The argument is a physical page number. We mimic x86 here by
+ * disallowing access to system RAM as well as device-exclusive MMIO regions.
+ * This effectively disable read()/write() on /dev/mem.
+ */
+int devmem_is_allowed(unsigned long pfn)
+{
+ if (iomem_is_exclusive(pfn << PAGE_SHIFT))
+ return 0;
+ if (!page_is_ram(pfn))
+ return 1;
+ return 0;
+}
+
+#endif
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
new file mode 100644
index 0000000..a6885d8
--- /dev/null
+++ b/arch/arm64/mm/mmu.c
@@ -0,0 +1,395 @@
+/*
+ * Based on arch/arm/mm/mmu.c
+ *
+ * Copyright (C) 1995-2005 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/export.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <linux/mman.h>
+#include <linux/nodemask.h>
+#include <linux/memblock.h>
+#include <linux/fs.h>
+
+#include <asm/cputype.h>
+#include <asm/sections.h>
+#include <asm/setup.h>
+#include <asm/sizes.h>
+#include <asm/tlb.h>
+#include <asm/mmu_context.h>
+
+#include "mm.h"
+
+/*
+ * Empty_zero_page is a special page that is used for zero-initialized data
+ * and COW.
+ */
+struct page *empty_zero_page;
+EXPORT_SYMBOL(empty_zero_page);
+
+pgprot_t pgprot_default;
+EXPORT_SYMBOL(pgprot_default);
+
+static pmdval_t prot_sect_kernel;
+
+struct cachepolicy {
+ const char policy[16];
+ u64 mair;
+ u64 tcr;
+};
+
+static struct cachepolicy cache_policies[] __initdata = {
+ {
+ .policy = "uncached",
+ .mair = 0x44, /* inner, outer non-cacheable */
+ .tcr = TCR_IRGN_NC | TCR_ORGN_NC,
+ }, {
+ .policy = "writethrough",
+ .mair = 0xaa, /* inner, outer write-through, read-allocate */
+ .tcr = TCR_IRGN_WT | TCR_ORGN_WT,
+ }, {
+ .policy = "writeback",
+ .mair = 0xee, /* inner, outer write-back, read-allocate */
+ .tcr = TCR_IRGN_WBnWA | TCR_ORGN_WBnWA,
+ }
+};
+
+/*
+ * These are useful for identifying cache coherency problems by allowing the
+ * cache or the cache and writebuffer to be turned off. It changes the Normal
+ * memory caching attributes in the MAIR_EL1 register.
+ */
+static int __init early_cachepolicy(char *p)
+{
+ int i;
+ u64 tmp;
+
+ for (i = 0; i < ARRAY_SIZE(cache_policies); i++) {
+ int len = strlen(cache_policies[i].policy);
+
+ if (memcmp(p, cache_policies[i].policy, len) == 0)
+ break;
+ }
+ if (i == ARRAY_SIZE(cache_policies)) {
+ pr_err("ERROR: unknown or unsupported cache policy: %s\n", p);
+ return 0;
+ }
+
+ flush_cache_all();
+
+ /*
+ * Modify MT_NORMAL attributes in MAIR_EL1.
+ */
+ asm volatile(
+ " mrs %0, mair_el1\n"
+ " bfi %0, %1, #%2, #8\n"
+ " msr mair_el1, %0\n"
+ " isb\n"
+ : "=&r" (tmp)
+ : "r" (cache_policies[i].mair), "i" (MT_NORMAL * 8));
+
+ /*
+ * Modify TCR PTW cacheability attributes.
+ */
+ asm volatile(
+ " mrs %0, tcr_el1\n"
+ " bic %0, %0, %2\n"
+ " orr %0, %0, %1\n"
+ " msr tcr_el1, %0\n"
+ " isb\n"
+ : "=&r" (tmp)
+ : "r" (cache_policies[i].tcr), "r" (TCR_IRGN_MASK | TCR_ORGN_MASK));
+
+ flush_cache_all();
+
+ return 0;
+}
+early_param("cachepolicy", early_cachepolicy);
+
+/*
+ * Adjust the PMD section entries according to the CPU in use.
+ */
+static void __init init_mem_pgprot(void)
+{
+ pteval_t default_pgprot;
+ int i;
+
+ default_pgprot = PTE_ATTRINDX(MT_NORMAL);
+ prot_sect_kernel = PMD_TYPE_SECT | PMD_SECT_AF | PMD_ATTRINDX(MT_NORMAL);
+
+#ifdef CONFIG_SMP
+ /*
+ * Mark memory with the "shared" attribute for SMP systems
+ */
+ default_pgprot |= PTE_SHARED;
+ prot_sect_kernel |= PMD_SECT_S;
+#endif
+
+ for (i = 0; i < 16; i++) {
+ unsigned long v = pgprot_val(protection_map[i]);
+ protection_map[i] = __pgprot(v | default_pgprot);
+ }
+
+ pgprot_default = __pgprot(PTE_TYPE_PAGE | PTE_AF | default_pgprot);
+}
+
+pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
+ unsigned long size, pgprot_t vma_prot)
+{
+ if (!pfn_valid(pfn))
+ return pgprot_noncached(vma_prot);
+ else if (file->f_flags & O_SYNC)
+ return pgprot_writecombine(vma_prot);
+ return vma_prot;
+}
+EXPORT_SYMBOL(phys_mem_access_prot);
+
+static void __init *early_alloc(unsigned long sz)
+{
+ void *ptr = __va(memblock_alloc(sz, sz));
+ memset(ptr, 0, sz);
+ return ptr;
+}
+
+static void __init alloc_init_pte(pmd_t *pmd, unsigned long addr,
+ unsigned long end, unsigned long pfn)
+{
+ pte_t *pte;
+
+ if (pmd_none(*pmd)) {
+ pte = early_alloc(PTRS_PER_PTE * sizeof(pte_t));
+ __pmd_populate(pmd, __pa(pte), PMD_TYPE_TABLE);
+ }
+ BUG_ON(pmd_bad(*pmd));
+
+ pte = pte_offset_kernel(pmd, addr);
+ do {
+ set_pte(pte, pfn_pte(pfn, PAGE_KERNEL_EXEC));
+ pfn++;
+ } while (pte++, addr += PAGE_SIZE, addr != end);
+}
+
+static void __init alloc_init_pmd(pud_t *pud, unsigned long addr,
+ unsigned long end, phys_addr_t phys)
+{
+ pmd_t *pmd;
+ unsigned long next;
+
+ /*
+ * Check for initial section mappings in the pgd/pud and remove them.
+ */
+ if (pud_none(*pud) || pud_bad(*pud)) {
+ pmd = early_alloc(PTRS_PER_PMD * sizeof(pmd_t));
+ pud_populate(&init_mm, pud, pmd);
+ }
+
+ pmd = pmd_offset(pud, addr);
+ do {
+ next = pmd_addr_end(addr, end);
+ /* try section mapping first */
+ if (((addr | next | phys) & ~SECTION_MASK) == 0)
+ set_pmd(pmd, __pmd(phys | prot_sect_kernel));
+ else
+ alloc_init_pte(pmd, addr, next, __phys_to_pfn(phys));
+ phys += next - addr;
+ } while (pmd++, addr = next, addr != end);
+}
+
+static void __init alloc_init_pud(pgd_t *pgd, unsigned long addr,
+ unsigned long end, unsigned long phys)
+{
+ pud_t *pud = pud_offset(pgd, addr);
+ unsigned long next;
+
+ do {
+ next = pud_addr_end(addr, end);
+ alloc_init_pmd(pud, addr, next, phys);
+ phys += next - addr;
+ } while (pud++, addr = next, addr != end);
+}
+
+/*
+ * Create the page directory entries and any necessary page tables for the
+ * mapping specified by 'md'.
+ */
+static void __init create_mapping(phys_addr_t phys, unsigned long virt,
+ phys_addr_t size)
+{
+ unsigned long addr, length, end, next;
+ pgd_t *pgd;
+
+ if (virt < VMALLOC_START) {
+ pr_warning("BUG: not creating mapping for 0x%016llx at 0x%016lx - outside kernel range\n",
+ phys, virt);
+ return;
+ }
+
+ addr = virt & PAGE_MASK;
+ length = PAGE_ALIGN(size + (virt & ~PAGE_MASK));
+
+ pgd = pgd_offset_k(addr);
+ end = addr + length;
+ do {
+ next = pgd_addr_end(addr, end);
+ alloc_init_pud(pgd, addr, next, phys);
+ phys += next - addr;
+ } while (pgd++, addr = next, addr != end);
+}
+
+static void __init map_mem(void)
+{
+ struct memblock_region *reg;
+
+ /* map all the memory banks */
+ for_each_memblock(memory, reg) {
+ phys_addr_t start = reg->base;
+ phys_addr_t end = start + reg->size;
+
+ if (start >= end)
+ break;
+
+ create_mapping(start, __phys_to_virt(start), end - start);
+ }
+}
+
+/*
+ * paging_init() sets up the page tables, initialises the zone memory
+ * maps and sets up the zero page.
+ */
+void __init paging_init(void)
+{
+ void *zero_page;
+
+ /*
+ * Maximum PGDIR_SIZE addressable via the initial direct kernel
+ * mapping in swapper_pg_dir.
+ */
+ memblock_set_current_limit((PHYS_OFFSET & PGDIR_MASK) + PGDIR_SIZE);
+
+ init_mem_pgprot();
+ map_mem();
+
+ /*
+ * Finally flush the caches and tlb to ensure that we're in a
+ * consistent state.
+ */
+ flush_cache_all();
+ flush_tlb_all();
+
+ /* allocate the zero page. */
+ zero_page = early_alloc(PAGE_SIZE);
+
+ bootmem_init();
+
+ empty_zero_page = virt_to_page(zero_page);
+ __flush_dcache_page(empty_zero_page);
+
+ /*
+ * TTBR0 is only used for the identity mapping at this stage. Make it
+ * point to zero page to avoid speculatively fetching new entries.
+ */
+ cpu_set_reserved_ttbr0();
+ flush_tlb_all();
+}
+
+/*
+ * Enable the identity mapping to allow the MMU disabling.
+ */
+void setup_mm_for_reboot(void)
+{
+ cpu_switch_mm(idmap_pg_dir, &init_mm);
+ flush_tlb_all();
+}
+
+/*
+ * Check whether a kernel address is valid (derived from arch/x86/).
+ */
+int kern_addr_valid(unsigned long addr)
+{
+ pgd_t *pgd;
+ pud_t *pud;
+ pmd_t *pmd;
+ pte_t *pte;
+
+ if ((((long)addr) >> VA_BITS) != -1UL)
+ return 0;
+
+ pgd = pgd_offset_k(addr);
+ if (pgd_none(*pgd))
+ return 0;
+
+ pud = pud_offset(pgd, addr);
+ if (pud_none(*pud))
+ return 0;
+
+ pmd = pmd_offset(pud, addr);
+ if (pmd_none(*pmd))
+ return 0;
+
+ pte = pte_offset_kernel(pmd, addr);
+ if (pte_none(*pte))
+ return 0;
+
+ return pfn_valid(pte_pfn(*pte));
+}
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+#ifdef CONFIG_ARM64_64K_PAGES
+int __meminit vmemmap_populate(struct page *start_page,
+ unsigned long size, int node)
+{
+ return vmemmap_populate_basepages(start_page, size, node);
+}
+#else /* !CONFIG_ARM64_64K_PAGES */
+int __meminit vmemmap_populate(struct page *start_page,
+ unsigned long size, int node)
+{
+ unsigned long addr = (unsigned long)start_page;
+ unsigned long end = (unsigned long)(start_page + size);
+ unsigned long next;
+ pgd_t *pgd;
+ pud_t *pud;
+ pmd_t *pmd;
+
+ do {
+ next = pmd_addr_end(addr, end);
+
+ pgd = vmemmap_pgd_populate(addr, node);
+ if (!pgd)
+ return -ENOMEM;
+
+ pud = vmemmap_pud_populate(pgd, addr, node);
+ if (!pud)
+ return -ENOMEM;
+
+ pmd = pmd_offset(pud, addr);
+ if (pmd_none(*pmd)) {
+ void *p = NULL;
+
+ p = vmemmap_alloc_block_buf(PMD_SIZE, node);
+ if (!p)
+ return -ENOMEM;
+
+ set_pmd(pmd, __pmd(__pa(p) | prot_sect_kernel));
+ } else
+ vmemmap_verify((pte_t *)pmd, node, addr, next);
+ } while (addr = next, addr != end);
+
+ return 0;
+}
+#endif /* CONFIG_ARM64_64K_PAGES */
+#endif /* CONFIG_SPARSEMEM_VMEMMAP */
diff --git a/arch/arm64/mm/pgd.c b/arch/arm64/mm/pgd.c
new file mode 100644
index 0000000..7083cda
--- /dev/null
+++ b/arch/arm64/mm/pgd.c
@@ -0,0 +1,54 @@
+/*
+ * PGD allocation/freeing
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/mm.h>
+#include <linux/gfp.h>
+#include <linux/highmem.h>
+#include <linux/slab.h>
+
+#include <asm/pgalloc.h>
+#include <asm/page.h>
+#include <asm/tlbflush.h>
+
+#include "mm.h"
+
+#define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t))
+
+pgd_t *pgd_alloc(struct mm_struct *mm)
+{
+ pgd_t *new_pgd;
+
+ if (PGD_SIZE == PAGE_SIZE)
+ new_pgd = (pgd_t *)get_zeroed_page(GFP_KERNEL);
+ else
+ new_pgd = kzalloc(PGD_SIZE, GFP_KERNEL);
+
+ if (!new_pgd)
+ return NULL;
+
+ return new_pgd;
+}
+
+void pgd_free(struct mm_struct *mm, pgd_t *pgd)
+{
+ if (PGD_SIZE == PAGE_SIZE)
+ free_page((unsigned long)pgd);
+ else
+ kfree(pgd);
+}
diff --git a/arch/arm64/mm/proc-macros.S b/arch/arm64/mm/proc-macros.S
new file mode 100644
index 0000000..8957b82
--- /dev/null
+++ b/arch/arm64/mm/proc-macros.S
@@ -0,0 +1,55 @@
+/*
+ * Based on arch/arm/mm/proc-macros.S
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <asm/asm-offsets.h>
+#include <asm/thread_info.h>
+
+/*
+ * vma_vm_mm - get mm pointer from vma pointer (vma->vm_mm)
+ */
+ .macro vma_vm_mm, rd, rn
+ ldr \rd, [\rn, #VMA_VM_MM]
+ .endm
+
+/*
+ * mmid - get context id from mm pointer (mm->context.id)
+ */
+ .macro mmid, rd, rn
+ ldr \rd, [\rn, #MM_CONTEXT_ID]
+ .endm
+
+/*
+ * dcache_line_size - get the minimum D-cache line size from the CTR register.
+ */
+ .macro dcache_line_size, reg, tmp
+ mrs \tmp, ctr_el0 // read CTR
+ lsr \tmp, \tmp, #16
+ and \tmp, \tmp, #0xf // cache line size encoding
+ mov \reg, #4 // bytes per word
+ lsl \reg, \reg, \tmp // actual cache line size
+ .endm
+
+/*
+ * icache_line_size - get the minimum I-cache line size from the CTR register.
+ */
+ .macro icache_line_size, reg, tmp
+ mrs \tmp, ctr_el0 // read CTR
+ and \tmp, \tmp, #0xf // cache line size encoding
+ mov \reg, #4 // bytes per word
+ lsl \reg, \reg, \tmp // actual cache line size
+ .endm
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
new file mode 100644
index 0000000..f1d8b9b
--- /dev/null
+++ b/arch/arm64/mm/proc.S
@@ -0,0 +1,175 @@
+/*
+ * Based on arch/arm/mm/proc.S
+ *
+ * Copyright (C) 2001 Deep Blue Solutions Ltd.
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+#include <asm/asm-offsets.h>
+#include <asm/hwcap.h>
+#include <asm/pgtable-hwdef.h>
+#include <asm/pgtable.h>
+
+#include "proc-macros.S"
+
+#ifndef CONFIG_SMP
+/* PTWs cacheable, inner/outer WBWA not shareable */
+#define TCR_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA
+#else
+/* PTWs cacheable, inner/outer WBWA shareable */
+#define TCR_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA | TCR_SHARED
+#endif
+
+#define MAIR(attr, mt) ((attr) << ((mt) * 8))
+
+/*
+ * cpu_cache_off()
+ *
+ * Turn the CPU D-cache off.
+ */
+ENTRY(cpu_cache_off)
+ mrs x0, sctlr_el1
+ bic x0, x0, #1 << 2 // clear SCTLR.C
+ msr sctlr_el1, x0
+ isb
+ ret
+ENDPROC(cpu_cache_off)
+
+/*
+ * cpu_reset(loc)
+ *
+ * Perform a soft reset of the system. Put the CPU into the same state
+ * as it would be if it had been reset, and branch to what would be the
+ * reset vector. It must be executed with the flat identity mapping.
+ *
+ * - loc - location to jump to for soft reset
+ */
+ .align 5
+ENTRY(cpu_reset)
+ mrs x1, sctlr_el1
+ bic x1, x1, #1
+ msr sctlr_el1, x1 // disable the MMU
+ isb
+ ret x0
+ENDPROC(cpu_reset)
+
+/*
+ * cpu_do_idle()
+ *
+ * Idle the processor (wait for interrupt).
+ */
+ENTRY(cpu_do_idle)
+ dsb sy // WFI may enter a low-power mode
+ wfi
+ ret
+ENDPROC(cpu_do_idle)
+
+/*
+ * cpu_switch_mm(pgd_phys, tsk)
+ *
+ * Set the translation table base pointer to be pgd_phys.
+ *
+ * - pgd_phys - physical address of new TTB
+ */
+ENTRY(cpu_do_switch_mm)
+ mmid w1, x1 // get mm->context.id
+ bfi x0, x1, #48, #16 // set the ASID
+ msr ttbr0_el1, x0 // set TTBR0
+ isb
+ ret
+ENDPROC(cpu_do_switch_mm)
+
+cpu_name:
+ .ascii "AArch64 Processor"
+ .align
+
+ .section ".text.init", #alloc, #execinstr
+
+/*
+ * __cpu_setup
+ *
+ * Initialise the processor for turning the MMU on. Return in x0 the
+ * value of the SCTLR_EL1 register.
+ */
+ENTRY(__cpu_setup)
+ /*
+ * Preserve the link register across the function call.
+ */
+ mov x28, lr
+ bl __flush_dcache_all
+ mov lr, x28
+ ic iallu // I+BTB cache invalidate
+ dsb sy
+
+ mov x0, #3 << 20
+ msr cpacr_el1, x0 // Enable FP/ASIMD
+ mov x0, #1
+ msr oslar_el1, x0 // Set the debug OS lock
+ tlbi vmalle1is // invalidate I + D TLBs
+ /*
+ * Memory region attributes for LPAE:
+ *
+ * n = AttrIndx[2:0]
+ * n MAIR
+ * DEVICE_nGnRnE 000 00000000
+ * DEVICE_nGnRE 001 00000100
+ * DEVICE_GRE 010 00001100
+ * NORMAL_NC 011 01000100
+ * NORMAL 100 11111111
+ */
+ ldr x5, =MAIR(0x00, MT_DEVICE_nGnRnE) | \
+ MAIR(0x04, MT_DEVICE_nGnRE) | \
+ MAIR(0x0c, MT_DEVICE_GRE) | \
+ MAIR(0x44, MT_NORMAL_NC) | \
+ MAIR(0xff, MT_NORMAL)
+ msr mair_el1, x5
+ /*
+ * Prepare SCTLR
+ */
+ adr x5, crval
+ ldp w5, w6, [x5]
+ mrs x0, sctlr_el1
+ bic x0, x0, x5 // clear bits
+ orr x0, x0, x6 // set bits
+ /*
+ * Set/prepare TCR and TTBR. We use 512GB (39-bit) address range for
+ * both user and kernel.
+ */
+ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_FLAGS | TCR_IPS_40BIT | \
+ TCR_ASID16 | (1 << 31)
+#ifdef CONFIG_ARM64_64K_PAGES
+ orr x10, x10, TCR_TG0_64K
+ orr x10, x10, TCR_TG1_64K
+#endif
+ msr tcr_el1, x10
+ ret // return to head.S
+ENDPROC(__cpu_setup)
+
+ /*
+ * n n T
+ * U E WT T UD US IHBS
+ * CE0 XWHW CZ ME TEEA S
+ * .... .IEE .... NEAI TE.I ..AD DEN0 ACAM
+ * 0011 0... 1101 ..0. ..0. 10.. .... .... < hardware reserved
+ * .... .100 .... 01.1 11.1 ..01 0001 1101 < software settings
+ */
+ .type crval, #object
+crval:
+ .word 0x030802e2 // clear
+ .word 0x0405d11d // set
diff --git a/arch/arm64/mm/tlb.S b/arch/arm64/mm/tlb.S
new file mode 100644
index 0000000..8ae80a1
--- /dev/null
+++ b/arch/arm64/mm/tlb.S
@@ -0,0 +1,71 @@
+/*
+ * Based on arch/arm/mm/tlb.S
+ *
+ * Copyright (C) 1997-2002 Russell King
+ * Copyright (C) 2012 ARM Ltd.
+ * Written by Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/linkage.h>
+#include <asm/assembler.h>
+#include <asm/asm-offsets.h>
+#include <asm/page.h>
+#include <asm/tlbflush.h>
+#include "proc-macros.S"
+
+/*
+ * __cpu_flush_user_tlb_range(start, end, vma)
+ *
+ * Invalidate a range of TLB entries in the specified address space.
+ *
+ * - start - start address (may not be aligned)
+ * - end - end address (exclusive, may not be aligned)
+ * - vma - vma_struct describing address range
+ */
+ENTRY(__cpu_flush_user_tlb_range)
+ vma_vm_mm x3, x2 // get vma->vm_mm
+ mmid x3, x3 // get vm_mm->context.id
+ dsb sy
+ lsr x0, x0, #12 // align address
+ lsr x1, x1, #12
+ bfi x0, x3, #48, #16 // start VA and ASID
+ bfi x1, x3, #48, #16 // end VA and ASID
+1: tlbi vae1is, x0 // TLB invalidate by address and ASID
+ add x0, x0, #1
+ cmp x0, x1
+ b.lo 1b
+ dsb sy
+ ret
+ENDPROC(__cpu_flush_user_tlb_range)
+
+/*
+ * __cpu_flush_kern_tlb_range(start,end)
+ *
+ * Invalidate a range of kernel TLB entries.
+ *
+ * - start - start address (may not be aligned)
+ * - end - end address (exclusive, may not be aligned)
+ */
+ENTRY(__cpu_flush_kern_tlb_range)
+ dsb sy
+ lsr x0, x0, #12 // align address
+ lsr x1, x1, #12
+1: tlbi vaae1is, x0 // TLB invalidate by address
+ add x0, x0, #1
+ cmp x0, x1
+ b.lo 1b
+ dsb sy
+ isb
+ ret
+ENDPROC(__cpu_flush_kern_tlb_range)
diff --git a/arch/blackfin/Kconfig b/arch/blackfin/Kconfig
index f348619..c7092e6 100644
--- a/arch/blackfin/Kconfig
+++ b/arch/blackfin/Kconfig
@@ -38,6 +38,7 @@
select GENERIC_ATOMIC64
select GENERIC_IRQ_PROBE
select IRQ_PER_CPU if SMP
+ select USE_GENERIC_SMP_HELPERS if SMP
select HAVE_NMI_WATCHDOG if NMI_WATCHDOG
select GENERIC_SMP_IDLE_THREAD
select ARCH_USES_GETTIMEOFFSET if !GENERIC_CLOCKEVENTS
diff --git a/arch/blackfin/Makefile b/arch/blackfin/Makefile
index d3d7e64..66cf000 100644
--- a/arch/blackfin/Makefile
+++ b/arch/blackfin/Makefile
@@ -20,7 +20,6 @@
KBUILD_AFLAGS += $(call cc-option,-mno-fdpic)
KBUILD_CFLAGS_MODULE += -mlong-calls
LDFLAGS += -m elf32bfin
-KALLSYMS += --symbol-prefix=_
KBUILD_DEFCONFIG := BF537-STAMP_defconfig
diff --git a/arch/blackfin/include/asm/smp.h b/arch/blackfin/include/asm/smp.h
index dc3d144..9631598 100644
--- a/arch/blackfin/include/asm/smp.h
+++ b/arch/blackfin/include/asm/smp.h
@@ -18,6 +18,8 @@
#define raw_smp_processor_id() blackfin_core_id()
extern void bfin_relocate_coreb_l1_mem(void);
+extern void arch_send_call_function_single_ipi(int cpu);
+extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
#if defined(CONFIG_SMP) && defined(CONFIG_ICACHE_FLUSH_L1)
asmlinkage void blackfin_icache_flush_range_l1(unsigned long *ptr);
diff --git a/arch/blackfin/mach-common/smp.c b/arch/blackfin/mach-common/smp.c
index 00bbe67..a401513 100644
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -48,10 +48,13 @@
struct blackfin_initial_pda __cpuinitdata initial_pda_coreb;
-#define BFIN_IPI_TIMER 0
-#define BFIN_IPI_RESCHEDULE 1
-#define BFIN_IPI_CALL_FUNC 2
-#define BFIN_IPI_CPU_STOP 3
+enum ipi_message_type {
+ BFIN_IPI_TIMER,
+ BFIN_IPI_RESCHEDULE,
+ BFIN_IPI_CALL_FUNC,
+ BFIN_IPI_CALL_FUNC_SINGLE,
+ BFIN_IPI_CPU_STOP,
+};
struct blackfin_flush_data {
unsigned long start;
@@ -60,35 +63,20 @@
void *secondary_stack;
-
-struct smp_call_struct {
- void (*func)(void *info);
- void *info;
- int wait;
- cpumask_t *waitmask;
-};
-
static struct blackfin_flush_data smp_flush_data;
static DEFINE_SPINLOCK(stop_lock);
-struct ipi_message {
- unsigned long type;
- struct smp_call_struct call_struct;
-};
-
/* A magic number - stress test shows this is safe for common cases */
#define BFIN_IPI_MSGQ_LEN 5
/* Simple FIFO buffer, overflow leads to panic */
-struct ipi_message_queue {
- spinlock_t lock;
+struct ipi_data {
unsigned long count;
- unsigned long head; /* head of the queue */
- struct ipi_message ipi_message[BFIN_IPI_MSGQ_LEN];
+ unsigned long bits;
};
-static DEFINE_PER_CPU(struct ipi_message_queue, ipi_msg_queue);
+static DEFINE_PER_CPU(struct ipi_data, bfin_ipi);
static void ipi_cpu_stop(unsigned int cpu)
{
@@ -129,28 +117,6 @@
blackfin_icache_flush_range(fdata->start, fdata->end);
}
-static void ipi_call_function(unsigned int cpu, struct ipi_message *msg)
-{
- int wait;
- void (*func)(void *info);
- void *info;
- func = msg->call_struct.func;
- info = msg->call_struct.info;
- wait = msg->call_struct.wait;
- func(info);
- if (wait) {
-#ifdef __ARCH_SYNC_CORE_DCACHE
- /*
- * 'wait' usually means synchronization between CPUs.
- * Invalidate D cache in case shared data was changed
- * by func() to ensure cache coherence.
- */
- resync_core_dcache();
-#endif
- cpumask_clear_cpu(cpu, msg->call_struct.waitmask);
- }
-}
-
/* Use IRQ_SUPPLE_0 to request reschedule.
* When returning from interrupt to user space,
* there is chance to reschedule */
@@ -172,152 +138,95 @@
static irqreturn_t ipi_handler_int1(int irq, void *dev_instance)
{
- struct ipi_message *msg;
- struct ipi_message_queue *msg_queue;
+ struct ipi_data *bfin_ipi_data;
unsigned int cpu = smp_processor_id();
- unsigned long flags;
+ unsigned long pending;
+ unsigned long msg;
platform_clear_ipi(cpu, IRQ_SUPPLE_1);
- msg_queue = &__get_cpu_var(ipi_msg_queue);
+ bfin_ipi_data = &__get_cpu_var(bfin_ipi);
- spin_lock_irqsave(&msg_queue->lock, flags);
+ while ((pending = xchg(&bfin_ipi_data->bits, 0)) != 0) {
+ msg = 0;
+ do {
+ msg = find_next_bit(&pending, BITS_PER_LONG, msg + 1);
+ switch (msg) {
+ case BFIN_IPI_TIMER:
+ ipi_timer();
+ break;
+ case BFIN_IPI_RESCHEDULE:
+ scheduler_ipi();
+ break;
+ case BFIN_IPI_CALL_FUNC:
+ generic_smp_call_function_interrupt();
+ break;
- while (msg_queue->count) {
- msg = &msg_queue->ipi_message[msg_queue->head];
- switch (msg->type) {
- case BFIN_IPI_TIMER:
- ipi_timer();
- break;
- case BFIN_IPI_RESCHEDULE:
- scheduler_ipi();
- break;
- case BFIN_IPI_CALL_FUNC:
- ipi_call_function(cpu, msg);
- break;
- case BFIN_IPI_CPU_STOP:
- ipi_cpu_stop(cpu);
- break;
- default:
- printk(KERN_CRIT "CPU%u: Unknown IPI message 0x%lx\n",
- cpu, msg->type);
- break;
- }
- msg_queue->head++;
- msg_queue->head %= BFIN_IPI_MSGQ_LEN;
- msg_queue->count--;
+ case BFIN_IPI_CALL_FUNC_SINGLE:
+ generic_smp_call_function_single_interrupt();
+ break;
+
+ case BFIN_IPI_CPU_STOP:
+ ipi_cpu_stop(cpu);
+ break;
+ }
+ } while (msg < BITS_PER_LONG);
+
+ smp_mb();
}
- spin_unlock_irqrestore(&msg_queue->lock, flags);
return IRQ_HANDLED;
}
-static void ipi_queue_init(void)
+static void bfin_ipi_init(void)
{
unsigned int cpu;
- struct ipi_message_queue *msg_queue;
+ struct ipi_data *bfin_ipi_data;
for_each_possible_cpu(cpu) {
- msg_queue = &per_cpu(ipi_msg_queue, cpu);
- spin_lock_init(&msg_queue->lock);
- msg_queue->count = 0;
- msg_queue->head = 0;
+ bfin_ipi_data = &per_cpu(bfin_ipi, cpu);
+ bfin_ipi_data->bits = 0;
+ bfin_ipi_data->count = 0;
}
}
-static inline void smp_send_message(cpumask_t callmap, unsigned long type,
- void (*func) (void *info), void *info, int wait)
+void send_ipi(const struct cpumask *cpumask, enum ipi_message_type msg)
{
unsigned int cpu;
- struct ipi_message_queue *msg_queue;
- struct ipi_message *msg;
- unsigned long flags, next_msg;
- cpumask_t waitmask; /* waitmask is shared by all cpus */
+ struct ipi_data *bfin_ipi_data;
+ unsigned long flags;
- cpumask_copy(&waitmask, &callmap);
- for_each_cpu(cpu, &callmap) {
- msg_queue = &per_cpu(ipi_msg_queue, cpu);
- spin_lock_irqsave(&msg_queue->lock, flags);
- if (msg_queue->count < BFIN_IPI_MSGQ_LEN) {
- next_msg = (msg_queue->head + msg_queue->count)
- % BFIN_IPI_MSGQ_LEN;
- msg = &msg_queue->ipi_message[next_msg];
- msg->type = type;
- if (type == BFIN_IPI_CALL_FUNC) {
- msg->call_struct.func = func;
- msg->call_struct.info = info;
- msg->call_struct.wait = wait;
- msg->call_struct.waitmask = &waitmask;
- }
- msg_queue->count++;
- } else
- panic("IPI message queue overflow\n");
- spin_unlock_irqrestore(&msg_queue->lock, flags);
+ local_irq_save(flags);
+
+ for_each_cpu(cpu, cpumask) {
+ bfin_ipi_data = &per_cpu(bfin_ipi, cpu);
+ smp_mb();
+ set_bit(msg, &bfin_ipi_data->bits);
+ bfin_ipi_data->count++;
platform_send_ipi_cpu(cpu, IRQ_SUPPLE_1);
}
- if (wait) {
- while (!cpumask_empty(&waitmask))
- blackfin_dcache_invalidate_range(
- (unsigned long)(&waitmask),
- (unsigned long)(&waitmask));
-#ifdef __ARCH_SYNC_CORE_DCACHE
- /*
- * Invalidate D cache in case shared data was changed by
- * other processors to ensure cache coherence.
- */
- resync_core_dcache();
-#endif
- }
+ local_irq_restore(flags);
}
-int smp_call_function(void (*func)(void *info), void *info, int wait)
+void arch_send_call_function_single_ipi(int cpu)
{
- cpumask_t callmap;
-
- preempt_disable();
- cpumask_copy(&callmap, cpu_online_mask);
- cpumask_clear_cpu(smp_processor_id(), &callmap);
- if (!cpumask_empty(&callmap))
- smp_send_message(callmap, BFIN_IPI_CALL_FUNC, func, info, wait);
-
- preempt_enable();
-
- return 0;
+ send_ipi(cpumask_of(cpu), BFIN_IPI_CALL_FUNC_SINGLE);
}
-EXPORT_SYMBOL_GPL(smp_call_function);
-int smp_call_function_single(int cpuid, void (*func) (void *info), void *info,
- int wait)
+void arch_send_call_function_ipi_mask(const struct cpumask *mask)
{
- unsigned int cpu = cpuid;
- cpumask_t callmap;
-
- if (cpu_is_offline(cpu))
- return 0;
- cpumask_clear(&callmap);
- cpumask_set_cpu(cpu, &callmap);
-
- smp_send_message(callmap, BFIN_IPI_CALL_FUNC, func, info, wait);
-
- return 0;
+ send_ipi(mask, BFIN_IPI_CALL_FUNC);
}
-EXPORT_SYMBOL_GPL(smp_call_function_single);
void smp_send_reschedule(int cpu)
{
- cpumask_t callmap;
- /* simply trigger an ipi */
-
- cpumask_clear(&callmap);
- cpumask_set_cpu(cpu, &callmap);
-
- smp_send_message(callmap, BFIN_IPI_RESCHEDULE, NULL, NULL, 0);
+ send_ipi(cpumask_of(cpu), BFIN_IPI_RESCHEDULE);
return;
}
void smp_send_msg(const struct cpumask *mask, unsigned long type)
{
- smp_send_message(*mask, type, NULL, NULL, 0);
+ send_ipi(mask, type);
}
void smp_timer_broadcast(const struct cpumask *mask)
@@ -333,7 +242,7 @@
cpumask_copy(&callmap, cpu_online_mask);
cpumask_clear_cpu(smp_processor_id(), &callmap);
if (!cpumask_empty(&callmap))
- smp_send_message(callmap, BFIN_IPI_CPU_STOP, NULL, NULL, 0);
+ send_ipi(&callmap, BFIN_IPI_CPU_STOP);
preempt_enable();
@@ -436,7 +345,7 @@
void __init smp_prepare_cpus(unsigned int max_cpus)
{
platform_prepare_cpus(max_cpus);
- ipi_queue_init();
+ bfin_ipi_init();
platform_request_ipi(IRQ_SUPPLE_0, ipi_handler_int0);
platform_request_ipi(IRQ_SUPPLE_1, ipi_handler_int1);
}
diff --git a/arch/c6x/include/asm/Kbuild b/arch/c6x/include/asm/Kbuild
index 3af601e..f08e891 100644
--- a/arch/c6x/include/asm/Kbuild
+++ b/arch/c6x/include/asm/Kbuild
@@ -2,6 +2,7 @@
generic-y += atomic.h
generic-y += auxvec.h
+generic-y += barrier.h
generic-y += bitsperlong.h
generic-y += bugs.h
generic-y += cputime.h
diff --git a/arch/c6x/include/asm/barrier.h b/arch/c6x/include/asm/barrier.h
deleted file mode 100644
index 538240e..0000000
--- a/arch/c6x/include/asm/barrier.h
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Port on Texas Instruments TMS320C6x architecture
- *
- * Copyright (C) 2004, 2009, 2010, 2011 Texas Instruments Incorporated
- * Author: Aurelien Jacquiot (aurelien.jacquiot@jaluna.com)
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#ifndef _ASM_C6X_BARRIER_H
-#define _ASM_C6X_BARRIER_H
-
-#define nop() asm("NOP\n");
-
-#define mb() barrier()
-#define rmb() barrier()
-#define wmb() barrier()
-#define set_mb(var, value) do { var = value; mb(); } while (0)
-#define set_wmb(var, value) do { var = value; wmb(); } while (0)
-
-#define smp_mb() barrier()
-#define smp_rmb() barrier()
-#define smp_wmb() barrier()
-#define smp_read_barrier_depends() do { } while (0)
-
-#endif /* _ASM_C6X_BARRIER_H */
diff --git a/arch/cris/kernel/process.c b/arch/cris/kernel/process.c
index 66fd017..7f65be6 100644
--- a/arch/cris/kernel/process.c
+++ b/arch/cris/kernel/process.c
@@ -25,6 +25,7 @@
#include <linux/elfcore.h>
#include <linux/mqueue.h>
#include <linux/reboot.h>
+#include <linux/rcupdate.h>
//#define DEBUG
@@ -74,6 +75,7 @@
{
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched()) {
void (*idle)(void);
/*
@@ -86,6 +88,7 @@
idle = default_idle;
idle();
}
+ rcu_idle_exit();
schedule_preempt_disabled();
}
}
diff --git a/arch/frv/kernel/process.c b/arch/frv/kernel/process.c
index ff95f50..2eb7fa5 100644
--- a/arch/frv/kernel/process.c
+++ b/arch/frv/kernel/process.c
@@ -25,6 +25,7 @@
#include <linux/reboot.h>
#include <linux/interrupt.h>
#include <linux/pagemap.h>
+#include <linux/rcupdate.h>
#include <asm/asm-offsets.h>
#include <asm/uaccess.h>
@@ -69,12 +70,14 @@
{
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched()) {
check_pgt_cache();
if (!frv_dma_inprogress && idle)
idle();
}
+ rcu_idle_exit();
schedule_preempt_disabled();
}
diff --git a/arch/frv/mb93090-mb00/pci-vdk.c b/arch/frv/mb93090-mb00/pci-vdk.c
index d04ed14..71e9bcf 100644
--- a/arch/frv/mb93090-mb00/pci-vdk.c
+++ b/arch/frv/mb93090-mb00/pci-vdk.c
@@ -330,10 +330,8 @@
pci_read_bridge_bases(bus);
if (bus->number == 0) {
- struct list_head *ln;
struct pci_dev *dev;
- for (ln=bus->devices.next; ln != &bus->devices; ln=ln->next) {
- dev = pci_dev_b(ln);
+ list_for_each_entry(dev, &bus->devices, bus_list) {
if (dev->devfn == 0) {
dev->resource[0].start = 0;
dev->resource[0].end = 0;
diff --git a/arch/h8300/kernel/process.c b/arch/h8300/kernel/process.c
index 0e9c315..f153ed1 100644
--- a/arch/h8300/kernel/process.c
+++ b/arch/h8300/kernel/process.c
@@ -36,6 +36,7 @@
#include <linux/reboot.h>
#include <linux/fs.h>
#include <linux/slab.h>
+#include <linux/rcupdate.h>
#include <asm/uaccess.h>
#include <asm/traps.h>
@@ -78,8 +79,10 @@
void cpu_idle(void)
{
while (1) {
+ rcu_idle_enter();
while (!need_resched())
idle();
+ rcu_idle_exit();
schedule_preempt_disabled();
}
}
diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index 310cf57..3c720ef 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -25,6 +25,7 @@
select HAVE_GENERIC_HARDIRQS
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP
+ select HAVE_VIRT_CPU_ACCOUNTING
select ARCH_DISCARD_MEMBLOCK
select GENERIC_IRQ_PROBE
select GENERIC_PENDING_IRQ if SMP
@@ -340,17 +341,6 @@
default "17" if HUGETLB_PAGE
default "11"
-config VIRT_CPU_ACCOUNTING
- bool "Deterministic task and CPU time accounting"
- default n
- help
- Select this option to enable more accurate task and CPU time
- accounting. This is done by reading a CPU counter on each
- kernel entry and exit and on transitions within the kernel
- between system, softirq and hardirq state, so there is a
- small performance impact.
- If in doubt, say N here.
-
config SMP
bool "Symmetric multi-processing support"
select USE_GENERIC_SMP_HELPERS
diff --git a/arch/ia64/include/asm/switch_to.h b/arch/ia64/include/asm/switch_to.h
index cb2412f..d38c7ea 100644
--- a/arch/ia64/include/asm/switch_to.h
+++ b/arch/ia64/include/asm/switch_to.h
@@ -30,13 +30,6 @@
extern void ia64_save_extra (struct task_struct *task);
extern void ia64_load_extra (struct task_struct *task);
-#ifdef CONFIG_VIRT_CPU_ACCOUNTING
-extern void ia64_account_on_switch (struct task_struct *prev, struct task_struct *next);
-# define IA64_ACCOUNT_ON_SWITCH(p,n) ia64_account_on_switch(p,n)
-#else
-# define IA64_ACCOUNT_ON_SWITCH(p,n)
-#endif
-
#ifdef CONFIG_PERFMON
DECLARE_PER_CPU(unsigned long, pfm_syst_info);
# define PERFMON_IS_SYSWIDE() (__get_cpu_var(pfm_syst_info) & 0x1)
@@ -49,7 +42,6 @@
|| PERFMON_IS_SYSWIDE())
#define __switch_to(prev,next,last) do { \
- IA64_ACCOUNT_ON_SWITCH(prev, next); \
if (IA64_HAS_EXTRA_STATE(prev)) \
ia64_save_extra(prev); \
if (IA64_HAS_EXTRA_STATE(next)) \
diff --git a/arch/ia64/kernel/process.c b/arch/ia64/kernel/process.c
index dd6fc14..3e316ec 100644
--- a/arch/ia64/kernel/process.c
+++ b/arch/ia64/kernel/process.c
@@ -29,6 +29,7 @@
#include <linux/kdebug.h>
#include <linux/utsname.h>
#include <linux/tracehook.h>
+#include <linux/rcupdate.h>
#include <asm/cpu.h>
#include <asm/delay.h>
@@ -279,6 +280,7 @@
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
if (can_do_pal_halt) {
current_thread_info()->status &= ~TS_POLLING;
/*
@@ -309,6 +311,7 @@
normal_xtp();
#endif
}
+ rcu_idle_exit();
schedule_preempt_disabled();
check_pgt_cache();
if (cpu_is_offline(cpu))
diff --git a/arch/ia64/kernel/time.c b/arch/ia64/kernel/time.c
index ecc904b..80ff9ac 100644
--- a/arch/ia64/kernel/time.c
+++ b/arch/ia64/kernel/time.c
@@ -83,32 +83,36 @@
extern cputime_t cycle_to_cputime(u64 cyc);
+static void vtime_account_user(struct task_struct *tsk)
+{
+ cputime_t delta_utime;
+ struct thread_info *ti = task_thread_info(tsk);
+
+ if (ti->ac_utime) {
+ delta_utime = cycle_to_cputime(ti->ac_utime);
+ account_user_time(tsk, delta_utime, delta_utime);
+ ti->ac_utime = 0;
+ }
+}
+
/*
* Called from the context switch with interrupts disabled, to charge all
* accumulated times to the current process, and to prepare accounting on
* the next process.
*/
-void ia64_account_on_switch(struct task_struct *prev, struct task_struct *next)
+void vtime_task_switch(struct task_struct *prev)
{
struct thread_info *pi = task_thread_info(prev);
- struct thread_info *ni = task_thread_info(next);
- cputime_t delta_stime, delta_utime;
- __u64 now;
+ struct thread_info *ni = task_thread_info(current);
- now = ia64_get_itc();
-
- delta_stime = cycle_to_cputime(pi->ac_stime + (now - pi->ac_stamp));
if (idle_task(smp_processor_id()) != prev)
- account_system_time(prev, 0, delta_stime, delta_stime);
+ vtime_account_system(prev);
else
- account_idle_time(delta_stime);
+ vtime_account_idle(prev);
- if (pi->ac_utime) {
- delta_utime = cycle_to_cputime(pi->ac_utime);
- account_user_time(prev, delta_utime, delta_utime);
- }
+ vtime_account_user(prev);
- pi->ac_stamp = ni->ac_stamp = now;
+ pi->ac_stamp = ni->ac_stamp;
ni->ac_stime = ni->ac_utime = 0;
}
@@ -116,29 +120,32 @@
* Account time for a transition between system, hard irq or soft irq state.
* Note that this function is called with interrupts enabled.
*/
-void account_system_vtime(struct task_struct *tsk)
+static cputime_t vtime_delta(struct task_struct *tsk)
{
struct thread_info *ti = task_thread_info(tsk);
- unsigned long flags;
cputime_t delta_stime;
__u64 now;
- local_irq_save(flags);
-
now = ia64_get_itc();
delta_stime = cycle_to_cputime(ti->ac_stime + (now - ti->ac_stamp));
- if (irq_count() || idle_task(smp_processor_id()) != tsk)
- account_system_time(tsk, 0, delta_stime, delta_stime);
- else
- account_idle_time(delta_stime);
ti->ac_stime = 0;
-
ti->ac_stamp = now;
- local_irq_restore(flags);
+ return delta_stime;
}
-EXPORT_SYMBOL_GPL(account_system_vtime);
+
+void vtime_account_system(struct task_struct *tsk)
+{
+ cputime_t delta = vtime_delta(tsk);
+
+ account_system_time(tsk, 0, delta, delta);
+}
+
+void vtime_account_idle(struct task_struct *tsk)
+{
+ account_idle_time(vtime_delta(tsk));
+}
/*
* Called from the timer interrupt handler to charge accumulated user time
@@ -146,14 +153,7 @@
*/
void account_process_tick(struct task_struct *p, int user_tick)
{
- struct thread_info *ti = task_thread_info(p);
- cputime_t delta_utime;
-
- if (ti->ac_utime) {
- delta_utime = cycle_to_cputime(ti->ac_utime);
- account_user_time(p, delta_utime, delta_utime);
- ti->ac_utime = 0;
- }
+ vtime_account_user(p);
}
#endif /* CONFIG_VIRT_CPU_ACCOUNTING */
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 81acc7a..5faa66c 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -295,7 +295,6 @@
window->resource.flags = flags;
window->resource.start = addr.minimum + offset;
window->resource.end = window->resource.start + addr.address_length - 1;
- window->resource.child = NULL;
window->offset = offset;
if (insert_resource(root, &window->resource)) {
@@ -357,7 +356,7 @@
&windows);
if (windows) {
controller->window =
- kmalloc_node(sizeof(*controller->window) * windows,
+ kzalloc_node(sizeof(*controller->window) * windows,
GFP_KERNEL, controller->node);
if (!controller->window)
goto out2;
@@ -461,14 +460,6 @@
/* No special bus mastering setup handling */
}
-void __devinit
-pcibios_update_irq (struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-
- /* ??? FIXME -- record old value for shutdown. */
-}
-
int
pcibios_enable_device (struct pci_dev *dev, int mask)
{
diff --git a/arch/ia64/sn/kernel/io_common.c b/arch/ia64/sn/kernel/io_common.c
index fbb5f2f..8630875 100644
--- a/arch/ia64/sn/kernel/io_common.c
+++ b/arch/ia64/sn/kernel/io_common.c
@@ -229,7 +229,6 @@
{
int segment = pci_domain_nr(dev->bus);
struct pcibus_bussoft *bs;
- struct pci_bus *host_pci_bus;
struct pci_dev *host_pci_dev;
unsigned int bus_no, devfn;
@@ -245,8 +244,7 @@
bus_no = (pcidev_info->pdi_slot_host_handle >> 32) & 0xff;
devfn = pcidev_info->pdi_slot_host_handle & 0xffffffff;
- host_pci_bus = pci_find_bus(segment, bus_no);
- host_pci_dev = pci_get_slot(host_pci_bus, devfn);
+ host_pci_dev = pci_get_domain_bus_and_slot(segment, bus_no, devfn);
pcidev_info->host_pci_dev = host_pci_dev;
pcidev_info->pdi_linux_pcidev = dev;
diff --git a/arch/m32r/kernel/process.c b/arch/m32r/kernel/process.c
index 3a4a32b..384e63f 100644
--- a/arch/m32r/kernel/process.c
+++ b/arch/m32r/kernel/process.c
@@ -26,6 +26,7 @@
#include <linux/ptrace.h>
#include <linux/unistd.h>
#include <linux/hardirq.h>
+#include <linux/rcupdate.h>
#include <asm/io.h>
#include <asm/uaccess.h>
@@ -82,6 +83,7 @@
{
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched()) {
void (*idle)(void) = pm_idle;
@@ -90,6 +92,7 @@
idle();
}
+ rcu_idle_exit();
schedule_preempt_disabled();
}
}
diff --git a/arch/m68k/kernel/pcibios.c b/arch/m68k/kernel/pcibios.c
index b2988aa..73fa0b5 100644
--- a/arch/m68k/kernel/pcibios.c
+++ b/arch/m68k/kernel/pcibios.c
@@ -87,11 +87,6 @@
return 0;
}
-void pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
void __devinit pcibios_fixup_bus(struct pci_bus *bus)
{
struct pci_dev *dev;
diff --git a/arch/m68k/kernel/process.c b/arch/m68k/kernel/process.c
index c488e3c..ac2892e 100644
--- a/arch/m68k/kernel/process.c
+++ b/arch/m68k/kernel/process.c
@@ -25,6 +25,7 @@
#include <linux/reboot.h>
#include <linux/init_task.h>
#include <linux/mqueue.h>
+#include <linux/rcupdate.h>
#include <asm/uaccess.h>
#include <asm/traps.h>
@@ -75,8 +76,10 @@
{
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched())
idle();
+ rcu_idle_exit();
schedule_preempt_disabled();
}
}
diff --git a/arch/m68k/platform/coldfire/clk.c b/arch/m68k/platform/coldfire/clk.c
index 75f9ee9..9cd13b4 100644
--- a/arch/m68k/platform/coldfire/clk.c
+++ b/arch/m68k/platform/coldfire/clk.c
@@ -146,9 +146,3 @@
};
#endif /* MCFPM_PPMCR1 */
#endif /* MCFPM_PPMCR0 */
-
-struct clk *devm_clk_get(struct device *dev, const char *id)
-{
- return NULL;
-}
-EXPORT_SYMBOL(devm_clk_get);
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 331d574..faf6528 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -89,6 +89,7 @@
select CEVT_R4K
select CSRC_R4K
select DMA_NONCOHERENT
+ select HAVE_CLK
select IRQ_CPU
select MIPS_MACHINE
select SYS_HAS_CPU_MIPS32_R2
diff --git a/arch/mips/alchemy/board-mtx1.c b/arch/mips/alchemy/board-mtx1.c
index 9996948..a124c25 100644
--- a/arch/mips/alchemy/board-mtx1.c
+++ b/arch/mips/alchemy/board-mtx1.c
@@ -228,6 +228,8 @@
* adapter on the mtx-1 "singleboard" variant. It triggers a custom
* logic chip connected to EXT_IO3 (GPIO1) to suppress IDSEL signals.
*/
+ udelay(1);
+
if (assert && devsel != 0)
/* Suppress signal to Cardbus */
alchemy_gpio_set_value(1, 0); /* set EXT_IO3 OFF */
diff --git a/arch/mips/ath79/dev-usb.c b/arch/mips/ath79/dev-usb.c
index 36e9570..b2a2311 100644
--- a/arch/mips/ath79/dev-usb.c
+++ b/arch/mips/ath79/dev-usb.c
@@ -145,6 +145,8 @@
ath79_ohci_resources[0].start = AR7240_OHCI_BASE;
ath79_ohci_resources[0].end = AR7240_OHCI_BASE + AR7240_OHCI_SIZE - 1;
+ ath79_ohci_resources[1].start = ATH79_CPU_IRQ_USB;
+ ath79_ohci_resources[1].end = ATH79_CPU_IRQ_USB;
platform_device_register(&ath79_ohci_device);
}
diff --git a/arch/mips/ath79/gpio.c b/arch/mips/ath79/gpio.c
index 29054f2..48fe762 100644
--- a/arch/mips/ath79/gpio.c
+++ b/arch/mips/ath79/gpio.c
@@ -188,8 +188,10 @@
if (soc_is_ar71xx())
ath79_gpio_count = AR71XX_GPIO_COUNT;
- else if (soc_is_ar724x())
- ath79_gpio_count = AR724X_GPIO_COUNT;
+ else if (soc_is_ar7240())
+ ath79_gpio_count = AR7240_GPIO_COUNT;
+ else if (soc_is_ar7241() || soc_is_ar7242())
+ ath79_gpio_count = AR7241_GPIO_COUNT;
else if (soc_is_ar913x())
ath79_gpio_count = AR913X_GPIO_COUNT;
else if (soc_is_ar933x())
diff --git a/arch/mips/bcm63xx/dev-spi.c b/arch/mips/bcm63xx/dev-spi.c
index e39f730..f1c9c3e 100644
--- a/arch/mips/bcm63xx/dev-spi.c
+++ b/arch/mips/bcm63xx/dev-spi.c
@@ -106,11 +106,15 @@
if (BCMCPU_IS_6338() || BCMCPU_IS_6348()) {
spi_resources[0].end += BCM_6338_RSET_SPI_SIZE - 1;
spi_pdata.fifo_size = SPI_6338_MSG_DATA_SIZE;
+ spi_pdata.msg_type_shift = SPI_6338_MSG_TYPE_SHIFT;
+ spi_pdata.msg_ctl_width = SPI_6338_MSG_CTL_WIDTH;
}
if (BCMCPU_IS_6358() || BCMCPU_IS_6368()) {
spi_resources[0].end += BCM_6358_RSET_SPI_SIZE - 1;
spi_pdata.fifo_size = SPI_6358_MSG_DATA_SIZE;
+ spi_pdata.msg_type_shift = SPI_6358_MSG_TYPE_SHIFT;
+ spi_pdata.msg_ctl_width = SPI_6358_MSG_CTL_WIDTH;
}
bcm63xx_spi_regs_init();
diff --git a/arch/mips/cavium-octeon/octeon-irq.c b/arch/mips/cavium-octeon/octeon-irq.c
index 7fb1f22..274cd4f 100644
--- a/arch/mips/cavium-octeon/octeon-irq.c
+++ b/arch/mips/cavium-octeon/octeon-irq.c
@@ -61,6 +61,12 @@
octeon_irq_ciu_to_irq[line][bit] = irq;
}
+static void octeon_irq_force_ciu_mapping(struct irq_domain *domain,
+ int irq, int line, int bit)
+{
+ irq_domain_associate(domain, irq, line << 6 | bit);
+}
+
static int octeon_coreid_for_cpu(int cpu)
{
#ifdef CONFIG_SMP
@@ -183,19 +189,9 @@
mutex_init(&cd->core_irq_mutex);
irq = OCTEON_IRQ_SW0 + i;
- switch (irq) {
- case OCTEON_IRQ_TIMER:
- case OCTEON_IRQ_SW0:
- case OCTEON_IRQ_SW1:
- case OCTEON_IRQ_5:
- case OCTEON_IRQ_PERF:
- irq_set_chip_data(irq, cd);
- irq_set_chip_and_handler(irq, &octeon_irq_chip_core,
- handle_percpu_irq);
- break;
- default:
- break;
- }
+ irq_set_chip_data(irq, cd);
+ irq_set_chip_and_handler(irq, &octeon_irq_chip_core,
+ handle_percpu_irq);
}
}
@@ -890,7 +886,6 @@
unsigned int type;
unsigned int pin;
unsigned int trigger;
- struct octeon_irq_gpio_domain_data *gpiod;
if (d->of_node != node)
return -EINVAL;
@@ -925,8 +920,7 @@
break;
}
*out_type = type;
- gpiod = d->host_data;
- *out_hwirq = gpiod->base_hwirq + pin;
+ *out_hwirq = pin;
return 0;
}
@@ -996,19 +990,21 @@
static int octeon_irq_gpio_map(struct irq_domain *d,
unsigned int virq, irq_hw_number_t hw)
{
- unsigned int line = hw >> 6;
- unsigned int bit = hw & 63;
+ struct octeon_irq_gpio_domain_data *gpiod = d->host_data;
+ unsigned int line, bit;
if (!octeon_irq_virq_in_range(virq))
return -EINVAL;
+ hw += gpiod->base_hwirq;
+ line = hw >> 6;
+ bit = hw & 63;
if (line > 1 || octeon_irq_ciu_to_irq[line][bit] != 0)
return -EINVAL;
octeon_irq_set_ciu_mapping(virq, line, bit,
octeon_irq_gpio_chip,
octeon_irq_handle_gpio);
-
return 0;
}
@@ -1149,6 +1145,7 @@
struct irq_chip *chip_wd;
struct device_node *gpio_node;
struct device_node *ciu_node;
+ struct irq_domain *ciu_domain = NULL;
octeon_irq_init_ciu_percpu();
octeon_irq_setup_secondary = octeon_irq_setup_secondary_ciu;
@@ -1177,31 +1174,6 @@
/* Mips internal */
octeon_irq_init_core();
- /* CIU_0 */
- for (i = 0; i < 16; i++)
- octeon_irq_set_ciu_mapping(i + OCTEON_IRQ_WORKQ0, 0, i + 0, chip, handle_level_irq);
-
- octeon_irq_set_ciu_mapping(OCTEON_IRQ_MBOX0, 0, 32, chip_mbox, handle_percpu_irq);
- octeon_irq_set_ciu_mapping(OCTEON_IRQ_MBOX1, 0, 33, chip_mbox, handle_percpu_irq);
-
- for (i = 0; i < 4; i++)
- octeon_irq_set_ciu_mapping(i + OCTEON_IRQ_PCI_INT0, 0, i + 36, chip, handle_level_irq);
- for (i = 0; i < 4; i++)
- octeon_irq_set_ciu_mapping(i + OCTEON_IRQ_PCI_MSI0, 0, i + 40, chip, handle_level_irq);
-
- octeon_irq_set_ciu_mapping(OCTEON_IRQ_RML, 0, 46, chip, handle_level_irq);
- for (i = 0; i < 4; i++)
- octeon_irq_set_ciu_mapping(i + OCTEON_IRQ_TIMER0, 0, i + 52, chip, handle_edge_irq);
-
- octeon_irq_set_ciu_mapping(OCTEON_IRQ_USB0, 0, 56, chip, handle_level_irq);
- octeon_irq_set_ciu_mapping(OCTEON_IRQ_BOOTDMA, 0, 63, chip, handle_level_irq);
-
- /* CIU_1 */
- for (i = 0; i < 16; i++)
- octeon_irq_set_ciu_mapping(i + OCTEON_IRQ_WDOG0, 1, i + 0, chip_wd, handle_level_irq);
-
- octeon_irq_set_ciu_mapping(OCTEON_IRQ_USB1, 1, 17, chip, handle_level_irq);
-
gpio_node = of_find_compatible_node(NULL, NULL, "cavium,octeon-3860-gpio");
if (gpio_node) {
struct octeon_irq_gpio_domain_data *gpiod;
@@ -1219,10 +1191,35 @@
ciu_node = of_find_compatible_node(NULL, NULL, "cavium,octeon-3860-ciu");
if (ciu_node) {
- irq_domain_add_tree(ciu_node, &octeon_irq_domain_ciu_ops, NULL);
+ ciu_domain = irq_domain_add_tree(ciu_node, &octeon_irq_domain_ciu_ops, NULL);
of_node_put(ciu_node);
} else
- pr_warn("Cannot find device node for cavium,octeon-3860-ciu.\n");
+ panic("Cannot find device node for cavium,octeon-3860-ciu.");
+
+ /* CIU_0 */
+ for (i = 0; i < 16; i++)
+ octeon_irq_force_ciu_mapping(ciu_domain, i + OCTEON_IRQ_WORKQ0, 0, i + 0);
+
+ octeon_irq_set_ciu_mapping(OCTEON_IRQ_MBOX0, 0, 32, chip_mbox, handle_percpu_irq);
+ octeon_irq_set_ciu_mapping(OCTEON_IRQ_MBOX1, 0, 33, chip_mbox, handle_percpu_irq);
+
+ for (i = 0; i < 4; i++)
+ octeon_irq_force_ciu_mapping(ciu_domain, i + OCTEON_IRQ_PCI_INT0, 0, i + 36);
+ for (i = 0; i < 4; i++)
+ octeon_irq_force_ciu_mapping(ciu_domain, i + OCTEON_IRQ_PCI_MSI0, 0, i + 40);
+
+ octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_RML, 0, 46);
+ for (i = 0; i < 4; i++)
+ octeon_irq_force_ciu_mapping(ciu_domain, i + OCTEON_IRQ_TIMER0, 0, i + 52);
+
+ octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_USB0, 0, 56);
+ octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_BOOTDMA, 0, 63);
+
+ /* CIU_1 */
+ for (i = 0; i < 16; i++)
+ octeon_irq_set_ciu_mapping(i + OCTEON_IRQ_WDOG0, 1, i + 0, chip_wd, handle_level_irq);
+
+ octeon_irq_force_ciu_mapping(ciu_domain, OCTEON_IRQ_USB1, 1, 17);
/* Enable the CIU lines */
set_c0_status(STATUSF_IP3 | STATUSF_IP2);
diff --git a/arch/mips/include/asm/mach-ath79/ar71xx_regs.h b/arch/mips/include/asm/mach-ath79/ar71xx_regs.h
index 1caa78a..dde5044 100644
--- a/arch/mips/include/asm/mach-ath79/ar71xx_regs.h
+++ b/arch/mips/include/asm/mach-ath79/ar71xx_regs.h
@@ -393,7 +393,8 @@
#define AR71XX_GPIO_REG_FUNC 0x28
#define AR71XX_GPIO_COUNT 16
-#define AR724X_GPIO_COUNT 18
+#define AR7240_GPIO_COUNT 18
+#define AR7241_GPIO_COUNT 20
#define AR913X_GPIO_COUNT 22
#define AR933X_GPIO_COUNT 30
#define AR934X_GPIO_COUNT 23
diff --git a/arch/mips/include/asm/mach-ath79/cpu-feature-overrides.h b/arch/mips/include/asm/mach-ath79/cpu-feature-overrides.h
index 4476fa0..6ddae92 100644
--- a/arch/mips/include/asm/mach-ath79/cpu-feature-overrides.h
+++ b/arch/mips/include/asm/mach-ath79/cpu-feature-overrides.h
@@ -42,7 +42,6 @@
#define cpu_has_mips64r1 0
#define cpu_has_mips64r2 0
-#define cpu_has_dsp 0
#define cpu_has_mipsmt 0
#define cpu_has_64bits 0
diff --git a/arch/mips/include/asm/mach-bcm63xx/bcm63xx_dev_spi.h b/arch/mips/include/asm/mach-bcm63xx/bcm63xx_dev_spi.h
index 7d98dbe..c9bae13 100644
--- a/arch/mips/include/asm/mach-bcm63xx/bcm63xx_dev_spi.h
+++ b/arch/mips/include/asm/mach-bcm63xx/bcm63xx_dev_spi.h
@@ -9,6 +9,8 @@
struct bcm63xx_spi_pdata {
unsigned int fifo_size;
+ unsigned int msg_type_shift;
+ unsigned int msg_ctl_width;
int bus_num;
int num_chipselect;
u32 speed_hz;
diff --git a/arch/mips/include/asm/mach-bcm63xx/bcm63xx_regs.h b/arch/mips/include/asm/mach-bcm63xx/bcm63xx_regs.h
index 4ccc2a7..61f2a2a 100644
--- a/arch/mips/include/asm/mach-bcm63xx/bcm63xx_regs.h
+++ b/arch/mips/include/asm/mach-bcm63xx/bcm63xx_regs.h
@@ -1054,7 +1054,8 @@
#define SPI_6338_FILL_BYTE 0x07
#define SPI_6338_MSG_TAIL 0x09
#define SPI_6338_RX_TAIL 0x0b
-#define SPI_6338_MSG_CTL 0x40
+#define SPI_6338_MSG_CTL 0x40 /* 8-bits register */
+#define SPI_6338_MSG_CTL_WIDTH 8
#define SPI_6338_MSG_DATA 0x41
#define SPI_6338_MSG_DATA_SIZE 0x3f
#define SPI_6338_RX_DATA 0x80
@@ -1070,7 +1071,8 @@
#define SPI_6348_FILL_BYTE 0x07
#define SPI_6348_MSG_TAIL 0x09
#define SPI_6348_RX_TAIL 0x0b
-#define SPI_6348_MSG_CTL 0x40
+#define SPI_6348_MSG_CTL 0x40 /* 8-bits register */
+#define SPI_6348_MSG_CTL_WIDTH 8
#define SPI_6348_MSG_DATA 0x41
#define SPI_6348_MSG_DATA_SIZE 0x3f
#define SPI_6348_RX_DATA 0x80
@@ -1078,6 +1080,7 @@
/* BCM 6358 SPI core */
#define SPI_6358_MSG_CTL 0x00 /* 16-bits register */
+#define SPI_6358_MSG_CTL_WIDTH 16
#define SPI_6358_MSG_DATA 0x02
#define SPI_6358_MSG_DATA_SIZE 0x21e
#define SPI_6358_RX_DATA 0x400
@@ -1094,6 +1097,7 @@
/* BCM 6358 SPI core */
#define SPI_6368_MSG_CTL 0x00 /* 16-bits register */
+#define SPI_6368_MSG_CTL_WIDTH 16
#define SPI_6368_MSG_DATA 0x02
#define SPI_6368_MSG_DATA_SIZE 0x21e
#define SPI_6368_RX_DATA 0x400
@@ -1115,7 +1119,10 @@
#define SPI_HD_W 0x01
#define SPI_HD_R 0x02
#define SPI_BYTE_CNT_SHIFT 0
-#define SPI_MSG_TYPE_SHIFT 14
+#define SPI_6338_MSG_TYPE_SHIFT 6
+#define SPI_6348_MSG_TYPE_SHIFT 6
+#define SPI_6358_MSG_TYPE_SHIFT 14
+#define SPI_6368_MSG_TYPE_SHIFT 14
/* Command */
#define SPI_CMD_NOOP 0x00
diff --git a/arch/mips/include/asm/mach-cavium-octeon/irq.h b/arch/mips/include/asm/mach-cavium-octeon/irq.h
index 4189920..c22a307 100644
--- a/arch/mips/include/asm/mach-cavium-octeon/irq.h
+++ b/arch/mips/include/asm/mach-cavium-octeon/irq.h
@@ -21,14 +21,10 @@
OCTEON_IRQ_TIMER,
/* sources in CIU_INTX_EN0 */
OCTEON_IRQ_WORKQ0,
- OCTEON_IRQ_GPIO0 = OCTEON_IRQ_WORKQ0 + 16,
- OCTEON_IRQ_WDOG0 = OCTEON_IRQ_GPIO0 + 16,
+ OCTEON_IRQ_WDOG0 = OCTEON_IRQ_WORKQ0 + 16,
OCTEON_IRQ_WDOG15 = OCTEON_IRQ_WDOG0 + 15,
OCTEON_IRQ_MBOX0 = OCTEON_IRQ_WDOG0 + 16,
OCTEON_IRQ_MBOX1,
- OCTEON_IRQ_UART0,
- OCTEON_IRQ_UART1,
- OCTEON_IRQ_UART2,
OCTEON_IRQ_PCI_INT0,
OCTEON_IRQ_PCI_INT1,
OCTEON_IRQ_PCI_INT2,
@@ -38,8 +34,6 @@
OCTEON_IRQ_PCI_MSI2,
OCTEON_IRQ_PCI_MSI3,
- OCTEON_IRQ_TWSI,
- OCTEON_IRQ_TWSI2,
OCTEON_IRQ_RML,
OCTEON_IRQ_TIMER0,
OCTEON_IRQ_TIMER1,
@@ -47,8 +41,6 @@
OCTEON_IRQ_TIMER3,
OCTEON_IRQ_USB0,
OCTEON_IRQ_USB1,
- OCTEON_IRQ_MII0,
- OCTEON_IRQ_MII1,
OCTEON_IRQ_BOOTDMA,
#ifndef CONFIG_PCI_MSI
OCTEON_IRQ_LAST = 127
diff --git a/arch/mips/include/asm/module.h b/arch/mips/include/asm/module.h
index 7531ecd..dca8bce 100644
--- a/arch/mips/include/asm/module.h
+++ b/arch/mips/include/asm/module.h
@@ -10,6 +10,7 @@
struct list_head dbe_list;
const struct exception_table_entry *dbe_start;
const struct exception_table_entry *dbe_end;
+ struct mips_hi16 *r_mips_hi16_list;
};
typedef uint8_t Elf64_Byte; /* Type for a 8-bit quantity. */
diff --git a/arch/mips/include/asm/r4k-timer.h b/arch/mips/include/asm/r4k-timer.h
index a37d12b..afe9e0e 100644
--- a/arch/mips/include/asm/r4k-timer.h
+++ b/arch/mips/include/asm/r4k-timer.h
@@ -12,16 +12,16 @@
#ifdef CONFIG_SYNC_R4K
-extern void synchronise_count_master(void);
-extern void synchronise_count_slave(void);
+extern void synchronise_count_master(int cpu);
+extern void synchronise_count_slave(int cpu);
#else
-static inline void synchronise_count_master(void)
+static inline void synchronise_count_master(int cpu)
{
}
-static inline void synchronise_count_slave(void)
+static inline void synchronise_count_slave(int cpu)
{
}
diff --git a/arch/mips/kernel/module.c b/arch/mips/kernel/module.c
index a5066b1..4f8c3cb 100644
--- a/arch/mips/kernel/module.c
+++ b/arch/mips/kernel/module.c
@@ -39,8 +39,6 @@
Elf_Addr value;
};
-static struct mips_hi16 *mips_hi16_list;
-
static LIST_HEAD(dbe_list);
static DEFINE_SPINLOCK(dbe_lock);
@@ -128,8 +126,8 @@
n->addr = (Elf_Addr *)location;
n->value = v;
- n->next = mips_hi16_list;
- mips_hi16_list = n;
+ n->next = me->arch.r_mips_hi16_list;
+ me->arch.r_mips_hi16_list = n;
return 0;
}
@@ -142,18 +140,28 @@
return 0;
}
+static void free_relocation_chain(struct mips_hi16 *l)
+{
+ struct mips_hi16 *next;
+
+ while (l) {
+ next = l->next;
+ kfree(l);
+ l = next;
+ }
+}
+
static int apply_r_mips_lo16_rel(struct module *me, u32 *location, Elf_Addr v)
{
unsigned long insnlo = *location;
+ struct mips_hi16 *l;
Elf_Addr val, vallo;
/* Sign extend the addend we extract from the lo insn. */
vallo = ((insnlo & 0xffff) ^ 0x8000) - 0x8000;
- if (mips_hi16_list != NULL) {
- struct mips_hi16 *l;
-
- l = mips_hi16_list;
+ if (me->arch.r_mips_hi16_list != NULL) {
+ l = me->arch.r_mips_hi16_list;
while (l != NULL) {
struct mips_hi16 *next;
unsigned long insn;
@@ -188,7 +196,7 @@
l = next;
}
- mips_hi16_list = NULL;
+ me->arch.r_mips_hi16_list = NULL;
}
/*
@@ -201,6 +209,9 @@
return 0;
out_danger:
+ free_relocation_chain(l);
+ me->arch.r_mips_hi16_list = NULL;
+
pr_err("module %s: dangerous R_MIPS_LO16 REL relocation\n", me->name);
return -ENOEXEC;
@@ -273,6 +284,7 @@
pr_debug("Applying relocate section %u to %u\n", relsec,
sechdrs[relsec].sh_info);
+ me->arch.r_mips_hi16_list = NULL;
for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) {
/* This is where to make the change */
location = (void *)sechdrs[sechdrs[relsec].sh_info].sh_addr
@@ -296,6 +308,19 @@
return res;
}
+ /*
+ * Normally the hi16 list should be deallocated at this point. A
+ * malformed binary however could contain a series of R_MIPS_HI16
+ * relocations not followed by a R_MIPS_LO16 relocation. In that
+ * case, free up the list and return an error.
+ */
+ if (me->arch.r_mips_hi16_list) {
+ free_relocation_chain(me->arch.r_mips_hi16_list);
+ me->arch.r_mips_hi16_list = NULL;
+
+ return -ENOEXEC;
+ }
+
return 0;
}
diff --git a/arch/mips/kernel/smp-cmp.c b/arch/mips/kernel/smp-cmp.c
index e7e03ec..afc379c 100644
--- a/arch/mips/kernel/smp-cmp.c
+++ b/arch/mips/kernel/smp-cmp.c
@@ -102,7 +102,7 @@
c->vpe_id = (read_c0_tcbind() >> TCBIND_CURVPE_SHIFT) & TCBIND_CURVPE;
#endif
#ifdef CONFIG_MIPS_MT_SMTC
- c->tc_id = (read_c0_tcbind() >> TCBIND_CURTC_SHIFT) & TCBIND_CURTC;
+ c->tc_id = (read_c0_tcbind() & TCBIND_CURTC) >> TCBIND_CURTC_SHIFT;
#endif
}
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 31637d8..9005bf9 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -130,7 +130,7 @@
cpu_set(cpu, cpu_callin_map);
- synchronise_count_slave();
+ synchronise_count_slave(cpu);
/*
* irq will be enabled in ->smp_finish(), enabling it too early
@@ -173,7 +173,6 @@
void __init smp_cpus_done(unsigned int max_cpus)
{
mp_ops->cpus_done();
- synchronise_count_master();
}
/* called from main before smp_init() */
@@ -206,6 +205,7 @@
while (!cpu_isset(cpu, cpu_callin_map))
udelay(100);
+ synchronise_count_master(cpu);
return 0;
}
diff --git a/arch/mips/kernel/sync-r4k.c b/arch/mips/kernel/sync-r4k.c
index 842d55e..7f1eca3 100644
--- a/arch/mips/kernel/sync-r4k.c
+++ b/arch/mips/kernel/sync-r4k.c
@@ -28,12 +28,11 @@
#define COUNTON 100
#define NR_LOOPS 5
-void __cpuinit synchronise_count_master(void)
+void __cpuinit synchronise_count_master(int cpu)
{
int i;
unsigned long flags;
unsigned int initcount;
- int nslaves;
#ifdef CONFIG_MIPS_MT_SMTC
/*
@@ -43,8 +42,7 @@
return;
#endif
- printk(KERN_INFO "Synchronize counters across %u CPUs: ",
- num_online_cpus());
+ printk(KERN_INFO "Synchronize counters for CPU %u: ", cpu);
local_irq_save(flags);
@@ -52,7 +50,7 @@
* Notify the slaves that it's time to start
*/
atomic_set(&count_reference, read_c0_count());
- atomic_set(&count_start_flag, 1);
+ atomic_set(&count_start_flag, cpu);
smp_wmb();
/* Count will be initialised to current timer for all CPU's */
@@ -69,10 +67,9 @@
* two CPUs.
*/
- nslaves = num_online_cpus()-1;
for (i = 0; i < NR_LOOPS; i++) {
- /* slaves loop on '!= ncpus' */
- while (atomic_read(&count_count_start) != nslaves)
+ /* slaves loop on '!= 2' */
+ while (atomic_read(&count_count_start) != 1)
mb();
atomic_set(&count_count_stop, 0);
smp_wmb();
@@ -89,7 +86,7 @@
/*
* Wait for all slaves to leave the synchronization point:
*/
- while (atomic_read(&count_count_stop) != nslaves)
+ while (atomic_read(&count_count_stop) != 1)
mb();
atomic_set(&count_count_start, 0);
smp_wmb();
@@ -97,6 +94,7 @@
}
/* Arrange for an interrupt in a short while */
write_c0_compare(read_c0_count() + COUNTON);
+ atomic_set(&count_start_flag, 0);
local_irq_restore(flags);
@@ -108,11 +106,10 @@
printk("done.\n");
}
-void __cpuinit synchronise_count_slave(void)
+void __cpuinit synchronise_count_slave(int cpu)
{
int i;
unsigned int initcount;
- int ncpus;
#ifdef CONFIG_MIPS_MT_SMTC
/*
@@ -127,16 +124,15 @@
* so we first wait for the master to say everyone is ready
*/
- while (!atomic_read(&count_start_flag))
+ while (atomic_read(&count_start_flag) != cpu)
mb();
/* Count will be initialised to next expire for all CPU's */
initcount = atomic_read(&count_reference);
- ncpus = num_online_cpus();
for (i = 0; i < NR_LOOPS; i++) {
atomic_inc(&count_count_start);
- while (atomic_read(&count_count_start) != ncpus)
+ while (atomic_read(&count_count_start) != 2)
mb();
/*
@@ -146,7 +142,7 @@
write_c0_count(initcount);
atomic_inc(&count_count_stop);
- while (atomic_read(&count_count_stop) != ncpus)
+ while (atomic_read(&count_count_stop) != 2)
mb();
}
/* Arrange for an interrupt in a short while */
diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c
index 33aadbc..dcfd573 100644
--- a/arch/mips/mm/gup.c
+++ b/arch/mips/mm/gup.c
@@ -152,6 +152,8 @@
do {
VM_BUG_ON(compound_head(page) != head);
pages[*nr] = page;
+ if (PageTail(page))
+ get_huge_page_tail(page);
(*nr)++;
page++;
refs++;
diff --git a/arch/mips/mti-malta/malta-int.c b/arch/mips/mti-malta/malta-int.c
index 7b13a4c..fea823f 100644
--- a/arch/mips/mti-malta/malta-int.c
+++ b/arch/mips/mti-malta/malta-int.c
@@ -273,16 +273,19 @@
unsigned int pending = read_c0_cause() & read_c0_status() & ST0_IM;
int irq;
+ if (unlikely(!pending)) {
+ spurious_interrupt();
+ return;
+ }
+
irq = irq_ffs(pending);
if (irq == MIPSCPU_INT_I8259A)
malta_hw0_irqdispatch();
else if (gic_present && ((1 << irq) & ipi_map[smp_processor_id()]))
malta_ipi_irqdispatch();
- else if (irq >= 0)
- do_IRQ(MIPS_CPU_IRQ_BASE + irq);
else
- spurious_interrupt();
+ do_IRQ(MIPS_CPU_IRQ_BASE + irq);
}
#ifdef CONFIG_MIPS_MT_SMP
diff --git a/arch/mips/mti-malta/malta-pci.c b/arch/mips/mti-malta/malta-pci.c
index 284dea5..2147cb3 100644
--- a/arch/mips/mti-malta/malta-pci.c
+++ b/arch/mips/mti-malta/malta-pci.c
@@ -252,16 +252,3 @@
register_pci_controller(controller);
}
-
-/* Enable PCI 2.1 compatibility in PIIX4 */
-static void __devinit quirk_dlcsetup(struct pci_dev *dev)
-{
- u8 odlc, ndlc;
- (void) pci_read_config_byte(dev, 0x82, &odlc);
- /* Enable passive releases and delayed transaction */
- ndlc = odlc | 7;
- (void) pci_write_config_byte(dev, 0x82, ndlc);
-}
-
-DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82371AB_0,
- quirk_dlcsetup);
diff --git a/arch/mips/mti-malta/malta-platform.c b/arch/mips/mti-malta/malta-platform.c
index 4c35301..80562b8 100644
--- a/arch/mips/mti-malta/malta-platform.c
+++ b/arch/mips/mti-malta/malta-platform.c
@@ -138,11 +138,6 @@
if (err)
return err;
- /*
- * Set RTC to BCD mode to support current alarm code.
- */
- CMOS_WRITE(CMOS_READ(RTC_CONTROL) & ~RTC_DM_BINARY, RTC_CONTROL);
-
return 0;
}
diff --git a/arch/mips/pci/pci-ar724x.c b/arch/mips/pci/pci-ar724x.c
index 414a745..86d77a6 100644
--- a/arch/mips/pci/pci-ar724x.c
+++ b/arch/mips/pci/pci-ar724x.c
@@ -23,9 +23,12 @@
#define AR724X_PCI_MEM_BASE 0x10000000
#define AR724X_PCI_MEM_SIZE 0x08000000
+#define AR724X_PCI_REG_RESET 0x18
#define AR724X_PCI_REG_INT_STATUS 0x4c
#define AR724X_PCI_REG_INT_MASK 0x50
+#define AR724X_PCI_RESET_LINK_UP BIT(0)
+
#define AR724X_PCI_INT_DEV0 BIT(14)
#define AR724X_PCI_IRQ_COUNT 1
@@ -38,6 +41,15 @@
static u32 ar724x_pci_bar0_value;
static bool ar724x_pci_bar0_is_cached;
+static bool ar724x_pci_link_up;
+
+static inline bool ar724x_pci_check_link(void)
+{
+ u32 reset;
+
+ reset = __raw_readl(ar724x_pci_ctrl_base + AR724X_PCI_REG_RESET);
+ return reset & AR724X_PCI_RESET_LINK_UP;
+}
static int ar724x_pci_read(struct pci_bus *bus, unsigned int devfn, int where,
int size, uint32_t *value)
@@ -46,6 +58,9 @@
void __iomem *base;
u32 data;
+ if (!ar724x_pci_link_up)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+
if (devfn)
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -96,6 +111,9 @@
u32 data;
int s;
+ if (!ar724x_pci_link_up)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+
if (devfn)
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -280,6 +298,10 @@
if (ar724x_pci_ctrl_base == NULL)
goto err_unmap_devcfg;
+ ar724x_pci_link_up = ar724x_pci_check_link();
+ if (!ar724x_pci_link_up)
+ pr_warn("ar724x: PCIe link is down\n");
+
ar724x_pci_irq_init(irq);
register_pci_controller(&ar724x_pci_controller);
diff --git a/arch/mips/pci/pci-octeon.c b/arch/mips/pci/pci-octeon.c
index 52a1ba7..c5dfb2c 100644
--- a/arch/mips/pci/pci-octeon.c
+++ b/arch/mips/pci/pci-octeon.c
@@ -117,16 +117,11 @@
}
/* Enable the PCIe normal error reporting */
- pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
- if (pos) {
- /* Update Device Control */
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &config);
- config |= PCI_EXP_DEVCTL_CERE; /* Correctable Error Reporting */
- config |= PCI_EXP_DEVCTL_NFERE; /* Non-Fatal Error Reporting */
- config |= PCI_EXP_DEVCTL_FERE; /* Fatal Error Reporting */
- config |= PCI_EXP_DEVCTL_URRE; /* Unsupported Request */
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, config);
- }
+ config = PCI_EXP_DEVCTL_CERE; /* Correctable Error Reporting */
+ config |= PCI_EXP_DEVCTL_NFERE; /* Non-Fatal Error Reporting */
+ config |= PCI_EXP_DEVCTL_FERE; /* Fatal Error Reporting */
+ config |= PCI_EXP_DEVCTL_URRE; /* Unsupported Request */
+ pcie_capability_set_word(dev, PCI_EXP_DEVCTL, config);
/* Find the Advanced Error Reporting capability */
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);
diff --git a/arch/mips/pci/pci.c b/arch/mips/pci/pci.c
index 6903568..04e35bc 100644
--- a/arch/mips/pci/pci.c
+++ b/arch/mips/pci/pci.c
@@ -313,12 +313,6 @@
}
}
-void __init
-pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
#ifdef CONFIG_HOTPLUG
EXPORT_SYMBOL(PCIBIOS_MIN_IO);
EXPORT_SYMBOL(PCIBIOS_MIN_MEM);
diff --git a/arch/mn10300/kernel/process.c b/arch/mn10300/kernel/process.c
index 7dab0cd..e9cceba 100644
--- a/arch/mn10300/kernel/process.c
+++ b/arch/mn10300/kernel/process.c
@@ -25,6 +25,7 @@
#include <linux/err.h>
#include <linux/fs.h>
#include <linux/slab.h>
+#include <linux/rcupdate.h>
#include <asm/uaccess.h>
#include <asm/pgtable.h>
#include <asm/io.h>
@@ -107,6 +108,7 @@
{
/* endless idle loop with no priority at all */
for (;;) {
+ rcu_idle_enter();
while (!need_resched()) {
void (*idle)(void);
@@ -121,6 +123,7 @@
}
idle();
}
+ rcu_idle_exit();
schedule_preempt_disabled();
}
diff --git a/arch/parisc/include/asm/atomic.h b/arch/parisc/include/asm/atomic.h
index 6c6defc..af9cf30 100644
--- a/arch/parisc/include/asm/atomic.h
+++ b/arch/parisc/include/asm/atomic.h
@@ -141,7 +141,7 @@
#define atomic_sub_and_test(i,v) (atomic_sub_return((i),(v)) == 0)
-#define ATOMIC_INIT(i) ((atomic_t) { (i) })
+#define ATOMIC_INIT(i) { (i) }
#define smp_mb__before_atomic_dec() smp_mb()
#define smp_mb__after_atomic_dec() smp_mb()
@@ -150,7 +150,7 @@
#ifdef CONFIG_64BIT
-#define ATOMIC64_INIT(i) ((atomic64_t) { (i) })
+#define ATOMIC64_INIT(i) { (i) }
static __inline__ s64
__atomic64_add_return(s64 i, atomic64_t *v)
diff --git a/arch/parisc/kernel/process.c b/arch/parisc/kernel/process.c
index d4b94b3..8c6b6b6 100644
--- a/arch/parisc/kernel/process.c
+++ b/arch/parisc/kernel/process.c
@@ -48,6 +48,7 @@
#include <linux/unistd.h>
#include <linux/kallsyms.h>
#include <linux/uaccess.h>
+#include <linux/rcupdate.h>
#include <asm/io.h>
#include <asm/asm-offsets.h>
@@ -69,8 +70,10 @@
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched())
barrier();
+ rcu_idle_exit();
schedule_preempt_disabled();
check_pgt_cache();
}
@@ -309,7 +312,7 @@
cregs->ksp = (unsigned long)stack
+ (pregs->gr[21] & (THREAD_SIZE - 1));
cregs->gr[30] = usp;
- if (p->personality == PER_HPUX) {
+ if (personality(p->personality) == PER_HPUX) {
#ifdef CONFIG_HPUX
cregs->kpc = (unsigned long) &hpux_child_return;
#else
diff --git a/arch/parisc/kernel/sys_parisc.c b/arch/parisc/kernel/sys_parisc.c
index c9b9322..7426e40 100644
--- a/arch/parisc/kernel/sys_parisc.c
+++ b/arch/parisc/kernel/sys_parisc.c
@@ -225,12 +225,12 @@
long err;
if (personality(current->personality) == PER_LINUX32
- && personality == PER_LINUX)
- personality = PER_LINUX32;
+ && personality(personality) == PER_LINUX)
+ personality = (personality & ~PER_MASK) | PER_LINUX32;
err = sys_personality(personality);
- if (err == PER_LINUX32)
- err = PER_LINUX;
+ if (personality(err) == PER_LINUX32)
+ err = (err & ~PER_MASK) | PER_LINUX;
return err;
}
diff --git a/arch/powerpc/boot/.gitignore b/arch/powerpc/boot/.gitignore
index 1c1aadc..c32ae5c 100644
--- a/arch/powerpc/boot/.gitignore
+++ b/arch/powerpc/boot/.gitignore
@@ -1,10 +1,6 @@
addnote
empty.c
hack-coff
-infblock.c
-infblock.h
-infcodes.c
-infcodes.h
inffast.c
inffast.h
inffixed.h
diff --git a/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi b/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi
index 8d35d2c..4f9c9f6 100644
--- a/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi
+++ b/arch/powerpc/boot/dts/fsl/p4080si-post.dtsi
@@ -345,6 +345,13 @@
/include/ "qoriq-duart-1.dtsi"
/include/ "qoriq-gpio-0.dtsi"
/include/ "qoriq-usb2-mph-0.dtsi"
+ usb@210000 {
+ compatible = "fsl-usb2-mph-v1.6", "fsl,mpc85xx-usb2-mph", "fsl-usb2-mph";
+ port0;
+ };
/include/ "qoriq-usb2-dr-0.dtsi"
+ usb@211000 {
+ compatible = "fsl-usb2-dr-v1.6", "fsl,mpc85xx-usb2-dr", "fsl-usb2-dr";
+ };
/include/ "qoriq-sec4.0-0.dtsi"
};
diff --git a/arch/powerpc/configs/85xx/p1023rds_defconfig b/arch/powerpc/configs/85xx/p1023rds_defconfig
index f4337ba..26e541c 100644
--- a/arch/powerpc/configs/85xx/p1023rds_defconfig
+++ b/arch/powerpc/configs/85xx/p1023rds_defconfig
@@ -6,28 +6,27 @@
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_AUDIT=y
-CONFIG_SPARSE_IRQ=y
+CONFIG_IRQ_DOMAIN_DEBUG=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=14
CONFIG_BLK_DEV_INITRD=y
-# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_KALLSYMS_ALL=y
-CONFIG_KALLSYMS_EXTRA_PASS=y
CONFIG_EMBEDDED=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PARTITION_ADVANCED=y
+CONFIG_MAC_PARTITION=y
CONFIG_P1023_RDS=y
CONFIG_QUICC_ENGINE=y
CONFIG_QE_GPIO=y
CONFIG_CPM2=y
-CONFIG_GPIO_MPC8XXX=y
CONFIG_HIGHMEM=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
CONFIG_BINFMT_MISC=m
CONFIG_MATH_EMULATION=y
@@ -63,11 +62,11 @@
CONFIG_IPV6=y
CONFIG_IP_SCTP=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
CONFIG_PROC_DEVICETREE=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=131072
-CONFIG_MISC_DEVICES=y
CONFIG_EEPROM_LEGACY=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
@@ -80,15 +79,14 @@
CONFIG_SATA_SIL24=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
+CONFIG_FS_ENET=y
+CONFIG_FSL_PQ_MDIO=y
+CONFIG_E1000E=y
CONFIG_MARVELL_PHY=y
CONFIG_DAVICOM_PHY=y
CONFIG_CICADA_PHY=y
CONFIG_VITESSE_PHY=y
CONFIG_FIXED_PHY=y
-CONFIG_NET_ETHERNET=y
-CONFIG_FS_ENET=y
-CONFIG_E1000E=y
-CONFIG_FSL_PQ_MDIO=y
CONFIG_INPUT_FF_MEMLESS=m
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_KEYBOARD is not set
@@ -98,16 +96,15 @@
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=2
CONFIG_SERIAL_8250_RUNTIME_UARTS=2
-CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y
CONFIG_SERIAL_QE=m
-CONFIG_HW_RANDOM=y
CONFIG_NVRAM=y
CONFIG_I2C=y
CONFIG_I2C_CPM=m
CONFIG_I2C_MPC=y
+CONFIG_GPIO_MPC8XXX=y
# CONFIG_HWMON is not set
CONFIG_VIDEO_OUTPUT_CONTROL=y
CONFIG_SOUND=y
@@ -123,7 +120,6 @@
CONFIG_FSL_DMA=y
# CONFIG_NET_DMA is not set
CONFIG_STAGING=y
-# CONFIG_STAGING_EXCLUDE_BUILD is not set
CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
@@ -150,22 +146,15 @@
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=y
-CONFIG_PARTITION_ADVANCED=y
-CONFIG_MAC_PARTITION=y
CONFIG_CRC_T10DIF=y
CONFIG_FRAME_WARN=8092
CONFIG_DEBUG_FS=y
-CONFIG_DEBUG_KERNEL=y
CONFIG_DETECT_HUNG_TASK=y
# CONFIG_DEBUG_BUGVERBOSE is not set
CONFIG_DEBUG_INFO=y
-# CONFIG_RCU_CPU_STALL_DETECTOR is not set
-CONFIG_SYSCTL_SYSCALL_CHECK=y
-CONFIG_IRQ_DOMAIN_DEBUG=y
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
diff --git a/arch/powerpc/configs/corenet32_smp_defconfig b/arch/powerpc/configs/corenet32_smp_defconfig
index cbb98c1..8b3d57c 100644
--- a/arch/powerpc/configs/corenet32_smp_defconfig
+++ b/arch/powerpc/configs/corenet32_smp_defconfig
@@ -6,8 +6,8 @@
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_AUDIT=y
-CONFIG_SPARSE_IRQ=y
-CONFIG_RCU_TRACE=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=14
@@ -21,23 +21,22 @@
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PARTITION_ADVANCED=y
+CONFIG_MAC_PARTITION=y
CONFIG_P2041_RDB=y
CONFIG_P3041_DS=y
CONFIG_P4080_DS=y
CONFIG_P5020_DS=y
CONFIG_HIGHMEM=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
CONFIG_BINFMT_MISC=m
CONFIG_KEXEC=y
CONFIG_IRQ_ALL_CPUS=y
CONFIG_FORCE_MAX_ZONEORDER=13
-CONFIG_FSL_LBC=y
CONFIG_PCI=y
CONFIG_PCIEPORTBUS=y
-CONFIG_PCI_MSI=y
# CONFIG_PCIEASPM is not set
+CONFIG_PCI_MSI=y
CONFIG_RAPIDIO=y
CONFIG_FSL_RIO=y
CONFIG_NET=y
@@ -70,6 +69,7 @@
CONFIG_IPV6=y
CONFIG_IP_SCTP=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_CHAR=y
@@ -77,17 +77,14 @@
CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_AMDSTD=y
CONFIG_MTD_PHYSMAP_OF=y
-CONFIG_MTD_NAND=y
-CONFIG_MTD_NAND_ECC=y
-CONFIG_MTD_NAND_IDS=y
-CONFIG_MTD_NAND_FSL_IFC=y
-CONFIG_MTD_NAND_FSL_ELBC=y
CONFIG_MTD_M25P80=y
+CONFIG_MTD_NAND=y
+CONFIG_MTD_NAND_FSL_ELBC=y
+CONFIG_MTD_NAND_FSL_IFC=y
CONFIG_PROC_DEVICETREE=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=131072
-CONFIG_MISC_DEVICES=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
CONFIG_BLK_DEV_SR=y
@@ -115,11 +112,9 @@
CONFIG_PPC_EPAPR_HV_BYTECHAN=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
-CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y
-CONFIG_HW_RANDOM=y
CONFIG_NVRAM=y
CONFIG_I2C=y
CONFIG_I2C_CHARDEV=y
@@ -132,7 +127,6 @@
CONFIG_VIDEO_OUTPUT_CONTROL=y
CONFIG_USB_HID=m
CONFIG_USB=y
-CONFIG_USB_DEVICEFS=y
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_FSL=y
@@ -142,8 +136,6 @@
CONFIG_USB_STORAGE=y
CONFIG_MMC=y
CONFIG_MMC_SDHCI=y
-CONFIG_MMC_SDHCI_OF=y
-CONFIG_MMC_SDHCI_OF_ESDHC=y
CONFIG_EDAC=y
CONFIG_EDAC_MM_EDAC=y
CONFIG_EDAC_MPC85XX=y
@@ -170,19 +162,16 @@
CONFIG_JFFS2_FS=y
CONFIG_CRAMFS=y
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=m
-CONFIG_PARTITION_ADVANCED=y
-CONFIG_MAC_PARTITION=y
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_UTF8=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_DEBUG_SHIRQ=y
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEBUG_INFO=y
-CONFIG_SYSCTL_SYSCALL_CHECK=y
+CONFIG_RCU_TRACE=y
CONFIG_CRYPTO_NULL=y
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_MD4=y
diff --git a/arch/powerpc/configs/corenet64_smp_defconfig b/arch/powerpc/configs/corenet64_smp_defconfig
index dd89de8..0516e22 100644
--- a/arch/powerpc/configs/corenet64_smp_defconfig
+++ b/arch/powerpc/configs/corenet64_smp_defconfig
@@ -56,6 +56,7 @@
CONFIG_IPV6=y
CONFIG_IP_SCTP=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_CHAR=y
diff --git a/arch/powerpc/configs/g5_defconfig b/arch/powerpc/configs/g5_defconfig
index 1513006..07b7f2a 100644
--- a/arch/powerpc/configs/g5_defconfig
+++ b/arch/powerpc/configs/g5_defconfig
@@ -1,8 +1,10 @@
+CONFIG_PPC64=y
+CONFIG_ALTIVEC=y
+CONFIG_SMP=y
+CONFIG_NR_CPUS=4
CONFIG_EXPERIMENTAL=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_BLK_DEV_INITRD=y
@@ -13,15 +15,16 @@
CONFIG_MODULE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SRCVERSION_ALL=y
-CONFIG_PARTITION_ADVANCED=y
-CONFIG_MAC_PARTITION=y
-CONFIG_SMP=y
-CONFIG_NR_CPUS=4
-CONFIG_KEXEC=y
-# CONFIG_RELOCATABLE is not set
+# CONFIG_PPC_PSERIES is not set
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
CONFIG_CPU_FREQ_GOV_USERSPACE=y
+CONFIG_CPU_FREQ_PMAC64=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_KEXEC=y
+CONFIG_IRQ_ALL_CPUS=y
+# CONFIG_MIGRATION is not set
CONFIG_PCI_MSI=y
CONFIG_NET=y
CONFIG_PACKET=y
@@ -49,6 +52,7 @@
CONFIG_NF_CONNTRACK_IPV4=m
CONFIG_IP_NF_QUEUE=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_PROC_DEVICETREE=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
@@ -56,6 +60,8 @@
CONFIG_CDROM_PKTCDVD=m
CONFIG_IDE=y
CONFIG_BLK_DEV_IDECD=y
+CONFIG_BLK_DEV_IDE_PMAC=y
+CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
CONFIG_BLK_DEV_SR=y
@@ -79,24 +85,33 @@
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
CONFIG_DM_ZERO=m
-CONFIG_MACINTOSH_DRIVERS=y
+CONFIG_IEEE1394=y
+CONFIG_IEEE1394_OHCI1394=y
+CONFIG_IEEE1394_SBP2=m
+CONFIG_IEEE1394_ETH1394=m
+CONFIG_IEEE1394_RAWIO=y
+CONFIG_IEEE1394_VIDEO1394=m
+CONFIG_IEEE1394_DV1394=m
+CONFIG_ADB_PMU=y
+CONFIG_PMAC_SMU=y
CONFIG_MAC_EMUMOUSEBTN=y
+CONFIG_THERM_PM72=y
+CONFIG_WINDFARM=y
+CONFIG_WINDFARM_PM81=y
+CONFIG_WINDFARM_PM91=y
+CONFIG_WINDFARM_PM112=y
+CONFIG_WINDFARM_PM121=y
CONFIG_NETDEVICES=y
-CONFIG_BONDING=m
CONFIG_DUMMY=m
-CONFIG_MII=y
+CONFIG_BONDING=m
CONFIG_TUN=m
+CONFIG_NET_ETHERNET=y
+CONFIG_MII=y
+CONFIG_SUNGEM=y
CONFIG_ACENIC=m
CONFIG_ACENIC_OMIT_TIGON_I=y
-CONFIG_TIGON3=y
CONFIG_E1000=y
-CONFIG_SUNGEM=y
-CONFIG_PPP=m
-CONFIG_PPP_BSDCOMP=m
-CONFIG_PPP_DEFLATE=m
-CONFIG_PPPOE=m
-CONFIG_PPP_ASYNC=m
-CONFIG_PPP_SYNC_TTY=m
+CONFIG_TIGON3=y
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
@@ -106,24 +121,36 @@
# CONFIG_USB_NET_NET1080 is not set
# CONFIG_USB_NET_CDC_SUBSET is not set
# CONFIG_USB_NET_ZAURUS is not set
+CONFIG_PPP=m
+CONFIG_PPP_ASYNC=m
+CONFIG_PPP_SYNC_TTY=m
+CONFIG_PPP_DEFLATE=m
+CONFIG_PPP_BSDCOMP=m
+CONFIG_PPPOE=m
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_JOYDEV=m
CONFIG_INPUT_EVDEV=y
+# CONFIG_KEYBOARD_ATKBD is not set
# CONFIG_MOUSE_PS2 is not set
+# CONFIG_SERIO_I8042 is not set
# CONFIG_SERIO_SERPORT is not set
-CONFIG_VT_HW_CONSOLE_BINDING=y
# CONFIG_HW_RANDOM is not set
CONFIG_GEN_RTC=y
CONFIG_RAW_DRIVER=y
CONFIG_I2C_CHARDEV=y
# CONFIG_HWMON is not set
-CONFIG_AGP=y
-CONFIG_DRM=y
-CONFIG_DRM_NOUVEAU=y
+CONFIG_AGP=m
+CONFIG_AGP_UNINORTH=m
CONFIG_VIDEO_OUTPUT_CONTROL=m
+CONFIG_FB=y
CONFIG_FIRMWARE_EDID=y
CONFIG_FB_TILEBLITTING=y
+CONFIG_FB_OF=y
+CONFIG_FB_NVIDIA=y
+CONFIG_FB_NVIDIA_I2C=y
CONFIG_FB_RADEON=y
+# CONFIG_VGA_CONSOLE is not set
+CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_LOGO=y
CONFIG_SOUND=m
CONFIG_SND=m
@@ -131,7 +158,15 @@
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
+CONFIG_SND_POWERMAC=m
+CONFIG_SND_AOA=m
+CONFIG_SND_AOA_FABRIC_LAYOUT=m
+CONFIG_SND_AOA_ONYX=m
+CONFIG_SND_AOA_TAS=m
+CONFIG_SND_AOA_TOONIE=m
CONFIG_SND_USB_AUDIO=m
+CONFIG_HID_PID=y
+CONFIG_USB_HIDDEV=y
CONFIG_HID_GYRATION=y
CONFIG_LOGITECH_FF=y
CONFIG_HID_PANTHERLORD=y
@@ -139,12 +174,13 @@
CONFIG_HID_SAMSUNG=y
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
-CONFIG_HID_PID=y
-CONFIG_USB_HIDDEV=y
CONFIG_USB=y
+CONFIG_USB_DEVICEFS=y
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
+# CONFIG_USB_EHCI_HCD_PPC_OF is not set
CONFIG_USB_OHCI_HCD=y
+CONFIG_USB_OHCI_HCD_PPC_OF_BE=y
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=y
CONFIG_USB_STORAGE=y
@@ -208,6 +244,8 @@
CONFIG_REISERFS_FS_SECURITY=y
CONFIG_XFS_FS=m
CONFIG_XFS_POSIX_ACL=y
+CONFIG_INOTIFY=y
+CONFIG_AUTOFS_FS=m
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
@@ -221,12 +259,14 @@
CONFIG_HFSPLUS_FS=m
CONFIG_CRAMFS=y
CONFIG_NFS_FS=y
+CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFSD=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_CIFS=m
+CONFIG_PARTITION_ADVANCED=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_1250=y
CONFIG_NLS_CODEPAGE_1251=y
@@ -234,23 +274,29 @@
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_ISO8859_15=y
CONFIG_NLS_UTF8=y
+CONFIG_CRC_T10DIF=y
+CONFIG_LIBCRC32C=m
CONFIG_MAGIC_SYSRQ=y
-# CONFIG_UNUSED_SYMBOLS is not set
CONFIG_DEBUG_FS=y
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_MUTEXES=y
+# CONFIG_RCU_CPU_STALL_DETECTOR is not set
CONFIG_LATENCYTOP=y
-CONFIG_STRICT_DEVMEM=y
+CONFIG_SYSCTL_SYSCALL_CHECK=y
+CONFIG_BOOTX_TEXT=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_TEST=m
+CONFIG_CRYPTO_ECB=m
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_HMAC=y
+CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_ANUBIS=m
+CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
@@ -260,6 +306,3 @@
CONFIG_CRYPTO_TWOFISH=m
# CONFIG_CRYPTO_ANSI_CPRNG is not set
# CONFIG_CRYPTO_HW is not set
-# CONFIG_VIRTUALIZATION is not set
-CONFIG_CRC_T10DIF=y
-CONFIG_LIBCRC32C=m
diff --git a/arch/powerpc/configs/mpc83xx_defconfig b/arch/powerpc/configs/mpc83xx_defconfig
index 5aac9a8..9352e44 100644
--- a/arch/powerpc/configs/mpc83xx_defconfig
+++ b/arch/powerpc/configs/mpc83xx_defconfig
@@ -2,12 +2,12 @@
CONFIG_SYSVIPC=y
CONFIG_LOG_BUF_SHIFT=14
CONFIG_BLK_DEV_INITRD=y
-# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_EXPERT=y
CONFIG_SLAB=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PARTITION_ADVANCED=y
# CONFIG_PPC_CHRP is not set
# CONFIG_PPC_PMAC is not set
CONFIG_PPC_83xx=y
@@ -25,7 +25,6 @@
CONFIG_QUICC_ENGINE=y
CONFIG_QE_GPIO=y
CONFIG_MATH_EMULATION=y
-CONFIG_SPARSE_IRQ=y
CONFIG_PCI=y
CONFIG_NET=y
CONFIG_PACKET=y
@@ -42,10 +41,9 @@
# CONFIG_INET_LRO is not set
# CONFIG_IPV6 is not set
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
-CONFIG_MTD_PARTITIONS=y
-CONFIG_MTD_OF_PARTS=y
CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
@@ -64,15 +62,14 @@
CONFIG_SATA_FSL=y
CONFIG_SATA_SIL=y
CONFIG_NETDEVICES=y
+CONFIG_MII=y
+CONFIG_UCC_GETH=y
+CONFIG_GIANFAR=y
CONFIG_MARVELL_PHY=y
CONFIG_DAVICOM_PHY=y
CONFIG_VITESSE_PHY=y
CONFIG_ICPLUS_PHY=y
CONFIG_FIXED_PHY=y
-CONFIG_NET_ETHERNET=y
-CONFIG_MII=y
-CONFIG_GIANFAR=y
-CONFIG_UCC_GETH=y
CONFIG_INPUT_FF_MEMLESS=m
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_KEYBOARD is not set
@@ -112,17 +109,12 @@
CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-CONFIG_INOTIFY=y
CONFIG_PROC_KCORE=y
CONFIG_TMPFS=y
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
-CONFIG_PARTITION_ADVANCED=y
CONFIG_CRC_T10DIF=y
-# CONFIG_RCU_CPU_STALL_DETECTOR is not set
-CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_CRYPTO_ECB=m
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_SHA256=y
diff --git a/arch/powerpc/configs/mpc85xx_defconfig b/arch/powerpc/configs/mpc85xx_defconfig
index 03ee911c..8b5bda2 100644
--- a/arch/powerpc/configs/mpc85xx_defconfig
+++ b/arch/powerpc/configs/mpc85xx_defconfig
@@ -5,7 +5,9 @@
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_AUDIT=y
-CONFIG_SPARSE_IRQ=y
+CONFIG_IRQ_DOMAIN_DEBUG=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=14
@@ -17,6 +19,8 @@
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PARTITION_ADVANCED=y
+CONFIG_MAC_PARTITION=y
CONFIG_MPC8540_ADS=y
CONFIG_MPC8560_ADS=y
CONFIG_MPC85xx_CDS=y
@@ -40,8 +44,6 @@
CONFIG_QUICC_ENGINE=y
CONFIG_QE_GPIO=y
CONFIG_HIGHMEM=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
CONFIG_BINFMT_MISC=m
CONFIG_MATH_EMULATION=y
CONFIG_FORCE_MAX_ZONEORDER=12
@@ -74,36 +76,25 @@
CONFIG_IPV6=y
CONFIG_IP_SCTP=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
-CONFIG_MTD_CFI=y
CONFIG_FTL=y
-CONFIG_MTD_GEN_PROBE=y
-CONFIG_MTD_MAP_BANK_WIDTH_1=y
-CONFIG_MTD_MAP_BANK_WIDTH_2=y
-CONFIG_MTD_MAP_BANK_WIDTH_4=y
-CONFIG_MTD_CFI_I1=y
-CONFIG_MTD_CFI_I2=y
+CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_INTELEXT=y
CONFIG_MTD_CFI_AMDSTD=y
-CONFIG_MTD_CFI_UTIL=y
CONFIG_MTD_PHYSMAP_OF=y
-CONFIG_MTD_PARTITIONS=y
-CONFIG_MTD_OF_PARTS=y
+CONFIG_MTD_M25P80=y
CONFIG_MTD_NAND=y
CONFIG_MTD_NAND_FSL_ELBC=y
CONFIG_MTD_NAND_FSL_IFC=y
-CONFIG_MTD_NAND_IDS=y
-CONFIG_MTD_NAND_ECC=y
-CONFIG_MTD_M25P80=y
CONFIG_PROC_DEVICETREE=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_NBD=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=131072
-CONFIG_MISC_DEVICES=y
CONFIG_EEPROM_LEGACY=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
@@ -115,6 +106,7 @@
CONFIG_SATA_AHCI=y
CONFIG_SATA_FSL=y
CONFIG_PATA_ALI=y
+CONFIG_PATA_VIA=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
CONFIG_FS_ENET=y
@@ -134,7 +126,6 @@
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=2
CONFIG_SERIAL_8250_RUNTIME_UARTS=2
-CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y
@@ -183,7 +174,6 @@
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
CONFIG_USB=y
-CONFIG_USB_DEVICEFS=y
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_FSL=y
@@ -229,18 +219,13 @@
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=y
-CONFIG_PARTITION_ADVANCED=y
-CONFIG_MAC_PARTITION=y
CONFIG_CRC_T10DIF=y
CONFIG_DEBUG_FS=y
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEBUG_INFO=y
-CONFIG_SYSCTL_SYSCALL_CHECK=y
-CONFIG_IRQ_DOMAIN_DEBUG=y
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
diff --git a/arch/powerpc/configs/mpc85xx_smp_defconfig b/arch/powerpc/configs/mpc85xx_smp_defconfig
index fdfa84d..b0974e7 100644
--- a/arch/powerpc/configs/mpc85xx_smp_defconfig
+++ b/arch/powerpc/configs/mpc85xx_smp_defconfig
@@ -7,7 +7,9 @@
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_AUDIT=y
-CONFIG_SPARSE_IRQ=y
+CONFIG_IRQ_DOMAIN_DEBUG=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=14
@@ -19,6 +21,8 @@
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
# CONFIG_BLK_DEV_BSG is not set
+CONFIG_PARTITION_ADVANCED=y
+CONFIG_MAC_PARTITION=y
CONFIG_MPC8540_ADS=y
CONFIG_MPC8560_ADS=y
CONFIG_MPC85xx_CDS=y
@@ -42,8 +46,6 @@
CONFIG_QUICC_ENGINE=y
CONFIG_QE_GPIO=y
CONFIG_HIGHMEM=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
CONFIG_BINFMT_MISC=m
CONFIG_MATH_EMULATION=y
CONFIG_IRQ_ALL_CPUS=y
@@ -77,36 +79,25 @@
CONFIG_IPV6=y
CONFIG_IP_SCTP=m
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
+CONFIG_DEVTMPFS=y
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_CHAR=y
CONFIG_MTD_BLOCK=y
-CONFIG_MTD_CFI=y
CONFIG_FTL=y
-CONFIG_MTD_GEN_PROBE=y
-CONFIG_MTD_MAP_BANK_WIDTH_1=y
-CONFIG_MTD_MAP_BANK_WIDTH_2=y
-CONFIG_MTD_MAP_BANK_WIDTH_4=y
-CONFIG_MTD_CFI_I1=y
-CONFIG_MTD_CFI_I2=y
+CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_INTELEXT=y
CONFIG_MTD_CFI_AMDSTD=y
-CONFIG_MTD_CFI_UTIL=y
CONFIG_MTD_PHYSMAP_OF=y
-CONFIG_MTD_PARTITIONS=y
-CONFIG_MTD_OF_PARTS=y
+CONFIG_MTD_M25P80=y
CONFIG_MTD_NAND=y
CONFIG_MTD_NAND_FSL_ELBC=y
CONFIG_MTD_NAND_FSL_IFC=y
-CONFIG_MTD_NAND_IDS=y
-CONFIG_MTD_NAND_ECC=y
-CONFIG_MTD_M25P80=y
CONFIG_PROC_DEVICETREE=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_NBD=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=131072
-CONFIG_MISC_DEVICES=y
CONFIG_EEPROM_LEGACY=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
@@ -137,7 +128,6 @@
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=2
CONFIG_SERIAL_8250_RUNTIME_UARTS=2
-CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_RSA=y
@@ -186,7 +176,6 @@
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
CONFIG_USB=y
-CONFIG_USB_DEVICEFS=y
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_EHCI_FSL=y
@@ -232,18 +221,13 @@
CONFIG_SYSV_FS=m
CONFIG_UFS_FS=m
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=y
-CONFIG_PARTITION_ADVANCED=y
-CONFIG_MAC_PARTITION=y
CONFIG_CRC_T10DIF=y
CONFIG_DEBUG_FS=y
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEBUG_INFO=y
-CONFIG_SYSCTL_SYSCALL_CHECK=y
-CONFIG_IRQ_DOMAIN_DEBUG=y
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
diff --git a/arch/powerpc/include/asm/cputable.h b/arch/powerpc/include/asm/cputable.h
index 50d82c8..b3c083d 100644
--- a/arch/powerpc/include/asm/cputable.h
+++ b/arch/powerpc/include/asm/cputable.h
@@ -553,9 +553,7 @@
& feature);
}
-#ifdef CONFIG_HAVE_HW_BREAKPOINT
#define HBP_NUM 1
-#endif /* CONFIG_HAVE_HW_BREAKPOINT */
#endif /* !__ASSEMBLY__ */
diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h
index 50ea12f..a8bf5c6 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -33,6 +33,7 @@
#include <asm/kvm_asm.h>
#include <asm/processor.h>
#include <asm/page.h>
+#include <asm/cacheflush.h>
#define KVM_MAX_VCPUS NR_CPUS
#define KVM_MAX_VCORES NR_CPUS
diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h
index 0124937..e006f0b 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -219,4 +219,16 @@
void kvmppc_free_lpid(long lpid);
void kvmppc_init_lpid(unsigned long nr_lpids);
+static inline void kvmppc_mmu_flush_icache(pfn_t pfn)
+{
+ /* Clear i-cache for new pages */
+ struct page *page;
+ page = pfn_to_page(pfn);
+ if (!test_bit(PG_arch_1, &page->flags)) {
+ flush_dcache_icache_page(page);
+ set_bit(PG_arch_1, &page->flags);
+ }
+}
+
+
#endif /* __POWERPC_KVM_PPC_H__ */
diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h
index 42ce570..f7706d7 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -214,6 +214,9 @@
/* Called after scan and before resource survey */
void (*pcibios_fixup_phb)(struct pci_controller *hose);
+ /* Called during PCI resource reassignment */
+ resource_size_t (*pcibios_window_alignment)(struct pci_bus *, unsigned long type);
+
/* Called to shutdown machine specific hardware not already controlled
* by other drivers.
*/
diff --git a/arch/powerpc/include/asm/mpic_msgr.h b/arch/powerpc/include/asm/mpic_msgr.h
index 326d33c..d4f471f 100644
--- a/arch/powerpc/include/asm/mpic_msgr.h
+++ b/arch/powerpc/include/asm/mpic_msgr.h
@@ -14,6 +14,7 @@
#include <linux/types.h>
#include <linux/spinlock.h>
#include <asm/smp.h>
+#include <asm/io.h>
struct mpic_msgr {
u32 __iomem *base;
diff --git a/arch/powerpc/include/asm/processor.h b/arch/powerpc/include/asm/processor.h
index 53b6dfa..54b73a2 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -386,6 +386,7 @@
enum idle_boot_override {IDLE_NO_OVERRIDE = 0, IDLE_POWERSAVE_OFF};
extern int powersave_nap; /* set if nap mode can be used in idle loop */
+extern void power7_nap(void);
#ifdef CONFIG_PSERIES_IDLE
extern void update_smt_snooze_delay(int snooze);
diff --git a/arch/powerpc/include/asm/time.h b/arch/powerpc/include/asm/time.h
index 3b4b4a8..c1f2676 100644
--- a/arch/powerpc/include/asm/time.h
+++ b/arch/powerpc/include/asm/time.h
@@ -197,12 +197,6 @@
DECLARE_PER_CPU(struct cpu_usage, cpu_usage_array);
-#if defined(CONFIG_VIRT_CPU_ACCOUNTING)
-#define account_process_vtime(tsk) account_process_tick(tsk, 0)
-#else
-#define account_process_vtime(tsk) do { } while (0)
-#endif
-
extern void secondary_cpu_time_init(void);
DECLARE_PER_CPU(u64, decrementers_next_tb);
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 85b05c4..e899572 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -76,6 +76,7 @@
DEFINE(SIGSEGV, SIGSEGV);
DEFINE(NMI_MASK, NMI_MASK);
DEFINE(THREAD_DSCR, offsetof(struct thread_struct, dscr));
+ DEFINE(THREAD_DSCR_INHERIT, offsetof(struct thread_struct, dscr_inherit));
#else
DEFINE(THREAD_INFO, offsetof(struct task_struct, stack));
#endif /* CONFIG_PPC64 */
diff --git a/arch/powerpc/kernel/dbell.c b/arch/powerpc/kernel/dbell.c
index 5b25c80..a892680 100644
--- a/arch/powerpc/kernel/dbell.c
+++ b/arch/powerpc/kernel/dbell.c
@@ -28,6 +28,8 @@
void doorbell_cause_ipi(int cpu, unsigned long data)
{
+ /* Order previous accesses vs. msgsnd, which is treated as a store */
+ mb();
ppc_msgsnd(PPC_DBELL, 0, data);
}
diff --git a/arch/powerpc/kernel/dma-iommu.c b/arch/powerpc/kernel/dma-iommu.c
index 2d7bb8c..e489752 100644
--- a/arch/powerpc/kernel/dma-iommu.c
+++ b/arch/powerpc/kernel/dma-iommu.c
@@ -83,11 +83,10 @@
return 0;
}
- if ((tbl->it_offset + tbl->it_size) > (mask >> IOMMU_PAGE_SHIFT)) {
- dev_info(dev, "Warning: IOMMU window too big for device mask\n");
- dev_info(dev, "mask: 0x%08llx, table end: 0x%08lx\n",
- mask, (tbl->it_offset + tbl->it_size) <<
- IOMMU_PAGE_SHIFT);
+ if (tbl->it_offset > (mask >> IOMMU_PAGE_SHIFT)) {
+ dev_info(dev, "Warning: IOMMU offset too big for device mask\n");
+ dev_info(dev, "mask: 0x%08llx, table offset: 0x%08lx\n",
+ mask, tbl->it_offset << IOMMU_PAGE_SHIFT);
return 0;
} else
return 1;
diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
index 4b01a25..b40e0b4 100644
--- a/arch/powerpc/kernel/entry_64.S
+++ b/arch/powerpc/kernel/entry_64.S
@@ -370,6 +370,12 @@
li r3,0
b syscall_exit
+ .section ".toc","aw"
+DSCR_DEFAULT:
+ .tc dscr_default[TC],dscr_default
+
+ .section ".text"
+
/*
* This routine switches between two different tasks. The process
* state of one is saved on its kernel stack. Then the state
@@ -509,9 +515,6 @@
mr r1,r8 /* start using new stack pointer */
std r7,PACAKSAVE(r13)
- ld r6,_CCR(r1)
- mtcrf 0xFF,r6
-
#ifdef CONFIG_ALTIVEC
BEGIN_FTR_SECTION
ld r0,THREAD_VRSAVE(r4)
@@ -520,14 +523,22 @@
#endif /* CONFIG_ALTIVEC */
#ifdef CONFIG_PPC64
BEGIN_FTR_SECTION
+ lwz r6,THREAD_DSCR_INHERIT(r4)
+ ld r7,DSCR_DEFAULT@toc(2)
ld r0,THREAD_DSCR(r4)
- cmpd r0,r25
- beq 1f
+ cmpwi r6,0
+ bne 1f
+ ld r0,0(r7)
+1: cmpd r0,r25
+ beq 2f
mtspr SPRN_DSCR,r0
-1:
+2:
END_FTR_SECTION_IFSET(CPU_FTR_DSCR)
#endif
+ ld r6,_CCR(r1)
+ mtcrf 0xFF,r6
+
/* r3-r13 are destroyed -- Cort */
REST_8GPRS(14, r1)
REST_10GPRS(22, r1)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index e894515..39aa97d3 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -186,7 +186,7 @@
KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x800)
MASKABLE_EXCEPTION_PSERIES(0x900, 0x900, decrementer)
- MASKABLE_EXCEPTION_HV(0x980, 0x982, decrementer)
+ STD_EXCEPTION_HV(0x980, 0x982, hdecrementer)
STD_EXCEPTION_PSERIES(0xa00, 0xa00, trap_0a)
KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xa00)
@@ -486,6 +486,7 @@
STD_EXCEPTION_COMMON_ASYNC(0x500, hardware_interrupt, do_IRQ)
STD_EXCEPTION_COMMON_ASYNC(0x900, decrementer, .timer_interrupt)
+ STD_EXCEPTION_COMMON(0x980, hdecrementer, .hdec_interrupt)
STD_EXCEPTION_COMMON(0xa00, trap_0a, .unknown_exception)
STD_EXCEPTION_COMMON(0xb00, trap_0b, .unknown_exception)
STD_EXCEPTION_COMMON(0xd00, single_step, .single_step_exception)
diff --git a/arch/powerpc/kernel/hw_breakpoint.c b/arch/powerpc/kernel/hw_breakpoint.c
index f3a82dd..956a4c4 100644
--- a/arch/powerpc/kernel/hw_breakpoint.c
+++ b/arch/powerpc/kernel/hw_breakpoint.c
@@ -253,7 +253,7 @@
/* Do not emulate user-space instructions, instead single-step them */
if (user_mode(regs)) {
- bp->ctx->task->thread.last_hit_ubp = bp;
+ current->thread.last_hit_ubp = bp;
regs->msr |= MSR_SE;
goto out;
}
diff --git a/arch/powerpc/kernel/idle_power7.S b/arch/powerpc/kernel/idle_power7.S
index 7140d83..e11863f 100644
--- a/arch/powerpc/kernel/idle_power7.S
+++ b/arch/powerpc/kernel/idle_power7.S
@@ -28,7 +28,9 @@
lwz r4,ADDROFF(powersave_nap)(r3)
cmpwi 0,r4,0
beqlr
+ /* fall through */
+_GLOBAL(power7_nap)
/* NAP is a state loss, we create a regs frame on the
* stack, fill it up with the state we care about and
* stick a pointer to it in PACAR1. We really only
diff --git a/arch/powerpc/kernel/kgdb.c b/arch/powerpc/kernel/kgdb.c
index 782bd0a..c470a40 100644
--- a/arch/powerpc/kernel/kgdb.c
+++ b/arch/powerpc/kernel/kgdb.c
@@ -25,6 +25,7 @@
#include <asm/processor.h>
#include <asm/machdep.h>
#include <asm/debug.h>
+#include <linux/slab.h>
/*
* This table contains the mapping between PowerPC hardware trap types, and
@@ -101,6 +102,21 @@
return SIGHUP; /* default for things we don't know about */
}
+/**
+ *
+ * kgdb_skipexception - Bail out of KGDB when we've been triggered.
+ * @exception: Exception vector number
+ * @regs: Current &struct pt_regs.
+ *
+ * On some architectures we need to skip a breakpoint exception when
+ * it occurs after a breakpoint has been removed.
+ *
+ */
+int kgdb_skipexception(int exception, struct pt_regs *regs)
+{
+ return kgdb_isremovedbreak(regs->nip);
+}
+
static int kgdb_call_nmi_hook(struct pt_regs *regs)
{
kgdb_nmicallback(raw_smp_processor_id(), regs);
@@ -138,6 +154,8 @@
static int kgdb_singlestep(struct pt_regs *regs)
{
struct thread_info *thread_info, *exception_thread_info;
+ struct thread_info *backup_current_thread_info = \
+ (struct thread_info *)kmalloc(sizeof(struct thread_info), GFP_KERNEL);
if (user_mode(regs))
return 0;
@@ -155,13 +173,17 @@
thread_info = (struct thread_info *)(regs->gpr[1] & ~(THREAD_SIZE-1));
exception_thread_info = current_thread_info();
- if (thread_info != exception_thread_info)
+ if (thread_info != exception_thread_info) {
+ /* Save the original current_thread_info. */
+ memcpy(backup_current_thread_info, exception_thread_info, sizeof *thread_info);
memcpy(exception_thread_info, thread_info, sizeof *thread_info);
+ }
kgdb_handle_exception(0, SIGTRAP, 0, regs);
if (thread_info != exception_thread_info)
- memcpy(thread_info, exception_thread_info, sizeof *thread_info);
+ /* Restore current_thread_info lastly. */
+ memcpy(exception_thread_info, backup_current_thread_info, sizeof *thread_info);
return 1;
}
@@ -410,7 +432,6 @@
#else
linux_regs->msr |= MSR_SE;
#endif
- kgdb_single_step = 1;
atomic_set(&kgdb_cpu_doing_single_step,
raw_smp_processor_id());
}
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 2aa04f2..43fea54 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -99,6 +99,26 @@
kfree(phb);
}
+/*
+ * The function is used to return the minimal alignment
+ * for memory or I/O windows of the associated P2P bridge.
+ * By default, 4KiB alignment for I/O windows and 1MiB for
+ * memory windows.
+ */
+resource_size_t pcibios_window_alignment(struct pci_bus *bus,
+ unsigned long type)
+{
+ if (ppc_md.pcibios_window_alignment)
+ return ppc_md.pcibios_window_alignment(bus, type);
+
+ /*
+ * PCI core will figure out the default
+ * alignment: 4KiB for I/O and 1MiB for
+ * memory window.
+ */
+ return 1;
+}
+
static resource_size_t pcibios_io_size(const struct pci_controller *hose)
{
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 710f400..e9cb51f 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -514,9 +514,6 @@
local_irq_save(flags);
- account_system_vtime(current);
- account_process_vtime(current);
-
/*
* We can't take a PMU exception inside _switch() since there is a
* window where the kernel stack SLB and the kernel stack are out
@@ -802,16 +799,8 @@
#endif /* CONFIG_PPC_STD_MMU_64 */
#ifdef CONFIG_PPC64
if (cpu_has_feature(CPU_FTR_DSCR)) {
- if (current->thread.dscr_inherit) {
- p->thread.dscr_inherit = 1;
- p->thread.dscr = current->thread.dscr;
- } else if (0 != dscr_default) {
- p->thread.dscr_inherit = 1;
- p->thread.dscr = dscr_default;
- } else {
- p->thread.dscr_inherit = 0;
- p->thread.dscr = 0;
- }
+ p->thread.dscr_inherit = current->thread.dscr_inherit;
+ p->thread.dscr = current->thread.dscr;
}
#endif
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index 0321007..8d4214a 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -198,8 +198,15 @@
struct cpu_messages *info = &per_cpu(ipi_message, cpu);
char *message = (char *)&info->messages;
+ /*
+ * Order previous accesses before accesses in the IPI handler.
+ */
+ smp_mb();
message[msg] = 1;
- mb();
+ /*
+ * cause_ipi functions are required to include a full barrier
+ * before doing whatever causes the IPI.
+ */
smp_ops->cause_ipi(cpu, info->data);
}
@@ -211,7 +218,7 @@
mb(); /* order any irq clear */
do {
- all = xchg_local(&info->messages, 0);
+ all = xchg(&info->messages, 0);
#ifdef __BIG_ENDIAN
if (all & (1 << (24 - 8 * PPC_MSG_CALL_FUNCTION)))
diff --git a/arch/powerpc/kernel/syscalls.c b/arch/powerpc/kernel/syscalls.c
index f2496f2..4e3cc47 100644
--- a/arch/powerpc/kernel/syscalls.c
+++ b/arch/powerpc/kernel/syscalls.c
@@ -107,11 +107,11 @@
long ret;
if (personality(current->personality) == PER_LINUX32
- && personality == PER_LINUX)
- personality = PER_LINUX32;
+ && personality(personality) == PER_LINUX)
+ personality = (personality & ~PER_MASK) | PER_LINUX32;
ret = sys_personality(personality);
- if (ret == PER_LINUX32)
- ret = PER_LINUX;
+ if (personality(ret) == PER_LINUX32)
+ ret = (ret & ~PER_MASK) | PER_LINUX;
return ret;
}
#endif
diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
index 3529446..8302af6 100644
--- a/arch/powerpc/kernel/sysfs.c
+++ b/arch/powerpc/kernel/sysfs.c
@@ -194,6 +194,14 @@
return sprintf(buf, "%lx\n", dscr_default);
}
+static void update_dscr(void *dummy)
+{
+ if (!current->thread.dscr_inherit) {
+ current->thread.dscr = dscr_default;
+ mtspr(SPRN_DSCR, dscr_default);
+ }
+}
+
static ssize_t __used store_dscr_default(struct device *dev,
struct device_attribute *attr, const char *buf,
size_t count)
@@ -206,6 +214,8 @@
return -EINVAL;
dscr_default = val;
+ on_each_cpu(update_dscr, NULL, 1);
+
return count;
}
diff --git a/arch/powerpc/kernel/time.c b/arch/powerpc/kernel/time.c
index be171ee..eaa9d0e 100644
--- a/arch/powerpc/kernel/time.c
+++ b/arch/powerpc/kernel/time.c
@@ -291,13 +291,12 @@
* Account time for a transition between system, hard irq
* or soft irq state.
*/
-void account_system_vtime(struct task_struct *tsk)
+static u64 vtime_delta(struct task_struct *tsk,
+ u64 *sys_scaled, u64 *stolen)
{
- u64 now, nowscaled, delta, deltascaled;
- unsigned long flags;
- u64 stolen, udelta, sys_scaled, user_scaled;
+ u64 now, nowscaled, deltascaled;
+ u64 udelta, delta, user_scaled;
- local_irq_save(flags);
now = mftb();
nowscaled = read_spurr(now);
get_paca()->system_time += now - get_paca()->starttime;
@@ -305,7 +304,7 @@
deltascaled = nowscaled - get_paca()->startspurr;
get_paca()->startspurr = nowscaled;
- stolen = calculate_stolen_time(now);
+ *stolen = calculate_stolen_time(now);
delta = get_paca()->system_time;
get_paca()->system_time = 0;
@@ -322,35 +321,45 @@
* the user ticks get saved up in paca->user_time_scaled to be
* used by account_process_tick.
*/
- sys_scaled = delta;
+ *sys_scaled = delta;
user_scaled = udelta;
if (deltascaled != delta + udelta) {
if (udelta) {
- sys_scaled = deltascaled * delta / (delta + udelta);
- user_scaled = deltascaled - sys_scaled;
+ *sys_scaled = deltascaled * delta / (delta + udelta);
+ user_scaled = deltascaled - *sys_scaled;
} else {
- sys_scaled = deltascaled;
+ *sys_scaled = deltascaled;
}
}
get_paca()->user_time_scaled += user_scaled;
- if (in_interrupt() || idle_task(smp_processor_id()) != tsk) {
- account_system_time(tsk, 0, delta, sys_scaled);
- if (stolen)
- account_steal_time(stolen);
- } else {
- account_idle_time(delta + stolen);
- }
- local_irq_restore(flags);
+ return delta;
}
-EXPORT_SYMBOL_GPL(account_system_vtime);
+
+void vtime_account_system(struct task_struct *tsk)
+{
+ u64 delta, sys_scaled, stolen;
+
+ delta = vtime_delta(tsk, &sys_scaled, &stolen);
+ account_system_time(tsk, 0, delta, sys_scaled);
+ if (stolen)
+ account_steal_time(stolen);
+}
+
+void vtime_account_idle(struct task_struct *tsk)
+{
+ u64 delta, sys_scaled, stolen;
+
+ delta = vtime_delta(tsk, &sys_scaled, &stolen);
+ account_idle_time(delta + stolen);
+}
/*
* Transfer the user and system times accumulated in the paca
* by the exception entry and exit code to the generic process
* user and system time records.
* Must be called with interrupts disabled.
- * Assumes that account_system_vtime() has been called recently
+ * Assumes that vtime_account() has been called recently
* (i.e. since the last entry from usermode) so that
* get_paca()->user_time_scaled is up to date.
*/
@@ -366,6 +375,12 @@
account_user_time(tsk, utime, utimescaled);
}
+void vtime_task_switch(struct task_struct *prev)
+{
+ vtime_account(prev);
+ account_process_tick(prev, 0);
+}
+
#else /* ! CONFIG_VIRT_CPU_ACCOUNTING */
#define calc_cputime_factors()
#endif
@@ -535,6 +550,15 @@
trace_timer_interrupt_exit(regs);
}
+/*
+ * Hypervisor decrementer interrupts shouldn't occur but are sometimes
+ * left pending on exit from a KVM guest. We don't need to do anything
+ * to clear them, as they are edge-triggered.
+ */
+void hdec_interrupt(struct pt_regs *regs)
+{
+}
+
#ifdef CONFIG_SUSPEND
static void generic_suspend_disable_irqs(void)
{
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index 1589723..ae0843f 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -972,8 +972,9 @@
cpu_has_feature(CPU_FTR_DSCR)) {
PPC_WARN_EMULATED(mtdscr, regs);
rd = (instword >> 21) & 0x1f;
- mtspr(SPRN_DSCR, regs->gpr[rd]);
+ current->thread.dscr = regs->gpr[rd];
current->thread.dscr_inherit = 1;
+ mtspr(SPRN_DSCR, current->thread.dscr);
return 0;
}
#endif
diff --git a/arch/powerpc/kvm/book3s_32_mmu_host.c b/arch/powerpc/kvm/book3s_32_mmu_host.c
index f922c29..837f13e 100644
--- a/arch/powerpc/kvm/book3s_32_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_32_mmu_host.c
@@ -211,6 +211,9 @@
pteg1 |= PP_RWRX;
}
+ if (orig_pte->may_execute)
+ kvmppc_mmu_flush_icache(hpaddr >> PAGE_SHIFT);
+
local_irq_disable();
if (pteg[rr]) {
diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c
index 10fc8ec..0688b6b 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_host.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_host.c
@@ -126,6 +126,8 @@
if (!orig_pte->may_execute)
rflags |= HPTE_R_N;
+ else
+ kvmppc_mmu_flush_icache(hpaddr >> PAGE_SHIFT);
hash = hpt_hash(va, PTE_SIZE, MMU_SEGSIZE_256M);
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 5a84c8d..44b72fe 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -1421,13 +1421,13 @@
sync /* order setting ceded vs. testing prodded */
lbz r5,VCPU_PRODDED(r3)
cmpwi r5,0
- bne 1f
+ bne kvm_cede_prodded
li r0,0 /* set trap to 0 to say hcall is handled */
stw r0,VCPU_TRAP(r3)
li r0,H_SUCCESS
std r0,VCPU_GPR(R3)(r3)
BEGIN_FTR_SECTION
- b 2f /* just send it up to host on 970 */
+ b kvm_cede_exit /* just send it up to host on 970 */
END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_206)
/*
@@ -1446,7 +1446,7 @@
or r4,r4,r0
PPC_POPCNTW(R7,R4)
cmpw r7,r8
- bge 2f
+ bge kvm_cede_exit
stwcx. r4,0,r6
bne 31b
li r0,1
@@ -1555,7 +1555,8 @@
b hcall_real_fallback
/* cede when already previously prodded case */
-1: li r0,0
+kvm_cede_prodded:
+ li r0,0
stb r0,VCPU_PRODDED(r3)
sync /* order testing prodded vs. clearing ceded */
stb r0,VCPU_CEDED(r3)
@@ -1563,7 +1564,8 @@
blr
/* we've ceded but we want to give control to the host */
-2: li r3,H_TOO_HARD
+kvm_cede_exit:
+ li r3,H_TOO_HARD
blr
secondary_too_late:
diff --git a/arch/powerpc/kvm/e500_tlb.c b/arch/powerpc/kvm/e500_tlb.c
index c510fc9..a2b6671 100644
--- a/arch/powerpc/kvm/e500_tlb.c
+++ b/arch/powerpc/kvm/e500_tlb.c
@@ -322,11 +322,11 @@
static void clear_tlb1_bitmap(struct kvmppc_vcpu_e500 *vcpu_e500)
{
if (vcpu_e500->g2h_tlb1_map)
- memset(vcpu_e500->g2h_tlb1_map,
- sizeof(u64) * vcpu_e500->gtlb_params[1].entries, 0);
+ memset(vcpu_e500->g2h_tlb1_map, 0,
+ sizeof(u64) * vcpu_e500->gtlb_params[1].entries);
if (vcpu_e500->h2g_tlb1_rmap)
- memset(vcpu_e500->h2g_tlb1_rmap,
- sizeof(unsigned int) * host_tlb_params[1].entries, 0);
+ memset(vcpu_e500->h2g_tlb1_rmap, 0,
+ sizeof(unsigned int) * host_tlb_params[1].entries);
}
static void clear_tlb_privs(struct kvmppc_vcpu_e500 *vcpu_e500)
@@ -539,6 +539,9 @@
kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize,
ref, gvaddr, stlbe);
+
+ /* Clear i-cache for new pages */
+ kvmppc_mmu_flush_icache(pfn);
}
/* XXX only map the one-one case, for now use TLB0 */
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index dd223b3..17e5b23 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -20,7 +20,7 @@
{
int err;
- err = __put_user(instr, addr);
+ __put_user_size(instr, addr, 4, err);
if (err)
return err;
asm ("dcbst 0, %0; sync; icbi 0,%0; sync; isync" : : "r" (addr));
diff --git a/arch/powerpc/lib/copyuser_power7.S b/arch/powerpc/lib/copyuser_power7.S
index f9ede7c..0d24ff1 100644
--- a/arch/powerpc/lib/copyuser_power7.S
+++ b/arch/powerpc/lib/copyuser_power7.S
@@ -288,7 +288,7 @@
std r0,16(r1)
stdu r1,-STACKFRAMESIZE(r1)
bl .enter_vmx_usercopy
- cmpwi r3,0
+ cmpwi cr1,r3,0
ld r0,STACKFRAMESIZE+16(r1)
ld r3,STACKFRAMESIZE+48(r1)
ld r4,STACKFRAMESIZE+56(r1)
@@ -326,38 +326,7 @@
dcbt r0,r8,0b01010 /* GO */
.machine pop
- /*
- * We prefetch both the source and destination using enhanced touch
- * instructions. We use a stream ID of 0 for the load side and
- * 1 for the store side.
- */
- clrrdi r6,r4,7
- clrrdi r9,r3,7
- ori r9,r9,1 /* stream=1 */
-
- srdi r7,r5,7 /* length in cachelines, capped at 0x3FF */
- cmpldi cr1,r7,0x3FF
- ble cr1,1f
- li r7,0x3FF
-1: lis r0,0x0E00 /* depth=7 */
- sldi r7,r7,7
- or r7,r7,r0
- ori r10,r7,1 /* stream=1 */
-
- lis r8,0x8000 /* GO=1 */
- clrldi r8,r8,32
-
-.machine push
-.machine "power4"
- dcbt r0,r6,0b01000
- dcbt r0,r7,0b01010
- dcbtst r0,r9,0b01000
- dcbtst r0,r10,0b01010
- eieio
- dcbt r0,r8,0b01010 /* GO */
-.machine pop
-
- beq .Lunwind_stack_nonvmx_copy
+ beq cr1,.Lunwind_stack_nonvmx_copy
/*
* If source and destination are not relatively aligned we use a
diff --git a/arch/powerpc/lib/memcpy_power7.S b/arch/powerpc/lib/memcpy_power7.S
index 0efdc51..7ba6c96 100644
--- a/arch/powerpc/lib/memcpy_power7.S
+++ b/arch/powerpc/lib/memcpy_power7.S
@@ -222,7 +222,7 @@
std r0,16(r1)
stdu r1,-STACKFRAMESIZE(r1)
bl .enter_vmx_copy
- cmpwi r3,0
+ cmpwi cr1,r3,0
ld r0,STACKFRAMESIZE+16(r1)
ld r3,STACKFRAMESIZE+48(r1)
ld r4,STACKFRAMESIZE+56(r1)
@@ -260,7 +260,7 @@
dcbt r0,r8,0b01010 /* GO */
.machine pop
- beq .Lunwind_stack_nonvmx_copy
+ beq cr1,.Lunwind_stack_nonvmx_copy
/*
* If source and destination are not relatively aligned we use a
diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
index baaafde..fbdad0e 100644
--- a/arch/powerpc/mm/mem.c
+++ b/arch/powerpc/mm/mem.c
@@ -469,6 +469,7 @@
__flush_dcache_icache_phys(page_to_pfn(page) << PAGE_SHIFT);
#endif
}
+EXPORT_SYMBOL(flush_dcache_icache_page);
void clear_user_page(void *page, unsigned long vaddr, struct page *pg)
{
diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index 39b1597..59213cf 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c
@@ -1436,11 +1436,11 @@
/*
* Update the node maps and sysfs entries for each cpu whose home node
- * has changed.
+ * has changed. Returns 1 when the topology has changed, and 0 otherwise.
*/
int arch_update_cpu_topology(void)
{
- int cpu, nid, old_nid;
+ int cpu, nid, old_nid, changed = 0;
unsigned int associativity[VPHN_ASSOC_BUFSIZE] = {0};
struct device *dev;
@@ -1466,9 +1466,10 @@
dev = get_cpu_device(cpu);
if (dev)
kobject_uevent(&dev->kobj, KOBJ_CHANGE);
+ changed = 1;
}
- return 1;
+ return changed;
}
static void topology_work_fn(struct work_struct *work)
diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c
index 77b49dd..7cd2dbd 100644
--- a/arch/powerpc/perf/core-book3s.c
+++ b/arch/powerpc/perf/core-book3s.c
@@ -1431,7 +1431,7 @@
if (!event->hw.idx || is_limited_pmc(event->hw.idx))
continue;
val = read_pmc(event->hw.idx);
- if ((int)val < 0) {
+ if (pmc_overflow(val)) {
/* event has overflowed */
found = 1;
record_and_restart(event, val, regs);
diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype
index 30fd01d..72afd28 100644
--- a/arch/powerpc/platforms/Kconfig.cputype
+++ b/arch/powerpc/platforms/Kconfig.cputype
@@ -1,6 +1,7 @@
config PPC64
bool "64-bit kernel"
default n
+ select HAVE_VIRT_CPU_ACCOUNTING
help
This option selects whether a 32-bit or a 64-bit kernel
will be built.
@@ -337,21 +338,6 @@
default y if (!PPC_FSL_BOOK3E && PPC64 && HUGETLB_PAGE) || (PPC_STD_MMU_64 && PPC_64K_PAGES)
default n
-config VIRT_CPU_ACCOUNTING
- bool "Deterministic task and CPU time accounting"
- depends on PPC64
- default y
- help
- Select this option to enable more accurate task and CPU time
- accounting. This is done by reading a CPU counter on each
- kernel entry and exit and on transitions within the kernel
- between system, softirq and hardirq state, so there is a
- small performance impact. This also enables accounting of
- stolen time on logically-partitioned systems running on
- IBM POWER5-based machines.
-
- If in doubt, say Y here.
-
config PPC_HAVE_PMU_SUPPORT
bool
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 9cda6a1..0e7eccc 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -855,7 +855,7 @@
if (pe == NULL)
continue;
/* Leaving the PCIe domain ... single PE# */
- if (dev->pcie_type == PCI_EXP_TYPE_PCI_BRIDGE)
+ if (pci_pcie_type(dev) == PCI_EXP_TYPE_PCI_BRIDGE)
pnv_ioda_setup_bus_PE(dev, pe);
else if (dev->subordinate)
pnv_ioda_setup_PEs(dev->subordinate);
@@ -1139,6 +1139,44 @@
}
}
+/*
+ * Returns the alignment for I/O or memory windows for P2P
+ * bridges. That actually depends on how PEs are segmented.
+ * For now, we return I/O or M32 segment size for PE sensitive
+ * P2P bridges. Otherwise, the default values (4KiB for I/O,
+ * 1MiB for memory) will be returned.
+ *
+ * The current PCI bus might be put into one PE, which was
+ * create against the parent PCI bridge. For that case, we
+ * needn't enlarge the alignment so that we can save some
+ * resources.
+ */
+static resource_size_t pnv_pci_window_alignment(struct pci_bus *bus,
+ unsigned long type)
+{
+ struct pci_dev *bridge;
+ struct pci_controller *hose = pci_bus_to_host(bus);
+ struct pnv_phb *phb = hose->private_data;
+ int num_pci_bridges = 0;
+
+ bridge = bus->self;
+ while (bridge) {
+ if (pci_pcie_type(bridge) == PCI_EXP_TYPE_PCI_BRIDGE) {
+ num_pci_bridges++;
+ if (num_pci_bridges >= 2)
+ return 1;
+ }
+
+ bridge = bridge->bus->self;
+ }
+
+ /* We need support prefetchable memory window later */
+ if (type & IORESOURCE_MEM)
+ return phb->ioda.m32_segsize;
+
+ return phb->ioda.io_segsize;
+}
+
/* Prevent enabling devices for which we couldn't properly
* assign a PE
*/
@@ -1306,6 +1344,7 @@
*/
ppc_md.pcibios_fixup_phb = pnv_pci_ioda_fixup_phb;
ppc_md.pcibios_enable_device_hook = pnv_pci_enable_device_hook;
+ ppc_md.pcibios_window_alignment = pnv_pci_window_alignment;
pci_add_flags(PCI_PROBE_ONLY | PCI_REASSIGN_ALL_RSRC);
/* Reset IODA tables to a clean state */
diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c
index 3ef4625..7698b6e 100644
--- a/arch/powerpc/platforms/powernv/smp.c
+++ b/arch/powerpc/platforms/powernv/smp.c
@@ -106,14 +106,6 @@
{
unsigned int cpu;
- /* If powersave_nap is enabled, use NAP mode, else just
- * spin aimlessly
- */
- if (!powersave_nap) {
- generic_mach_cpu_die();
- return;
- }
-
/* Standard hot unplug procedure */
local_irq_disable();
idle_task_exit();
@@ -128,7 +120,7 @@
*/
mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1);
while (!generic_check_cpu_restart(cpu)) {
- power7_idle();
+ power7_nap();
if (!generic_check_cpu_restart(cpu)) {
DBG("CPU%d Unexpected exit while offline !\n", cpu);
/* We may be getting an IPI, so we re-enable
diff --git a/arch/powerpc/sysdev/fsl_pci.c b/arch/powerpc/sysdev/fsl_pci.c
index a7b2a60..c37f461 100644
--- a/arch/powerpc/sysdev/fsl_pci.c
+++ b/arch/powerpc/sysdev/fsl_pci.c
@@ -465,7 +465,7 @@
iounmap(hose->cfg_data);
iounmap(hose->cfg_addr);
pcibios_free_controller(hose);
- return 0;
+ return -ENODEV;
}
setup_pci_cmd(hose);
@@ -827,6 +827,7 @@
void __devinit fsl_pci_init(void)
{
+ int ret;
struct device_node *node;
struct pci_controller *hose;
dma_addr_t max = 0xffffffff;
@@ -855,10 +856,12 @@
if (!fsl_pci_primary)
fsl_pci_primary = node;
- fsl_add_bridge(node, fsl_pci_primary == node);
- hose = pci_find_hose_for_OF_device(node);
- max = min(max, hose->dma_window_base_cur +
- hose->dma_window_size);
+ ret = fsl_add_bridge(node, fsl_pci_primary == node);
+ if (ret == 0) {
+ hose = pci_find_hose_for_OF_device(node);
+ max = min(max, hose->dma_window_base_cur +
+ hose->dma_window_size);
+ }
}
}
diff --git a/arch/powerpc/sysdev/mpic_msgr.c b/arch/powerpc/sysdev/mpic_msgr.c
index 483d8fa..e961f8c 100644
--- a/arch/powerpc/sysdev/mpic_msgr.c
+++ b/arch/powerpc/sysdev/mpic_msgr.c
@@ -14,6 +14,9 @@
#include <linux/list.h>
#include <linux/of_platform.h>
#include <linux/errno.h>
+#include <linux/err.h>
+#include <linux/export.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/hw_irq.h>
#include <asm/ppc-pci.h>
diff --git a/arch/powerpc/sysdev/xics/icp-hv.c b/arch/powerpc/sysdev/xics/icp-hv.c
index 14469cf..df0fc58 100644
--- a/arch/powerpc/sysdev/xics/icp-hv.c
+++ b/arch/powerpc/sysdev/xics/icp-hv.c
@@ -65,7 +65,11 @@
static inline void icp_hv_set_qirr(int n_cpu , u8 value)
{
int hw_cpu = get_hard_smp_processor_id(n_cpu);
- long rc = plpar_hcall_norets(H_IPI, hw_cpu, value);
+ long rc;
+
+ /* Make sure all previous accesses are ordered before IPI sending */
+ mb();
+ rc = plpar_hcall_norets(H_IPI, hw_cpu, value);
if (rc != H_SUCCESS) {
pr_err("%s: bad return code qirr cpu=%d hw_cpu=%d mfrr=0x%x "
"returned %ld\n", __func__, n_cpu, hw_cpu, value, rc);
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index eab3492..9b49c65e 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -17,6 +17,7 @@
#include <linux/reboot.h>
#include <linux/delay.h>
#include <linux/kallsyms.h>
+#include <linux/kmsg_dump.h>
#include <linux/cpumask.h>
#include <linux/export.h>
#include <linux/sysrq.h>
@@ -894,13 +895,13 @@
#endif
default:
printf("Unrecognized command: ");
- do {
+ do {
if (' ' < cmd && cmd <= '~')
putchar(cmd);
else
printf("\\x%x", cmd);
cmd = inchar();
- } while (cmd != '\n');
+ } while (cmd != '\n');
printf(" (type ? for help)\n");
break;
}
@@ -1097,7 +1098,7 @@
return 1;
}
-static char *breakpoint_help_string =
+static char *breakpoint_help_string =
"Breakpoint command usage:\n"
"b show breakpoints\n"
"b <addr> [cnt] set breakpoint at given instr addr\n"
@@ -1193,7 +1194,7 @@
default:
termch = cmd;
- cmd = skipbl();
+ cmd = skipbl();
if (cmd == '?') {
printf(breakpoint_help_string);
break;
@@ -1359,7 +1360,7 @@
sp + REGS_OFFSET);
break;
}
- printf("--- Exception: %lx %s at ", regs.trap,
+ printf("--- Exception: %lx %s at ", regs.trap,
getvecname(TRAP(®s)));
pc = regs.nip;
lr = regs.link;
@@ -1623,14 +1624,14 @@
cmd = skipbl();
if (cmd == '\n') {
- unsigned long sp, toc;
+ unsigned long sp, toc;
asm("mr %0,1" : "=r" (sp) :);
asm("mr %0,2" : "=r" (toc) :);
printf("msr = "REG" sprg0= "REG"\n",
mfmsr(), mfspr(SPRN_SPRG0));
printf("pvr = "REG" sprg1= "REG"\n",
- mfspr(SPRN_PVR), mfspr(SPRN_SPRG1));
+ mfspr(SPRN_PVR), mfspr(SPRN_SPRG1));
printf("dec = "REG" sprg2= "REG"\n",
mfspr(SPRN_DEC), mfspr(SPRN_SPRG2));
printf("sp = "REG" sprg3= "REG"\n", sp, mfspr(SPRN_SPRG3));
@@ -1783,7 +1784,7 @@
static int brev;
static int mnoread;
-static char *memex_help_string =
+static char *memex_help_string =
"Memory examine command usage:\n"
"m [addr] [flags] examine/change memory\n"
" addr is optional. will start where left off.\n"
@@ -1798,7 +1799,7 @@
"NOTE: flags are saved as defaults\n"
"";
-static char *memex_subcmd_help_string =
+static char *memex_subcmd_help_string =
"Memory examine subcommands:\n"
" hexval write this val to current location\n"
" 'string' write chars from string to this location\n"
@@ -2064,7 +2065,7 @@
nr = mread(adrs, temp, r);
adrs += nr;
for (m = 0; m < r; ++m) {
- if ((m & (sizeof(long) - 1)) == 0 && m > 0)
+ if ((m & (sizeof(long) - 1)) == 0 && m > 0)
putchar(' ');
if (m < nr)
printf("%.2x", temp[m]);
@@ -2072,7 +2073,7 @@
printf("%s", fault_chars[fault_type]);
}
for (; m < 16; ++m) {
- if ((m & (sizeof(long) - 1)) == 0)
+ if ((m & (sizeof(long) - 1)) == 0)
putchar(' ');
printf(" ");
}
@@ -2148,45 +2149,28 @@
void
dump_log_buf(void)
{
- const unsigned long size = 128;
- unsigned long end, addr;
- unsigned char buf[size + 1];
+ struct kmsg_dumper dumper = { .active = 1 };
+ unsigned char buf[128];
+ size_t len;
- addr = 0;
- buf[size] = '\0';
+ if (setjmp(bus_error_jmp) != 0) {
+ printf("Error dumping printk buffer!\n");
+ return;
+ }
- if (setjmp(bus_error_jmp) != 0) {
- printf("Unable to lookup symbol __log_buf!\n");
- return;
- }
+ catch_memory_errors = 1;
+ sync();
- catch_memory_errors = 1;
- sync();
- addr = kallsyms_lookup_name("__log_buf");
+ kmsg_dump_rewind_nolock(&dumper);
+ while (kmsg_dump_get_line_nolock(&dumper, false, buf, sizeof(buf), &len)) {
+ buf[len] = '\0';
+ printf("%s", buf);
+ }
- if (! addr)
- printf("Symbol __log_buf not found!\n");
- else {
- end = addr + (1 << CONFIG_LOG_BUF_SHIFT);
- while (addr < end) {
- if (! mread(addr, buf, size)) {
- printf("Can't read memory at address 0x%lx\n", addr);
- break;
- }
-
- printf("%s", buf);
-
- if (strlen(buf) < size)
- break;
-
- addr += size;
- }
- }
-
- sync();
- /* wait a little while to see if we get a machine check */
- __delay(200);
- catch_memory_errors = 0;
+ sync();
+ /* wait a little while to see if we get a machine check */
+ __delay(200);
+ catch_memory_errors = 0;
}
/*
diff --git a/arch/s390/Kbuild b/arch/s390/Kbuild
index 9858476..cc45d25 100644
--- a/arch/s390/Kbuild
+++ b/arch/s390/Kbuild
@@ -5,3 +5,4 @@
obj-$(CONFIG_S390_HYPFS_FS) += hypfs/
obj-$(CONFIG_APPLDATA_BASE) += appldata/
obj-$(CONFIG_MATHEMU) += math-emu/
+obj-y += net/
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 107610e..f9acddd 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -49,10 +49,13 @@
config PGSTE
def_bool y if KVM
-config VIRT_CPU_ACCOUNTING
+config ARCH_SUPPORTS_DEBUG_PAGEALLOC
def_bool y
-config ARCH_SUPPORTS_DEBUG_PAGEALLOC
+config KEXEC
+ def_bool y
+
+config AUDIT_ARCH
def_bool y
config S390
@@ -84,11 +87,15 @@
select HAVE_KERNEL_XZ
select HAVE_ARCH_MUTEX_CPU_RELAX
select HAVE_ARCH_JUMP_LABEL if !MARCH_G5
+ select HAVE_BPF_JIT if 64BIT && PACK_STACK
select ARCH_SAVE_PAGE_KEYS if HIBERNATION
select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP
select HAVE_CMPXCHG_LOCAL
+ select HAVE_CMPXCHG_DOUBLE
+ select HAVE_VIRT_CPU_ACCOUNTING
+ select VIRT_CPU_ACCOUNTING
select ARCH_DISCARD_MEMBLOCK
select BUILDTIME_EXTABLE_SORT
select ARCH_INLINE_SPIN_TRYLOCK
@@ -133,9 +140,79 @@
source "kernel/Kconfig.freezer"
-menu "Base setup"
+menu "Processor type and features"
-comment "Processor type and features"
+config HAVE_MARCH_Z900_FEATURES
+ def_bool n
+
+config HAVE_MARCH_Z990_FEATURES
+ def_bool n
+ select HAVE_MARCH_Z900_FEATURES
+
+config HAVE_MARCH_Z9_109_FEATURES
+ def_bool n
+ select HAVE_MARCH_Z990_FEATURES
+
+config HAVE_MARCH_Z10_FEATURES
+ def_bool n
+ select HAVE_MARCH_Z9_109_FEATURES
+
+config HAVE_MARCH_Z196_FEATURES
+ def_bool n
+ select HAVE_MARCH_Z10_FEATURES
+
+choice
+ prompt "Processor type"
+ default MARCH_G5
+
+config MARCH_G5
+ bool "System/390 model G5 and G6"
+ depends on !64BIT
+ help
+ Select this to build a 31 bit kernel that works
+ on all ESA/390 and z/Architecture machines.
+
+config MARCH_Z900
+ bool "IBM zSeries model z800 and z900"
+ select HAVE_MARCH_Z900_FEATURES if 64BIT
+ help
+ Select this to enable optimizations for model z800/z900 (2064 and
+ 2066 series). This will enable some optimizations that are not
+ available on older ESA/390 (31 Bit) only CPUs.
+
+config MARCH_Z990
+ bool "IBM zSeries model z890 and z990"
+ select HAVE_MARCH_Z990_FEATURES if 64BIT
+ help
+ Select this to enable optimizations for model z890/z990 (2084 and
+ 2086 series). The kernel will be slightly faster but will not work
+ on older machines.
+
+config MARCH_Z9_109
+ bool "IBM System z9"
+ select HAVE_MARCH_Z9_109_FEATURES if 64BIT
+ help
+ Select this to enable optimizations for IBM System z9 (2094 and
+ 2096 series). The kernel will be slightly faster but will not work
+ on older machines.
+
+config MARCH_Z10
+ bool "IBM System z10"
+ select HAVE_MARCH_Z10_FEATURES if 64BIT
+ help
+ Select this to enable optimizations for IBM System z10 (2097 and
+ 2098 series). The kernel will be slightly faster but will not work
+ on older machines.
+
+config MARCH_Z196
+ bool "IBM zEnterprise 114 and 196"
+ select HAVE_MARCH_Z196_FEATURES if 64BIT
+ help
+ Select this to enable optimizations for IBM zEnterprise 114 and 196
+ (2818 and 2817 series). The kernel will be slightly faster but will
+ not work on older machines.
+
+endchoice
config 64BIT
def_bool y
@@ -147,6 +224,24 @@
config 32BIT
def_bool y if !64BIT
+config COMPAT
+ def_bool y
+ prompt "Kernel support for 31 bit emulation"
+ depends on 64BIT
+ select COMPAT_BINFMT_ELF if BINFMT_ELF
+ select ARCH_WANT_OLD_COMPAT_IPC
+ help
+ Select this option if you want to enable your system kernel to
+ handle system-calls from ELF binaries for 31 bit ESA. This option
+ (and some other stuff like libraries and such) is needed for
+ executing 31 bit applications. It is safe to say "Y".
+
+config SYSVIPC_COMPAT
+ def_bool y if COMPAT && SYSVIPC
+
+config KEYS_COMPAT
+ def_bool y if COMPAT && KEYS
+
config SMP
def_bool y
prompt "Symmetric multi-processing support"
@@ -202,6 +297,8 @@
Book scheduler support improves the CPU scheduler's decision making
when dealing with machines that have several books.
+source kernel/Kconfig.preempt
+
config MATHEMU
def_bool y
prompt "IEEE FPU emulation"
@@ -211,100 +308,35 @@
on older ESA/390 machines. Say Y unless you know your machine doesn't
need this.
-config COMPAT
+source kernel/Kconfig.hz
+
+endmenu
+
+menu "Memory setup"
+
+config ARCH_SPARSEMEM_ENABLE
def_bool y
- prompt "Kernel support for 31 bit emulation"
- depends on 64BIT
- select COMPAT_BINFMT_ELF if BINFMT_ELF
- select ARCH_WANT_OLD_COMPAT_IPC
- help
- Select this option if you want to enable your system kernel to
- handle system-calls from ELF binaries for 31 bit ESA. This option
- (and some other stuff like libraries and such) is needed for
- executing 31 bit applications. It is safe to say "Y".
+ select SPARSEMEM_VMEMMAP_ENABLE
+ select SPARSEMEM_VMEMMAP
+ select SPARSEMEM_STATIC if !64BIT
-config SYSVIPC_COMPAT
- def_bool y if COMPAT && SYSVIPC
-
-config KEYS_COMPAT
- def_bool y if COMPAT && KEYS
-
-config AUDIT_ARCH
+config ARCH_SPARSEMEM_DEFAULT
def_bool y
-config HAVE_MARCH_Z900_FEATURES
- def_bool n
+config ARCH_SELECT_MEMORY_MODEL
+ def_bool y
-config HAVE_MARCH_Z990_FEATURES
- def_bool n
- select HAVE_MARCH_Z900_FEATURES
+config ARCH_ENABLE_MEMORY_HOTPLUG
+ def_bool y if SPARSEMEM
-config HAVE_MARCH_Z9_109_FEATURES
- def_bool n
- select HAVE_MARCH_Z990_FEATURES
+config ARCH_ENABLE_MEMORY_HOTREMOVE
+ def_bool y
-config HAVE_MARCH_Z10_FEATURES
- def_bool n
- select HAVE_MARCH_Z9_109_FEATURES
+config FORCE_MAX_ZONEORDER
+ int
+ default "9"
-config HAVE_MARCH_Z196_FEATURES
- def_bool n
- select HAVE_MARCH_Z10_FEATURES
-
-comment "Code generation options"
-
-choice
- prompt "Processor type"
- default MARCH_G5
-
-config MARCH_G5
- bool "System/390 model G5 and G6"
- depends on !64BIT
- help
- Select this to build a 31 bit kernel that works
- on all ESA/390 and z/Architecture machines.
-
-config MARCH_Z900
- bool "IBM zSeries model z800 and z900"
- select HAVE_MARCH_Z900_FEATURES if 64BIT
- help
- Select this to enable optimizations for model z800/z900 (2064 and
- 2066 series). This will enable some optimizations that are not
- available on older ESA/390 (31 Bit) only CPUs.
-
-config MARCH_Z990
- bool "IBM zSeries model z890 and z990"
- select HAVE_MARCH_Z990_FEATURES if 64BIT
- help
- Select this to enable optimizations for model z890/z990 (2084 and
- 2086 series). The kernel will be slightly faster but will not work
- on older machines.
-
-config MARCH_Z9_109
- bool "IBM System z9"
- select HAVE_MARCH_Z9_109_FEATURES if 64BIT
- help
- Select this to enable optimizations for IBM System z9 (2094 and
- 2096 series). The kernel will be slightly faster but will not work
- on older machines.
-
-config MARCH_Z10
- bool "IBM System z10"
- select HAVE_MARCH_Z10_FEATURES if 64BIT
- help
- Select this to enable optimizations for IBM System z10 (2097 and
- 2098 series). The kernel will be slightly faster but will not work
- on older machines.
-
-config MARCH_Z196
- bool "IBM zEnterprise 114 and 196"
- select HAVE_MARCH_Z196_FEATURES if 64BIT
- help
- Select this to enable optimizations for IBM zEnterprise 114 and 196
- (2818 and 2817 series). The kernel will be slightly faster but will
- not work on older machines.
-
-endchoice
+source "mm/Kconfig"
config PACK_STACK
def_bool y
@@ -368,34 +400,9 @@
Say N if you are unsure.
-comment "Kernel preemption"
+endmenu
-source "kernel/Kconfig.preempt"
-
-config ARCH_SPARSEMEM_ENABLE
- def_bool y
- select SPARSEMEM_VMEMMAP_ENABLE
- select SPARSEMEM_VMEMMAP
- select SPARSEMEM_STATIC if !64BIT
-
-config ARCH_SPARSEMEM_DEFAULT
- def_bool y
-
-config ARCH_SELECT_MEMORY_MODEL
- def_bool y
-
-config ARCH_ENABLE_MEMORY_HOTPLUG
- def_bool y if SPARSEMEM
-
-config ARCH_ENABLE_MEMORY_HOTREMOVE
- def_bool y
-
-config ARCH_HIBERNATION_POSSIBLE
- def_bool y if 64BIT
-
-source "mm/Kconfig"
-
-comment "I/O subsystem configuration"
+menu "I/O subsystem"
config QDIO
def_tristate y
@@ -426,13 +433,102 @@
If unsure, say N.
-comment "Misc"
+config SCM_BUS
+ def_bool y
+ depends on 64BIT
+ prompt "SCM bus driver"
+ help
+ Bus driver for Storage Class Memory.
+
+config EADM_SCH
+ def_tristate m
+ prompt "Support for EADM subchannels"
+ depends on SCM_BUS
+ help
+ This driver allows usage of EADM subchannels. EADM subchannels act
+ as a communication vehicle for SCM increments.
+
+ To compile this driver as a module, choose M here: the
+ module will be called eadm_sch.
+
+endmenu
+
+menu "Dump support"
+
+config CRASH_DUMP
+ bool "kernel crash dumps"
+ depends on 64BIT && SMP
+ select KEXEC
+ help
+ Generate crash dump after being started by kexec.
+ Crash dump kernels are loaded in the main kernel with kexec-tools
+ into a specially reserved region and then later executed after
+ a crash by kdump/kexec.
+ For more details see Documentation/kdump/kdump.txt
+
+config ZFCPDUMP
+ def_bool n
+ prompt "zfcpdump support"
+ select SMP
+ help
+ Select this option if you want to build an zfcpdump enabled kernel.
+ Refer to <file:Documentation/s390/zfcpdump.txt> for more details on this.
+
+endmenu
+
+menu "Executable file formats / Emulations"
source "fs/Kconfig.binfmt"
-config FORCE_MAX_ZONEORDER
- int
- default "9"
+config SECCOMP
+ def_bool y
+ prompt "Enable seccomp to safely compute untrusted bytecode"
+ depends on PROC_FS
+ help
+ This kernel feature is useful for number crunching applications
+ that may need to compute untrusted bytecode during their
+ execution. By using pipes or other transports made available to
+ the process as file descriptors supporting the read/write
+ syscalls, it's possible to isolate those applications in
+ their own address space using seccomp. Once seccomp is
+ enabled via /proc/<pid>/seccomp, it cannot be disabled
+ and the task is only allowed to execute a few safe syscalls
+ defined by each seccomp mode.
+
+ If unsure, say Y.
+
+endmenu
+
+menu "Power Management"
+
+config ARCH_HIBERNATION_POSSIBLE
+ def_bool y if 64BIT
+
+source "kernel/power/Kconfig"
+
+endmenu
+
+source "net/Kconfig"
+
+config PCMCIA
+ def_bool n
+
+config CCW
+ def_bool y
+
+source "drivers/Kconfig"
+
+source "fs/Kconfig"
+
+source "arch/s390/Kconfig.debug"
+
+source "security/Kconfig"
+
+source "crypto/Kconfig"
+
+source "lib/Kconfig"
+
+menu "Virtualization"
config PFAULT
def_bool y
@@ -448,8 +544,8 @@
this option.
config SHARED_KERNEL
- def_bool y
- prompt "VM shared kernel support"
+ bool "VM shared kernel support"
+ depends on !JUMP_LABEL
help
Select this option, if you want to share the text segment of the
Linux kernel between different VM guests. This reduces memory
@@ -544,8 +640,6 @@
This can also be compiled as a module, which will be called
appldata_net_sum.o.
-source kernel/Kconfig.hz
-
config S390_HYPFS_FS
def_bool y
prompt "s390 hypervisor file system support"
@@ -554,90 +648,21 @@
This is a virtual file system intended to provide accounting
information in an s390 hypervisor environment.
-config KEXEC
- def_bool n
- prompt "kexec system call"
- help
- kexec is a system call that implements the ability to shutdown your
- current kernel, and to start another kernel. It is like a reboot
- but is independent of hardware/microcode support.
-
-config CRASH_DUMP
- bool "kernel crash dumps"
- depends on 64BIT && SMP
- select KEXEC
- help
- Generate crash dump after being started by kexec.
- Crash dump kernels are loaded in the main kernel with kexec-tools
- into a specially reserved region and then later executed after
- a crash by kdump/kexec.
- For more details see Documentation/kdump/kdump.txt
-
-config ZFCPDUMP
- def_bool n
- prompt "zfcpdump support"
- select SMP
- help
- Select this option if you want to build an zfcpdump enabled kernel.
- Refer to <file:Documentation/s390/zfcpdump.txt> for more details on this.
+source "arch/s390/kvm/Kconfig"
config S390_GUEST
def_bool y
- prompt "s390 guest support for KVM (EXPERIMENTAL)"
+ prompt "s390 support for virtio devices (EXPERIMENTAL)"
depends on 64BIT && EXPERIMENTAL
select VIRTUALIZATION
select VIRTIO
select VIRTIO_RING
select VIRTIO_CONSOLE
help
+ Enabling this option adds support for virtio based paravirtual device
+ drivers on s390.
+
Select this option if you want to run the kernel as a guest under
- the KVM hypervisor. This will add detection for KVM as well as a
- virtio transport. If KVM is detected, the virtio console will be
- the default console.
-
-config SECCOMP
- def_bool y
- prompt "Enable seccomp to safely compute untrusted bytecode"
- depends on PROC_FS
- help
- This kernel feature is useful for number crunching applications
- that may need to compute untrusted bytecode during their
- execution. By using pipes or other transports made available to
- the process as file descriptors supporting the read/write
- syscalls, it's possible to isolate those applications in
- their own address space using seccomp. Once seccomp is
- enabled via /proc/<pid>/seccomp, it cannot be disabled
- and the task is only allowed to execute a few safe syscalls
- defined by each seccomp mode.
-
- If unsure, say Y.
+ the KVM hypervisor.
endmenu
-
-menu "Power Management"
-
-source "kernel/power/Kconfig"
-
-endmenu
-
-source "net/Kconfig"
-
-config PCMCIA
- def_bool n
-
-config CCW
- def_bool y
-
-source "drivers/Kconfig"
-
-source "fs/Kconfig"
-
-source "arch/s390/Kconfig.debug"
-
-source "security/Kconfig"
-
-source "crypto/Kconfig"
-
-source "lib/Kconfig"
-
-source "arch/s390/kvm/Kconfig"
diff --git a/arch/s390/boot/compressed/Makefile b/arch/s390/boot/compressed/Makefile
index 10e22c4..3ad8f61 100644
--- a/arch/s390/boot/compressed/Makefile
+++ b/arch/s390/boot/compressed/Makefile
@@ -11,6 +11,7 @@
sizes.h head$(BITS).o
KBUILD_CFLAGS := -m$(BITS) -D__KERNEL__ $(LINUX_INCLUDE) -O2
+KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
KBUILD_CFLAGS += $(cflags-y)
KBUILD_CFLAGS += $(call cc-option,-mpacked-stack)
KBUILD_CFLAGS += $(call cc-option,-ffreestanding)
diff --git a/arch/s390/boot/compressed/misc.c b/arch/s390/boot/compressed/misc.c
index 465eca7..c4c6a1c 100644
--- a/arch/s390/boot/compressed/misc.c
+++ b/arch/s390/boot/compressed/misc.c
@@ -71,34 +71,37 @@
{
char *xs;
- if (c == 0)
- return __builtin_memset(s, 0, n);
-
- xs = (char *) s;
- if (n > 0)
- do {
- *xs++ = c;
- } while (--n > 0);
+ xs = s;
+ while (n--)
+ *xs++ = c;
return s;
}
-void *memcpy(void *__dest, __const void *__src, size_t __n)
+void *memcpy(void *dest, const void *src, size_t n)
{
- return __builtin_memcpy(__dest, __src, __n);
+ const char *s = src;
+ char *d = dest;
+
+ while (n--)
+ *d++ = *s++;
+ return dest;
}
-void *memmove(void *__dest, __const void *__src, size_t __n)
+void *memmove(void *dest, const void *src, size_t n)
{
- char *d;
- const char *s;
+ const char *s = src;
+ char *d = dest;
- if (__dest <= __src)
- return __builtin_memcpy(__dest, __src, __n);
- d = __dest + __n;
- s = __src + __n;
- while (__n--)
- *--d = *--s;
- return __dest;
+ if (d <= s) {
+ while (n--)
+ *d++ = *s++;
+ } else {
+ d += n;
+ s += n;
+ while (n--)
+ *--d = *--s;
+ }
+ return dest;
}
static void error(char *x)
diff --git a/arch/s390/defconfig b/arch/s390/defconfig
index f39cd71..b74400e 100644
--- a/arch/s390/defconfig
+++ b/arch/s390/defconfig
@@ -16,8 +16,8 @@
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_RESOURCE_COUNTERS=y
-CONFIG_CGROUP_MEMCG=y
-CONFIG_CGROUP_MEM_RES_CTLR_SWAP=y
+CONFIG_MEMCG=y
+CONFIG_MEMCG_SWAP=y
CONFIG_CGROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_BLK_CGROUP=y
@@ -32,20 +32,19 @@
CONFIG_PROFILING=y
CONFIG_OPROFILE=y
CONFIG_KPROBES=y
+CONFIG_JUMP_LABEL=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_PARTITION_ADVANCED=y
CONFIG_IBM_PARTITION=y
CONFIG_DEFAULT_DEADLINE=y
-CONFIG_PREEMPT=y
+CONFIG_HZ_100=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_MEMORY_HOTREMOVE=y
CONFIG_KSM=y
-CONFIG_BINFMT_MISC=m
-CONFIG_CMM=m
-CONFIG_HZ_100=y
CONFIG_CRASH_DUMP=y
+CONFIG_BINFMT_MISC=m
CONFIG_HIBERNATION=y
CONFIG_PACKET=y
CONFIG_UNIX=y
@@ -75,6 +74,7 @@
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=y
+CONFIG_BPF_JIT=y
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_BLK_DEV_LOOP=m
@@ -121,7 +121,6 @@
CONFIG_RCU_TRACE=y
CONFIG_KPROBES_SANITY_TEST=y
CONFIG_DEBUG_FORCE_WEAK_PER_CPU=y
-CONFIG_CPU_NOTIFIER_ERROR_INJECT=m
CONFIG_LATENCYTOP=y
CONFIG_DEBUG_PAGEALLOC=y
CONFIG_BLK_DEV_IO_TRACE=y
@@ -173,3 +172,4 @@
CONFIG_CRYPTO_DES_S390=m
CONFIG_CRYPTO_AES_S390=m
CONFIG_CRC7=m
+CONFIG_CMM=m
diff --git a/arch/s390/include/asm/appldata.h b/arch/s390/include/asm/appldata.h
index f328294..32a7059 100644
--- a/arch/s390/include/asm/appldata.h
+++ b/arch/s390/include/asm/appldata.h
@@ -70,7 +70,7 @@
int ry;
if (!MACHINE_IS_VM)
- return -ENOSYS;
+ return -EOPNOTSUPP;
parm_list.diag = 0xdc;
parm_list.function = fn;
parm_list.parlist_length = sizeof(parm_list);
diff --git a/arch/s390/include/asm/chsc.h b/arch/s390/include/asm/chsc.h
index bf115b4..aea451f 100644
--- a/arch/s390/include/asm/chsc.h
+++ b/arch/s390/include/asm/chsc.h
@@ -125,32 +125,4 @@
#define CHSC_INFO_CPD _IOWR(CHSC_IOCTL_MAGIC, 0x87, struct chsc_cpd_info)
#define CHSC_INFO_DCAL _IOWR(CHSC_IOCTL_MAGIC, 0x88, struct chsc_dcal)
-#ifdef __KERNEL__
-
-struct css_general_char {
- u64 : 12;
- u32 dynio : 1; /* bit 12 */
- u32 : 28;
- u32 aif : 1; /* bit 41 */
- u32 : 3;
- u32 mcss : 1; /* bit 45 */
- u32 fcs : 1; /* bit 46 */
- u32 : 1;
- u32 ext_mb : 1; /* bit 48 */
- u32 : 7;
- u32 aif_tdd : 1; /* bit 56 */
- u32 : 1;
- u32 qebsm : 1; /* bit 58 */
- u32 : 8;
- u32 aif_osa : 1; /* bit 67 */
- u32 : 14;
- u32 cib : 1; /* bit 82 */
- u32 : 5;
- u32 fcx : 1; /* bit 88 */
- u32 : 7;
-}__attribute__((packed));
-
-extern struct css_general_char css_general_characteristics;
-
-#endif /* __KERNEL__ */
#endif
diff --git a/arch/s390/include/asm/cio.h b/arch/s390/include/asm/cio.h
index 77043aa..55bde60 100644
--- a/arch/s390/include/asm/cio.h
+++ b/arch/s390/include/asm/cio.h
@@ -80,6 +80,18 @@
} __attribute__ ((packed));
/**
+ * struct erw_eadm - EADM Subchannel extended report word
+ * @b: aob error
+ * @r: arsb error
+ */
+struct erw_eadm {
+ __u32 : 16;
+ __u32 b : 1;
+ __u32 r : 1;
+ __u32 : 14;
+} __packed;
+
+/**
* struct sublog - subchannel logout area
* @res0: reserved
* @esf: extended status flags
@@ -170,9 +182,22 @@
} __attribute__ ((packed));
/**
+ * struct esw_eadm - EADM Subchannel Extended Status Word (ESW)
+ * @sublog: subchannel logout
+ * @erw: extended report word
+ */
+struct esw_eadm {
+ __u32 sublog;
+ struct erw_eadm erw;
+ __u32 : 32;
+ __u32 : 32;
+ __u32 : 32;
+} __packed;
+
+/**
* struct irb - interruption response block
* @scsw: subchannel status word
- * @esw: extened status word, 4 formats
+ * @esw: extened status word
* @ecw: extended control word
*
* The irb that is handed to the device driver when an interrupt occurs. For
@@ -191,6 +216,7 @@
struct esw1 esw1;
struct esw2 esw2;
struct esw3 esw3;
+ struct esw_eadm eadm;
} esw;
__u8 ecw[32];
} __attribute__ ((packed,aligned(4)));
diff --git a/arch/s390/include/asm/cmpxchg.h b/arch/s390/include/asm/cmpxchg.h
index 8d798e9..0f636cb 100644
--- a/arch/s390/include/asm/cmpxchg.h
+++ b/arch/s390/include/asm/cmpxchg.h
@@ -7,7 +7,9 @@
#ifndef __ASM_CMPXCHG_H
#define __ASM_CMPXCHG_H
+#include <linux/mmdebug.h>
#include <linux/types.h>
+#include <linux/bug.h>
extern void __xchg_called_with_bad_pointer(void);
@@ -203,6 +205,65 @@
})
#endif /* CONFIG_64BIT */
+#define __cmpxchg_double_op(p1, p2, o1, o2, n1, n2, insn) \
+({ \
+ register __typeof__(*(p1)) __old1 asm("2") = (o1); \
+ register __typeof__(*(p2)) __old2 asm("3") = (o2); \
+ register __typeof__(*(p1)) __new1 asm("4") = (n1); \
+ register __typeof__(*(p2)) __new2 asm("5") = (n2); \
+ int cc; \
+ asm volatile( \
+ insn " %[old],%[new],%[ptr]\n" \
+ " ipm %[cc]\n" \
+ " srl %[cc],28" \
+ : [cc] "=d" (cc), [old] "+d" (__old1), "+d" (__old2) \
+ : [new] "d" (__new1), "d" (__new2), \
+ [ptr] "Q" (*(p1)), "Q" (*(p2)) \
+ : "memory", "cc"); \
+ !cc; \
+})
+
+#define __cmpxchg_double_4(p1, p2, o1, o2, n1, n2) \
+ __cmpxchg_double_op(p1, p2, o1, o2, n1, n2, "cds")
+
+#define __cmpxchg_double_8(p1, p2, o1, o2, n1, n2) \
+ __cmpxchg_double_op(p1, p2, o1, o2, n1, n2, "cdsg")
+
+extern void __cmpxchg_double_called_with_bad_pointer(void);
+
+#define __cmpxchg_double(p1, p2, o1, o2, n1, n2) \
+({ \
+ int __ret; \
+ switch (sizeof(*(p1))) { \
+ case 4: \
+ __ret = __cmpxchg_double_4(p1, p2, o1, o2, n1, n2); \
+ break; \
+ case 8: \
+ __ret = __cmpxchg_double_8(p1, p2, o1, o2, n1, n2); \
+ break; \
+ default: \
+ __cmpxchg_double_called_with_bad_pointer(); \
+ } \
+ __ret; \
+})
+
+#define cmpxchg_double(p1, p2, o1, o2, n1, n2) \
+({ \
+ __typeof__(p1) __p1 = (p1); \
+ __typeof__(p2) __p2 = (p2); \
+ int __ret; \
+ BUILD_BUG_ON(sizeof(*(p1)) != sizeof(long)); \
+ BUILD_BUG_ON(sizeof(*(p2)) != sizeof(long)); \
+ VM_BUG_ON((unsigned long)((__p1) + 1) != (unsigned long)(__p2));\
+ if (sizeof(long) == 4) \
+ __ret = __cmpxchg_double_4(__p1, __p2, o1, o2, n1, n2); \
+ else \
+ __ret = __cmpxchg_double_8(__p1, __p2, o1, o2, n1, n2); \
+ __ret; \
+})
+
+#define system_has_cmpxchg_double() 1
+
#include <asm-generic/cmpxchg-local.h>
static inline unsigned long __cmpxchg_local(void *ptr,
diff --git a/arch/s390/include/asm/cpu_mf.h b/arch/s390/include/asm/cpu_mf.h
index a3afecd..35f0020 100644
--- a/arch/s390/include/asm/cpu_mf.h
+++ b/arch/s390/include/asm/cpu_mf.h
@@ -21,11 +21,15 @@
#define CPU_MF_INT_SF_LSDA (1 << 22) /* loss of sample data alert */
#define CPU_MF_INT_CF_CACA (1 << 7) /* counter auth. change alert */
#define CPU_MF_INT_CF_LCDA (1 << 6) /* loss of counter data alert */
+#define CPU_MF_INT_RI_HALTED (1 << 5) /* run-time instr. halted */
+#define CPU_MF_INT_RI_BUF_FULL (1 << 4) /* run-time instr. program
+ buffer full */
#define CPU_MF_INT_CF_MASK (CPU_MF_INT_CF_CACA|CPU_MF_INT_CF_LCDA)
#define CPU_MF_INT_SF_MASK (CPU_MF_INT_SF_IAE|CPU_MF_INT_SF_ISE| \
CPU_MF_INT_SF_PRA|CPU_MF_INT_SF_SACA| \
CPU_MF_INT_SF_LSDA)
+#define CPU_MF_INT_RI_MASK (CPU_MF_INT_RI_HALTED|CPU_MF_INT_RI_BUF_FULL)
/* CPU measurement facility support */
static inline int cpum_cf_avail(void)
diff --git a/arch/s390/include/asm/cputime.h b/arch/s390/include/asm/cputime.h
index 8709bde..023d5ae 100644
--- a/arch/s390/include/asm/cputime.h
+++ b/arch/s390/include/asm/cputime.h
@@ -12,6 +12,9 @@
#include <linux/spinlock.h>
#include <asm/div64.h>
+
+#define __ARCH_HAS_VTIME_ACCOUNT
+
/* We want to use full resolution of the CPU timer: 2**-12 micro-seconds. */
typedef unsigned long long __nocast cputime_t;
diff --git a/arch/s390/include/asm/css_chars.h b/arch/s390/include/asm/css_chars.h
new file mode 100644
index 0000000..a06ebc2
--- /dev/null
+++ b/arch/s390/include/asm/css_chars.h
@@ -0,0 +1,39 @@
+#ifndef _ASM_CSS_CHARS_H
+#define _ASM_CSS_CHARS_H
+
+#include <linux/types.h>
+
+#ifdef __KERNEL__
+
+struct css_general_char {
+ u64 : 12;
+ u32 dynio : 1; /* bit 12 */
+ u32 : 4;
+ u32 eadm : 1; /* bit 17 */
+ u32 : 23;
+ u32 aif : 1; /* bit 41 */
+ u32 : 3;
+ u32 mcss : 1; /* bit 45 */
+ u32 fcs : 1; /* bit 46 */
+ u32 : 1;
+ u32 ext_mb : 1; /* bit 48 */
+ u32 : 7;
+ u32 aif_tdd : 1; /* bit 56 */
+ u32 : 1;
+ u32 qebsm : 1; /* bit 58 */
+ u32 : 8;
+ u32 aif_osa : 1; /* bit 67 */
+ u32 : 12;
+ u32 eadm_rf : 1; /* bit 80 */
+ u32 : 1;
+ u32 cib : 1; /* bit 82 */
+ u32 : 5;
+ u32 fcx : 1; /* bit 88 */
+ u32 : 19;
+ u32 alt_ssi : 1; /* bit 108 */
+} __packed;
+
+extern struct css_general_char css_general_characteristics;
+
+#endif /* __KERNEL__ */
+#endif
diff --git a/arch/s390/include/asm/eadm.h b/arch/s390/include/asm/eadm.h
new file mode 100644
index 0000000..8d48471
--- /dev/null
+++ b/arch/s390/include/asm/eadm.h
@@ -0,0 +1,124 @@
+#ifndef _ASM_S390_EADM_H
+#define _ASM_S390_EADM_H
+
+#include <linux/types.h>
+#include <linux/device.h>
+
+struct arqb {
+ u64 data;
+ u16 fmt:4;
+ u16:12;
+ u16 cmd_code;
+ u16:16;
+ u16 msb_count;
+ u32 reserved[12];
+} __packed;
+
+#define ARQB_CMD_MOVE 1
+
+struct arsb {
+ u16 fmt:4;
+ u32:28;
+ u8 ef;
+ u8:8;
+ u8 ecbi;
+ u8:8;
+ u8 fvf;
+ u16:16;
+ u8 eqc;
+ u32:32;
+ u64 fail_msb;
+ u64 fail_aidaw;
+ u64 fail_ms;
+ u64 fail_scm;
+ u32 reserved[4];
+} __packed;
+
+struct msb {
+ u8 fmt:4;
+ u8 oc:4;
+ u8 flags;
+ u16:12;
+ u16 bs:4;
+ u32 blk_count;
+ u64 data_addr;
+ u64 scm_addr;
+ u64:64;
+} __packed;
+
+struct aidaw {
+ u8 flags;
+ u32 :24;
+ u32 :32;
+ u64 data_addr;
+} __packed;
+
+#define MSB_OC_CLEAR 0
+#define MSB_OC_READ 1
+#define MSB_OC_WRITE 2
+#define MSB_OC_RELEASE 3
+
+#define MSB_FLAG_BNM 0x80
+#define MSB_FLAG_IDA 0x40
+
+#define MSB_BS_4K 0
+#define MSB_BS_1M 1
+
+#define AOB_NR_MSB 124
+
+struct aob {
+ struct arqb request;
+ struct arsb response;
+ struct msb msb[AOB_NR_MSB];
+} __packed __aligned(PAGE_SIZE);
+
+struct aob_rq_header {
+ struct scm_device *scmdev;
+ char data[0];
+};
+
+struct scm_device {
+ u64 address;
+ u64 size;
+ unsigned int nr_max_block;
+ struct device dev;
+ struct {
+ unsigned int persistence:4;
+ unsigned int oper_state:4;
+ unsigned int data_state:4;
+ unsigned int rank:4;
+ unsigned int release:1;
+ unsigned int res_id:8;
+ } __packed attrs;
+};
+
+#define OP_STATE_GOOD 1
+#define OP_STATE_TEMP_ERR 2
+#define OP_STATE_PERM_ERR 3
+
+struct scm_driver {
+ struct device_driver drv;
+ int (*probe) (struct scm_device *scmdev);
+ int (*remove) (struct scm_device *scmdev);
+ void (*notify) (struct scm_device *scmdev);
+ void (*handler) (struct scm_device *scmdev, void *data, int error);
+};
+
+int scm_driver_register(struct scm_driver *scmdrv);
+void scm_driver_unregister(struct scm_driver *scmdrv);
+
+int scm_start_aob(struct aob *aob);
+void scm_irq_handler(struct aob *aob, int error);
+
+struct eadm_ops {
+ int (*eadm_start) (struct aob *aob);
+ struct module *owner;
+};
+
+int scm_get_ref(void);
+void scm_put_ref(void);
+
+void register_eadm_ops(struct eadm_ops *ops);
+void unregister_eadm_ops(struct eadm_ops *ops);
+
+#endif /* _ASM_S390_EADM_H */
diff --git a/arch/s390/include/asm/elf.h b/arch/s390/include/asm/elf.h
index 32e8449..178ff96 100644
--- a/arch/s390/include/asm/elf.h
+++ b/arch/s390/include/asm/elf.h
@@ -101,6 +101,7 @@
#define HWCAP_S390_HPAGE 128
#define HWCAP_S390_ETF3EH 256
#define HWCAP_S390_HIGH_GPRS 512
+#define HWCAP_S390_TE 1024
/*
* These are used to set parameters in the core dumps.
@@ -180,7 +181,8 @@
#define ELF_PLATFORM (elf_platform)
#ifndef CONFIG_64BIT
-#define SET_PERSONALITY(ex) set_personality(PER_LINUX)
+#define SET_PERSONALITY(ex) \
+ set_personality(PER_LINUX | (current->personality & (~PER_MASK)))
#else /* CONFIG_64BIT */
#define SET_PERSONALITY(ex) \
do { \
@@ -211,4 +213,6 @@
extern unsigned long arch_randomize_brk(struct mm_struct *mm);
#define arch_randomize_brk arch_randomize_brk
+void *fill_cpu_elf_notes(void *ptr, struct save_area *sa);
+
#endif
diff --git a/arch/s390/include/asm/etr.h b/arch/s390/include/asm/etr.h
index a24b03b..629b79a 100644
--- a/arch/s390/include/asm/etr.h
+++ b/arch/s390/include/asm/etr.h
@@ -140,7 +140,7 @@
/* Inline assembly helper functions */
static inline int etr_setr(struct etr_eacr *ctrl)
{
- int rc = -ENOSYS;
+ int rc = -EOPNOTSUPP;
asm volatile(
" .insn s,0xb2160000,%1\n"
@@ -154,7 +154,7 @@
/* Stores a format 1 aib with 64 bytes */
static inline int etr_stetr(struct etr_aib *aib)
{
- int rc = -ENOSYS;
+ int rc = -EOPNOTSUPP;
asm volatile(
" .insn s,0xb2170000,%1\n"
@@ -169,7 +169,7 @@
static inline int etr_steai(struct etr_aib *aib, unsigned int func)
{
register unsigned int reg0 asm("0") = func;
- int rc = -ENOSYS;
+ int rc = -EOPNOTSUPP;
asm volatile(
" .insn s,0xb2b30000,%1\n"
@@ -190,7 +190,7 @@
{
register unsigned int reg0 asm("0") = func;
register unsigned long reg1 asm("1") = (unsigned long) ptff_block;
- int rc = -ENOSYS;
+ int rc = -EOPNOTSUPP;
asm volatile(
" .word 0x0104\n"
diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h
index 799ed0f..2d6e6e3 100644
--- a/arch/s390/include/asm/hugetlb.h
+++ b/arch/s390/include/asm/hugetlb.h
@@ -66,16 +66,6 @@
return pte;
}
-static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep)
-{
- pte_t pte = huge_ptep_get(ptep);
-
- mm->context.flush_mm = 1;
- pmd_clear((pmd_t *) ptep);
- return pte;
-}
-
static inline void __pmd_csp(pmd_t *pmdp)
{
register unsigned long reg2 asm("2") = pmd_val(*pmdp);
@@ -117,6 +107,15 @@
__pmd_csp(pmdp);
}
+static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+ unsigned long addr, pte_t *ptep)
+{
+ pte_t pte = huge_ptep_get(ptep);
+
+ huge_ptep_invalidate(mm, addr, ptep);
+ return pte;
+}
+
#define huge_ptep_set_access_flags(__vma, __addr, __ptep, __entry, __dirty) \
({ \
int __changed = !pte_same(huge_ptep_get(__ptep), __entry); \
@@ -131,10 +130,7 @@
({ \
pte_t __pte = huge_ptep_get(__ptep); \
if (pte_write(__pte)) { \
- (__mm)->context.flush_mm = 1; \
- if (atomic_read(&(__mm)->context.attach_count) > 1 || \
- (__mm) != current->active_mm) \
- huge_ptep_invalidate(__mm, __addr, __ptep); \
+ huge_ptep_invalidate(__mm, __addr, __ptep); \
set_huge_pte_at(__mm, __addr, __ptep, \
huge_pte_wrprotect(__pte)); \
} \
diff --git a/arch/s390/include/asm/irq.h b/arch/s390/include/asm/irq.h
index 2b9d418..6703dd9 100644
--- a/arch/s390/include/asm/irq.h
+++ b/arch/s390/include/asm/irq.h
@@ -19,6 +19,7 @@
EXTINT_IUC,
EXTINT_CMS,
EXTINT_CMC,
+ EXTINT_CMR,
IOINT_CIO,
IOINT_QAI,
IOINT_DAS,
@@ -30,6 +31,7 @@
IOINT_CLW,
IOINT_CTC,
IOINT_APB,
+ IOINT_ADM,
IOINT_CSC,
NMI_NMI,
NR_IRQS,
diff --git a/arch/s390/include/asm/isc.h b/arch/s390/include/asm/isc.h
index 1420a11..5ae6064 100644
--- a/arch/s390/include/asm/isc.h
+++ b/arch/s390/include/asm/isc.h
@@ -14,6 +14,7 @@
/* Regular I/O interrupts. */
#define IO_SCH_ISC 3 /* regular I/O subchannels */
#define CONSOLE_ISC 1 /* console I/O subchannel */
+#define EADM_SCH_ISC 4 /* EADM subchannels */
#define CHSC_SCH_ISC 7 /* CHSC subchannels */
/* Adapter interrupts. */
#define QDIO_AIRQ_ISC IO_SCH_ISC /* I/O subchannel in qdio mode */
diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcore.h
index aab5555..bbf8141 100644
--- a/arch/s390/include/asm/lowcore.h
+++ b/arch/s390/include/asm/lowcore.h
@@ -329,9 +329,13 @@
__u8 pad_0x1338[0x1340-0x1338]; /* 0x1338 */
__u32 access_regs_save_area[16]; /* 0x1340 */
__u64 cregs_save_area[16]; /* 0x1380 */
+ __u8 pad_0x1400[0x1800-0x1400]; /* 0x1400 */
+
+ /* Transaction abort diagnostic block */
+ __u8 pgm_tdb[256]; /* 0x1800 */
/* align to the top of the prefix area */
- __u8 pad_0x1400[0x2000-0x1400]; /* 0x1400 */
+ __u8 pad_0x1900[0x2000-0x1900]; /* 0x1900 */
} __packed;
#endif /* CONFIG_32BIT */
diff --git a/arch/s390/include/asm/mmu_context.h b/arch/s390/include/asm/mmu_context.h
index b749c57..084e775 100644
--- a/arch/s390/include/asm/mmu_context.h
+++ b/arch/s390/include/asm/mmu_context.h
@@ -57,7 +57,7 @@
pgd_t *pgd = mm->pgd;
S390_lowcore.user_asce = mm->context.asce_bits | __pa(pgd);
- if (addressing_mode != HOME_SPACE_MODE) {
+ if (s390_user_mode != HOME_SPACE_MODE) {
/* Load primary space page table origin. */
asm volatile(LCTL_OPCODE" 1,1,%0\n"
: : "m" (S390_lowcore.user_asce) );
diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h
index 6537e72..86fe0ee 100644
--- a/arch/s390/include/asm/percpu.h
+++ b/arch/s390/include/asm/percpu.h
@@ -20,7 +20,7 @@
#endif
#define arch_this_cpu_to_op(pcp, val, op) \
-do { \
+({ \
typedef typeof(pcp) pcp_op_T__; \
pcp_op_T__ old__, new__, prev__; \
pcp_op_T__ *ptr__; \
@@ -39,13 +39,19 @@
} \
} while (prev__ != old__); \
preempt_enable(); \
-} while (0)
+ new__; \
+})
#define this_cpu_add_1(pcp, val) arch_this_cpu_to_op(pcp, val, +)
#define this_cpu_add_2(pcp, val) arch_this_cpu_to_op(pcp, val, +)
#define this_cpu_add_4(pcp, val) arch_this_cpu_to_op(pcp, val, +)
#define this_cpu_add_8(pcp, val) arch_this_cpu_to_op(pcp, val, +)
+#define this_cpu_add_return_1(pcp, val) arch_this_cpu_to_op(pcp, val, +)
+#define this_cpu_add_return_2(pcp, val) arch_this_cpu_to_op(pcp, val, +)
+#define this_cpu_add_return_4(pcp, val) arch_this_cpu_to_op(pcp, val, +)
+#define this_cpu_add_return_8(pcp, val) arch_this_cpu_to_op(pcp, val, +)
+
#define this_cpu_and_1(pcp, val) arch_this_cpu_to_op(pcp, val, &)
#define this_cpu_and_2(pcp, val) arch_this_cpu_to_op(pcp, val, &)
#define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, &)
@@ -61,7 +67,7 @@
#define this_cpu_xor_4(pcp, val) arch_this_cpu_to_op(pcp, val, ^)
#define this_cpu_xor_8(pcp, val) arch_this_cpu_to_op(pcp, val, ^)
-#define arch_this_cpu_cmpxchg(pcp, oval, nval) \
+#define arch_this_cpu_cmpxchg(pcp, oval, nval) \
({ \
typedef typeof(pcp) pcp_op_T__; \
pcp_op_T__ ret__; \
@@ -84,6 +90,44 @@
#define this_cpu_cmpxchg_4(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, oval, nval)
#define this_cpu_cmpxchg_8(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, oval, nval)
+#define arch_this_cpu_xchg(pcp, nval) \
+({ \
+ typeof(pcp) *ptr__; \
+ typeof(pcp) ret__; \
+ preempt_disable(); \
+ ptr__ = __this_cpu_ptr(&(pcp)); \
+ ret__ = xchg(ptr__, nval); \
+ preempt_enable(); \
+ ret__; \
+})
+
+#define this_cpu_xchg_1(pcp, nval) arch_this_cpu_xchg(pcp, nval)
+#define this_cpu_xchg_2(pcp, nval) arch_this_cpu_xchg(pcp, nval)
+#define this_cpu_xchg_4(pcp, nval) arch_this_cpu_xchg(pcp, nval)
+#ifdef CONFIG_64BIT
+#define this_cpu_xchg_8(pcp, nval) arch_this_cpu_xchg(pcp, nval)
+#endif
+
+#define arch_this_cpu_cmpxchg_double(pcp1, pcp2, o1, o2, n1, n2) \
+({ \
+ typeof(pcp1) o1__ = (o1), n1__ = (n1); \
+ typeof(pcp2) o2__ = (o2), n2__ = (n2); \
+ typeof(pcp1) *p1__; \
+ typeof(pcp2) *p2__; \
+ int ret__; \
+ preempt_disable(); \
+ p1__ = __this_cpu_ptr(&(pcp1)); \
+ p2__ = __this_cpu_ptr(&(pcp2)); \
+ ret__ = __cmpxchg_double(p1__, p2__, o1__, o2__, n1__, n2__); \
+ preempt_enable(); \
+ ret__; \
+})
+
+#define this_cpu_cmpxchg_double_4 arch_this_cpu_cmpxchg_double
+#ifdef CONFIG_64BIT
+#define this_cpu_cmpxchg_double_8 arch_this_cpu_cmpxchg_double
+#endif
+
#include <asm-generic/percpu.h>
#endif /* __ARCH_S390_PERCPU__ */
diff --git a/arch/s390/include/asm/posix_types.h b/arch/s390/include/asm/posix_types.h
index 7bcc14e..bf2a2ad 100644
--- a/arch/s390/include/asm/posix_types.h
+++ b/arch/s390/include/asm/posix_types.h
@@ -13,6 +13,7 @@
*/
typedef unsigned long __kernel_size_t;
+typedef long __kernel_ssize_t;
#define __kernel_size_t __kernel_size_t
typedef unsigned short __kernel_old_dev_t;
@@ -25,7 +26,6 @@
typedef unsigned short __kernel_ipc_pid_t;
typedef unsigned short __kernel_uid_t;
typedef unsigned short __kernel_gid_t;
-typedef int __kernel_ssize_t;
typedef int __kernel_ptrdiff_t;
#else /* __s390x__ */
@@ -35,7 +35,6 @@
typedef int __kernel_ipc_pid_t;
typedef unsigned int __kernel_uid_t;
typedef unsigned int __kernel_gid_t;
-typedef long __kernel_ssize_t;
typedef long __kernel_ptrdiff_t;
typedef unsigned long __kernel_sigset_t; /* at least 32 bits */
diff --git a/arch/s390/include/asm/processor.h b/arch/s390/include/asm/processor.h
index 11e4e32..f3e0aab 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -11,12 +11,15 @@
#ifndef __ASM_S390_PROCESSOR_H
#define __ASM_S390_PROCESSOR_H
+#ifndef __ASSEMBLY__
+
#include <linux/linkage.h>
#include <linux/irqflags.h>
#include <asm/cpu.h>
#include <asm/page.h>
#include <asm/ptrace.h>
#include <asm/setup.h>
+#include <asm/runtime_instr.h>
/*
* Default implementation of macro that returns current
@@ -75,11 +78,20 @@
unsigned long gmap_addr; /* address of last gmap fault. */
struct per_regs per_user; /* User specified PER registers */
struct per_event per_event; /* Cause of the last PER trap */
+ unsigned long per_flags; /* Flags to control debug behavior */
/* pfault_wait is used to block the process on a pfault event */
unsigned long pfault_wait;
struct list_head list;
+ /* cpu runtime instrumentation */
+ struct runtime_instr_cb *ri_cb;
+ int ri_signum;
+#ifdef CONFIG_64BIT
+ unsigned char trap_tdb[256]; /* Transaction abort diagnose block */
+#endif
};
+#define PER_FLAG_NO_TE 1UL /* Flag to disable transactions. */
+
typedef struct thread_struct thread_struct;
/*
@@ -130,6 +142,12 @@
struct mm_struct;
struct seq_file;
+#ifdef CONFIG_64BIT
+extern void show_cacheinfo(struct seq_file *m);
+#else
+static inline void show_cacheinfo(struct seq_file *m) { }
+#endif
+
/* Free all resources held by a thread. */
extern void release_thread(struct task_struct *);
extern int kernel_thread(int (*fn)(void *), void * arg, unsigned long flags);
@@ -140,6 +158,7 @@
extern unsigned long thread_saved_pc(struct task_struct *t);
extern void show_code(struct pt_regs *regs);
+extern void print_fn_code(unsigned char *code, unsigned long len);
unsigned long get_wchan(struct task_struct *p);
#define task_pt_regs(tsk) ((struct pt_regs *) \
@@ -331,23 +350,6 @@
#define ARCH_LOW_ADDRESS_LIMIT 0x7fffffffUL
-/*
- * Helper macro for exception table entries
- */
-#ifndef CONFIG_64BIT
-#define EX_TABLE(_fault,_target) \
- ".section __ex_table,\"a\"\n" \
- " .align 4\n" \
- " .long " #_fault "," #_target "\n" \
- ".previous\n"
-#else
-#define EX_TABLE(_fault,_target) \
- ".section __ex_table,\"a\"\n" \
- " .align 8\n" \
- " .quad " #_fault "," #_target "\n" \
- ".previous\n"
-#endif
-
extern int memcpy_real(void *, void *, size_t);
extern void memcpy_absolute(void *, void *, size_t);
@@ -358,4 +360,25 @@
memcpy_absolute(&(dest), &__tmp, sizeof(__tmp)); \
}
-#endif /* __ASM_S390_PROCESSOR_H */
+/*
+ * Helper macro for exception table entries
+ */
+#define EX_TABLE(_fault, _target) \
+ ".section __ex_table,\"a\"\n" \
+ ".align 4\n" \
+ ".long (" #_fault ") - .\n" \
+ ".long (" #_target ") - .\n" \
+ ".previous\n"
+
+#else /* __ASSEMBLY__ */
+
+#define EX_TABLE(_fault, _target) \
+ .section __ex_table,"a" ; \
+ .align 4 ; \
+ .long (_fault) - . ; \
+ .long (_target) - . ; \
+ .previous
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_S390_PROCESSOR_H */
diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h
index d5f08ea..ce20a53 100644
--- a/arch/s390/include/asm/ptrace.h
+++ b/arch/s390/include/asm/ptrace.h
@@ -235,6 +235,7 @@
#define PSW_MASK_ASC 0x0000C000UL
#define PSW_MASK_CC 0x00003000UL
#define PSW_MASK_PM 0x00000F00UL
+#define PSW_MASK_RI 0x00000000UL
#define PSW_MASK_EA 0x00000000UL
#define PSW_MASK_BA 0x00000000UL
@@ -264,10 +265,11 @@
#define PSW_MASK_ASC 0x0000C00000000000UL
#define PSW_MASK_CC 0x0000300000000000UL
#define PSW_MASK_PM 0x00000F0000000000UL
+#define PSW_MASK_RI 0x0000008000000000UL
#define PSW_MASK_EA 0x0000000100000000UL
#define PSW_MASK_BA 0x0000000080000000UL
-#define PSW_MASK_USER 0x00003F0180000000UL
+#define PSW_MASK_USER 0x00003F8180000000UL
#define PSW_ADDR_AMODE 0x0000000000000000UL
#define PSW_ADDR_INSN 0xFFFFFFFFFFFFFFFFUL
@@ -359,17 +361,19 @@
unsigned char access_id; /* PER trap access identification */
};
-#define PER_EVENT_MASK 0xE9000000UL
+#define PER_EVENT_MASK 0xEB000000UL
#define PER_EVENT_BRANCH 0x80000000UL
#define PER_EVENT_IFETCH 0x40000000UL
#define PER_EVENT_STORE 0x20000000UL
#define PER_EVENT_STORE_REAL 0x08000000UL
+#define PER_EVENT_TRANSACTION_END 0x02000000UL
#define PER_EVENT_NULLIFICATION 0x01000000UL
-#define PER_CONTROL_MASK 0x00a00000UL
+#define PER_CONTROL_MASK 0x00e00000UL
#define PER_CONTROL_BRANCH_ADDRESS 0x00800000UL
+#define PER_CONTROL_SUSPENSION 0x00400000UL
#define PER_CONTROL_ALTERATION 0x00200000UL
#endif
@@ -483,6 +487,8 @@
#define PTRACE_GET_LAST_BREAK 0x5006
#define PTRACE_PEEK_SYSTEM_CALL 0x5007
#define PTRACE_POKE_SYSTEM_CALL 0x5008
+#define PTRACE_ENABLE_TE 0x5009
+#define PTRACE_DISABLE_TE 0x5010
/*
* PT_PROT definition is loosely based on hppa bsd definition in
diff --git a/arch/s390/include/asm/runtime_instr.h b/arch/s390/include/asm/runtime_instr.h
new file mode 100644
index 0000000..830da73
--- /dev/null
+++ b/arch/s390/include/asm/runtime_instr.h
@@ -0,0 +1,98 @@
+#ifndef _RUNTIME_INSTR_H
+#define _RUNTIME_INSTR_H
+
+#define S390_RUNTIME_INSTR_START 0x1
+#define S390_RUNTIME_INSTR_STOP 0x2
+
+struct runtime_instr_cb {
+ __u64 buf_current;
+ __u64 buf_origin;
+ __u64 buf_limit;
+
+ __u32 valid : 1;
+ __u32 pstate : 1;
+ __u32 pstate_set_buf : 1;
+ __u32 home_space : 1;
+ __u32 altered : 1;
+ __u32 : 3;
+ __u32 pstate_sample : 1;
+ __u32 sstate_sample : 1;
+ __u32 pstate_collect : 1;
+ __u32 sstate_collect : 1;
+ __u32 : 1;
+ __u32 halted_int : 1;
+ __u32 int_requested : 1;
+ __u32 buffer_full_int : 1;
+ __u32 key : 4;
+ __u32 : 9;
+ __u32 rgs : 3;
+
+ __u32 mode : 4;
+ __u32 next : 1;
+ __u32 mae : 1;
+ __u32 : 2;
+ __u32 call_type_br : 1;
+ __u32 return_type_br : 1;
+ __u32 other_type_br : 1;
+ __u32 bc_other_type : 1;
+ __u32 emit : 1;
+ __u32 tx_abort : 1;
+ __u32 : 2;
+ __u32 bp_xn : 1;
+ __u32 bp_xt : 1;
+ __u32 bp_ti : 1;
+ __u32 bp_ni : 1;
+ __u32 suppr_y : 1;
+ __u32 suppr_z : 1;
+
+ __u32 dc_miss_extra : 1;
+ __u32 lat_lev_ignore : 1;
+ __u32 ic_lat_lev : 4;
+ __u32 dc_lat_lev : 4;
+
+ __u64 reserved1;
+ __u64 scaling_factor;
+ __u64 rsic;
+ __u64 reserved2;
+} __packed __aligned(8);
+
+extern struct runtime_instr_cb runtime_instr_empty_cb;
+
+static inline void load_runtime_instr_cb(struct runtime_instr_cb *cb)
+{
+ asm volatile(".insn rsy,0xeb0000000060,0,0,%0" /* LRIC */
+ : : "Q" (*cb));
+}
+
+static inline void store_runtime_instr_cb(struct runtime_instr_cb *cb)
+{
+ asm volatile(".insn rsy,0xeb0000000061,0,0,%0" /* STRIC */
+ : "=Q" (*cb) : : "cc");
+}
+
+static inline void save_ri_cb(struct runtime_instr_cb *cb_prev)
+{
+#ifdef CONFIG_64BIT
+ if (cb_prev)
+ store_runtime_instr_cb(cb_prev);
+#endif
+}
+
+static inline void restore_ri_cb(struct runtime_instr_cb *cb_next,
+ struct runtime_instr_cb *cb_prev)
+{
+#ifdef CONFIG_64BIT
+ if (cb_next)
+ load_runtime_instr_cb(cb_next);
+ else if (cb_prev)
+ load_runtime_instr_cb(&runtime_instr_empty_cb);
+#endif
+}
+
+#ifdef CONFIG_64BIT
+extern void exit_thread_runtime_instr(void);
+#else
+static inline void exit_thread_runtime_instr(void) { }
+#endif
+
+#endif /* _RUNTIME_INSTR_H */
diff --git a/arch/s390/include/asm/scsw.h b/arch/s390/include/asm/scsw.h
index 4071d00..4af99cd 100644
--- a/arch/s390/include/asm/scsw.h
+++ b/arch/s390/include/asm/scsw.h
@@ -1,7 +1,7 @@
/*
* Helper functions for scsw access.
*
- * Copyright IBM Corp. 2008, 2009
+ * Copyright IBM Corp. 2008, 2012
* Author(s): Peter Oberparleiter <peter.oberparleiter@de.ibm.com>
*/
@@ -9,7 +9,7 @@
#define _ASM_S390_SCSW_H_
#include <linux/types.h>
-#include <asm/chsc.h>
+#include <asm/css_chars.h>
#include <asm/cio.h>
/**
@@ -100,14 +100,46 @@
} __attribute__ ((packed));
/**
+ * struct eadm_scsw - subchannel status word for eadm subchannels
+ * @key: subchannel key
+ * @eswf: esw format
+ * @cc: deferred condition code
+ * @ectl: extended control
+ * @fctl: function control
+ * @actl: activity control
+ * @stctl: status control
+ * @aob: AOB address
+ * @dstat: device status
+ * @cstat: subchannel status
+ */
+struct eadm_scsw {
+ u32 key:4;
+ u32:1;
+ u32 eswf:1;
+ u32 cc:2;
+ u32:6;
+ u32 ectl:1;
+ u32:2;
+ u32 fctl:3;
+ u32 actl:7;
+ u32 stctl:5;
+ u32 aob;
+ u32 dstat:8;
+ u32 cstat:8;
+ u32:16;
+} __packed;
+
+/**
* union scsw - subchannel status word
* @cmd: command-mode SCSW
* @tm: transport-mode SCSW
+ * @eadm: eadm SCSW
*/
union scsw {
struct cmd_scsw cmd;
struct tm_scsw tm;
-} __attribute__ ((packed));
+ struct eadm_scsw eadm;
+} __packed;
#define SCSW_FCTL_CLEAR_FUNC 0x1
#define SCSW_FCTL_HALT_FUNC 0x2
diff --git a/arch/s390/include/asm/setup.h b/arch/s390/include/asm/setup.h
index e6859d1..87b47ca 100644
--- a/arch/s390/include/asm/setup.h
+++ b/arch/s390/include/asm/setup.h
@@ -60,7 +60,7 @@
#define SECONDARY_SPACE_MODE 2
#define HOME_SPACE_MODE 3
-extern unsigned int addressing_mode;
+extern unsigned int s390_user_mode;
/*
* Machine features detected in head.S
@@ -80,6 +80,7 @@
#define MACHINE_FLAG_LPAR (1UL << 12)
#define MACHINE_FLAG_SPP (1UL << 13)
#define MACHINE_FLAG_TOPOLOGY (1UL << 14)
+#define MACHINE_FLAG_TE (1UL << 15)
#define MACHINE_IS_VM (S390_lowcore.machine_flags & MACHINE_FLAG_VM)
#define MACHINE_IS_KVM (S390_lowcore.machine_flags & MACHINE_FLAG_KVM)
@@ -98,6 +99,7 @@
#define MACHINE_HAS_PFMF (0)
#define MACHINE_HAS_SPP (0)
#define MACHINE_HAS_TOPOLOGY (0)
+#define MACHINE_HAS_TE (0)
#else /* CONFIG_64BIT */
#define MACHINE_HAS_IEEE (1)
#define MACHINE_HAS_CSP (1)
@@ -109,6 +111,7 @@
#define MACHINE_HAS_PFMF (S390_lowcore.machine_flags & MACHINE_FLAG_PFMF)
#define MACHINE_HAS_SPP (S390_lowcore.machine_flags & MACHINE_FLAG_SPP)
#define MACHINE_HAS_TOPOLOGY (S390_lowcore.machine_flags & MACHINE_FLAG_TOPOLOGY)
+#define MACHINE_HAS_TE (S390_lowcore.machine_flags & MACHINE_FLAG_TE)
#endif /* CONFIG_64BIT */
#define ZFCPDUMP_HSA_SIZE (32UL<<20)
diff --git a/arch/s390/include/asm/smp.h b/arch/s390/include/asm/smp.h
index a0a8340..b64f15c 100644
--- a/arch/s390/include/asm/smp.h
+++ b/arch/s390/include/asm/smp.h
@@ -30,6 +30,8 @@
extern void smp_yield_cpu(int cpu);
extern void smp_yield(void);
extern void smp_stop_cpu(void);
+extern void smp_cpu_set_polarization(int cpu, int val);
+extern int smp_cpu_get_polarization(int cpu);
#else /* CONFIG_SMP */
@@ -43,7 +45,8 @@
func(data);
}
-static inline int smp_find_processor_id(int address) { return 0; }
+static inline int smp_find_processor_id(u16 address) { return 0; }
+static inline int smp_store_status(int cpu) { return 0; }
static inline int smp_vcpu_scheduled(int cpu) { return 1; }
static inline void smp_yield_cpu(int cpu) { }
static inline void smp_yield(void) { }
diff --git a/arch/s390/include/asm/string.h b/arch/s390/include/asm/string.h
index 1bd1352..7e2dcd7 100644
--- a/arch/s390/include/asm/string.h
+++ b/arch/s390/include/asm/string.h
@@ -96,7 +96,6 @@
static inline char *strcpy(char *dst, const char *src)
{
-#if __GNUC__ < 4
register int r0 asm("0") = 0;
char *ret = dst;
@@ -106,14 +105,10 @@
: "+&a" (dst), "+&a" (src) : "d" (r0)
: "cc", "memory");
return ret;
-#else
- return __builtin_strcpy(dst, src);
-#endif
}
static inline size_t strlen(const char *s)
{
-#if __GNUC__ < 4
register unsigned long r0 asm("0") = 0;
const char *tmp = s;
@@ -122,9 +117,6 @@
" jo 0b"
: "+d" (r0), "+a" (tmp) : : "cc");
return r0 - (unsigned long) s;
-#else
- return __builtin_strlen(s);
-#endif
}
static inline size_t strnlen(const char * s, size_t n)
diff --git a/arch/s390/include/asm/switch_to.h b/arch/s390/include/asm/switch_to.h
index f223068..f3a9e0f 100644
--- a/arch/s390/include/asm/switch_to.h
+++ b/arch/s390/include/asm/switch_to.h
@@ -80,21 +80,19 @@
if (prev->mm) { \
save_fp_regs(&prev->thread.fp_regs); \
save_access_regs(&prev->thread.acrs[0]); \
+ save_ri_cb(prev->thread.ri_cb); \
} \
if (next->mm) { \
restore_fp_regs(&next->thread.fp_regs); \
restore_access_regs(&next->thread.acrs[0]); \
+ restore_ri_cb(next->thread.ri_cb, prev->thread.ri_cb); \
update_per_regs(next); \
} \
prev = __switch_to(prev,next); \
} while (0)
-extern void account_vtime(struct task_struct *, struct task_struct *);
-extern void account_tick_vtime(struct task_struct *);
-
#define finish_arch_switch(prev) do { \
set_fs(current->thread.mm_segment); \
- account_vtime(prev, current); \
} while (0)
#endif /* __ASM_SWITCH_TO_H */
diff --git a/arch/s390/include/asm/sysinfo.h b/arch/s390/include/asm/sysinfo.h
index 282ee36..f92428e 100644
--- a/arch/s390/include/asm/sysinfo.h
+++ b/arch/s390/include/asm/sysinfo.h
@@ -17,7 +17,10 @@
#include <asm/bitsperlong.h>
struct sysinfo_1_1_1 {
- unsigned short :16;
+ unsigned char p:1;
+ unsigned char :6;
+ unsigned char t:1;
+ unsigned char :8;
unsigned char ccr;
unsigned char cai;
char reserved_0[28];
@@ -30,9 +33,14 @@
char model[16];
char model_perm_cap[16];
char model_temp_cap[16];
- char model_cap_rating[4];
- char model_perm_cap_rating[4];
- char model_temp_cap_rating[4];
+ unsigned int model_cap_rating;
+ unsigned int model_perm_cap_rating;
+ unsigned int model_temp_cap_rating;
+ unsigned char typepct[5];
+ unsigned char reserved_2[3];
+ unsigned int ncr;
+ unsigned int npr;
+ unsigned int ntr;
};
struct sysinfo_1_2_1 {
@@ -47,8 +55,9 @@
char format;
char reserved_0[1];
unsigned short acc_offset;
- char reserved_1[24];
- unsigned int secondary_capability;
+ char reserved_1[20];
+ unsigned int nominal_cap;
+ unsigned int secondary_cap;
unsigned int capability;
unsigned short cpus_total;
unsigned short cpus_configured;
@@ -109,6 +118,8 @@
char reserved_544[3552];
};
+extern int topology_max_mnest;
+
#define TOPOLOGY_CPU_BITS 64
#define TOPOLOGY_NR_MAG 6
@@ -142,21 +153,7 @@
union topology_entry tle[0];
};
-static inline int stsi(void *sysinfo, int fc, int sel1, int sel2)
-{
- register int r0 asm("0") = (fc << 28) | sel1;
- register int r1 asm("1") = sel2;
-
- asm volatile(
- " stsi 0(%2)\n"
- "0: jz 2f\n"
- "1: lhi %0,%3\n"
- "2:\n"
- EX_TABLE(0b, 1b)
- : "+d" (r0) : "d" (r1), "a" (sysinfo), "K" (-ENOSYS)
- : "cc", "memory");
- return r0;
-}
+int stsi(void *sysinfo, int fc, int sel1, int sel2);
/*
* Service level reporting interface.
diff --git a/arch/s390/include/asm/tlbflush.h b/arch/s390/include/asm/tlbflush.h
index 9fde315..1d8fe2b 100644
--- a/arch/s390/include/asm/tlbflush.h
+++ b/arch/s390/include/asm/tlbflush.h
@@ -90,12 +90,10 @@
static inline void __tlb_flush_mm_cond(struct mm_struct * mm)
{
- spin_lock(&mm->page_table_lock);
if (mm->context.flush_mm) {
__tlb_flush_mm(mm);
mm->context.flush_mm = 0;
}
- spin_unlock(&mm->page_table_lock);
}
/*
diff --git a/arch/s390/include/asm/topology.h b/arch/s390/include/asm/topology.h
index 0837de8..9ca3053 100644
--- a/arch/s390/include/asm/topology.h
+++ b/arch/s390/include/asm/topology.h
@@ -2,8 +2,8 @@
#define _ASM_S390_TOPOLOGY_H
#include <linux/cpumask.h>
-#include <asm/sysinfo.h>
+struct sysinfo_15_1_x;
struct cpu;
#ifdef CONFIG_SCHED_BOOK
@@ -51,24 +51,6 @@
#define POLARIZATION_VM (2)
#define POLARIZATION_VH (3)
-extern int cpu_polarization[];
-
-static inline void cpu_set_polarization(int cpu, int val)
-{
-#ifdef CONFIG_SCHED_BOOK
- cpu_polarization[cpu] = val;
-#endif
-}
-
-static inline int cpu_read_polarization(int cpu)
-{
-#ifdef CONFIG_SCHED_BOOK
- return cpu_polarization[cpu];
-#else
- return POLARIZATION_HRZ;
-#endif
-}
-
#ifdef CONFIG_SCHED_BOOK
void s390_init_cpu_topology(void);
#else
diff --git a/arch/s390/include/asm/uaccess.h b/arch/s390/include/asm/uaccess.h
index a8ab18b..34268df 100644
--- a/arch/s390/include/asm/uaccess.h
+++ b/arch/s390/include/asm/uaccess.h
@@ -76,9 +76,22 @@
struct exception_table_entry
{
- unsigned long insn, fixup;
+ int insn, fixup;
};
+static inline unsigned long extable_insn(const struct exception_table_entry *x)
+{
+ return (unsigned long)&x->insn + x->insn;
+}
+
+static inline unsigned long extable_fixup(const struct exception_table_entry *x)
+{
+ return (unsigned long)&x->fixup + x->fixup;
+}
+
+#define ARCH_HAS_SORT_EXTABLE
+#define ARCH_HAS_SEARCH_EXTABLE
+
struct uaccess_ops {
size_t (*copy_from_user)(size_t, const void __user *, void *);
size_t (*copy_from_user_small)(size_t, const void __user *, void *);
diff --git a/arch/s390/include/asm/unistd.h b/arch/s390/include/asm/unistd.h
index 6756e78..4e64b5c 100644
--- a/arch/s390/include/asm/unistd.h
+++ b/arch/s390/include/asm/unistd.h
@@ -277,7 +277,9 @@
#define __NR_setns 339
#define __NR_process_vm_readv 340
#define __NR_process_vm_writev 341
-#define NR_syscalls 342
+#define __NR_s390_runtime_instr 342
+#define __NR_kcmp 343
+#define NR_syscalls 344
/*
* There are some system calls that are not present on 64 bit, some
diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
index 9733b3f..4da52fe 100644
--- a/arch/s390/kernel/Makefile
+++ b/arch/s390/kernel/Makefile
@@ -23,10 +23,11 @@
obj-y := bitmap.o traps.o time.o process.o base.o early.o setup.o vtime.o \
processor.o sys_s390.o ptrace.o signal.o cpcmd.o ebcdic.o nmi.o \
debug.o irq.o ipl.o dis.o diag.o mem_detect.o sclp.o vdso.o \
- sysinfo.o jump_label.o lgr.o os_info.o
+ sysinfo.o jump_label.o lgr.o os_info.o machine_kexec.o
obj-y += $(if $(CONFIG_64BIT),entry64.o,entry.o)
obj-y += $(if $(CONFIG_64BIT),reipl64.o,reipl.o)
+obj-y += $(if $(CONFIG_64BIT),relocate_kernel64.o,relocate_kernel.o)
extra-y += head.o vmlinux.lds
extra-y += $(if $(CONFIG_64BIT),head64.o,head31.o)
@@ -48,12 +49,11 @@
obj-$(CONFIG_FUNCTION_GRAPH_TRACER) += ftrace.o
obj-$(CONFIG_FTRACE_SYSCALLS) += ftrace.o
obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
-obj-$(CONFIG_PERF_EVENTS) += perf_event.o perf_cpum_cf.o
-# Kexec part
-S390_KEXEC_OBJS := machine_kexec.o crash.o
-S390_KEXEC_OBJS += $(if $(CONFIG_64BIT),relocate_kernel64.o,relocate_kernel.o)
-obj-$(CONFIG_KEXEC) += $(S390_KEXEC_OBJS)
+ifdef CONFIG_64BIT
+obj-$(CONFIG_PERF_EVENTS) += perf_event.o perf_cpum_cf.o
+obj-y += runtime_instr.o cache.o
+endif
# vdso
obj-$(CONFIG_64BIT) += vdso64/
diff --git a/arch/s390/kernel/asm-offsets.c b/arch/s390/kernel/asm-offsets.c
index 45ef1a7..fface87 100644
--- a/arch/s390/kernel/asm-offsets.c
+++ b/arch/s390/kernel/asm-offsets.c
@@ -157,6 +157,8 @@
DEFINE(__LC_LAST_BREAK, offsetof(struct _lowcore, breaking_event_addr));
DEFINE(__LC_VDSO_PER_CPU, offsetof(struct _lowcore, vdso_per_cpu_data));
DEFINE(__LC_GMAP, offsetof(struct _lowcore, gmap));
+ DEFINE(__LC_PGM_TDB, offsetof(struct _lowcore, pgm_tdb));
+ DEFINE(__THREAD_trap_tdb, offsetof(struct task_struct, thread.trap_tdb));
DEFINE(__GMAP_ASCE, offsetof(struct gmap, asce));
#endif /* CONFIG_32BIT */
return 0;
diff --git a/arch/s390/kernel/cache.c b/arch/s390/kernel/cache.c
new file mode 100644
index 0000000..8df8d8a1
--- /dev/null
+++ b/arch/s390/kernel/cache.c
@@ -0,0 +1,385 @@
+/*
+ * Extract CPU cache information and expose them via sysfs.
+ *
+ * Copyright IBM Corp. 2012
+ * Author(s): Heiko Carstens <heiko.carstens@de.ibm.com>
+ */
+
+#include <linux/notifier.h>
+#include <linux/seq_file.h>
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/cpu.h>
+#include <asm/facility.h>
+
+struct cache {
+ unsigned long size;
+ unsigned int line_size;
+ unsigned int associativity;
+ unsigned int nr_sets;
+ unsigned int level : 3;
+ unsigned int type : 2;
+ unsigned int private : 1;
+ struct list_head list;
+};
+
+struct cache_dir {
+ struct kobject *kobj;
+ struct cache_index_dir *index;
+};
+
+struct cache_index_dir {
+ struct kobject kobj;
+ int cpu;
+ struct cache *cache;
+ struct cache_index_dir *next;
+};
+
+enum {
+ CACHE_SCOPE_NOTEXISTS,
+ CACHE_SCOPE_PRIVATE,
+ CACHE_SCOPE_SHARED,
+ CACHE_SCOPE_RESERVED,
+};
+
+enum {
+ CACHE_TYPE_SEPARATE,
+ CACHE_TYPE_DATA,
+ CACHE_TYPE_INSTRUCTION,
+ CACHE_TYPE_UNIFIED,
+};
+
+enum {
+ EXTRACT_TOPOLOGY,
+ EXTRACT_LINE_SIZE,
+ EXTRACT_SIZE,
+ EXTRACT_ASSOCIATIVITY,
+};
+
+enum {
+ CACHE_TI_UNIFIED = 0,
+ CACHE_TI_INSTRUCTION = 0,
+ CACHE_TI_DATA,
+};
+
+struct cache_info {
+ unsigned char : 4;
+ unsigned char scope : 2;
+ unsigned char type : 2;
+};
+
+#define CACHE_MAX_LEVEL 8
+
+union cache_topology {
+ struct cache_info ci[CACHE_MAX_LEVEL];
+ unsigned long long raw;
+};
+
+static const char * const cache_type_string[] = {
+ "Data",
+ "Instruction",
+ "Unified",
+};
+
+static struct cache_dir *cache_dir_cpu[NR_CPUS];
+static LIST_HEAD(cache_list);
+
+void show_cacheinfo(struct seq_file *m)
+{
+ struct cache *cache;
+ int index = 0;
+
+ list_for_each_entry(cache, &cache_list, list) {
+ seq_printf(m, "cache%-11d: ", index);
+ seq_printf(m, "level=%d ", cache->level);
+ seq_printf(m, "type=%s ", cache_type_string[cache->type]);
+ seq_printf(m, "scope=%s ", cache->private ? "Private" : "Shared");
+ seq_printf(m, "size=%luK ", cache->size >> 10);
+ seq_printf(m, "line_size=%u ", cache->line_size);
+ seq_printf(m, "associativity=%d", cache->associativity);
+ seq_puts(m, "\n");
+ index++;
+ }
+}
+
+static inline unsigned long ecag(int ai, int li, int ti)
+{
+ unsigned long cmd, val;
+
+ cmd = ai << 4 | li << 1 | ti;
+ asm volatile(".insn rsy,0xeb000000004c,%0,0,0(%1)" /* ecag */
+ : "=d" (val) : "a" (cmd));
+ return val;
+}
+
+static int __init cache_add(int level, int private, int type)
+{
+ struct cache *cache;
+ int ti;
+
+ cache = kzalloc(sizeof(*cache), GFP_KERNEL);
+ if (!cache)
+ return -ENOMEM;
+ ti = type == CACHE_TYPE_DATA ? CACHE_TI_DATA : CACHE_TI_UNIFIED;
+ cache->size = ecag(EXTRACT_SIZE, level, ti);
+ cache->line_size = ecag(EXTRACT_LINE_SIZE, level, ti);
+ cache->associativity = ecag(EXTRACT_ASSOCIATIVITY, level, ti);
+ cache->nr_sets = cache->size / cache->associativity;
+ cache->nr_sets /= cache->line_size;
+ cache->private = private;
+ cache->level = level + 1;
+ cache->type = type - 1;
+ list_add_tail(&cache->list, &cache_list);
+ return 0;
+}
+
+static void __init cache_build_info(void)
+{
+ struct cache *cache, *next;
+ union cache_topology ct;
+ int level, private, rc;
+
+ ct.raw = ecag(EXTRACT_TOPOLOGY, 0, 0);
+ for (level = 0; level < CACHE_MAX_LEVEL; level++) {
+ switch (ct.ci[level].scope) {
+ case CACHE_SCOPE_NOTEXISTS:
+ case CACHE_SCOPE_RESERVED:
+ return;
+ case CACHE_SCOPE_SHARED:
+ private = 0;
+ break;
+ case CACHE_SCOPE_PRIVATE:
+ private = 1;
+ break;
+ }
+ if (ct.ci[level].type == CACHE_TYPE_SEPARATE) {
+ rc = cache_add(level, private, CACHE_TYPE_DATA);
+ rc |= cache_add(level, private, CACHE_TYPE_INSTRUCTION);
+ } else {
+ rc = cache_add(level, private, ct.ci[level].type);
+ }
+ if (rc)
+ goto error;
+ }
+ return;
+error:
+ list_for_each_entry_safe(cache, next, &cache_list, list) {
+ list_del(&cache->list);
+ kfree(cache);
+ }
+}
+
+static struct cache_dir *__cpuinit cache_create_cache_dir(int cpu)
+{
+ struct cache_dir *cache_dir;
+ struct kobject *kobj = NULL;
+ struct device *dev;
+
+ dev = get_cpu_device(cpu);
+ if (!dev)
+ goto out;
+ kobj = kobject_create_and_add("cache", &dev->kobj);
+ if (!kobj)
+ goto out;
+ cache_dir = kzalloc(sizeof(*cache_dir), GFP_KERNEL);
+ if (!cache_dir)
+ goto out;
+ cache_dir->kobj = kobj;
+ cache_dir_cpu[cpu] = cache_dir;
+ return cache_dir;
+out:
+ kobject_put(kobj);
+ return NULL;
+}
+
+static struct cache_index_dir *kobj_to_cache_index_dir(struct kobject *kobj)
+{
+ return container_of(kobj, struct cache_index_dir, kobj);
+}
+
+static void cache_index_release(struct kobject *kobj)
+{
+ struct cache_index_dir *index;
+
+ index = kobj_to_cache_index_dir(kobj);
+ kfree(index);
+}
+
+static ssize_t cache_index_show(struct kobject *kobj,
+ struct attribute *attr, char *buf)
+{
+ struct kobj_attribute *kobj_attr;
+
+ kobj_attr = container_of(attr, struct kobj_attribute, attr);
+ return kobj_attr->show(kobj, kobj_attr, buf);
+}
+
+#define DEFINE_CACHE_ATTR(_name, _format, _value) \
+static ssize_t cache_##_name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, \
+ char *buf) \
+{ \
+ struct cache_index_dir *index; \
+ \
+ index = kobj_to_cache_index_dir(kobj); \
+ return sprintf(buf, _format, _value); \
+} \
+static struct kobj_attribute cache_##_name##_attr = \
+ __ATTR(_name, 0444, cache_##_name##_show, NULL);
+
+DEFINE_CACHE_ATTR(size, "%luK\n", index->cache->size >> 10);
+DEFINE_CACHE_ATTR(coherency_line_size, "%u\n", index->cache->line_size);
+DEFINE_CACHE_ATTR(number_of_sets, "%u\n", index->cache->nr_sets);
+DEFINE_CACHE_ATTR(ways_of_associativity, "%u\n", index->cache->associativity);
+DEFINE_CACHE_ATTR(type, "%s\n", cache_type_string[index->cache->type]);
+DEFINE_CACHE_ATTR(level, "%d\n", index->cache->level);
+
+static ssize_t shared_cpu_map_func(struct kobject *kobj, int type, char *buf)
+{
+ struct cache_index_dir *index;
+ int len;
+
+ index = kobj_to_cache_index_dir(kobj);
+ len = type ?
+ cpulist_scnprintf(buf, PAGE_SIZE - 2, cpumask_of(index->cpu)) :
+ cpumask_scnprintf(buf, PAGE_SIZE - 2, cpumask_of(index->cpu));
+ len += sprintf(&buf[len], "\n");
+ return len;
+}
+
+static ssize_t shared_cpu_map_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return shared_cpu_map_func(kobj, 0, buf);
+}
+static struct kobj_attribute cache_shared_cpu_map_attr =
+ __ATTR(shared_cpu_map, 0444, shared_cpu_map_show, NULL);
+
+static ssize_t shared_cpu_list_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ return shared_cpu_map_func(kobj, 1, buf);
+}
+static struct kobj_attribute cache_shared_cpu_list_attr =
+ __ATTR(shared_cpu_list, 0444, shared_cpu_list_show, NULL);
+
+static struct attribute *cache_index_default_attrs[] = {
+ &cache_type_attr.attr,
+ &cache_size_attr.attr,
+ &cache_number_of_sets_attr.attr,
+ &cache_ways_of_associativity_attr.attr,
+ &cache_level_attr.attr,
+ &cache_coherency_line_size_attr.attr,
+ &cache_shared_cpu_map_attr.attr,
+ &cache_shared_cpu_list_attr.attr,
+ NULL,
+};
+
+static const struct sysfs_ops cache_index_ops = {
+ .show = cache_index_show,
+};
+
+static struct kobj_type cache_index_type = {
+ .sysfs_ops = &cache_index_ops,
+ .release = cache_index_release,
+ .default_attrs = cache_index_default_attrs,
+};
+
+static int __cpuinit cache_create_index_dir(struct cache_dir *cache_dir,
+ struct cache *cache, int index,
+ int cpu)
+{
+ struct cache_index_dir *index_dir;
+ int rc;
+
+ index_dir = kzalloc(sizeof(*index_dir), GFP_KERNEL);
+ if (!index_dir)
+ return -ENOMEM;
+ index_dir->cache = cache;
+ index_dir->cpu = cpu;
+ rc = kobject_init_and_add(&index_dir->kobj, &cache_index_type,
+ cache_dir->kobj, "index%d", index);
+ if (rc)
+ goto out;
+ index_dir->next = cache_dir->index;
+ cache_dir->index = index_dir;
+ return 0;
+out:
+ kfree(index_dir);
+ return rc;
+}
+
+static int __cpuinit cache_add_cpu(int cpu)
+{
+ struct cache_dir *cache_dir;
+ struct cache *cache;
+ int rc, index = 0;
+
+ if (list_empty(&cache_list))
+ return 0;
+ cache_dir = cache_create_cache_dir(cpu);
+ if (!cache_dir)
+ return -ENOMEM;
+ list_for_each_entry(cache, &cache_list, list) {
+ if (!cache->private)
+ break;
+ rc = cache_create_index_dir(cache_dir, cache, index, cpu);
+ if (rc)
+ return rc;
+ index++;
+ }
+ return 0;
+}
+
+static void __cpuinit cache_remove_cpu(int cpu)
+{
+ struct cache_index_dir *index, *next;
+ struct cache_dir *cache_dir;
+
+ cache_dir = cache_dir_cpu[cpu];
+ if (!cache_dir)
+ return;
+ index = cache_dir->index;
+ while (index) {
+ next = index->next;
+ kobject_put(&index->kobj);
+ index = next;
+ }
+ kobject_put(cache_dir->kobj);
+ kfree(cache_dir);
+ cache_dir_cpu[cpu] = NULL;
+}
+
+static int __cpuinit cache_hotplug(struct notifier_block *nfb,
+ unsigned long action, void *hcpu)
+{
+ int cpu = (long)hcpu;
+ int rc = 0;
+
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_ONLINE:
+ rc = cache_add_cpu(cpu);
+ if (rc)
+ cache_remove_cpu(cpu);
+ break;
+ case CPU_DEAD:
+ cache_remove_cpu(cpu);
+ break;
+ }
+ return rc ? NOTIFY_BAD : NOTIFY_OK;
+}
+
+static int __init cache_init(void)
+{
+ int cpu;
+
+ if (!test_facility(34))
+ return 0;
+ cache_build_info();
+ for_each_online_cpu(cpu)
+ cache_add_cpu(cpu);
+ hotcpu_notifier(cache_hotplug, 0);
+ return 0;
+}
+device_initcall(cache_init);
diff --git a/arch/s390/kernel/compat_wrapper.S b/arch/s390/kernel/compat_wrapper.S
index 2d82cfc..3afba80 100644
--- a/arch/s390/kernel/compat_wrapper.S
+++ b/arch/s390/kernel/compat_wrapper.S
@@ -1646,3 +1646,16 @@
llgf %r0,164(%r15) # unsigned long
stg %r0,160(%r15)
jg compat_sys_process_vm_writev
+
+ENTRY(sys_s390_runtime_instr_wrapper)
+ lgfr %r2,%r2 # int
+ lgfr %r3,%r3 # int
+ jg sys_s390_runtime_instr
+
+ENTRY(sys_kcmp_wrapper)
+ lgfr %r2,%r2 # pid_t
+ lgfr %r3,%r3 # pid_t
+ lgfr %r4,%r4 # int
+ llgfr %r5,%r5 # unsigned long
+ llgfr %r6,%r6 # unsigned long
+ jg sys_kcmp
diff --git a/arch/s390/kernel/crash.c b/arch/s390/kernel/crash.c
deleted file mode 100644
index 3819153..0000000
--- a/arch/s390/kernel/crash.c
+++ /dev/null
@@ -1,14 +0,0 @@
-/*
- * Copyright IBM Corp. 2005
- *
- * Author(s): Heiko Carstens <heiko.carstens@de.ibm.com>
- *
- */
-
-#include <linux/threads.h>
-#include <linux/kexec.h>
-#include <linux/reboot.h>
-
-void machine_crash_shutdown(struct pt_regs *regs)
-{
-}
diff --git a/arch/s390/kernel/crash_dump.c b/arch/s390/kernel/crash_dump.c
index cc1172b..fb8d878 100644
--- a/arch/s390/kernel/crash_dump.c
+++ b/arch/s390/kernel/crash_dump.c
@@ -13,8 +13,9 @@
#include <linux/slab.h>
#include <linux/bootmem.h>
#include <linux/elf.h>
-#include <asm/ipl.h>
#include <asm/os_info.h>
+#include <asm/elf.h>
+#include <asm/ipl.h>
#define PTR_ADD(x, y) (((char *) (x)) + ((unsigned long) (y)))
#define PTR_SUB(x, y) (((char *) (x)) - ((unsigned long) (y)))
diff --git a/arch/s390/kernel/dis.c b/arch/s390/kernel/dis.c
index 619c5d3..cc84a24 100644
--- a/arch/s390/kernel/dis.c
+++ b/arch/s390/kernel/dis.c
@@ -315,6 +315,11 @@
LONG_INSN_POPCNT,
LONG_INSN_RISBHG,
LONG_INSN_RISBLG,
+ LONG_INSN_RINEXT,
+ LONG_INSN_RIEMIT,
+ LONG_INSN_TABORT,
+ LONG_INSN_TBEGIN,
+ LONG_INSN_TBEGINC,
};
static char *long_insn_name[] = {
@@ -329,7 +334,12 @@
[LONG_INSN_LLGHRL] = "llghrl",
[LONG_INSN_POPCNT] = "popcnt",
[LONG_INSN_RISBHG] = "risbhg",
- [LONG_INSN_RISBLG] = "risblk",
+ [LONG_INSN_RISBLG] = "risblg",
+ [LONG_INSN_RINEXT] = "rinext",
+ [LONG_INSN_RIEMIT] = "riemit",
+ [LONG_INSN_TABORT] = "tabort",
+ [LONG_INSN_TBEGIN] = "tbegin",
+ [LONG_INSN_TBEGINC] = "tbeginc",
};
static struct insn opcode[] = {
@@ -582,6 +592,17 @@
{ "", 0, INSTR_INVALID }
};
+static struct insn opcode_aa[] = {
+#ifdef CONFIG_64BIT
+ { { 0, LONG_INSN_RINEXT }, 0x00, INSTR_RI_RI },
+ { "rion", 0x01, INSTR_RI_RI },
+ { "tric", 0x02, INSTR_RI_RI },
+ { "rioff", 0x03, INSTR_RI_RI },
+ { { 0, LONG_INSN_RIEMIT }, 0x04, INSTR_RI_RI },
+#endif
+ { "", 0, INSTR_INVALID }
+};
+
static struct insn opcode_b2[] = {
#ifdef CONFIG_64BIT
{ "sske", 0x2b, INSTR_RRF_M0RR },
@@ -594,6 +615,9 @@
{ "lpswe", 0xb2, INSTR_S_RD },
{ "srnmt", 0xb9, INSTR_S_RD },
{ "lfas", 0xbd, INSTR_S_RD },
+ { "etndg", 0xec, INSTR_RRE_R0 },
+ { { 0, LONG_INSN_TABORT }, 0xfc, INSTR_S_RD },
+ { "tend", 0xf8, INSTR_S_RD },
#endif
{ "stidp", 0x02, INSTR_S_RD },
{ "sck", 0x04, INSTR_S_RD },
@@ -1150,6 +1174,7 @@
{ "stfh", 0xcb, INSTR_RXY_RRRD },
{ "chf", 0xcd, INSTR_RXY_RRRD },
{ "clhf", 0xcf, INSTR_RXY_RRRD },
+ { "ntstg", 0x25, INSTR_RXY_RRRD },
#endif
{ "lrv", 0x1e, INSTR_RXY_RRRD },
{ "lrvh", 0x1f, INSTR_RXY_RRRD },
@@ -1173,6 +1198,8 @@
{ "mvhhi", 0x44, INSTR_SIL_RDI },
{ "mvhi", 0x4c, INSTR_SIL_RDI },
{ "mvghi", 0x48, INSTR_SIL_RDI },
+ { { 0, LONG_INSN_TBEGIN }, 0x60, INSTR_SIL_RDU },
+ { { 0, LONG_INSN_TBEGINC }, 0x61, INSTR_SIL_RDU },
#endif
{ "lasp", 0x00, INSTR_SSE_RDRD },
{ "tprot", 0x01, INSTR_SSE_RDRD },
@@ -1210,6 +1237,9 @@
{ "cliy", 0x55, INSTR_SIY_URD },
{ "oiy", 0x56, INSTR_SIY_URD },
{ "xiy", 0x57, INSTR_SIY_URD },
+ { "lric", 0x60, INSTR_RSY_RDRM },
+ { "stric", 0x61, INSTR_RSY_RDRM },
+ { "mric", 0x62, INSTR_RSY_RDRM },
{ "icmh", 0x80, INSTR_RSE_RURD },
{ "icmh", 0x80, INSTR_RSY_RURD },
{ "icmy", 0x81, INSTR_RSY_RURD },
@@ -1408,6 +1438,9 @@
case 0xa7:
table = opcode_a7;
break;
+ case 0xaa:
+ table = opcode_aa;
+ break;
case 0xb2:
table = opcode_b2;
break;
@@ -1601,3 +1634,26 @@
}
printk("\n");
}
+
+void print_fn_code(unsigned char *code, unsigned long len)
+{
+ char buffer[64], *ptr;
+ int opsize, i;
+
+ while (len) {
+ ptr = buffer;
+ opsize = insn_length(*code);
+ ptr += sprintf(ptr, "%p: ", code);
+ for (i = 0; i < opsize; i++)
+ ptr += sprintf(ptr, "%02x", code[i]);
+ *ptr++ = '\t';
+ if (i < 4)
+ *ptr++ = '\t';
+ ptr += print_insn(ptr, code, (unsigned long) code);
+ *ptr++ = '\n';
+ *ptr++ = 0;
+ printk(buffer);
+ code += opsize;
+ len -= opsize;
+ }
+}
diff --git a/arch/s390/kernel/early.c b/arch/s390/kernel/early.c
index 83c3271..7f47176 100644
--- a/arch/s390/kernel/early.c
+++ b/arch/s390/kernel/early.c
@@ -215,36 +215,54 @@
PAGE_DEFAULT_KEY, 0);
}
-static __initdata struct sysinfo_3_2_2 vmms __aligned(PAGE_SIZE);
+static __initdata char sysinfo_page[PAGE_SIZE] __aligned(PAGE_SIZE);
static noinline __init void detect_machine_type(void)
{
+ struct sysinfo_3_2_2 *vmms = (struct sysinfo_3_2_2 *)&sysinfo_page;
+
/* Check current-configuration-level */
- if ((stsi(NULL, 0, 0, 0) >> 28) <= 2) {
+ if (stsi(NULL, 0, 0, 0) <= 2) {
S390_lowcore.machine_flags |= MACHINE_FLAG_LPAR;
return;
}
/* Get virtual-machine cpu information. */
- if (stsi(&vmms, 3, 2, 2) == -ENOSYS || !vmms.count)
+ if (stsi(vmms, 3, 2, 2) || !vmms->count)
return;
/* Running under KVM? If not we assume z/VM */
- if (!memcmp(vmms.vm[0].cpi, "\xd2\xe5\xd4", 3))
+ if (!memcmp(vmms->vm[0].cpi, "\xd2\xe5\xd4", 3))
S390_lowcore.machine_flags |= MACHINE_FLAG_KVM;
else
S390_lowcore.machine_flags |= MACHINE_FLAG_VM;
}
+static __init void setup_topology(void)
+{
+#ifdef CONFIG_64BIT
+ int max_mnest;
+
+ if (!test_facility(11))
+ return;
+ S390_lowcore.machine_flags |= MACHINE_FLAG_TOPOLOGY;
+ for (max_mnest = 6; max_mnest > 1; max_mnest--) {
+ if (stsi(&sysinfo_page, 15, 1, max_mnest) == 0)
+ break;
+ }
+ topology_max_mnest = max_mnest;
+#endif
+}
+
static void early_pgm_check_handler(void)
{
- unsigned long addr;
const struct exception_table_entry *fixup;
+ unsigned long addr;
addr = S390_lowcore.program_old_psw.addr;
fixup = search_exception_tables(addr & PSW_ADDR_INSN);
if (!fixup)
disabled_wait(0);
- S390_lowcore.program_old_psw.addr = fixup->fixup | PSW_ADDR_AMODE;
+ S390_lowcore.program_old_psw.addr = extable_fixup(fixup)|PSW_ADDR_AMODE;
}
static noinline __init void setup_lowcore_early(void)
@@ -267,12 +285,10 @@
static noinline __init void setup_hpage(void)
{
-#ifndef CONFIG_DEBUG_PAGEALLOC
if (!test_facility(2) || !test_facility(8))
return;
S390_lowcore.machine_flags |= MACHINE_FLAG_HPAGE;
__ctl_set_bit(0, 23);
-#endif
}
static __init void detect_mvpg(void)
@@ -366,12 +382,12 @@
S390_lowcore.machine_flags |= MACHINE_FLAG_IDTE;
if (test_facility(8))
S390_lowcore.machine_flags |= MACHINE_FLAG_PFMF;
- if (test_facility(11))
- S390_lowcore.machine_flags |= MACHINE_FLAG_TOPOLOGY;
if (test_facility(27))
S390_lowcore.machine_flags |= MACHINE_FLAG_MVCOS;
if (test_facility(40))
S390_lowcore.machine_flags |= MACHINE_FLAG_SPP;
+ if (test_facility(50) && test_facility(73))
+ S390_lowcore.machine_flags |= MACHINE_FLAG_TE;
#endif
}
@@ -441,7 +457,6 @@
append_to_cmdline(append_ipl_scpdata);
}
-
/*
* Save ipl parameters, clear bss memory, initialize storage keys
* and create a kernel NSS at startup if the SAVESYS= parm is defined
@@ -468,6 +483,7 @@
detect_diag44();
detect_machine_facilities();
setup_hpage();
+ setup_topology();
sclp_facilities_detect();
detect_memory_layout(memory_chunk);
#ifdef CONFIG_DYNAMIC_FTRACE
diff --git a/arch/s390/kernel/entry64.S b/arch/s390/kernel/entry64.S
index 349b7ee..7549985 100644
--- a/arch/s390/kernel/entry64.S
+++ b/arch/s390/kernel/entry64.S
@@ -10,6 +10,7 @@
#include <linux/init.h>
#include <linux/linkage.h>
+#include <asm/processor.h>
#include <asm/cache.h>
#include <asm/errno.h>
#include <asm/ptrace.h>
@@ -412,6 +413,11 @@
1: UPDATE_VTIME %r14,__LC_SYNC_ENTER_TIMER
LAST_BREAK %r14
lg %r15,__LC_KERNEL_STACK
+ lg %r14,__TI_task(%r12)
+ lghi %r13,__LC_PGM_TDB
+ tm __LC_PGM_ILC+2,0x02 # check for transaction abort
+ jz 2f
+ mvc __THREAD_trap_tdb(256,%r14),0(%r13)
2: aghi %r15,-(STACK_FRAME_OVERHEAD + __PT_SIZE)
la %r11,STACK_FRAME_OVERHEAD(%r15)
stmg %r0,%r7,__PT_R0(%r11)
@@ -422,13 +428,12 @@
stg %r10,__PT_ARGS(%r11)
tm __LC_PGM_ILC+3,0x80 # check for per exception
jz 0f
- lg %r1,__TI_task(%r12)
tmhh %r8,0x0001 # kernel per event ?
jz pgm_kprobe
oi __TI_flags+7(%r12),_TIF_PER_TRAP
- mvc __THREAD_per_address(8,%r1),__LC_PER_ADDRESS
- mvc __THREAD_per_cause(2,%r1),__LC_PER_CAUSE
- mvc __THREAD_per_paid(1,%r1),__LC_PER_PAID
+ mvc __THREAD_per_address(8,%r14),__LC_PER_ADDRESS
+ mvc __THREAD_per_cause(2,%r14),__LC_PER_CAUSE
+ mvc __THREAD_per_paid(1,%r14),__LC_PER_PAID
0: REENABLE_IRQS
xc __SF_BACKCHAIN(8,%r15),__SF_BACKCHAIN(%r15)
larl %r1,pgm_check_table
@@ -1004,9 +1009,7 @@
.Lhost_id:
.quad 0
- .section __ex_table,"a"
- .quad sie_loop,sie_fault
- .previous
+ EX_TABLE(sie_loop,sie_fault)
#endif
.section .rodata, "a"
diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c
index dd7630d..6cdc55b 100644
--- a/arch/s390/kernel/irq.c
+++ b/arch/s390/kernel/irq.c
@@ -30,33 +30,35 @@
};
static const struct irq_class intrclass_names[] = {
- {.name = "EXT" },
- {.name = "I/O" },
- {.name = "CLK", .desc = "[EXT] Clock Comparator" },
- {.name = "EXC", .desc = "[EXT] External Call" },
- {.name = "EMS", .desc = "[EXT] Emergency Signal" },
- {.name = "TMR", .desc = "[EXT] CPU Timer" },
- {.name = "TAL", .desc = "[EXT] Timing Alert" },
- {.name = "PFL", .desc = "[EXT] Pseudo Page Fault" },
- {.name = "DSD", .desc = "[EXT] DASD Diag" },
- {.name = "VRT", .desc = "[EXT] Virtio" },
- {.name = "SCP", .desc = "[EXT] Service Call" },
- {.name = "IUC", .desc = "[EXT] IUCV" },
- {.name = "CMS", .desc = "[EXT] CPU-Measurement: Sampling" },
- {.name = "CMC", .desc = "[EXT] CPU-Measurement: Counter" },
- {.name = "CIO", .desc = "[I/O] Common I/O Layer Interrupt" },
- {.name = "QAI", .desc = "[I/O] QDIO Adapter Interrupt" },
- {.name = "DAS", .desc = "[I/O] DASD" },
- {.name = "C15", .desc = "[I/O] 3215" },
- {.name = "C70", .desc = "[I/O] 3270" },
- {.name = "TAP", .desc = "[I/O] Tape" },
- {.name = "VMR", .desc = "[I/O] Unit Record Devices" },
- {.name = "LCS", .desc = "[I/O] LCS" },
- {.name = "CLW", .desc = "[I/O] CLAW" },
- {.name = "CTC", .desc = "[I/O] CTC" },
- {.name = "APB", .desc = "[I/O] AP Bus" },
- {.name = "CSC", .desc = "[I/O] CHSC Subchannel" },
- {.name = "NMI", .desc = "[NMI] Machine Check" },
+ [EXTERNAL_INTERRUPT] = {.name = "EXT"},
+ [IO_INTERRUPT] = {.name = "I/O"},
+ [EXTINT_CLK] = {.name = "CLK", .desc = "[EXT] Clock Comparator"},
+ [EXTINT_EXC] = {.name = "EXC", .desc = "[EXT] External Call"},
+ [EXTINT_EMS] = {.name = "EMS", .desc = "[EXT] Emergency Signal"},
+ [EXTINT_TMR] = {.name = "TMR", .desc = "[EXT] CPU Timer"},
+ [EXTINT_TLA] = {.name = "TAL", .desc = "[EXT] Timing Alert"},
+ [EXTINT_PFL] = {.name = "PFL", .desc = "[EXT] Pseudo Page Fault"},
+ [EXTINT_DSD] = {.name = "DSD", .desc = "[EXT] DASD Diag"},
+ [EXTINT_VRT] = {.name = "VRT", .desc = "[EXT] Virtio"},
+ [EXTINT_SCP] = {.name = "SCP", .desc = "[EXT] Service Call"},
+ [EXTINT_IUC] = {.name = "IUC", .desc = "[EXT] IUCV"},
+ [EXTINT_CMS] = {.name = "CMS", .desc = "[EXT] CPU-Measurement: Sampling"},
+ [EXTINT_CMC] = {.name = "CMC", .desc = "[EXT] CPU-Measurement: Counter"},
+ [EXTINT_CMR] = {.name = "CMR", .desc = "[EXT] CPU-Measurement: RI"},
+ [IOINT_CIO] = {.name = "CIO", .desc = "[I/O] Common I/O Layer Interrupt"},
+ [IOINT_QAI] = {.name = "QAI", .desc = "[I/O] QDIO Adapter Interrupt"},
+ [IOINT_DAS] = {.name = "DAS", .desc = "[I/O] DASD"},
+ [IOINT_C15] = {.name = "C15", .desc = "[I/O] 3215"},
+ [IOINT_C70] = {.name = "C70", .desc = "[I/O] 3270"},
+ [IOINT_TAP] = {.name = "TAP", .desc = "[I/O] Tape"},
+ [IOINT_VMR] = {.name = "VMR", .desc = "[I/O] Unit Record Devices"},
+ [IOINT_LCS] = {.name = "LCS", .desc = "[I/O] LCS"},
+ [IOINT_CLW] = {.name = "CLW", .desc = "[I/O] CLAW"},
+ [IOINT_CTC] = {.name = "CTC", .desc = "[I/O] CTC"},
+ [IOINT_APB] = {.name = "APB", .desc = "[I/O] AP Bus"},
+ [IOINT_ADM] = {.name = "ADM", .desc = "[I/O] EADM Subchannel"},
+ [IOINT_CSC] = {.name = "CSC", .desc = "[I/O] CHSC Subchannel"},
+ [NMI_NMI] = {.name = "NMI", .desc = "[NMI] Machine Check"},
};
/*
diff --git a/arch/s390/kernel/kprobes.c b/arch/s390/kernel/kprobes.c
index 8aa634f..d1c7214 100644
--- a/arch/s390/kernel/kprobes.c
+++ b/arch/s390/kernel/kprobes.c
@@ -547,7 +547,7 @@
*/
entry = search_exception_tables(regs->psw.addr & PSW_ADDR_INSN);
if (entry) {
- regs->psw.addr = entry->fixup | PSW_ADDR_AMODE;
+ regs->psw.addr = extable_fixup(entry) | PSW_ADDR_AMODE;
return 1;
}
diff --git a/arch/s390/kernel/lgr.c b/arch/s390/kernel/lgr.c
index eca94e7..6ea6d69 100644
--- a/arch/s390/kernel/lgr.c
+++ b/arch/s390/kernel/lgr.c
@@ -51,16 +51,6 @@
static struct debug_info *lgr_dbf;
/*
- * Return number of valid stsi levels
- */
-static inline int stsi_0(void)
-{
- int rc = stsi(NULL, 0, 0, 0);
-
- return rc == -ENOSYS ? rc : (((unsigned int) rc) >> 28);
-}
-
-/*
* Copy buffer and then convert it to ASCII
*/
static void cpascii(char *dst, char *src, int size)
@@ -76,7 +66,7 @@
{
struct sysinfo_1_1_1 *si = (void *) lgr_page;
- if (stsi(si, 1, 1, 1) == -ENOSYS)
+ if (stsi(si, 1, 1, 1))
return;
cpascii(lgr_info->manufacturer, si->manufacturer,
sizeof(si->manufacturer));
@@ -93,7 +83,7 @@
{
struct sysinfo_2_2_2 *si = (void *) lgr_page;
- if (stsi(si, 2, 2, 2) == -ENOSYS)
+ if (stsi(si, 2, 2, 2))
return;
cpascii(lgr_info->name, si->name, sizeof(si->name));
memcpy(&lgr_info->lpar_number, &si->lpar_number,
@@ -108,7 +98,7 @@
struct sysinfo_3_2_2 *si = (void *) lgr_page;
int i;
- if (stsi(si, 3, 2, 2) == -ENOSYS)
+ if (stsi(si, 3, 2, 2))
return;
for (i = 0; i < min_t(u8, si->count, VM_LEVEL_MAX); i++) {
cpascii(lgr_info->vm[i].name, si->vm[i].name,
@@ -124,16 +114,17 @@
*/
static void lgr_info_get(struct lgr_info *lgr_info)
{
+ int level;
+
memset(lgr_info, 0, sizeof(*lgr_info));
stfle(lgr_info->stfle_fac_list, ARRAY_SIZE(lgr_info->stfle_fac_list));
- lgr_info->level = stsi_0();
- if (lgr_info->level == -ENOSYS)
- return;
- if (lgr_info->level >= 1)
+ level = stsi(NULL, 0, 0, 0);
+ lgr_info->level = level;
+ if (level >= 1)
lgr_stsi_1_1_1(lgr_info);
- if (lgr_info->level >= 2)
+ if (level >= 2)
lgr_stsi_2_2_2(lgr_info);
- if (lgr_info->level >= 3)
+ if (level >= 3)
lgr_stsi_3_2_2(lgr_info);
}
diff --git a/arch/s390/kernel/machine_kexec.c b/arch/s390/kernel/machine_kexec.c
index 493304b..b3de277 100644
--- a/arch/s390/kernel/machine_kexec.c
+++ b/arch/s390/kernel/machine_kexec.c
@@ -21,6 +21,7 @@
#include <asm/reset.h>
#include <asm/ipl.h>
#include <asm/diag.h>
+#include <asm/elf.h>
#include <asm/asm-offsets.h>
#include <asm/os_info.h>
@@ -31,8 +32,6 @@
#ifdef CONFIG_CRASH_DUMP
-void *fill_cpu_elf_notes(void *ptr, struct save_area *sa);
-
/*
* Create ELF notes for one CPU
*/
@@ -159,7 +158,7 @@
/* Can't replace kernel image since it is read-only. */
if (ipl_flags & IPL_NSS_VALID)
- return -ENOSYS;
+ return -EOPNOTSUPP;
if (image->type == KEXEC_TYPE_CRASH)
return machine_kexec_prepare_kdump();
@@ -191,6 +190,10 @@
{
}
+void machine_crash_shutdown(struct pt_regs *regs)
+{
+}
+
/*
* Do normal kexec
*/
diff --git a/arch/s390/kernel/process.c b/arch/s390/kernel/process.c
index 7331753..5024be2 100644
--- a/arch/s390/kernel/process.c
+++ b/arch/s390/kernel/process.c
@@ -26,10 +26,12 @@
#include <asm/io.h>
#include <asm/processor.h>
#include <asm/vtimer.h>
+#include <asm/exec.h>
#include <asm/irq.h>
#include <asm/nmi.h>
#include <asm/smp.h>
#include <asm/switch_to.h>
+#include <asm/runtime_instr.h>
#include "entry.h"
asmlinkage void ret_from_fork(void) asm ("ret_from_fork");
@@ -132,6 +134,7 @@
*/
void exit_thread(void)
{
+ exit_thread_runtime_instr();
}
void flush_thread(void)
@@ -170,6 +173,11 @@
/* Save access registers to new thread structure. */
save_access_regs(&p->thread.acrs[0]);
+ /* Don't copy runtime instrumentation info */
+ p->thread.ri_cb = NULL;
+ p->thread.ri_signum = 0;
+ frame->childregs.psw.mask &= ~PSW_MASK_RI;
+
#ifndef CONFIG_64BIT
/*
* save fprs to current->thread.fp_regs to merge them with
diff --git a/arch/s390/kernel/processor.c b/arch/s390/kernel/processor.c
index 572d4c9..753c41d 100644
--- a/arch/s390/kernel/processor.c
+++ b/arch/s390/kernel/processor.c
@@ -39,9 +39,9 @@
*/
static int show_cpuinfo(struct seq_file *m, void *v)
{
- static const char *hwcap_str[10] = {
+ static const char *hwcap_str[] = {
"esan3", "zarch", "stfle", "msa", "ldisp", "eimm", "dfp",
- "edat", "etf3eh", "highgprs"
+ "edat", "etf3eh", "highgprs", "te"
};
unsigned long n = (unsigned long) v - 1;
int i;
@@ -54,10 +54,11 @@
num_online_cpus(), loops_per_jiffy/(500000/HZ),
(loops_per_jiffy/(5000/HZ))%100);
seq_puts(m, "features\t: ");
- for (i = 0; i < 10; i++)
+ for (i = 0; i < ARRAY_SIZE(hwcap_str); i++)
if (hwcap_str[i] && (elf_hwcap & (1UL << i)))
seq_printf(m, "%s ", hwcap_str[i]);
seq_puts(m, "\n");
+ show_cacheinfo(m);
}
get_online_cpus();
if (cpu_online(n)) {
diff --git a/arch/s390/kernel/ptrace.c b/arch/s390/kernel/ptrace.c
index e4be113..a314c57 100644
--- a/arch/s390/kernel/ptrace.c
+++ b/arch/s390/kernel/ptrace.c
@@ -42,6 +42,7 @@
REGSET_GENERAL,
REGSET_FP,
REGSET_LAST_BREAK,
+ REGSET_TDB,
REGSET_SYSTEM_CALL,
REGSET_GENERAL_EXTENDED,
};
@@ -52,6 +53,22 @@
struct thread_struct *thread = &task->thread;
struct per_regs old, new;
+#ifdef CONFIG_64BIT
+ /* Take care of the enable/disable of transactional execution. */
+ if (MACHINE_HAS_TE) {
+ unsigned long cr0, cr0_new;
+
+ __ctl_store(cr0, 0, 0);
+ /* set or clear transaction execution bits 8 and 9. */
+ if (task->thread.per_flags & PER_FLAG_NO_TE)
+ cr0_new = cr0 & ~(3UL << 54);
+ else
+ cr0_new = cr0 | (3UL << 54);
+ /* Only load control register 0 if necessary. */
+ if (cr0 != cr0_new)
+ __ctl_load(cr0_new, 0, 0);
+ }
+#endif
/* Copy user specified PER registers */
new.control = thread->per_user.control;
new.start = thread->per_user.start;
@@ -60,6 +77,10 @@
/* merge TIF_SINGLE_STEP into user specified PER registers. */
if (test_tsk_thread_flag(task, TIF_SINGLE_STEP)) {
new.control |= PER_EVENT_IFETCH;
+#ifdef CONFIG_64BIT
+ new.control |= PER_CONTROL_SUSPENSION;
+ new.control |= PER_EVENT_TRANSACTION_END;
+#endif
new.start = 0;
new.end = PSW_ADDR_INSN;
}
@@ -100,6 +121,7 @@
memset(&task->thread.per_event, 0, sizeof(task->thread.per_event));
clear_tsk_thread_flag(task, TIF_SINGLE_STEP);
clear_tsk_thread_flag(task, TIF_PER_TRAP);
+ task->thread.per_flags = 0;
}
#ifndef CONFIG_64BIT
@@ -416,6 +438,16 @@
put_user(task_thread_info(child)->last_break,
(unsigned long __user *) data);
return 0;
+ case PTRACE_ENABLE_TE:
+ if (!MACHINE_HAS_TE)
+ return -EIO;
+ child->thread.per_flags &= ~PER_FLAG_NO_TE;
+ return 0;
+ case PTRACE_DISABLE_TE:
+ if (!MACHINE_HAS_TE)
+ return -EIO;
+ child->thread.per_flags |= PER_FLAG_NO_TE;
+ return 0;
default:
/* Removing high order bit from addr (only for 31 bit). */
addr &= PSW_ADDR_INSN;
@@ -903,6 +935,28 @@
return 0;
}
+static int s390_tdb_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ struct pt_regs *regs = task_pt_regs(target);
+ unsigned char *data;
+
+ if (!(regs->int_code & 0x200))
+ return -ENODATA;
+ data = target->thread.trap_tdb;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, data, 0, 256);
+}
+
+static int s390_tdb_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ return 0;
+}
+
#endif
static int s390_system_call_get(struct task_struct *target,
@@ -951,6 +1005,14 @@
.get = s390_last_break_get,
.set = s390_last_break_set,
},
+ [REGSET_TDB] = {
+ .core_note_type = NT_S390_TDB,
+ .n = 1,
+ .size = 256,
+ .align = 1,
+ .get = s390_tdb_get,
+ .set = s390_tdb_set,
+ },
#endif
[REGSET_SYSTEM_CALL] = {
.core_note_type = NT_S390_SYSTEM_CALL,
@@ -1148,6 +1210,14 @@
.get = s390_compat_last_break_get,
.set = s390_compat_last_break_set,
},
+ [REGSET_TDB] = {
+ .core_note_type = NT_S390_TDB,
+ .n = 1,
+ .size = 256,
+ .align = 1,
+ .get = s390_tdb_get,
+ .set = s390_tdb_set,
+ },
[REGSET_SYSTEM_CALL] = {
.core_note_type = NT_S390_SYSTEM_CALL,
.n = 1,
diff --git a/arch/s390/kernel/runtime_instr.c b/arch/s390/kernel/runtime_instr.c
new file mode 100644
index 0000000..61066f6
--- /dev/null
+++ b/arch/s390/kernel/runtime_instr.c
@@ -0,0 +1,150 @@
+/*
+ * Copyright IBM Corp. 2012
+ * Author(s): Jan Glauber <jang@linux.vnet.ibm.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/syscalls.h>
+#include <linux/signal.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/errno.h>
+#include <linux/kernel_stat.h>
+#include <asm/runtime_instr.h>
+#include <asm/cpu_mf.h>
+#include <asm/irq.h>
+
+/* empty control block to disable RI by loading it */
+struct runtime_instr_cb runtime_instr_empty_cb;
+
+static int runtime_instr_avail(void)
+{
+ return test_facility(64);
+}
+
+static void disable_runtime_instr(void)
+{
+ struct pt_regs *regs = task_pt_regs(current);
+
+ load_runtime_instr_cb(&runtime_instr_empty_cb);
+
+ /*
+ * Make sure the RI bit is deleted from the PSW. If the user did not
+ * switch off RI before the system call the process will get a
+ * specification exception otherwise.
+ */
+ regs->psw.mask &= ~PSW_MASK_RI;
+}
+
+static void init_runtime_instr_cb(struct runtime_instr_cb *cb)
+{
+ cb->buf_limit = 0xfff;
+ if (s390_user_mode == HOME_SPACE_MODE)
+ cb->home_space = 1;
+ cb->int_requested = 1;
+ cb->pstate = 1;
+ cb->pstate_set_buf = 1;
+ cb->pstate_sample = 1;
+ cb->pstate_collect = 1;
+ cb->key = PAGE_DEFAULT_KEY;
+ cb->valid = 1;
+}
+
+void exit_thread_runtime_instr(void)
+{
+ struct task_struct *task = current;
+
+ if (!task->thread.ri_cb)
+ return;
+ disable_runtime_instr();
+ kfree(task->thread.ri_cb);
+ task->thread.ri_signum = 0;
+ task->thread.ri_cb = NULL;
+}
+
+static void runtime_instr_int_handler(struct ext_code ext_code,
+ unsigned int param32, unsigned long param64)
+{
+ struct siginfo info;
+
+ if (!(param32 & CPU_MF_INT_RI_MASK))
+ return;
+
+ kstat_cpu(smp_processor_id()).irqs[EXTINT_CMR]++;
+
+ if (!current->thread.ri_cb)
+ return;
+ if (current->thread.ri_signum < SIGRTMIN ||
+ current->thread.ri_signum > SIGRTMAX) {
+ WARN_ON_ONCE(1);
+ return;
+ }
+
+ memset(&info, 0, sizeof(info));
+ info.si_signo = current->thread.ri_signum;
+ info.si_code = SI_QUEUE;
+ if (param32 & CPU_MF_INT_RI_BUF_FULL)
+ info.si_int = ENOBUFS;
+ else if (param32 & CPU_MF_INT_RI_HALTED)
+ info.si_int = ECANCELED;
+ else
+ return; /* unknown reason */
+
+ send_sig_info(current->thread.ri_signum, &info, current);
+}
+
+SYSCALL_DEFINE2(s390_runtime_instr, int, command, int, signum)
+{
+ struct runtime_instr_cb *cb;
+
+ if (!runtime_instr_avail())
+ return -EOPNOTSUPP;
+
+ if (command == S390_RUNTIME_INSTR_STOP) {
+ preempt_disable();
+ exit_thread_runtime_instr();
+ preempt_enable();
+ return 0;
+ }
+
+ if (command != S390_RUNTIME_INSTR_START ||
+ (signum < SIGRTMIN || signum > SIGRTMAX))
+ return -EINVAL;
+
+ if (!current->thread.ri_cb) {
+ cb = kzalloc(sizeof(*cb), GFP_KERNEL);
+ if (!cb)
+ return -ENOMEM;
+ } else {
+ cb = current->thread.ri_cb;
+ memset(cb, 0, sizeof(*cb));
+ }
+
+ init_runtime_instr_cb(cb);
+ current->thread.ri_signum = signum;
+
+ /* now load the control block to make it available */
+ preempt_disable();
+ current->thread.ri_cb = cb;
+ load_runtime_instr_cb(cb);
+ preempt_enable();
+ return 0;
+}
+
+static int __init runtime_instr_init(void)
+{
+ int rc;
+
+ if (!runtime_instr_avail())
+ return 0;
+
+ measurement_alert_subclass_register();
+ rc = register_external_interrupt(0x1407, runtime_instr_int_handler);
+ if (rc)
+ measurement_alert_subclass_unregister();
+ else
+ pr_info("Runtime instrumentation facility initialized\n");
+ return rc;
+}
+device_initcall(runtime_instr_init);
diff --git a/arch/s390/kernel/s390_ksyms.c b/arch/s390/kernel/s390_ksyms.c
index 57b5366..9bdbcef 100644
--- a/arch/s390/kernel/s390_ksyms.c
+++ b/arch/s390/kernel/s390_ksyms.c
@@ -8,3 +8,5 @@
#if defined(CONFIG_KVM) || defined(CONFIG_KVM_MODULE)
EXPORT_SYMBOL(sie64a);
#endif
+EXPORT_SYMBOL(memcpy);
+EXPORT_SYMBOL(memset);
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index f86c81e..afa9fdb 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -302,10 +302,10 @@
}
early_param("vmalloc", parse_vmalloc);
-unsigned int addressing_mode = HOME_SPACE_MODE;
-EXPORT_SYMBOL_GPL(addressing_mode);
+unsigned int s390_user_mode = PRIMARY_SPACE_MODE;
+EXPORT_SYMBOL_GPL(s390_user_mode);
-static int set_amode_primary(void)
+static void __init set_user_mode_primary(void)
{
psw_kernel_bits = (psw_kernel_bits & ~PSW_MASK_ASC) | PSW_ASC_HOME;
psw_user_bits = (psw_user_bits & ~PSW_MASK_ASC) | PSW_ASC_PRIMARY;
@@ -313,48 +313,30 @@
psw32_user_bits =
(psw32_user_bits & ~PSW32_MASK_ASC) | PSW32_ASC_PRIMARY;
#endif
-
- if (MACHINE_HAS_MVCOS) {
- memcpy(&uaccess, &uaccess_mvcos_switch, sizeof(uaccess));
- return 1;
- } else {
- memcpy(&uaccess, &uaccess_pt, sizeof(uaccess));
- return 0;
- }
+ uaccess = MACHINE_HAS_MVCOS ? uaccess_mvcos_switch : uaccess_pt;
}
-/*
- * Switch kernel/user addressing modes?
- */
-static int __init early_parse_switch_amode(char *p)
-{
- addressing_mode = PRIMARY_SPACE_MODE;
- return 0;
-}
-early_param("switch_amode", early_parse_switch_amode);
-
static int __init early_parse_user_mode(char *p)
{
if (p && strcmp(p, "primary") == 0)
- addressing_mode = PRIMARY_SPACE_MODE;
+ s390_user_mode = PRIMARY_SPACE_MODE;
else if (!p || strcmp(p, "home") == 0)
- addressing_mode = HOME_SPACE_MODE;
+ s390_user_mode = HOME_SPACE_MODE;
else
return 1;
return 0;
}
early_param("user_mode", early_parse_user_mode);
-static void setup_addressing_mode(void)
+static void __init setup_addressing_mode(void)
{
- if (addressing_mode == PRIMARY_SPACE_MODE) {
- if (set_amode_primary())
- pr_info("Address spaces switched, "
- "mvcos available\n");
- else
- pr_info("Address spaces switched, "
- "mvcos not available\n");
- }
+ if (s390_user_mode != PRIMARY_SPACE_MODE)
+ return;
+ set_user_mode_primary();
+ if (MACHINE_HAS_MVCOS)
+ pr_info("Address spaces switched, mvcos available\n");
+ else
+ pr_info("Address spaces switched, mvcos not available\n");
}
void *restart_stack __attribute__((__section__(".data")));
@@ -602,9 +584,7 @@
static void __init setup_vmcoreinfo(void)
{
-#ifdef CONFIG_KEXEC
mem_assign_absolute(S390_lowcore.vmcore_info, paddr_vmcoreinfo_note());
-#endif
}
#ifdef CONFIG_CRASH_DUMP
@@ -974,12 +954,20 @@
if (MACHINE_HAS_HPAGE)
elf_hwcap |= HWCAP_S390_HPAGE;
+#if defined(CONFIG_64BIT)
/*
* 64-bit register support for 31-bit processes
* HWCAP_S390_HIGH_GPRS is bit 9.
*/
elf_hwcap |= HWCAP_S390_HIGH_GPRS;
+ /*
+ * Transactional execution support HWCAP_S390_TE is bit 10.
+ */
+ if (test_facility(50) && test_facility(73))
+ elf_hwcap |= HWCAP_S390_TE;
+#endif
+
get_cpu_id(&cpu_id);
switch (cpu_id.machine) {
case 0x9672:
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 720fda1..ea431e5 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -66,7 +66,7 @@
unsigned long panic_stack; /* panic stack for the cpu */
unsigned long ec_mask; /* bit mask for ec_xxx functions */
int state; /* physical cpu state */
- u32 status; /* last status received via sigp */
+ int polarization; /* physical polarization */
u16 address; /* physical cpu address */
};
@@ -74,6 +74,10 @@
static u16 boot_cpu_address;
static struct pcpu pcpu_devices[NR_CPUS];
+/*
+ * The smp_cpu_state_mutex must be held when changing the state or polarization
+ * member of a pcpu data structure within the pcpu_devices arreay.
+ */
DEFINE_MUTEX(smp_cpu_state_mutex);
/*
@@ -99,7 +103,7 @@
int cc;
while (1) {
- cc = __pcpu_sigp(addr, order, parm, status);
+ cc = __pcpu_sigp(addr, order, parm, NULL);
if (cc != SIGP_CC_BUSY)
return cc;
cpu_relax();
@@ -111,7 +115,7 @@
int cc, retry;
for (retry = 0; ; retry++) {
- cc = __pcpu_sigp(pcpu->address, order, parm, &pcpu->status);
+ cc = __pcpu_sigp(pcpu->address, order, parm, NULL);
if (cc != SIGP_CC_BUSY)
break;
if (retry >= 3)
@@ -122,16 +126,18 @@
static inline int pcpu_stopped(struct pcpu *pcpu)
{
+ u32 uninitialized_var(status);
+
if (__pcpu_sigp(pcpu->address, SIGP_SENSE,
- 0, &pcpu->status) != SIGP_CC_STATUS_STORED)
+ 0, &status) != SIGP_CC_STATUS_STORED)
return 0;
- return !!(pcpu->status & (SIGP_STATUS_CHECK_STOP|SIGP_STATUS_STOPPED));
+ return !!(status & (SIGP_STATUS_CHECK_STOP|SIGP_STATUS_STOPPED));
}
static inline int pcpu_running(struct pcpu *pcpu)
{
if (__pcpu_sigp(pcpu->address, SIGP_SENSE_RUNNING,
- 0, &pcpu->status) != SIGP_CC_STATUS_STORED)
+ 0, NULL) != SIGP_CC_STATUS_STORED)
return 1;
/* Status stored condition code is equivalent to cpu not running. */
return 0;
@@ -586,6 +592,16 @@
#endif /* CONFIG_ZFCPDUMP || CONFIG_CRASH_DUMP */
+void smp_cpu_set_polarization(int cpu, int val)
+{
+ pcpu_devices[cpu].polarization = val;
+}
+
+int smp_cpu_get_polarization(int cpu)
+{
+ return pcpu_devices[cpu].polarization;
+}
+
static struct sclp_cpu_info *smp_get_cpu_info(void)
{
static int use_sigp_detection;
@@ -628,7 +644,7 @@
pcpu->address = info->cpu[i].address;
pcpu->state = (cpu >= info->configured) ?
CPU_STATE_STANDBY : CPU_STATE_CONFIGURED;
- cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
+ smp_cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
set_cpu_present(cpu, true);
if (sysfs_add && smp_add_present_cpu(cpu) != 0)
set_cpu_present(cpu, false);
@@ -796,7 +812,7 @@
pcpu->async_stack = S390_lowcore.async_stack - ASYNC_SIZE;
pcpu->panic_stack = S390_lowcore.panic_stack - PAGE_SIZE;
S390_lowcore.percpu_offset = __per_cpu_offset[0];
- cpu_set_polarization(0, POLARIZATION_UNKNOWN);
+ smp_cpu_set_polarization(0, POLARIZATION_UNKNOWN);
set_cpu_present(0, true);
set_cpu_online(0, true);
}
@@ -862,7 +878,7 @@
if (rc)
break;
pcpu->state = CPU_STATE_STANDBY;
- cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
+ smp_cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
topology_expect_change();
break;
case 1:
@@ -872,7 +888,7 @@
if (rc)
break;
pcpu->state = CPU_STATE_CONFIGURED;
- cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
+ smp_cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
topology_expect_change();
break;
default:
@@ -959,23 +975,17 @@
struct device *s = &c->dev;
int err = 0;
- switch (action) {
+ switch (action & ~CPU_TASKS_FROZEN) {
case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
err = sysfs_create_group(&s->kobj, &cpu_online_attr_group);
break;
case CPU_DEAD:
- case CPU_DEAD_FROZEN:
sysfs_remove_group(&s->kobj, &cpu_online_attr_group);
break;
}
return notifier_from_errno(err);
}
-static struct notifier_block __cpuinitdata smp_cpu_nb = {
- .notifier_call = smp_cpu_notify,
-};
-
static int __devinit smp_add_present_cpu(int cpu)
{
struct cpu *c = &pcpu_devices[cpu].cpu;
@@ -1050,7 +1060,7 @@
{
int cpu, rc;
- register_cpu_notifier(&smp_cpu_nb);
+ hotcpu_notifier(smp_cpu_notify, 0);
#ifdef CONFIG_HOTPLUG_CPU
rc = device_create_file(cpu_subsys.dev_root, &dev_attr_rescan);
if (rc)
diff --git a/arch/s390/kernel/syscalls.S b/arch/s390/kernel/syscalls.S
index bcab2f0..4817485 100644
--- a/arch/s390/kernel/syscalls.S
+++ b/arch/s390/kernel/syscalls.S
@@ -350,3 +350,5 @@
SYSCALL(sys_setns,sys_setns,sys_setns_wrapper)
SYSCALL(sys_process_vm_readv,sys_process_vm_readv,compat_sys_process_vm_readv_wrapper) /* 340 */
SYSCALL(sys_process_vm_writev,sys_process_vm_writev,compat_sys_process_vm_writev_wrapper)
+SYSCALL(sys_ni_syscall,sys_s390_runtime_instr,sys_s390_runtime_instr_wrapper)
+SYSCALL(sys_kcmp,sys_kcmp,sys_kcmp_wrapper)
diff --git a/arch/s390/kernel/sysinfo.c b/arch/s390/kernel/sysinfo.c
index fa0eb23..62f89d9 100644
--- a/arch/s390/kernel/sysinfo.c
+++ b/arch/s390/kernel/sysinfo.c
@@ -22,17 +22,41 @@
#include <math-emu/soft-fp.h>
#include <math-emu/single.h>
-static inline int stsi_0(void)
+int topology_max_mnest;
+
+/*
+ * stsi - store system information
+ *
+ * Returns the current configuration level if function code 0 was specified.
+ * Otherwise returns 0 on success or a negative value on error.
+ */
+int stsi(void *sysinfo, int fc, int sel1, int sel2)
{
- int rc = stsi(NULL, 0, 0, 0);
- return rc == -ENOSYS ? rc : (((unsigned int) rc) >> 28);
+ register int r0 asm("0") = (fc << 28) | sel1;
+ register int r1 asm("1") = sel2;
+ int rc = 0;
+
+ asm volatile(
+ " stsi 0(%3)\n"
+ "0: jz 2f\n"
+ "1: lhi %1,%4\n"
+ "2:\n"
+ EX_TABLE(0b, 1b)
+ : "+d" (r0), "+d" (rc)
+ : "d" (r1), "a" (sysinfo), "K" (-EOPNOTSUPP)
+ : "cc", "memory");
+ if (rc)
+ return rc;
+ return fc ? 0 : ((unsigned int) r0) >> 28;
}
+EXPORT_SYMBOL(stsi);
-static int stsi_1_1_1(struct sysinfo_1_1_1 *info, char *page, int len)
+static void stsi_1_1_1(struct seq_file *m, struct sysinfo_1_1_1 *info)
{
- if (stsi(info, 1, 1, 1) == -ENOSYS)
- return len;
+ int i;
+ if (stsi(info, 1, 1, 1))
+ return;
EBCASC(info->manufacturer, sizeof(info->manufacturer));
EBCASC(info->type, sizeof(info->type));
EBCASC(info->model, sizeof(info->model));
@@ -41,242 +65,197 @@
EBCASC(info->model_capacity, sizeof(info->model_capacity));
EBCASC(info->model_perm_cap, sizeof(info->model_perm_cap));
EBCASC(info->model_temp_cap, sizeof(info->model_temp_cap));
- len += sprintf(page + len, "Manufacturer: %-16.16s\n",
- info->manufacturer);
- len += sprintf(page + len, "Type: %-4.4s\n",
- info->type);
+ seq_printf(m, "Manufacturer: %-16.16s\n", info->manufacturer);
+ seq_printf(m, "Type: %-4.4s\n", info->type);
+ /*
+ * Sigh: the model field has been renamed with System z9
+ * to model_capacity and a new model field has been added
+ * after the plant field. To avoid confusing older programs
+ * the "Model:" prints "model_capacity model" or just
+ * "model_capacity" if the model string is empty .
+ */
+ seq_printf(m, "Model: %-16.16s", info->model_capacity);
if (info->model[0] != '\0')
- /*
- * Sigh: the model field has been renamed with System z9
- * to model_capacity and a new model field has been added
- * after the plant field. To avoid confusing older programs
- * the "Model:" prints "model_capacity model" or just
- * "model_capacity" if the model string is empty .
- */
- len += sprintf(page + len,
- "Model: %-16.16s %-16.16s\n",
- info->model_capacity, info->model);
- else
- len += sprintf(page + len, "Model: %-16.16s\n",
- info->model_capacity);
- len += sprintf(page + len, "Sequence Code: %-16.16s\n",
- info->sequence);
- len += sprintf(page + len, "Plant: %-4.4s\n",
- info->plant);
- len += sprintf(page + len, "Model Capacity: %-16.16s %08u\n",
- info->model_capacity, *(u32 *) info->model_cap_rating);
- if (info->model_perm_cap[0] != '\0')
- len += sprintf(page + len,
- "Model Perm. Capacity: %-16.16s %08u\n",
- info->model_perm_cap,
- *(u32 *) info->model_perm_cap_rating);
- if (info->model_temp_cap[0] != '\0')
- len += sprintf(page + len,
- "Model Temp. Capacity: %-16.16s %08u\n",
- info->model_temp_cap,
- *(u32 *) info->model_temp_cap_rating);
+ seq_printf(m, " %-16.16s", info->model);
+ seq_putc(m, '\n');
+ seq_printf(m, "Sequence Code: %-16.16s\n", info->sequence);
+ seq_printf(m, "Plant: %-4.4s\n", info->plant);
+ seq_printf(m, "Model Capacity: %-16.16s %08u\n",
+ info->model_capacity, info->model_cap_rating);
+ if (info->model_perm_cap_rating)
+ seq_printf(m, "Model Perm. Capacity: %-16.16s %08u\n",
+ info->model_perm_cap,
+ info->model_perm_cap_rating);
+ if (info->model_temp_cap_rating)
+ seq_printf(m, "Model Temp. Capacity: %-16.16s %08u\n",
+ info->model_temp_cap,
+ info->model_temp_cap_rating);
+ if (info->ncr)
+ seq_printf(m, "Nominal Cap. Rating: %08u\n", info->ncr);
+ if (info->npr)
+ seq_printf(m, "Nominal Perm. Rating: %08u\n", info->npr);
+ if (info->ntr)
+ seq_printf(m, "Nominal Temp. Rating: %08u\n", info->ntr);
if (info->cai) {
- len += sprintf(page + len,
- "Capacity Adj. Ind.: %d\n",
- info->cai);
- len += sprintf(page + len, "Capacity Ch. Reason: %d\n",
- info->ccr);
+ seq_printf(m, "Capacity Adj. Ind.: %d\n", info->cai);
+ seq_printf(m, "Capacity Ch. Reason: %d\n", info->ccr);
+ seq_printf(m, "Capacity Transient: %d\n", info->t);
}
- return len;
+ if (info->p) {
+ for (i = 1; i <= ARRAY_SIZE(info->typepct); i++) {
+ seq_printf(m, "Type %d Percentage: %d\n",
+ i, info->typepct[i - 1]);
+ }
+ }
}
-static int stsi_15_1_x(struct sysinfo_15_1_x *info, char *page, int len)
+static void stsi_15_1_x(struct seq_file *m, struct sysinfo_15_1_x *info)
{
static int max_mnest;
int i, rc;
- len += sprintf(page + len, "\n");
+ seq_putc(m, '\n');
if (!MACHINE_HAS_TOPOLOGY)
- return len;
- if (max_mnest) {
- stsi(info, 15, 1, max_mnest);
- } else {
- for (max_mnest = 6; max_mnest > 1; max_mnest--) {
- rc = stsi(info, 15, 1, max_mnest);
- if (rc != -ENOSYS)
- break;
- }
- }
- len += sprintf(page + len, "CPU Topology HW: ");
+ return;
+ if (stsi(info, 15, 1, topology_max_mnest))
+ return;
+ seq_printf(m, "CPU Topology HW: ");
for (i = 0; i < TOPOLOGY_NR_MAG; i++)
- len += sprintf(page + len, " %d", info->mag[i]);
- len += sprintf(page + len, "\n");
+ seq_printf(m, " %d", info->mag[i]);
+ seq_putc(m, '\n');
#ifdef CONFIG_SCHED_MC
store_topology(info);
- len += sprintf(page + len, "CPU Topology SW: ");
+ seq_printf(m, "CPU Topology SW: ");
for (i = 0; i < TOPOLOGY_NR_MAG; i++)
- len += sprintf(page + len, " %d", info->mag[i]);
- len += sprintf(page + len, "\n");
+ seq_printf(m, " %d", info->mag[i]);
+ seq_putc(m, '\n');
#endif
- return len;
}
-static int stsi_1_2_2(struct sysinfo_1_2_2 *info, char *page, int len)
+static void stsi_1_2_2(struct seq_file *m, struct sysinfo_1_2_2 *info)
{
struct sysinfo_1_2_2_extension *ext;
int i;
- if (stsi(info, 1, 2, 2) == -ENOSYS)
- return len;
+ if (stsi(info, 1, 2, 2))
+ return;
ext = (struct sysinfo_1_2_2_extension *)
((unsigned long) info + info->acc_offset);
-
- len += sprintf(page + len, "CPUs Total: %d\n",
- info->cpus_total);
- len += sprintf(page + len, "CPUs Configured: %d\n",
- info->cpus_configured);
- len += sprintf(page + len, "CPUs Standby: %d\n",
- info->cpus_standby);
- len += sprintf(page + len, "CPUs Reserved: %d\n",
- info->cpus_reserved);
-
- if (info->format == 1) {
- /*
- * Sigh 2. According to the specification the alternate
- * capability field is a 32 bit floating point number
- * if the higher order 8 bits are not zero. Printing
- * a floating point number in the kernel is a no-no,
- * always print the number as 32 bit unsigned integer.
- * The user-space needs to know about the strange
- * encoding of the alternate cpu capability.
- */
- len += sprintf(page + len, "Capability: %u %u\n",
- info->capability, ext->alt_capability);
- for (i = 2; i <= info->cpus_total; i++)
- len += sprintf(page + len,
- "Adjustment %02d-way: %u %u\n",
- i, info->adjustment[i-2],
- ext->alt_adjustment[i-2]);
-
- } else {
- len += sprintf(page + len, "Capability: %u\n",
- info->capability);
- for (i = 2; i <= info->cpus_total; i++)
- len += sprintf(page + len,
- "Adjustment %02d-way: %u\n",
- i, info->adjustment[i-2]);
+ seq_printf(m, "CPUs Total: %d\n", info->cpus_total);
+ seq_printf(m, "CPUs Configured: %d\n", info->cpus_configured);
+ seq_printf(m, "CPUs Standby: %d\n", info->cpus_standby);
+ seq_printf(m, "CPUs Reserved: %d\n", info->cpus_reserved);
+ /*
+ * Sigh 2. According to the specification the alternate
+ * capability field is a 32 bit floating point number
+ * if the higher order 8 bits are not zero. Printing
+ * a floating point number in the kernel is a no-no,
+ * always print the number as 32 bit unsigned integer.
+ * The user-space needs to know about the strange
+ * encoding of the alternate cpu capability.
+ */
+ seq_printf(m, "Capability: %u", info->capability);
+ if (info->format == 1)
+ seq_printf(m, " %u", ext->alt_capability);
+ seq_putc(m, '\n');
+ if (info->nominal_cap)
+ seq_printf(m, "Nominal Capability: %d\n", info->nominal_cap);
+ if (info->secondary_cap)
+ seq_printf(m, "Secondary Capability: %d\n", info->secondary_cap);
+ for (i = 2; i <= info->cpus_total; i++) {
+ seq_printf(m, "Adjustment %02d-way: %u",
+ i, info->adjustment[i-2]);
+ if (info->format == 1)
+ seq_printf(m, " %u", ext->alt_adjustment[i-2]);
+ seq_putc(m, '\n');
}
-
- if (info->secondary_capability != 0)
- len += sprintf(page + len, "Secondary Capability: %d\n",
- info->secondary_capability);
- return len;
}
-static int stsi_2_2_2(struct sysinfo_2_2_2 *info, char *page, int len)
+static void stsi_2_2_2(struct seq_file *m, struct sysinfo_2_2_2 *info)
{
- if (stsi(info, 2, 2, 2) == -ENOSYS)
- return len;
-
+ if (stsi(info, 2, 2, 2))
+ return;
EBCASC(info->name, sizeof(info->name));
-
- len += sprintf(page + len, "\n");
- len += sprintf(page + len, "LPAR Number: %d\n",
- info->lpar_number);
-
- len += sprintf(page + len, "LPAR Characteristics: ");
+ seq_putc(m, '\n');
+ seq_printf(m, "LPAR Number: %d\n", info->lpar_number);
+ seq_printf(m, "LPAR Characteristics: ");
if (info->characteristics & LPAR_CHAR_DEDICATED)
- len += sprintf(page + len, "Dedicated ");
+ seq_printf(m, "Dedicated ");
if (info->characteristics & LPAR_CHAR_SHARED)
- len += sprintf(page + len, "Shared ");
+ seq_printf(m, "Shared ");
if (info->characteristics & LPAR_CHAR_LIMITED)
- len += sprintf(page + len, "Limited ");
- len += sprintf(page + len, "\n");
-
- len += sprintf(page + len, "LPAR Name: %-8.8s\n",
- info->name);
-
- len += sprintf(page + len, "LPAR Adjustment: %d\n",
- info->caf);
-
- len += sprintf(page + len, "LPAR CPUs Total: %d\n",
- info->cpus_total);
- len += sprintf(page + len, "LPAR CPUs Configured: %d\n",
- info->cpus_configured);
- len += sprintf(page + len, "LPAR CPUs Standby: %d\n",
- info->cpus_standby);
- len += sprintf(page + len, "LPAR CPUs Reserved: %d\n",
- info->cpus_reserved);
- len += sprintf(page + len, "LPAR CPUs Dedicated: %d\n",
- info->cpus_dedicated);
- len += sprintf(page + len, "LPAR CPUs Shared: %d\n",
- info->cpus_shared);
- return len;
+ seq_printf(m, "Limited ");
+ seq_putc(m, '\n');
+ seq_printf(m, "LPAR Name: %-8.8s\n", info->name);
+ seq_printf(m, "LPAR Adjustment: %d\n", info->caf);
+ seq_printf(m, "LPAR CPUs Total: %d\n", info->cpus_total);
+ seq_printf(m, "LPAR CPUs Configured: %d\n", info->cpus_configured);
+ seq_printf(m, "LPAR CPUs Standby: %d\n", info->cpus_standby);
+ seq_printf(m, "LPAR CPUs Reserved: %d\n", info->cpus_reserved);
+ seq_printf(m, "LPAR CPUs Dedicated: %d\n", info->cpus_dedicated);
+ seq_printf(m, "LPAR CPUs Shared: %d\n", info->cpus_shared);
}
-static int stsi_3_2_2(struct sysinfo_3_2_2 *info, char *page, int len)
+static void stsi_3_2_2(struct seq_file *m, struct sysinfo_3_2_2 *info)
{
int i;
- if (stsi(info, 3, 2, 2) == -ENOSYS)
- return len;
+ if (stsi(info, 3, 2, 2))
+ return;
for (i = 0; i < info->count; i++) {
EBCASC(info->vm[i].name, sizeof(info->vm[i].name));
EBCASC(info->vm[i].cpi, sizeof(info->vm[i].cpi));
- len += sprintf(page + len, "\n");
- len += sprintf(page + len, "VM%02d Name: %-8.8s\n",
- i, info->vm[i].name);
- len += sprintf(page + len, "VM%02d Control Program: %-16.16s\n",
- i, info->vm[i].cpi);
-
- len += sprintf(page + len, "VM%02d Adjustment: %d\n",
- i, info->vm[i].caf);
-
- len += sprintf(page + len, "VM%02d CPUs Total: %d\n",
- i, info->vm[i].cpus_total);
- len += sprintf(page + len, "VM%02d CPUs Configured: %d\n",
- i, info->vm[i].cpus_configured);
- len += sprintf(page + len, "VM%02d CPUs Standby: %d\n",
- i, info->vm[i].cpus_standby);
- len += sprintf(page + len, "VM%02d CPUs Reserved: %d\n",
- i, info->vm[i].cpus_reserved);
+ seq_putc(m, '\n');
+ seq_printf(m, "VM%02d Name: %-8.8s\n", i, info->vm[i].name);
+ seq_printf(m, "VM%02d Control Program: %-16.16s\n", i, info->vm[i].cpi);
+ seq_printf(m, "VM%02d Adjustment: %d\n", i, info->vm[i].caf);
+ seq_printf(m, "VM%02d CPUs Total: %d\n", i, info->vm[i].cpus_total);
+ seq_printf(m, "VM%02d CPUs Configured: %d\n", i, info->vm[i].cpus_configured);
+ seq_printf(m, "VM%02d CPUs Standby: %d\n", i, info->vm[i].cpus_standby);
+ seq_printf(m, "VM%02d CPUs Reserved: %d\n", i, info->vm[i].cpus_reserved);
}
- return len;
}
-static int proc_read_sysinfo(char *page, char **start,
- off_t off, int count,
- int *eof, void *data)
+static int sysinfo_show(struct seq_file *m, void *v)
{
- unsigned long info = get_zeroed_page(GFP_KERNEL);
- int level, len;
+ void *info = (void *)get_zeroed_page(GFP_KERNEL);
+ int level;
if (!info)
return 0;
-
- len = 0;
- level = stsi_0();
+ level = stsi(NULL, 0, 0, 0);
if (level >= 1)
- len = stsi_1_1_1((struct sysinfo_1_1_1 *) info, page, len);
-
+ stsi_1_1_1(m, info);
if (level >= 1)
- len = stsi_15_1_x((struct sysinfo_15_1_x *) info, page, len);
-
+ stsi_15_1_x(m, info);
if (level >= 1)
- len = stsi_1_2_2((struct sysinfo_1_2_2 *) info, page, len);
-
+ stsi_1_2_2(m, info);
if (level >= 2)
- len = stsi_2_2_2((struct sysinfo_2_2_2 *) info, page, len);
-
+ stsi_2_2_2(m, info);
if (level >= 3)
- len = stsi_3_2_2((struct sysinfo_3_2_2 *) info, page, len);
-
- free_page(info);
- return len;
-}
-
-static __init int create_proc_sysinfo(void)
-{
- create_proc_read_entry("sysinfo", 0444, NULL,
- proc_read_sysinfo, NULL);
+ stsi_3_2_2(m, info);
+ free_page((unsigned long)info);
return 0;
}
-device_initcall(create_proc_sysinfo);
+
+static int sysinfo_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, sysinfo_show, NULL);
+}
+
+static const struct file_operations sysinfo_fops = {
+ .open = sysinfo_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int __init sysinfo_create_proc(void)
+{
+ proc_create("sysinfo", 0444, NULL, &sysinfo_fops);
+ return 0;
+}
+device_initcall(sysinfo_create_proc);
/*
* Service levels interface.
@@ -407,7 +386,7 @@
if (!info)
return;
- if (stsi(info, 1, 2, 2) != -ENOSYS) {
+ if (stsi(info, 1, 2, 2) == 0) {
/*
* Major sigh. The cpu capability encoding is "special".
* If the first 9 bits of info->capability are 0 then it
diff --git a/arch/s390/kernel/time.c b/arch/s390/kernel/time.c
index dcec960..2db1011 100644
--- a/arch/s390/kernel/time.c
+++ b/arch/s390/kernel/time.c
@@ -329,7 +329,7 @@
* The synchronous get_clock function. It will write the current clock
* value to the clock pointer and return 0 if the clock is in sync with
* the external time source. If the clock mode is local it will return
- * -ENOSYS and -EAGAIN if the clock is not in sync with the external
+ * -EOPNOTSUPP and -EAGAIN if the clock is not in sync with the external
* reference.
*/
int get_sync_clock(unsigned long long *clock)
@@ -347,7 +347,7 @@
return 0;
if (!test_bit(CLOCK_SYNC_HAS_ETR, &clock_sync_flags) &&
!test_bit(CLOCK_SYNC_HAS_STP, &clock_sync_flags))
- return -ENOSYS;
+ return -EOPNOTSUPP;
if (!test_bit(CLOCK_SYNC_ETR, &clock_sync_flags) &&
!test_bit(CLOCK_SYNC_STP, &clock_sync_flags))
return -EACCES;
diff --git a/arch/s390/kernel/topology.c b/arch/s390/kernel/topology.c
index 05151e0..54d93f4 100644
--- a/arch/s390/kernel/topology.c
+++ b/arch/s390/kernel/topology.c
@@ -17,6 +17,7 @@
#include <linux/cpu.h>
#include <linux/smp.h>
#include <linux/mm.h>
+#include <asm/sysinfo.h>
#define PTF_HORIZONTAL (0UL)
#define PTF_VERTICAL (1UL)
@@ -44,9 +45,6 @@
cpumask_t cpu_book_map[NR_CPUS];
unsigned char cpu_book_id[NR_CPUS];
-/* smp_cpu_state_mutex must be held when accessing this array */
-int cpu_polarization[NR_CPUS];
-
static cpumask_t cpu_group_map(struct mask_info *info, unsigned int cpu)
{
cpumask_t mask;
@@ -75,10 +73,7 @@
{
unsigned int cpu;
- for (cpu = find_first_bit(&tl_cpu->mask[0], TOPOLOGY_CPU_BITS);
- cpu < TOPOLOGY_CPU_BITS;
- cpu = find_next_bit(&tl_cpu->mask[0], TOPOLOGY_CPU_BITS, cpu + 1))
- {
+ for_each_set_bit(cpu, &tl_cpu->mask[0], TOPOLOGY_CPU_BITS) {
unsigned int rcpu;
int lcpu;
@@ -94,7 +89,7 @@
} else {
cpu_core_id[lcpu] = core->id;
}
- cpu_set_polarization(lcpu, tl_cpu->pp);
+ smp_cpu_set_polarization(lcpu, tl_cpu->pp);
}
}
return core;
@@ -201,7 +196,7 @@
mutex_lock(&smp_cpu_state_mutex);
for_each_possible_cpu(cpu)
- cpu_set_polarization(cpu, POLARIZATION_HRZ);
+ smp_cpu_set_polarization(cpu, POLARIZATION_HRZ);
mutex_unlock(&smp_cpu_state_mutex);
}
@@ -231,7 +226,7 @@
if (rc)
return -EBUSY;
for_each_possible_cpu(cpu)
- cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
+ smp_cpu_set_polarization(cpu, POLARIZATION_UNKNOWN);
return rc;
}
@@ -250,12 +245,10 @@
void store_topology(struct sysinfo_15_1_x *info)
{
- int rc;
-
- rc = stsi(info, 15, 1, 3);
- if (rc != -ENOSYS)
- return;
- stsi(info, 15, 1, 2);
+ if (topology_max_mnest >= 3)
+ stsi(info, 15, 1, 3);
+ else
+ stsi(info, 15, 1, 2);
}
int arch_update_cpu_topology(void)
@@ -415,7 +408,7 @@
ssize_t count;
mutex_lock(&smp_cpu_state_mutex);
- switch (cpu_read_polarization(cpu)) {
+ switch (smp_cpu_get_polarization(cpu)) {
case POLARIZATION_HRZ:
count = sprintf(buf, "horizontal\n");
break;
diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c
index 01775c0..3d2b0fa3 100644
--- a/arch/s390/kernel/traps.c
+++ b/arch/s390/kernel/traps.c
@@ -57,6 +57,23 @@
static int kstack_depth_to_print = 20;
#endif /* CONFIG_64BIT */
+static inline void __user *get_trap_ip(struct pt_regs *regs)
+{
+#ifdef CONFIG_64BIT
+ unsigned long address;
+
+ if (regs->int_code & 0x200)
+ address = *(unsigned long *)(current->thread.trap_tdb + 24);
+ else
+ address = regs->psw.addr;
+ return (void __user *)
+ ((address - (regs->int_code >> 16)) & PSW_ADDR_INSN);
+#else
+ return (void __user *)
+ ((regs->psw.addr - (regs->int_code >> 16)) & PSW_ADDR_INSN);
+#endif
+}
+
/*
* For show_trace we have tree different stack to consider:
* - the panic stack which is used if the kernel stack has overflown
@@ -214,7 +231,6 @@
void show_regs(struct pt_regs *regs)
{
- print_modules();
printk("CPU: %d %s %s %.*s\n",
task_thread_info(current)->cpu, print_tainted(),
init_utsname()->release,
@@ -254,6 +270,7 @@
#endif
printk("\n");
notify_die(DIE_OOPS, str, regs, 0, regs->int_code & 0xffff, SIGSEGV);
+ print_modules();
show_regs(regs);
bust_spinlocks(0);
add_taint(TAINT_DIE);
@@ -285,12 +302,6 @@
return 1;
}
-static inline void __user *get_psw_address(struct pt_regs *regs)
-{
- return (void __user *)
- ((regs->psw.addr - (regs->int_code >> 16)) & PSW_ADDR_INSN);
-}
-
static void __kprobes do_trap(struct pt_regs *regs,
int si_signo, int si_code, char *str)
{
@@ -304,14 +315,14 @@
info.si_signo = si_signo;
info.si_errno = 0;
info.si_code = si_code;
- info.si_addr = get_psw_address(regs);
+ info.si_addr = get_trap_ip(regs);
force_sig_info(si_signo, &info, current);
report_user_fault(regs, si_signo);
} else {
const struct exception_table_entry *fixup;
fixup = search_exception_tables(regs->psw.addr & PSW_ADDR_INSN);
if (fixup)
- regs->psw.addr = fixup->fixup | PSW_ADDR_AMODE;
+ regs->psw.addr = extable_fixup(fixup) | PSW_ADDR_AMODE;
else {
enum bug_trap_type btt;
@@ -381,6 +392,11 @@
DO_ERROR_INFO(translation_exception, SIGILL, ILL_ILLOPN,
"translation exception")
+#ifdef CONFIG_64BIT
+DO_ERROR_INFO(transaction_exception, SIGILL, ILL_ILLOPN,
+ "transaction constraint exception")
+#endif
+
static inline void do_fp_trap(struct pt_regs *regs, int fpc)
{
int si_code = 0;
@@ -408,7 +424,7 @@
__u16 __user *location;
int signal = 0;
- location = get_psw_address(regs);
+ location = get_trap_ip(regs);
if (user_mode(regs)) {
if (get_user(*((__u16 *) opcode), (__u16 __user *) location))
@@ -476,7 +492,7 @@
__u16 __user *location = NULL;
int signal = 0;
- location = (__u16 __user *) get_psw_address(regs);
+ location = (__u16 __user *) get_trap_ip(regs);
if (user_mode(regs)) {
get_user(*((__u16 *) opcode), location);
@@ -525,7 +541,7 @@
__u16 __user *location;
int signal = 0;
- location = get_psw_address(regs);
+ location = get_trap_ip(regs);
if (MACHINE_HAS_IEEE)
asm volatile("stfpc %0" : "=m" (current->thread.fp_regs.fpc));
@@ -641,6 +657,7 @@
pgm_check_table[0x12] = &translation_exception;
pgm_check_table[0x13] = &special_op_exception;
#ifdef CONFIG_64BIT
+ pgm_check_table[0x18] = &transaction_exception;
pgm_check_table[0x38] = &do_asce_exception;
pgm_check_table[0x39] = &do_dat_exception;
pgm_check_table[0x3A] = &do_dat_exception;
diff --git a/arch/s390/kernel/vdso.c b/arch/s390/kernel/vdso.c
index 9a19ca3..d777628 100644
--- a/arch/s390/kernel/vdso.c
+++ b/arch/s390/kernel/vdso.c
@@ -85,7 +85,7 @@
static void vdso_init_data(struct vdso_data *vd)
{
vd->ectg_available =
- addressing_mode != HOME_SPACE_MODE && test_facility(31);
+ s390_user_mode != HOME_SPACE_MODE && test_facility(31);
}
#ifdef CONFIG_64BIT
@@ -102,7 +102,7 @@
lowcore->vdso_per_cpu_data = __LC_PASTE;
- if (addressing_mode == HOME_SPACE_MODE || !vdso_enabled)
+ if (s390_user_mode == HOME_SPACE_MODE || !vdso_enabled)
return 0;
segment_table = __get_free_pages(GFP_KERNEL, SEGMENT_ORDER);
@@ -147,7 +147,7 @@
unsigned long segment_table, page_table, page_frame;
u32 *psal, *aste;
- if (addressing_mode == HOME_SPACE_MODE || !vdso_enabled)
+ if (s390_user_mode == HOME_SPACE_MODE || !vdso_enabled)
return;
psal = (u32 *)(addr_t) lowcore->paste[4];
@@ -165,7 +165,7 @@
{
unsigned long cr5;
- if (addressing_mode == HOME_SPACE_MODE || !vdso_enabled)
+ if (s390_user_mode == HOME_SPACE_MODE || !vdso_enabled)
return;
cr5 = offsetof(struct _lowcore, paste);
__ctl_load(cr5, 5, 5);
diff --git a/arch/s390/kernel/vtime.c b/arch/s390/kernel/vtime.c
index 4fc97b4..7903344 100644
--- a/arch/s390/kernel/vtime.c
+++ b/arch/s390/kernel/vtime.c
@@ -99,7 +99,7 @@
return virt_timer_forward(user + system);
}
-void account_vtime(struct task_struct *prev, struct task_struct *next)
+void vtime_task_switch(struct task_struct *prev)
{
struct thread_info *ti;
@@ -107,7 +107,7 @@
ti = task_thread_info(prev);
ti->user_timer = S390_lowcore.user_timer;
ti->system_timer = S390_lowcore.system_timer;
- ti = task_thread_info(next);
+ ti = task_thread_info(current);
S390_lowcore.user_timer = ti->user_timer;
S390_lowcore.system_timer = ti->system_timer;
}
@@ -122,7 +122,7 @@
* Update process times based on virtual cpu times stored by entry.S
* to the lowcore fields user_timer, system_timer & steal_clock.
*/
-void account_system_vtime(struct task_struct *tsk)
+void vtime_account(struct task_struct *tsk)
{
struct thread_info *ti = task_thread_info(tsk);
u64 timer, system;
@@ -138,7 +138,7 @@
virt_timer_forward(system);
}
-EXPORT_SYMBOL_GPL(account_system_vtime);
+EXPORT_SYMBOL_GPL(vtime_account);
void __kprobes vtime_stop_cpu(void)
{
@@ -378,9 +378,8 @@
long cpu = (long) hcpu;
idle = &per_cpu(s390_idle, cpu);
- switch (action) {
+ switch (action & ~CPU_TASKS_FROZEN) {
case CPU_DYING:
- case CPU_DYING_FROZEN:
idle->nohz_delay = 0;
default:
break;
diff --git a/arch/s390/kvm/Kconfig b/arch/s390/kvm/Kconfig
index 78eb984..9b04a32 100644
--- a/arch/s390/kvm/Kconfig
+++ b/arch/s390/kvm/Kconfig
@@ -5,7 +5,7 @@
menuconfig VIRTUALIZATION
def_bool y
- prompt "Virtualization"
+ prompt "KVM"
---help---
Say Y here to get to see options for using your Linux host to run other
operating systems inside virtual machines (guests).
diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
index 60da903..310be61 100644
--- a/arch/s390/kvm/priv.c
+++ b/arch/s390/kvm/priv.c
@@ -211,7 +211,7 @@
spin_unlock(&fi->lock);
/* deal with other level 3 hypervisors */
- if (stsi(mem, 3, 2, 2) == -ENOSYS)
+ if (stsi(mem, 3, 2, 2))
mem->count = 0;
if (mem->count < 8)
mem->count++;
@@ -259,7 +259,7 @@
mem = get_zeroed_page(GFP_KERNEL);
if (!mem)
goto out_fail;
- if (stsi((void *) mem, fc, sel1, sel2) == -ENOSYS)
+ if (stsi((void *) mem, fc, sel1, sel2))
goto out_mem;
break;
case 3:
diff --git a/arch/s390/lib/Makefile b/arch/s390/lib/Makefile
index 761ab8b..6ab0d0b 100644
--- a/arch/s390/lib/Makefile
+++ b/arch/s390/lib/Makefile
@@ -4,6 +4,7 @@
lib-y += delay.o string.o uaccess_std.o uaccess_pt.o
obj-y += usercopy.o
-obj-$(CONFIG_32BIT) += div64.o qrnnd.o ucmpdi2.o
+obj-$(CONFIG_32BIT) += div64.o qrnnd.o ucmpdi2.o mem32.o
+obj-$(CONFIG_64BIT) += mem64.o
lib-$(CONFIG_64BIT) += uaccess_mvcos.o
lib-$(CONFIG_SMP) += spinlock.o
diff --git a/arch/s390/lib/mem32.S b/arch/s390/lib/mem32.S
new file mode 100644
index 0000000..14ca924
--- /dev/null
+++ b/arch/s390/lib/mem32.S
@@ -0,0 +1,92 @@
+/*
+ * String handling functions.
+ *
+ * Copyright IBM Corp. 2012
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * memset implementation
+ *
+ * This code corresponds to the C construct below. We do distinguish
+ * between clearing (c == 0) and setting a memory array (c != 0) simply
+ * because nearly all memset invocations in the kernel clear memory and
+ * the xc instruction is preferred in such cases.
+ *
+ * void *memset(void *s, int c, size_t n)
+ * {
+ * if (likely(c == 0))
+ * return __builtin_memset(s, 0, n);
+ * return __builtin_memset(s, c, n);
+ * }
+ */
+ENTRY(memset)
+ basr %r5,%r0
+.Lmemset_base:
+ ltr %r4,%r4
+ bzr %r14
+ ltr %r3,%r3
+ jnz .Lmemset_fill
+ ahi %r4,-1
+ lr %r3,%r4
+ srl %r3,8
+ ltr %r3,%r3
+ lr %r1,%r2
+ je .Lmemset_clear_rest
+.Lmemset_clear_loop:
+ xc 0(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brct %r3,.Lmemset_clear_loop
+.Lmemset_clear_rest:
+ ex %r4,.Lmemset_xc-.Lmemset_base(%r5)
+ br %r14
+.Lmemset_fill:
+ stc %r3,0(%r2)
+ chi %r4,1
+ lr %r1,%r2
+ ber %r14
+ ahi %r4,-2
+ lr %r3,%r4
+ srl %r3,8
+ ltr %r3,%r3
+ je .Lmemset_fill_rest
+.Lmemset_fill_loop:
+ mvc 1(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brct %r3,.Lmemset_fill_loop
+.Lmemset_fill_rest:
+ ex %r4,.Lmemset_mvc-.Lmemset_base(%r5)
+ br %r14
+.Lmemset_xc:
+ xc 0(1,%r1),0(%r1)
+.Lmemset_mvc:
+ mvc 1(1,%r1),0(%r1)
+
+/*
+ * memcpy implementation
+ *
+ * void *memcpy(void *dest, const void *src, size_t n)
+ */
+ENTRY(memcpy)
+ basr %r5,%r0
+.Lmemcpy_base:
+ ltr %r4,%r4
+ bzr %r14
+ ahi %r4,-1
+ lr %r0,%r4
+ srl %r0,8
+ ltr %r0,%r0
+ lr %r1,%r2
+ jnz .Lmemcpy_loop
+.Lmemcpy_rest:
+ ex %r4,.Lmemcpy_mvc-.Lmemcpy_base(%r5)
+ br %r14
+.Lmemcpy_loop:
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brct %r0,.Lmemcpy_loop
+ j .Lmemcpy_rest
+.Lmemcpy_mvc:
+ mvc 0(1,%r1),0(%r3)
diff --git a/arch/s390/lib/mem64.S b/arch/s390/lib/mem64.S
new file mode 100644
index 0000000..c6d553e
--- /dev/null
+++ b/arch/s390/lib/mem64.S
@@ -0,0 +1,88 @@
+/*
+ * String handling functions.
+ *
+ * Copyright IBM Corp. 2012
+ */
+
+#include <linux/linkage.h>
+
+/*
+ * memset implementation
+ *
+ * This code corresponds to the C construct below. We do distinguish
+ * between clearing (c == 0) and setting a memory array (c != 0) simply
+ * because nearly all memset invocations in the kernel clear memory and
+ * the xc instruction is preferred in such cases.
+ *
+ * void *memset(void *s, int c, size_t n)
+ * {
+ * if (likely(c == 0))
+ * return __builtin_memset(s, 0, n);
+ * return __builtin_memset(s, c, n);
+ * }
+ */
+ENTRY(memset)
+ ltgr %r4,%r4
+ bzr %r14
+ ltgr %r3,%r3
+ jnz .Lmemset_fill
+ aghi %r4,-1
+ srlg %r3,%r4,8
+ ltgr %r3,%r3
+ lgr %r1,%r2
+ jz .Lmemset_clear_rest
+.Lmemset_clear_loop:
+ xc 0(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brctg %r3,.Lmemset_clear_loop
+.Lmemset_clear_rest:
+ larl %r3,.Lmemset_xc
+ ex %r4,0(%r3)
+ br %r14
+.Lmemset_fill:
+ stc %r3,0(%r2)
+ cghi %r4,1
+ lgr %r1,%r2
+ ber %r14
+ aghi %r4,-2
+ srlg %r3,%r4,8
+ ltgr %r3,%r3
+ jz .Lmemset_fill_rest
+.Lmemset_fill_loop:
+ mvc 1(256,%r1),0(%r1)
+ la %r1,256(%r1)
+ brctg %r3,.Lmemset_fill_loop
+.Lmemset_fill_rest:
+ larl %r3,.Lmemset_mvc
+ ex %r4,0(%r3)
+ br %r14
+.Lmemset_xc:
+ xc 0(1,%r1),0(%r1)
+.Lmemset_mvc:
+ mvc 1(1,%r1),0(%r1)
+
+/*
+ * memcpy implementation
+ *
+ * void *memcpy(void *dest, const void *src, size_t n)
+ */
+ENTRY(memcpy)
+ ltgr %r4,%r4
+ bzr %r14
+ aghi %r4,-1
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ lgr %r1,%r2
+ jnz .Lmemcpy_loop
+.Lmemcpy_rest:
+ larl %r5,.Lmemcpy_mvc
+ ex %r4,0(%r5)
+ br %r14
+.Lmemcpy_loop:
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brctg %r5,.Lmemcpy_loop
+ j .Lmemcpy_rest
+.Lmemcpy_mvc:
+ mvc 0(1,%r1),0(%r3)
diff --git a/arch/s390/lib/string.c b/arch/s390/lib/string.c
index 846ec64..b647d5f 100644
--- a/arch/s390/lib/string.c
+++ b/arch/s390/lib/string.c
@@ -43,11 +43,7 @@
*/
size_t strlen(const char *s)
{
-#if __GNUC__ < 4
return __strend(s) - s;
-#else
- return __builtin_strlen(s);
-#endif
}
EXPORT_SYMBOL(strlen);
@@ -73,7 +69,6 @@
*/
char *strcpy(char *dest, const char *src)
{
-#if __GNUC__ < 4
register int r0 asm("0") = 0;
char *ret = dest;
@@ -82,9 +77,6 @@
: "+&a" (dest), "+&a" (src) : "d" (r0)
: "cc", "memory" );
return ret;
-#else
- return __builtin_strcpy(dest, src);
-#endif
}
EXPORT_SYMBOL(strcpy);
@@ -106,7 +98,7 @@
if (size) {
size_t len = (ret >= size) ? size-1 : ret;
dest[len] = '\0';
- __builtin_memcpy(dest, src, len);
+ memcpy(dest, src, len);
}
return ret;
}
@@ -124,8 +116,8 @@
char *strncpy(char *dest, const char *src, size_t n)
{
size_t len = __strnend(src, n) - src;
- __builtin_memset(dest + len, 0, n - len);
- __builtin_memcpy(dest, src, len);
+ memset(dest + len, 0, n - len);
+ memcpy(dest, src, len);
return dest;
}
EXPORT_SYMBOL(strncpy);
@@ -171,7 +163,7 @@
if (len >= n)
len = n - 1;
dest[len] = '\0';
- __builtin_memcpy(dest, src, len);
+ memcpy(dest, src, len);
}
return res;
}
@@ -194,7 +186,7 @@
char *p = __strend(dest);
p[len] = '\0';
- __builtin_memcpy(p, src, len);
+ memcpy(p, src, len);
return dest;
}
EXPORT_SYMBOL(strncat);
@@ -348,41 +340,3 @@
return (void *) ret;
}
EXPORT_SYMBOL(memscan);
-
-/**
- * memcpy - Copy one area of memory to another
- * @dest: Where to copy to
- * @src: Where to copy from
- * @n: The size of the area.
- *
- * returns a pointer to @dest
- */
-void *memcpy(void *dest, const void *src, size_t n)
-{
- return __builtin_memcpy(dest, src, n);
-}
-EXPORT_SYMBOL(memcpy);
-
-/**
- * memset - Fill a region of memory with the given value
- * @s: Pointer to the start of the area.
- * @c: The byte to fill the area with
- * @n: The size of the area.
- *
- * returns a pointer to @s
- */
-void *memset(void *s, int c, size_t n)
-{
- char *xs;
-
- if (c == 0)
- return __builtin_memset(s, 0, n);
-
- xs = (char *) s;
- if (n > 0)
- do {
- *xs++ = c;
- } while (--n > 0);
- return s;
-}
-EXPORT_SYMBOL(memset);
diff --git a/arch/s390/lib/uaccess_pt.c b/arch/s390/lib/uaccess_pt.c
index 60ee2b8..2d37bb8 100644
--- a/arch/s390/lib/uaccess_pt.c
+++ b/arch/s390/lib/uaccess_pt.c
@@ -2,69 +2,82 @@
* User access functions based on page table walks for enhanced
* system layout without hardware support.
*
- * Copyright IBM Corp. 2006
+ * Copyright IBM Corp. 2006, 2012
* Author(s): Gerald Schaefer (gerald.schaefer@de.ibm.com)
*/
#include <linux/errno.h>
#include <linux/hardirq.h>
#include <linux/mm.h>
+#include <linux/hugetlb.h>
#include <asm/uaccess.h>
#include <asm/futex.h>
#include "uaccess.h"
-static inline pte_t *follow_table(struct mm_struct *mm, unsigned long addr)
+
+/*
+ * Returns kernel address for user virtual address. If the returned address is
+ * >= -4095 (IS_ERR_VALUE(x) returns true), a fault has occured and the address
+ * contains the (negative) exception code.
+ */
+static __always_inline unsigned long follow_table(struct mm_struct *mm,
+ unsigned long addr, int write)
{
pgd_t *pgd;
pud_t *pud;
pmd_t *pmd;
+ pte_t *ptep;
pgd = pgd_offset(mm, addr);
if (pgd_none(*pgd) || unlikely(pgd_bad(*pgd)))
- return (pte_t *) 0x3a;
+ return -0x3aUL;
pud = pud_offset(pgd, addr);
if (pud_none(*pud) || unlikely(pud_bad(*pud)))
- return (pte_t *) 0x3b;
+ return -0x3bUL;
pmd = pmd_offset(pud, addr);
- if (pmd_none(*pmd) || unlikely(pmd_bad(*pmd)))
- return (pte_t *) 0x10;
+ if (pmd_none(*pmd))
+ return -0x10UL;
+ if (pmd_huge(*pmd)) {
+ if (write && (pmd_val(*pmd) & _SEGMENT_ENTRY_RO))
+ return -0x04UL;
+ return (pmd_val(*pmd) & HPAGE_MASK) + (addr & ~HPAGE_MASK);
+ }
+ if (unlikely(pmd_bad(*pmd)))
+ return -0x10UL;
- return pte_offset_map(pmd, addr);
+ ptep = pte_offset_map(pmd, addr);
+ if (!pte_present(*ptep))
+ return -0x11UL;
+ if (write && !pte_write(*ptep))
+ return -0x04UL;
+
+ return (pte_val(*ptep) & PAGE_MASK) + (addr & ~PAGE_MASK);
}
static __always_inline size_t __user_copy_pt(unsigned long uaddr, void *kptr,
size_t n, int write_user)
{
struct mm_struct *mm = current->mm;
- unsigned long offset, pfn, done, size;
- pte_t *pte;
+ unsigned long offset, done, size, kaddr;
void *from, *to;
done = 0;
retry:
spin_lock(&mm->page_table_lock);
do {
- pte = follow_table(mm, uaddr);
- if ((unsigned long) pte < 0x1000)
+ kaddr = follow_table(mm, uaddr, write_user);
+ if (IS_ERR_VALUE(kaddr))
goto fault;
- if (!pte_present(*pte)) {
- pte = (pte_t *) 0x11;
- goto fault;
- } else if (write_user && !pte_write(*pte)) {
- pte = (pte_t *) 0x04;
- goto fault;
- }
- pfn = pte_pfn(*pte);
- offset = uaddr & (PAGE_SIZE - 1);
+ offset = uaddr & ~PAGE_MASK;
size = min(n - done, PAGE_SIZE - offset);
if (write_user) {
- to = (void *)((pfn << PAGE_SHIFT) + offset);
+ to = (void *) kaddr;
from = kptr + done;
} else {
- from = (void *)((pfn << PAGE_SHIFT) + offset);
+ from = (void *) kaddr;
to = kptr + done;
}
memcpy(to, from, size);
@@ -75,7 +88,7 @@
return n - done;
fault:
spin_unlock(&mm->page_table_lock);
- if (__handle_fault(uaddr, (unsigned long) pte, write_user))
+ if (__handle_fault(uaddr, -kaddr, write_user))
return n - done;
goto retry;
}
@@ -84,27 +97,22 @@
* Do DAT for user address by page table walk, return kernel address.
* This function needs to be called with current->mm->page_table_lock held.
*/
-static __always_inline unsigned long __dat_user_addr(unsigned long uaddr)
+static __always_inline unsigned long __dat_user_addr(unsigned long uaddr,
+ int write)
{
struct mm_struct *mm = current->mm;
- unsigned long pfn;
- pte_t *pte;
+ unsigned long kaddr;
int rc;
retry:
- pte = follow_table(mm, uaddr);
- if ((unsigned long) pte < 0x1000)
+ kaddr = follow_table(mm, uaddr, write);
+ if (IS_ERR_VALUE(kaddr))
goto fault;
- if (!pte_present(*pte)) {
- pte = (pte_t *) 0x11;
- goto fault;
- }
- pfn = pte_pfn(*pte);
- return (pfn << PAGE_SHIFT) + (uaddr & (PAGE_SIZE - 1));
+ return kaddr;
fault:
spin_unlock(&mm->page_table_lock);
- rc = __handle_fault(uaddr, (unsigned long) pte, 0);
+ rc = __handle_fault(uaddr, -kaddr, write);
spin_lock(&mm->page_table_lock);
if (!rc)
goto retry;
@@ -159,11 +167,9 @@
static size_t strnlen_user_pt(size_t count, const char __user *src)
{
- char *addr;
unsigned long uaddr = (unsigned long) src;
struct mm_struct *mm = current->mm;
- unsigned long offset, pfn, done, len;
- pte_t *pte;
+ unsigned long offset, done, len, kaddr;
size_t len_str;
if (segment_eq(get_fs(), KERNEL_DS))
@@ -172,19 +178,13 @@
retry:
spin_lock(&mm->page_table_lock);
do {
- pte = follow_table(mm, uaddr);
- if ((unsigned long) pte < 0x1000)
+ kaddr = follow_table(mm, uaddr, 0);
+ if (IS_ERR_VALUE(kaddr))
goto fault;
- if (!pte_present(*pte)) {
- pte = (pte_t *) 0x11;
- goto fault;
- }
- pfn = pte_pfn(*pte);
- offset = uaddr & (PAGE_SIZE-1);
- addr = (char *)(pfn << PAGE_SHIFT) + offset;
+ offset = uaddr & ~PAGE_MASK;
len = min(count - done, PAGE_SIZE - offset);
- len_str = strnlen(addr, len);
+ len_str = strnlen((char *) kaddr, len);
done += len_str;
uaddr += len_str;
} while ((len_str == len) && (done < count));
@@ -192,7 +192,7 @@
return done + 1;
fault:
spin_unlock(&mm->page_table_lock);
- if (__handle_fault(uaddr, (unsigned long) pte, 0))
+ if (__handle_fault(uaddr, -kaddr, 0))
return 0;
goto retry;
}
@@ -225,11 +225,10 @@
const void __user *from)
{
struct mm_struct *mm = current->mm;
- unsigned long offset_from, offset_to, offset_max, pfn_from, pfn_to,
- uaddr, done, size, error_code;
+ unsigned long offset_max, uaddr, done, size, error_code;
unsigned long uaddr_from = (unsigned long) from;
unsigned long uaddr_to = (unsigned long) to;
- pte_t *pte_from, *pte_to;
+ unsigned long kaddr_to, kaddr_from;
int write_user;
if (segment_eq(get_fs(), KERNEL_DS)) {
@@ -242,38 +241,23 @@
do {
write_user = 0;
uaddr = uaddr_from;
- pte_from = follow_table(mm, uaddr_from);
- error_code = (unsigned long) pte_from;
- if (error_code < 0x1000)
+ kaddr_from = follow_table(mm, uaddr_from, 0);
+ error_code = kaddr_from;
+ if (IS_ERR_VALUE(error_code))
goto fault;
- if (!pte_present(*pte_from)) {
- error_code = 0x11;
- goto fault;
- }
write_user = 1;
uaddr = uaddr_to;
- pte_to = follow_table(mm, uaddr_to);
- error_code = (unsigned long) pte_to;
- if (error_code < 0x1000)
+ kaddr_to = follow_table(mm, uaddr_to, 1);
+ error_code = (unsigned long) kaddr_to;
+ if (IS_ERR_VALUE(error_code))
goto fault;
- if (!pte_present(*pte_to)) {
- error_code = 0x11;
- goto fault;
- } else if (!pte_write(*pte_to)) {
- error_code = 0x04;
- goto fault;
- }
- pfn_from = pte_pfn(*pte_from);
- pfn_to = pte_pfn(*pte_to);
- offset_from = uaddr_from & (PAGE_SIZE-1);
- offset_to = uaddr_from & (PAGE_SIZE-1);
- offset_max = max(offset_from, offset_to);
+ offset_max = max(uaddr_from & ~PAGE_MASK,
+ uaddr_to & ~PAGE_MASK);
size = min(n - done, PAGE_SIZE - offset_max);
- memcpy((void *)(pfn_to << PAGE_SHIFT) + offset_to,
- (void *)(pfn_from << PAGE_SHIFT) + offset_from, size);
+ memcpy((void *) kaddr_to, (void *) kaddr_from, size);
done += size;
uaddr_from += size;
uaddr_to += size;
@@ -282,7 +266,7 @@
return n - done;
fault:
spin_unlock(&mm->page_table_lock);
- if (__handle_fault(uaddr, error_code, write_user))
+ if (__handle_fault(uaddr, -error_code, write_user))
return n - done;
goto retry;
}
@@ -341,7 +325,7 @@
return __futex_atomic_op_pt(op, uaddr, oparg, old);
spin_lock(¤t->mm->page_table_lock);
uaddr = (u32 __force __user *)
- __dat_user_addr((__force unsigned long) uaddr);
+ __dat_user_addr((__force unsigned long) uaddr, 1);
if (!uaddr) {
spin_unlock(¤t->mm->page_table_lock);
return -EFAULT;
@@ -378,7 +362,7 @@
return __futex_atomic_cmpxchg_pt(uval, uaddr, oldval, newval);
spin_lock(¤t->mm->page_table_lock);
uaddr = (u32 __force __user *)
- __dat_user_addr((__force unsigned long) uaddr);
+ __dat_user_addr((__force unsigned long) uaddr, 1);
if (!uaddr) {
spin_unlock(¤t->mm->page_table_lock);
return -EFAULT;
diff --git a/arch/s390/mm/Makefile b/arch/s390/mm/Makefile
index d98fe90..0f5536b 100644
--- a/arch/s390/mm/Makefile
+++ b/arch/s390/mm/Makefile
@@ -3,7 +3,7 @@
#
obj-y := init.o fault.o extmem.o mmap.o vmem.o pgtable.o maccess.o \
- page-states.o gup.o
+ page-states.o gup.o extable.o
obj-$(CONFIG_CMM) += cmm.o
obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
obj-$(CONFIG_DEBUG_SET_MODULE_RONX) += pageattr.o
diff --git a/arch/s390/mm/extable.c b/arch/s390/mm/extable.c
new file mode 100644
index 0000000..4d1ee88
--- /dev/null
+++ b/arch/s390/mm/extable.c
@@ -0,0 +1,81 @@
+#include <linux/module.h>
+#include <linux/sort.h>
+#include <asm/uaccess.h>
+
+/*
+ * Search one exception table for an entry corresponding to the
+ * given instruction address, and return the address of the entry,
+ * or NULL if none is found.
+ * We use a binary search, and thus we assume that the table is
+ * already sorted.
+ */
+const struct exception_table_entry *
+search_extable(const struct exception_table_entry *first,
+ const struct exception_table_entry *last,
+ unsigned long value)
+{
+ const struct exception_table_entry *mid;
+ unsigned long addr;
+
+ while (first <= last) {
+ mid = ((last - first) >> 1) + first;
+ addr = extable_insn(mid);
+ if (addr < value)
+ first = mid + 1;
+ else if (addr > value)
+ last = mid - 1;
+ else
+ return mid;
+ }
+ return NULL;
+}
+
+/*
+ * The exception table needs to be sorted so that the binary
+ * search that we use to find entries in it works properly.
+ * This is used both for the kernel exception table and for
+ * the exception tables of modules that get loaded.
+ *
+ */
+static int cmp_ex(const void *a, const void *b)
+{
+ const struct exception_table_entry *x = a, *y = b;
+
+ /* This compare is only valid after normalization. */
+ return x->insn - y->insn;
+}
+
+void sort_extable(struct exception_table_entry *start,
+ struct exception_table_entry *finish)
+{
+ struct exception_table_entry *p;
+ int i;
+
+ /* Normalize entries to being relative to the start of the section */
+ for (p = start, i = 0; p < finish; p++, i += 8)
+ p->insn += i;
+ sort(start, finish - start, sizeof(*start), cmp_ex, NULL);
+ /* Denormalize all entries */
+ for (p = start, i = 0; p < finish; p++, i += 8)
+ p->insn -= i;
+}
+
+#ifdef CONFIG_MODULES
+/*
+ * If the exception table is sorted, any referring to the module init
+ * will be at the beginning or the end.
+ */
+void trim_init_extable(struct module *m)
+{
+ /* Trim the beginning */
+ while (m->num_exentries &&
+ within_module_init(extable_insn(&m->extable[0]), m)) {
+ m->extable++;
+ m->num_exentries--;
+ }
+ /* Trim the end */
+ while (m->num_exentries &&
+ within_module_init(extable_insn(&m->extable[m->num_exentries-1]), m))
+ m->num_exentries--;
+}
+#endif /* CONFIG_MODULES */
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c
index 6c013f5..ac9122c 100644
--- a/arch/s390/mm/fault.c
+++ b/arch/s390/mm/fault.c
@@ -111,7 +111,7 @@
if (trans_exc_code == 2)
/* Access via secondary space, set_fs setting decides */
return current->thread.mm_segment.ar4;
- if (addressing_mode == HOME_SPACE_MODE)
+ if (s390_user_mode == HOME_SPACE_MODE)
/* User space if the access has been done via home space. */
return trans_exc_code == 3;
/*
@@ -163,7 +163,7 @@
/* Are we prepared to handle this kernel fault? */
fixup = search_exception_tables(regs->psw.addr & PSW_ADDR_INSN);
if (fixup) {
- regs->psw.addr = fixup->fixup | PSW_ADDR_AMODE;
+ regs->psw.addr = extable_fixup(fixup) | PSW_ADDR_AMODE;
return;
}
@@ -628,9 +628,8 @@
struct thread_struct *thread, *next;
struct task_struct *tsk;
- switch (action) {
+ switch (action & ~CPU_TASKS_FROZEN) {
case CPU_DEAD:
- case CPU_DEAD_FROZEN:
spin_lock_irq(&pfault_lock);
list_for_each_entry_safe(thread, next, &pfault_list, list) {
thread->pfault_wait = 0;
diff --git a/arch/s390/mm/gup.c b/arch/s390/mm/gup.c
index 65cb06e..eeaf802 100644
--- a/arch/s390/mm/gup.c
+++ b/arch/s390/mm/gup.c
@@ -154,6 +154,43 @@
return 1;
}
+/*
+ * Like get_user_pages_fast() except its IRQ-safe in that it won't fall
+ * back to the regular GUP.
+ */
+int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
+ struct page **pages)
+{
+ struct mm_struct *mm = current->mm;
+ unsigned long addr, len, end;
+ unsigned long next, flags;
+ pgd_t *pgdp, pgd;
+ int nr = 0;
+
+ start &= PAGE_MASK;
+ addr = start;
+ len = (unsigned long) nr_pages << PAGE_SHIFT;
+ end = start + len;
+ if (unlikely(!access_ok(write ? VERIFY_WRITE : VERIFY_READ,
+ (void __user *)start, len)))
+ return 0;
+
+ local_irq_save(flags);
+ pgdp = pgd_offset(mm, addr);
+ do {
+ pgd = *pgdp;
+ barrier();
+ next = pgd_addr_end(addr, end);
+ if (pgd_none(pgd))
+ break;
+ if (!gup_pud_range(pgdp, pgd, addr, next, write, pages, &nr))
+ break;
+ } while (pgdp++, addr = next, addr != end);
+ local_irq_restore(flags);
+
+ return nr;
+}
+
/**
* get_user_pages_fast() - pin user pages in memory
* @start: starting user address
diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
index 6adbc08..81e596c 100644
--- a/arch/s390/mm/init.c
+++ b/arch/s390/mm/init.c
@@ -42,7 +42,7 @@
unsigned long empty_zero_page, zero_page_mask;
EXPORT_SYMBOL(empty_zero_page);
-static unsigned long setup_zero_pages(void)
+static unsigned long __init setup_zero_pages(void)
{
struct cpuid cpu_id;
unsigned int order;
@@ -212,7 +212,7 @@
}
#ifdef CONFIG_BLK_DEV_INITRD
-void free_initrd_mem(unsigned long start, unsigned long end)
+void __init free_initrd_mem(unsigned long start, unsigned long end)
{
free_init_pages("initrd memory", start, end);
}
diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c
index 18df31d..b402991 100644
--- a/arch/s390/mm/pgtable.c
+++ b/arch/s390/mm/pgtable.c
@@ -609,8 +609,8 @@
*/
unsigned long *page_table_alloc(struct mm_struct *mm, unsigned long vmaddr)
{
- struct page *page;
- unsigned long *table;
+ unsigned long *uninitialized_var(table);
+ struct page *uninitialized_var(page);
unsigned int mask, bit;
if (mm_has_pgste(mm))
@@ -796,7 +796,7 @@
struct mm_struct *mm, *old_mm;
/* Do we have switched amode? If no, we cannot do sie */
- if (addressing_mode == HOME_SPACE_MODE)
+ if (s390_user_mode == HOME_SPACE_MODE)
return -EINVAL;
/* Do we have pgstes? if yes, we are done */
diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
index 6f896e7..c22abf9 100644
--- a/arch/s390/mm/vmem.c
+++ b/arch/s390/mm/vmem.c
@@ -107,7 +107,7 @@
pte = mk_pte_phys(address, __pgprot(ro ? _PAGE_RO : 0));
pm_dir = pmd_offset(pu_dir, address);
-#ifdef CONFIG_64BIT
+#if defined(CONFIG_64BIT) && !defined(CONFIG_DEBUG_PAGEALLOC)
if (MACHINE_HAS_HPAGE && !(address & ~HPAGE_MASK) &&
(address + HPAGE_SIZE <= start + size) &&
(address >= HPAGE_SIZE)) {
diff --git a/arch/s390/net/Makefile b/arch/s390/net/Makefile
new file mode 100644
index 0000000..90568c3
--- /dev/null
+++ b/arch/s390/net/Makefile
@@ -0,0 +1,4 @@
+#
+# Arch-specific network modules
+#
+obj-$(CONFIG_BPF_JIT) += bpf_jit.o bpf_jit_comp.o
diff --git a/arch/s390/net/bpf_jit.S b/arch/s390/net/bpf_jit.S
new file mode 100644
index 0000000..7e45d13
--- /dev/null
+++ b/arch/s390/net/bpf_jit.S
@@ -0,0 +1,130 @@
+/*
+ * BPF Jit compiler for s390, help functions.
+ *
+ * Copyright IBM Corp. 2012
+ *
+ * Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
+ */
+#include <linux/linkage.h>
+
+/*
+ * Calling convention:
+ * registers %r2, %r6-%r8, %r10-%r11, %r13, %r15 are call saved
+ * %r2: skb pointer
+ * %r3: offset parameter
+ * %r5: BPF A accumulator
+ * %r8: return address
+ * %r9: save register for skb pointer
+ * %r10: skb->data
+ * %r11: skb->len - skb->data_len (headlen)
+ * %r12: BPF X accumulator
+ *
+ * skb_copy_bits takes 4 parameters:
+ * %r2 = skb pointer
+ * %r3 = offset into skb data
+ * %r4 = length to copy
+ * %r5 = pointer to temp buffer
+ */
+#define SKBDATA %r8
+
+ /* A = *(u32 *) (skb->data+K+X) */
+ENTRY(sk_load_word_ind)
+ ar %r3,%r12 # offset += X
+ bmr %r8 # < 0 -> return with cc
+
+ /* A = *(u32 *) (skb->data+K) */
+ENTRY(sk_load_word)
+ llgfr %r1,%r3 # extend offset
+ ahi %r3,4 # offset + 4
+ clr %r11,%r3 # hlen <= offset + 4 ?
+ jl sk_load_word_slow
+ l %r5,0(%r1,%r10) # get word from skb
+ xr %r1,%r1 # set cc to zero
+ br %r8
+
+sk_load_word_slow:
+ lgr %r9,%r2 # save %r2
+ lhi %r4,4 # 4 bytes
+ la %r5,160(%r15) # pointer to temp buffer
+ brasl %r14,skb_copy_bits # get data from skb
+ l %r5,160(%r15) # load result from temp buffer
+ ltgr %r2,%r2 # set cc to (%r2 != 0)
+ lgr %r2,%r9 # restore %r2
+ br %r8
+
+ /* A = *(u16 *) (skb->data+K+X) */
+ENTRY(sk_load_half_ind)
+ ar %r3,%r12 # offset += X
+ bmr %r8 # < 0 -> return with cc
+
+ /* A = *(u16 *) (skb->data+K) */
+ENTRY(sk_load_half)
+ llgfr %r1,%r3 # extend offset
+ ahi %r3,2 # offset + 2
+ clr %r11,%r3 # hlen <= offset + 2 ?
+ jl sk_load_half_slow
+ llgh %r5,0(%r1,%r10) # get half from skb
+ xr %r1,%r1 # set cc to zero
+ br %r8
+
+sk_load_half_slow:
+ lgr %r9,%r2 # save %r2
+ lhi %r4,2 # 2 bytes
+ la %r5,162(%r15) # pointer to temp buffer
+ brasl %r14,skb_copy_bits # get data from skb
+ xc 160(2,%r15),160(%r15)
+ l %r5,160(%r15) # load result from temp buffer
+ ltgr %r2,%r2 # set cc to (%r2 != 0)
+ lgr %r2,%r9 # restore %r2
+ br %r8
+
+ /* A = *(u8 *) (skb->data+K+X) */
+ENTRY(sk_load_byte_ind)
+ ar %r3,%r12 # offset += X
+ bmr %r8 # < 0 -> return with cc
+
+ /* A = *(u8 *) (skb->data+K) */
+ENTRY(sk_load_byte)
+ llgfr %r1,%r3 # extend offset
+ clr %r11,%r3 # hlen < offset ?
+ jle sk_load_byte_slow
+ lhi %r5,0
+ ic %r5,0(%r1,%r10) # get byte from skb
+ xr %r1,%r1 # set cc to zero
+ br %r8
+
+sk_load_byte_slow:
+ lgr %r9,%r2 # save %r2
+ lhi %r4,1 # 1 bytes
+ la %r5,163(%r15) # pointer to temp buffer
+ brasl %r14,skb_copy_bits # get data from skb
+ xc 160(3,%r15),160(%r15)
+ l %r5,160(%r15) # load result from temp buffer
+ ltgr %r2,%r2 # set cc to (%r2 != 0)
+ lgr %r2,%r9 # restore %r2
+ br %r8
+
+ /* A = (*(u8 *)(skb->data+K) & 0xf) << 2 */
+ENTRY(sk_load_byte_msh)
+ llgfr %r1,%r3 # extend offset
+ clr %r11,%r3 # hlen < offset ?
+ jle sk_load_byte_slow
+ lhi %r12,0
+ ic %r12,0(%r1,%r10) # get byte from skb
+ nill %r12,0x0f
+ sll %r12,2
+ xr %r1,%r1 # set cc to zero
+ br %r8
+
+sk_load_byte_msh_slow:
+ lgr %r9,%r2 # save %r2
+ lhi %r4,2 # 2 bytes
+ la %r5,162(%r15) # pointer to temp buffer
+ brasl %r14,skb_copy_bits # get data from skb
+ xc 160(3,%r15),160(%r15)
+ l %r12,160(%r15) # load result from temp buffer
+ nill %r12,0x0f
+ sll %r12,2
+ ltgr %r2,%r2 # set cc to (%r2 != 0)
+ lgr %r2,%r9 # restore %r2
+ br %r8
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c
new file mode 100644
index 0000000..9b355b4
--- /dev/null
+++ b/arch/s390/net/bpf_jit_comp.c
@@ -0,0 +1,776 @@
+/*
+ * BPF Jit compiler for s390.
+ *
+ * Copyright IBM Corp. 2012
+ *
+ * Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
+ */
+#include <linux/moduleloader.h>
+#include <linux/netdevice.h>
+#include <linux/filter.h>
+#include <asm/cacheflush.h>
+#include <asm/processor.h>
+#include <asm/facility.h>
+
+/*
+ * Conventions:
+ * %r2 = skb pointer
+ * %r3 = offset parameter
+ * %r4 = scratch register / length parameter
+ * %r5 = BPF A accumulator
+ * %r8 = return address
+ * %r9 = save register for skb pointer
+ * %r10 = skb->data
+ * %r11 = skb->len - skb->data_len (headlen)
+ * %r12 = BPF X accumulator
+ * %r13 = literal pool pointer
+ * 0(%r15) - 63(%r15) scratch memory array with BPF_MEMWORDS
+ */
+int bpf_jit_enable __read_mostly;
+
+/*
+ * assembly code in arch/x86/net/bpf_jit.S
+ */
+extern u8 sk_load_word[], sk_load_half[], sk_load_byte[], sk_load_byte_msh[];
+extern u8 sk_load_word_ind[], sk_load_half_ind[], sk_load_byte_ind[];
+
+struct bpf_jit {
+ unsigned int seen;
+ u8 *start;
+ u8 *prg;
+ u8 *mid;
+ u8 *lit;
+ u8 *end;
+ u8 *base_ip;
+ u8 *ret0_ip;
+ u8 *exit_ip;
+ unsigned int off_load_word;
+ unsigned int off_load_half;
+ unsigned int off_load_byte;
+ unsigned int off_load_bmsh;
+ unsigned int off_load_iword;
+ unsigned int off_load_ihalf;
+ unsigned int off_load_ibyte;
+};
+
+#define BPF_SIZE_MAX 4096 /* Max size for program */
+
+#define SEEN_DATAREF 1 /* might call external helpers */
+#define SEEN_XREG 2 /* ebx is used */
+#define SEEN_MEM 4 /* use mem[] for temporary storage */
+#define SEEN_RET0 8 /* pc_ret0 points to a valid return 0 */
+#define SEEN_LITERAL 16 /* code uses literals */
+#define SEEN_LOAD_WORD 32 /* code uses sk_load_word */
+#define SEEN_LOAD_HALF 64 /* code uses sk_load_half */
+#define SEEN_LOAD_BYTE 128 /* code uses sk_load_byte */
+#define SEEN_LOAD_BMSH 256 /* code uses sk_load_byte_msh */
+#define SEEN_LOAD_IWORD 512 /* code uses sk_load_word_ind */
+#define SEEN_LOAD_IHALF 1024 /* code uses sk_load_half_ind */
+#define SEEN_LOAD_IBYTE 2048 /* code uses sk_load_byte_ind */
+
+#define EMIT2(op) \
+({ \
+ if (jit->prg + 2 <= jit->mid) \
+ *(u16 *) jit->prg = op; \
+ jit->prg += 2; \
+})
+
+#define EMIT4(op) \
+({ \
+ if (jit->prg + 4 <= jit->mid) \
+ *(u32 *) jit->prg = op; \
+ jit->prg += 4; \
+})
+
+#define EMIT4_DISP(op, disp) \
+({ \
+ unsigned int __disp = (disp) & 0xfff; \
+ EMIT4(op | __disp); \
+})
+
+#define EMIT4_IMM(op, imm) \
+({ \
+ unsigned int __imm = (imm) & 0xffff; \
+ EMIT4(op | __imm); \
+})
+
+#define EMIT4_PCREL(op, pcrel) \
+({ \
+ long __pcrel = ((pcrel) >> 1) & 0xffff; \
+ EMIT4(op | __pcrel); \
+})
+
+#define EMIT6(op1, op2) \
+({ \
+ if (jit->prg + 6 <= jit->mid) { \
+ *(u32 *) jit->prg = op1; \
+ *(u16 *) (jit->prg + 4) = op2; \
+ } \
+ jit->prg += 6; \
+})
+
+#define EMIT6_DISP(op1, op2, disp) \
+({ \
+ unsigned int __disp = (disp) & 0xfff; \
+ EMIT6(op1 | __disp, op2); \
+})
+
+#define EMIT6_IMM(op, imm) \
+({ \
+ unsigned int __imm = (imm); \
+ EMIT6(op | (__imm >> 16), __imm & 0xffff); \
+})
+
+#define EMIT_CONST(val) \
+({ \
+ unsigned int ret; \
+ ret = (unsigned int) (jit->lit - jit->base_ip); \
+ jit->seen |= SEEN_LITERAL; \
+ if (jit->lit + 4 <= jit->end) \
+ *(u32 *) jit->lit = val; \
+ jit->lit += 4; \
+ ret; \
+})
+
+#define EMIT_FN_CONST(bit, fn) \
+({ \
+ unsigned int ret; \
+ ret = (unsigned int) (jit->lit - jit->base_ip); \
+ if (jit->seen & bit) { \
+ jit->seen |= SEEN_LITERAL; \
+ if (jit->lit + 8 <= jit->end) \
+ *(void **) jit->lit = fn; \
+ jit->lit += 8; \
+ } \
+ ret; \
+})
+
+static void bpf_jit_prologue(struct bpf_jit *jit)
+{
+ /* Save registers and create stack frame if necessary */
+ if (jit->seen & SEEN_DATAREF) {
+ /* stmg %r8,%r15,88(%r15) */
+ EMIT6(0xeb8ff058, 0x0024);
+ /* lgr %r14,%r15 */
+ EMIT4(0xb90400ef);
+ /* ahi %r15,<offset> */
+ EMIT4_IMM(0xa7fa0000, (jit->seen & SEEN_MEM) ? -112 : -80);
+ /* stg %r14,152(%r15) */
+ EMIT6(0xe3e0f098, 0x0024);
+ } else if ((jit->seen & SEEN_XREG) && (jit->seen & SEEN_LITERAL))
+ /* stmg %r12,%r13,120(%r15) */
+ EMIT6(0xebcdf078, 0x0024);
+ else if (jit->seen & SEEN_XREG)
+ /* stg %r12,120(%r15) */
+ EMIT6(0xe3c0f078, 0x0024);
+ else if (jit->seen & SEEN_LITERAL)
+ /* stg %r13,128(%r15) */
+ EMIT6(0xe3d0f080, 0x0024);
+
+ /* Setup literal pool */
+ if (jit->seen & SEEN_LITERAL) {
+ /* basr %r13,0 */
+ EMIT2(0x0dd0);
+ jit->base_ip = jit->prg;
+ }
+ jit->off_load_word = EMIT_FN_CONST(SEEN_LOAD_WORD, sk_load_word);
+ jit->off_load_half = EMIT_FN_CONST(SEEN_LOAD_HALF, sk_load_half);
+ jit->off_load_byte = EMIT_FN_CONST(SEEN_LOAD_BYTE, sk_load_byte);
+ jit->off_load_bmsh = EMIT_FN_CONST(SEEN_LOAD_BMSH, sk_load_byte_msh);
+ jit->off_load_iword = EMIT_FN_CONST(SEEN_LOAD_IWORD, sk_load_word_ind);
+ jit->off_load_ihalf = EMIT_FN_CONST(SEEN_LOAD_IHALF, sk_load_half_ind);
+ jit->off_load_ibyte = EMIT_FN_CONST(SEEN_LOAD_IBYTE, sk_load_byte_ind);
+
+ /* Filter needs to access skb data */
+ if (jit->seen & SEEN_DATAREF) {
+ /* l %r11,<len>(%r2) */
+ EMIT4_DISP(0x58b02000, offsetof(struct sk_buff, len));
+ /* s %r11,<data_len>(%r2) */
+ EMIT4_DISP(0x5bb02000, offsetof(struct sk_buff, data_len));
+ /* lg %r10,<data>(%r2) */
+ EMIT6_DISP(0xe3a02000, 0x0004,
+ offsetof(struct sk_buff, data));
+ }
+}
+
+static void bpf_jit_epilogue(struct bpf_jit *jit)
+{
+ /* Return 0 */
+ if (jit->seen & SEEN_RET0) {
+ jit->ret0_ip = jit->prg;
+ /* lghi %r2,0 */
+ EMIT4(0xa7290000);
+ }
+ jit->exit_ip = jit->prg;
+ /* Restore registers */
+ if (jit->seen & SEEN_DATAREF)
+ /* lmg %r8,%r15,<offset>(%r15) */
+ EMIT6_DISP(0xeb8ff000, 0x0004,
+ (jit->seen & SEEN_MEM) ? 200 : 168);
+ else if ((jit->seen & SEEN_XREG) && (jit->seen & SEEN_LITERAL))
+ /* lmg %r12,%r13,120(%r15) */
+ EMIT6(0xebcdf078, 0x0004);
+ else if (jit->seen & SEEN_XREG)
+ /* lg %r12,120(%r15) */
+ EMIT6(0xe3c0f078, 0x0004);
+ else if (jit->seen & SEEN_LITERAL)
+ /* lg %r13,128(%r15) */
+ EMIT6(0xe3d0f080, 0x0004);
+ /* br %r14 */
+ EMIT2(0x07fe);
+}
+
+/*
+ * make sure we dont leak kernel information to user
+ */
+static void bpf_jit_noleaks(struct bpf_jit *jit, struct sock_filter *filter)
+{
+ /* Clear temporary memory if (seen & SEEN_MEM) */
+ if (jit->seen & SEEN_MEM)
+ /* xc 0(64,%r15),0(%r15) */
+ EMIT6(0xd73ff000, 0xf000);
+ /* Clear X if (seen & SEEN_XREG) */
+ if (jit->seen & SEEN_XREG)
+ /* lhi %r12,0 */
+ EMIT4(0xa7c80000);
+ /* Clear A if the first register does not set it. */
+ switch (filter[0].code) {
+ case BPF_S_LD_W_ABS:
+ case BPF_S_LD_H_ABS:
+ case BPF_S_LD_B_ABS:
+ case BPF_S_LD_W_LEN:
+ case BPF_S_LD_W_IND:
+ case BPF_S_LD_H_IND:
+ case BPF_S_LD_B_IND:
+ case BPF_S_LDX_B_MSH:
+ case BPF_S_LD_IMM:
+ case BPF_S_LD_MEM:
+ case BPF_S_MISC_TXA:
+ case BPF_S_ANC_PROTOCOL:
+ case BPF_S_ANC_PKTTYPE:
+ case BPF_S_ANC_IFINDEX:
+ case BPF_S_ANC_MARK:
+ case BPF_S_ANC_QUEUE:
+ case BPF_S_ANC_HATYPE:
+ case BPF_S_ANC_RXHASH:
+ case BPF_S_ANC_CPU:
+ case BPF_S_RET_K:
+ /* first instruction sets A register */
+ break;
+ default: /* A = 0 */
+ /* lhi %r5,0 */
+ EMIT4(0xa7580000);
+ }
+}
+
+static int bpf_jit_insn(struct bpf_jit *jit, struct sock_filter *filter,
+ unsigned int *addrs, int i, int last)
+{
+ unsigned int K;
+ int offset;
+ unsigned int mask;
+
+ K = filter->k;
+ switch (filter->code) {
+ case BPF_S_ALU_ADD_X: /* A += X */
+ jit->seen |= SEEN_XREG;
+ /* ar %r5,%r12 */
+ EMIT2(0x1a5c);
+ break;
+ case BPF_S_ALU_ADD_K: /* A += K */
+ if (!K)
+ break;
+ if (K <= 16383)
+ /* ahi %r5,<K> */
+ EMIT4_IMM(0xa75a0000, K);
+ else if (test_facility(21))
+ /* alfi %r5,<K> */
+ EMIT6_IMM(0xc25b0000, K);
+ else
+ /* a %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x5a50d000, EMIT_CONST(K));
+ break;
+ case BPF_S_ALU_SUB_X: /* A -= X */
+ jit->seen |= SEEN_XREG;
+ /* sr %r5,%r12 */
+ EMIT2(0x1b5c);
+ break;
+ case BPF_S_ALU_SUB_K: /* A -= K */
+ if (!K)
+ break;
+ if (K <= 16384)
+ /* ahi %r5,-K */
+ EMIT4_IMM(0xa75a0000, -K);
+ else if (test_facility(21))
+ /* alfi %r5,-K */
+ EMIT6_IMM(0xc25b0000, -K);
+ else
+ /* s %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x5b50d000, EMIT_CONST(K));
+ break;
+ case BPF_S_ALU_MUL_X: /* A *= X */
+ jit->seen |= SEEN_XREG;
+ /* msr %r5,%r12 */
+ EMIT4(0xb252005c);
+ break;
+ case BPF_S_ALU_MUL_K: /* A *= K */
+ if (K <= 16383)
+ /* mhi %r5,K */
+ EMIT4_IMM(0xa75c0000, K);
+ else if (test_facility(34))
+ /* msfi %r5,<K> */
+ EMIT6_IMM(0xc2510000, K);
+ else
+ /* ms %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x7150d000, EMIT_CONST(K));
+ break;
+ case BPF_S_ALU_DIV_X: /* A /= X */
+ jit->seen |= SEEN_XREG | SEEN_RET0;
+ /* ltr %r12,%r12 */
+ EMIT2(0x12cc);
+ /* jz <ret0> */
+ EMIT4_PCREL(0xa7840000, (jit->ret0_ip - jit->prg));
+ /* lhi %r4,0 */
+ EMIT4(0xa7480000);
+ /* dr %r4,%r12 */
+ EMIT2(0x1d4c);
+ break;
+ case BPF_S_ALU_DIV_K: /* A = reciprocal_divide(A, K) */
+ /* m %r4,<d(K)>(%r13) */
+ EMIT4_DISP(0x5c40d000, EMIT_CONST(K));
+ /* lr %r5,%r4 */
+ EMIT2(0x1854);
+ break;
+ case BPF_S_ALU_AND_X: /* A &= X */
+ jit->seen |= SEEN_XREG;
+ /* nr %r5,%r12 */
+ EMIT2(0x145c);
+ break;
+ case BPF_S_ALU_AND_K: /* A &= K */
+ if (test_facility(21))
+ /* nilf %r5,<K> */
+ EMIT6_IMM(0xc05b0000, K);
+ else
+ /* n %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x5450d000, EMIT_CONST(K));
+ break;
+ case BPF_S_ALU_OR_X: /* A |= X */
+ jit->seen |= SEEN_XREG;
+ /* or %r5,%r12 */
+ EMIT2(0x165c);
+ break;
+ case BPF_S_ALU_OR_K: /* A |= K */
+ if (test_facility(21))
+ /* oilf %r5,<K> */
+ EMIT6_IMM(0xc05d0000, K);
+ else
+ /* o %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x5650d000, EMIT_CONST(K));
+ break;
+ case BPF_S_ANC_ALU_XOR_X: /* A ^= X; */
+ jit->seen |= SEEN_XREG;
+ /* xr %r5,%r12 */
+ EMIT2(0x175c);
+ break;
+ case BPF_S_ALU_LSH_X: /* A <<= X; */
+ jit->seen |= SEEN_XREG;
+ /* sll %r5,0(%r12) */
+ EMIT4(0x8950c000);
+ break;
+ case BPF_S_ALU_LSH_K: /* A <<= K */
+ if (K == 0)
+ break;
+ /* sll %r5,K */
+ EMIT4_DISP(0x89500000, K);
+ break;
+ case BPF_S_ALU_RSH_X: /* A >>= X; */
+ jit->seen |= SEEN_XREG;
+ /* srl %r5,0(%r12) */
+ EMIT4(0x8850c000);
+ break;
+ case BPF_S_ALU_RSH_K: /* A >>= K; */
+ if (K == 0)
+ break;
+ /* srl %r5,K */
+ EMIT4_DISP(0x88500000, K);
+ break;
+ case BPF_S_ALU_NEG: /* A = -A */
+ /* lnr %r5,%r5 */
+ EMIT2(0x1155);
+ break;
+ case BPF_S_JMP_JA: /* ip += K */
+ offset = addrs[i + K] + jit->start - jit->prg;
+ EMIT4_PCREL(0xa7f40000, offset);
+ break;
+ case BPF_S_JMP_JGT_K: /* ip += (A > K) ? jt : jf */
+ mask = 0x200000; /* jh */
+ goto kbranch;
+ case BPF_S_JMP_JGE_K: /* ip += (A >= K) ? jt : jf */
+ mask = 0xa00000; /* jhe */
+ goto kbranch;
+ case BPF_S_JMP_JEQ_K: /* ip += (A == K) ? jt : jf */
+ mask = 0x800000; /* je */
+kbranch: /* Emit compare if the branch targets are different */
+ if (filter->jt != filter->jf) {
+ if (K <= 16383)
+ /* chi %r5,<K> */
+ EMIT4_IMM(0xa75e0000, K);
+ else if (test_facility(21))
+ /* clfi %r5,<K> */
+ EMIT6_IMM(0xc25f0000, K);
+ else
+ /* c %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x5950d000, EMIT_CONST(K));
+ }
+branch: if (filter->jt == filter->jf) {
+ if (filter->jt == 0)
+ break;
+ /* j <jt> */
+ offset = addrs[i + filter->jt] + jit->start - jit->prg;
+ EMIT4_PCREL(0xa7f40000, offset);
+ break;
+ }
+ if (filter->jt != 0) {
+ /* brc <mask>,<jt> */
+ offset = addrs[i + filter->jt] + jit->start - jit->prg;
+ EMIT4_PCREL(0xa7040000 | mask, offset);
+ }
+ if (filter->jf != 0) {
+ /* brc <mask^15>,<jf> */
+ offset = addrs[i + filter->jf] + jit->start - jit->prg;
+ EMIT4_PCREL(0xa7040000 | (mask ^ 0xf00000), offset);
+ }
+ break;
+ case BPF_S_JMP_JSET_K: /* ip += (A & K) ? jt : jf */
+ mask = 0x700000; /* jnz */
+ /* Emit test if the branch targets are different */
+ if (filter->jt != filter->jf) {
+ if (K > 65535) {
+ /* lr %r4,%r5 */
+ EMIT2(0x1845);
+ /* n %r4,<d(K)>(%r13) */
+ EMIT4_DISP(0x5440d000, EMIT_CONST(K));
+ } else
+ /* tmll %r5,K */
+ EMIT4_IMM(0xa7510000, K);
+ }
+ goto branch;
+ case BPF_S_JMP_JGT_X: /* ip += (A > X) ? jt : jf */
+ mask = 0x200000; /* jh */
+ goto xbranch;
+ case BPF_S_JMP_JGE_X: /* ip += (A >= X) ? jt : jf */
+ mask = 0xa00000; /* jhe */
+ goto xbranch;
+ case BPF_S_JMP_JEQ_X: /* ip += (A == X) ? jt : jf */
+ mask = 0x800000; /* je */
+xbranch: /* Emit compare if the branch targets are different */
+ if (filter->jt != filter->jf) {
+ jit->seen |= SEEN_XREG;
+ /* cr %r5,%r12 */
+ EMIT2(0x195c);
+ }
+ goto branch;
+ case BPF_S_JMP_JSET_X: /* ip += (A & X) ? jt : jf */
+ mask = 0x700000; /* jnz */
+ /* Emit test if the branch targets are different */
+ if (filter->jt != filter->jf) {
+ jit->seen |= SEEN_XREG;
+ /* lr %r4,%r5 */
+ EMIT2(0x1845);
+ /* nr %r4,%r12 */
+ EMIT2(0x144c);
+ }
+ goto branch;
+ case BPF_S_LD_W_ABS: /* A = *(u32 *) (skb->data+K) */
+ jit->seen |= SEEN_DATAREF | SEEN_RET0 | SEEN_LOAD_WORD;
+ offset = jit->off_load_word;
+ goto load_abs;
+ case BPF_S_LD_H_ABS: /* A = *(u16 *) (skb->data+K) */
+ jit->seen |= SEEN_DATAREF | SEEN_RET0 | SEEN_LOAD_HALF;
+ offset = jit->off_load_half;
+ goto load_abs;
+ case BPF_S_LD_B_ABS: /* A = *(u8 *) (skb->data+K) */
+ jit->seen |= SEEN_DATAREF | SEEN_RET0 | SEEN_LOAD_BYTE;
+ offset = jit->off_load_byte;
+load_abs: if ((int) K < 0)
+ goto out;
+call_fn: /* lg %r1,<d(function)>(%r13) */
+ EMIT6_DISP(0xe310d000, 0x0004, offset);
+ /* l %r3,<d(K)>(%r13) */
+ EMIT4_DISP(0x5830d000, EMIT_CONST(K));
+ /* basr %r8,%r1 */
+ EMIT2(0x0d81);
+ /* jnz <ret0> */
+ EMIT4_PCREL(0xa7740000, (jit->ret0_ip - jit->prg));
+ break;
+ case BPF_S_LD_W_IND: /* A = *(u32 *) (skb->data+K+X) */
+ jit->seen |= SEEN_DATAREF | SEEN_RET0 | SEEN_LOAD_IWORD;
+ offset = jit->off_load_iword;
+ goto call_fn;
+ case BPF_S_LD_H_IND: /* A = *(u16 *) (skb->data+K+X) */
+ jit->seen |= SEEN_DATAREF | SEEN_RET0 | SEEN_LOAD_IHALF;
+ offset = jit->off_load_ihalf;
+ goto call_fn;
+ case BPF_S_LD_B_IND: /* A = *(u8 *) (skb->data+K+X) */
+ jit->seen |= SEEN_DATAREF | SEEN_RET0 | SEEN_LOAD_IBYTE;
+ offset = jit->off_load_ibyte;
+ goto call_fn;
+ case BPF_S_LDX_B_MSH:
+ /* X = (*(u8 *)(skb->data+K) & 0xf) << 2 */
+ jit->seen |= SEEN_RET0;
+ if ((int) K < 0) {
+ /* j <ret0> */
+ EMIT4_PCREL(0xa7f40000, (jit->ret0_ip - jit->prg));
+ break;
+ }
+ jit->seen |= SEEN_DATAREF | SEEN_LOAD_BMSH;
+ offset = jit->off_load_bmsh;
+ goto call_fn;
+ case BPF_S_LD_W_LEN: /* A = skb->len; */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, len) != 4);
+ /* l %r5,<d(len)>(%r2) */
+ EMIT4_DISP(0x58502000, offsetof(struct sk_buff, len));
+ break;
+ case BPF_S_LDX_W_LEN: /* X = skb->len; */
+ jit->seen |= SEEN_XREG;
+ /* l %r12,<d(len)>(%r2) */
+ EMIT4_DISP(0x58c02000, offsetof(struct sk_buff, len));
+ break;
+ case BPF_S_LD_IMM: /* A = K */
+ if (K <= 16383)
+ /* lhi %r5,K */
+ EMIT4_IMM(0xa7580000, K);
+ else if (test_facility(21))
+ /* llilf %r5,<K> */
+ EMIT6_IMM(0xc05f0000, K);
+ else
+ /* l %r5,<d(K)>(%r13) */
+ EMIT4_DISP(0x5850d000, EMIT_CONST(K));
+ break;
+ case BPF_S_LDX_IMM: /* X = K */
+ jit->seen |= SEEN_XREG;
+ if (K <= 16383)
+ /* lhi %r12,<K> */
+ EMIT4_IMM(0xa7c80000, K);
+ else if (test_facility(21))
+ /* llilf %r12,<K> */
+ EMIT6_IMM(0xc0cf0000, K);
+ else
+ /* l %r12,<d(K)>(%r13) */
+ EMIT4_DISP(0x58c0d000, EMIT_CONST(K));
+ break;
+ case BPF_S_LD_MEM: /* A = mem[K] */
+ jit->seen |= SEEN_MEM;
+ /* l %r5,<K>(%r15) */
+ EMIT4_DISP(0x5850f000,
+ (jit->seen & SEEN_DATAREF) ? 160 + K*4 : K*4);
+ break;
+ case BPF_S_LDX_MEM: /* X = mem[K] */
+ jit->seen |= SEEN_XREG | SEEN_MEM;
+ /* l %r12,<K>(%r15) */
+ EMIT4_DISP(0x58c0f000,
+ (jit->seen & SEEN_DATAREF) ? 160 + K*4 : K*4);
+ break;
+ case BPF_S_MISC_TAX: /* X = A */
+ jit->seen |= SEEN_XREG;
+ /* lr %r12,%r5 */
+ EMIT2(0x18c5);
+ break;
+ case BPF_S_MISC_TXA: /* A = X */
+ jit->seen |= SEEN_XREG;
+ /* lr %r5,%r12 */
+ EMIT2(0x185c);
+ break;
+ case BPF_S_RET_K:
+ if (K == 0) {
+ jit->seen |= SEEN_RET0;
+ if (last)
+ break;
+ /* j <ret0> */
+ EMIT4_PCREL(0xa7f40000, jit->ret0_ip - jit->prg);
+ } else {
+ if (K <= 16383)
+ /* lghi %r2,K */
+ EMIT4_IMM(0xa7290000, K);
+ else
+ /* llgf %r2,<K>(%r13) */
+ EMIT6_DISP(0xe320d000, 0x0016, EMIT_CONST(K));
+ /* j <exit> */
+ if (last && !(jit->seen & SEEN_RET0))
+ break;
+ EMIT4_PCREL(0xa7f40000, jit->exit_ip - jit->prg);
+ }
+ break;
+ case BPF_S_RET_A:
+ /* llgfr %r2,%r5 */
+ EMIT4(0xb9160025);
+ /* j <exit> */
+ EMIT4_PCREL(0xa7f40000, jit->exit_ip - jit->prg);
+ break;
+ case BPF_S_ST: /* mem[K] = A */
+ jit->seen |= SEEN_MEM;
+ /* st %r5,<K>(%r15) */
+ EMIT4_DISP(0x5050f000,
+ (jit->seen & SEEN_DATAREF) ? 160 + K*4 : K*4);
+ break;
+ case BPF_S_STX: /* mem[K] = X : mov %ebx,off8(%rbp) */
+ jit->seen |= SEEN_XREG | SEEN_MEM;
+ /* st %r12,<K>(%r15) */
+ EMIT4_DISP(0x50c0f000,
+ (jit->seen & SEEN_DATAREF) ? 160 + K*4 : K*4);
+ break;
+ case BPF_S_ANC_PROTOCOL: /* A = ntohs(skb->protocol); */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, protocol) != 2);
+ /* lhi %r5,0 */
+ EMIT4(0xa7580000);
+ /* icm %r5,3,<d(protocol)>(%r2) */
+ EMIT4_DISP(0xbf532000, offsetof(struct sk_buff, protocol));
+ break;
+ case BPF_S_ANC_IFINDEX: /* if (!skb->dev) return 0;
+ * A = skb->dev->ifindex */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct net_device, ifindex) != 4);
+ jit->seen |= SEEN_RET0;
+ /* lg %r1,<d(dev)>(%r2) */
+ EMIT6_DISP(0xe3102000, 0x0004, offsetof(struct sk_buff, dev));
+ /* ltgr %r1,%r1 */
+ EMIT4(0xb9020011);
+ /* jz <ret0> */
+ EMIT4_PCREL(0xa7840000, jit->ret0_ip - jit->prg);
+ /* l %r5,<d(ifindex)>(%r1) */
+ EMIT4_DISP(0x58501000, offsetof(struct net_device, ifindex));
+ break;
+ case BPF_S_ANC_MARK: /* A = skb->mark */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, mark) != 4);
+ /* l %r5,<d(mark)>(%r2) */
+ EMIT4_DISP(0x58502000, offsetof(struct sk_buff, mark));
+ break;
+ case BPF_S_ANC_QUEUE: /* A = skb->queue_mapping */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, queue_mapping) != 2);
+ /* lhi %r5,0 */
+ EMIT4(0xa7580000);
+ /* icm %r5,3,<d(queue_mapping)>(%r2) */
+ EMIT4_DISP(0xbf532000, offsetof(struct sk_buff, queue_mapping));
+ break;
+ case BPF_S_ANC_HATYPE: /* if (!skb->dev) return 0;
+ * A = skb->dev->type */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct net_device, type) != 2);
+ jit->seen |= SEEN_RET0;
+ /* lg %r1,<d(dev)>(%r2) */
+ EMIT6_DISP(0xe3102000, 0x0004, offsetof(struct sk_buff, dev));
+ /* ltgr %r1,%r1 */
+ EMIT4(0xb9020011);
+ /* jz <ret0> */
+ EMIT4_PCREL(0xa7840000, jit->ret0_ip - jit->prg);
+ /* lhi %r5,0 */
+ EMIT4(0xa7580000);
+ /* icm %r5,3,<d(type)>(%r1) */
+ EMIT4_DISP(0xbf531000, offsetof(struct net_device, type));
+ break;
+ case BPF_S_ANC_RXHASH: /* A = skb->rxhash */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, rxhash) != 4);
+ /* l %r5,<d(rxhash)>(%r2) */
+ EMIT4_DISP(0x58502000, offsetof(struct sk_buff, rxhash));
+ break;
+ case BPF_S_ANC_CPU: /* A = smp_processor_id() */
+#ifdef CONFIG_SMP
+ /* l %r5,<d(cpu_nr)> */
+ EMIT4_DISP(0x58500000, offsetof(struct _lowcore, cpu_nr));
+#else
+ /* lhi %r5,0 */
+ EMIT4(0xa7580000);
+#endif
+ break;
+ default: /* too complex, give up */
+ goto out;
+ }
+ addrs[i] = jit->prg - jit->start;
+ return 0;
+out:
+ return -1;
+}
+
+void bpf_jit_compile(struct sk_filter *fp)
+{
+ unsigned long size, prg_len, lit_len;
+ struct bpf_jit jit, cjit;
+ unsigned int *addrs;
+ int pass, i;
+
+ if (!bpf_jit_enable)
+ return;
+ addrs = kmalloc(fp->len * sizeof(*addrs), GFP_KERNEL);
+ if (addrs == NULL)
+ return;
+ memset(addrs, 0, fp->len * sizeof(*addrs));
+ memset(&jit, 0, sizeof(cjit));
+ memset(&cjit, 0, sizeof(cjit));
+
+ for (pass = 0; pass < 10; pass++) {
+ jit.prg = jit.start;
+ jit.lit = jit.mid;
+
+ bpf_jit_prologue(&jit);
+ bpf_jit_noleaks(&jit, fp->insns);
+ for (i = 0; i < fp->len; i++) {
+ if (bpf_jit_insn(&jit, fp->insns + i, addrs, i,
+ i == fp->len - 1))
+ goto out;
+ }
+ bpf_jit_epilogue(&jit);
+ if (jit.start) {
+ WARN_ON(jit.prg > cjit.prg || jit.lit > cjit.lit);
+ if (memcmp(&jit, &cjit, sizeof(jit)) == 0)
+ break;
+ } else if (jit.prg == cjit.prg && jit.lit == cjit.lit) {
+ prg_len = jit.prg - jit.start;
+ lit_len = jit.lit - jit.mid;
+ size = max_t(unsigned long, prg_len + lit_len,
+ sizeof(struct work_struct));
+ if (size >= BPF_SIZE_MAX)
+ goto out;
+ jit.start = module_alloc(size);
+ if (!jit.start)
+ goto out;
+ jit.prg = jit.mid = jit.start + prg_len;
+ jit.lit = jit.end = jit.start + prg_len + lit_len;
+ jit.base_ip += (unsigned long) jit.start;
+ jit.exit_ip += (unsigned long) jit.start;
+ jit.ret0_ip += (unsigned long) jit.start;
+ }
+ cjit = jit;
+ }
+ if (bpf_jit_enable > 1) {
+ pr_err("flen=%d proglen=%lu pass=%d image=%p\n",
+ fp->len, jit.end - jit.start, pass, jit.start);
+ if (jit.start) {
+ printk(KERN_ERR "JIT code:\n");
+ print_fn_code(jit.start, jit.mid - jit.start);
+ print_hex_dump(KERN_ERR, "JIT literals:\n",
+ DUMP_PREFIX_ADDRESS, 16, 1,
+ jit.mid, jit.end - jit.mid, false);
+ }
+ }
+ if (jit.start)
+ fp->bpf_func = (void *) jit.start;
+out:
+ kfree(addrs);
+}
+
+static void jit_free_defer(struct work_struct *arg)
+{
+ module_free(NULL, arg);
+}
+
+/* run from softirq, we must use a work_struct to call
+ * module_free() from process context
+ */
+void bpf_jit_free(struct sk_filter *fp)
+{
+ struct work_struct *work;
+
+ if (fp->bpf_func == sk_run_filter)
+ return;
+ work = (struct work_struct *)fp->bpf_func;
+ INIT_WORK(work, jit_free_defer);
+ schedule_work(work);
+}
diff --git a/arch/s390/oprofile/init.c b/arch/s390/oprofile/init.c
index a1e9d69..584b936 100644
--- a/arch/s390/oprofile/init.c
+++ b/arch/s390/oprofile/init.c
@@ -169,7 +169,7 @@
if (*offset)
return -EINVAL;
retval = oprofilefs_ulong_from_user(&val, buf, count);
- if (retval)
+ if (retval <= 0)
return retval;
if (val < oprofile_min_interval)
oprofile_hw_interval = oprofile_min_interval;
@@ -212,7 +212,7 @@
return -EINVAL;
retval = oprofilefs_ulong_from_user(&val, buf, count);
- if (retval)
+ if (retval <= 0)
return retval;
if (val != 0)
return -EINVAL;
@@ -243,7 +243,7 @@
return -EINVAL;
retval = oprofilefs_ulong_from_user(&val, buf, count);
- if (retval)
+ if (retval <= 0)
return retval;
if (val != 0 && val != 1)
@@ -278,7 +278,7 @@
return -EINVAL;
retval = oprofilefs_ulong_from_user(&val, buf, count);
- if (retval)
+ if (retval <= 0)
return retval;
if (val != 0 && val != 1)
@@ -317,7 +317,7 @@
return -EINVAL;
retval = oprofilefs_ulong_from_user(&val, buf, count);
- if (retval)
+ if (retval <= 0)
return retval;
if (val != 0 && val != 1)
diff --git a/arch/score/kernel/process.c b/arch/score/kernel/process.c
index 2707023..637970c 100644
--- a/arch/score/kernel/process.c
+++ b/arch/score/kernel/process.c
@@ -27,6 +27,7 @@
#include <linux/reboot.h>
#include <linux/elfcore.h>
#include <linux/pm.h>
+#include <linux/rcupdate.h>
void (*pm_power_off)(void);
EXPORT_SYMBOL(pm_power_off);
@@ -50,9 +51,10 @@
{
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched())
barrier();
-
+ rcu_idle_exit();
schedule_preempt_disabled();
}
}
diff --git a/arch/sh/drivers/pci/pci.c b/arch/sh/drivers/pci/pci.c
index 40db2d0..a7e078f 100644
--- a/arch/sh/drivers/pci/pci.c
+++ b/arch/sh/drivers/pci/pci.c
@@ -192,11 +192,6 @@
return pci_enable_resources(dev, mask);
}
-void __init pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
static void __init
pcibios_bus_report_status_early(struct pci_channel *hose,
int top_bus, int current_bus,
diff --git a/arch/sh/kernel/cpu/sh5/entry.S b/arch/sh/kernel/cpu/sh5/entry.S
index b7cf6a5..7e605b9 100644
--- a/arch/sh/kernel/cpu/sh5/entry.S
+++ b/arch/sh/kernel/cpu/sh5/entry.S
@@ -933,7 +933,7 @@
pta restore_all, tr1
- movi _TIF_SIGPENDING, r8
+ movi (_TIF_SIGPENDING|_TIF_NOTIFY_RESUME), r8
and r8, r7, r8
pta work_notifysig, tr0
bne r8, ZERO, tr0
diff --git a/arch/sh/kernel/entry-common.S b/arch/sh/kernel/entry-common.S
index f67601c..b96489d 100644
--- a/arch/sh/kernel/entry-common.S
+++ b/arch/sh/kernel/entry-common.S
@@ -139,7 +139,7 @@
! r8: current_thread_info
! t: result of "tst #_TIF_NEED_RESCHED, r0"
bf/s work_resched
- tst #_TIF_SIGPENDING, r0
+ tst #(_TIF_SIGPENDING | _TIF_NOTIFY_RESUME), r0
work_notifysig:
bt/s __restore_all
mov r15, r4
diff --git a/arch/sparc/kernel/leon_pci.c b/arch/sparc/kernel/leon_pci.c
index 21dcda7..fc05211 100644
--- a/arch/sparc/kernel/leon_pci.c
+++ b/arch/sparc/kernel/leon_pci.c
@@ -102,15 +102,6 @@
return pci_enable_resources(dev, mask);
}
-void __devinit pcibios_update_irq(struct pci_dev *dev, int irq)
-{
-#ifdef CONFIG_PCI_DEBUG
- printk(KERN_DEBUG "LEONPCI: Assigning IRQ %02d to %s\n", irq,
- pci_name(dev));
-#endif
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
/* in/out routines taken from pcic.c
*
* This probably belongs here rather than ioport.c because
diff --git a/arch/sparc/kernel/module.c b/arch/sparc/kernel/module.c
index 15e0a16..f1ddc0d 100644
--- a/arch/sparc/kernel/module.c
+++ b/arch/sparc/kernel/module.c
@@ -48,9 +48,7 @@
return NULL;
ret = module_map(size);
- if (!ret)
- ret = ERR_PTR(-ENOMEM);
- else
+ if (ret)
memset(ret, 0, size);
return ret;
@@ -116,6 +114,10 @@
v = sym->st_value + rel[i].r_addend;
switch (ELF_R_TYPE(rel[i].r_info) & 0xff) {
+ case R_SPARC_DISP32:
+ v -= (Elf_Addr) location;
+ *loc32 = v;
+ break;
#ifdef CONFIG_SPARC64
case R_SPARC_64:
location[0] = v >> 56;
@@ -128,11 +130,6 @@
location[7] = v >> 0;
break;
- case R_SPARC_DISP32:
- v -= (Elf_Addr) location;
- *loc32 = v;
- break;
-
case R_SPARC_WDISP19:
v -= (Elf_Addr) location;
*loc32 = (*loc32 & ~0x7ffff) |
diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index 065b88c..acc8c83 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -622,10 +622,6 @@
{
}
-void pcibios_update_irq(struct pci_dev *pdev, int irq)
-{
-}
-
resource_size_t pcibios_align_resource(void *data, const struct resource *res,
resource_size_t size, resource_size_t align)
{
diff --git a/arch/tile/include/asm/topology.h b/arch/tile/include/asm/topology.h
index 7a7ce39..d5e86c9 100644
--- a/arch/tile/include/asm/topology.h
+++ b/arch/tile/include/asm/topology.h
@@ -69,7 +69,6 @@
| 1*SD_BALANCE_FORK \
| 0*SD_BALANCE_WAKE \
| 0*SD_WAKE_AFFINE \
- | 0*SD_PREFER_LOCAL \
| 0*SD_SHARE_CPUPOWER \
| 0*SD_SHARE_PKG_RESOURCES \
| 0*SD_SERIALIZE \
diff --git a/arch/tile/include/gxio/iorpc_trio.h b/arch/tile/include/gxio/iorpc_trio.h
index 15fb779..58105c3 100644
--- a/arch/tile/include/gxio/iorpc_trio.h
+++ b/arch/tile/include/gxio/iorpc_trio.h
@@ -25,21 +25,23 @@
#include <linux/module.h>
#include <asm/pgtable.h>
-#define GXIO_TRIO_OP_ALLOC_ASIDS IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1400)
+#define GXIO_TRIO_OP_DEALLOC_ASID IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1400)
+#define GXIO_TRIO_OP_ALLOC_ASIDS IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1401)
-#define GXIO_TRIO_OP_ALLOC_MEMORY_MAPS IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1402)
+#define GXIO_TRIO_OP_ALLOC_MEMORY_MAPS IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1404)
-#define GXIO_TRIO_OP_ALLOC_PIO_REGIONS IORPC_OPCODE(IORPC_FORMAT_NONE, 0x140e)
-#define GXIO_TRIO_OP_INIT_PIO_REGION_AUX IORPC_OPCODE(IORPC_FORMAT_NONE, 0x140f)
+#define GXIO_TRIO_OP_ALLOC_PIO_REGIONS IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1412)
-#define GXIO_TRIO_OP_INIT_MEMORY_MAP_MMU_AUX IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x1417)
-#define GXIO_TRIO_OP_GET_PORT_PROPERTY IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x1418)
-#define GXIO_TRIO_OP_CONFIG_LEGACY_INTR IORPC_OPCODE(IORPC_FORMAT_KERNEL_INTERRUPT, 0x1419)
-#define GXIO_TRIO_OP_CONFIG_MSI_INTR IORPC_OPCODE(IORPC_FORMAT_KERNEL_INTERRUPT, 0x141a)
+#define GXIO_TRIO_OP_INIT_PIO_REGION_AUX IORPC_OPCODE(IORPC_FORMAT_NONE, 0x1414)
-#define GXIO_TRIO_OP_SET_MPS_MRS IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x141c)
-#define GXIO_TRIO_OP_FORCE_RC_LINK_UP IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x141d)
-#define GXIO_TRIO_OP_FORCE_EP_LINK_UP IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x141e)
+#define GXIO_TRIO_OP_INIT_MEMORY_MAP_MMU_AUX IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x141e)
+#define GXIO_TRIO_OP_GET_PORT_PROPERTY IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x141f)
+#define GXIO_TRIO_OP_CONFIG_LEGACY_INTR IORPC_OPCODE(IORPC_FORMAT_KERNEL_INTERRUPT, 0x1420)
+#define GXIO_TRIO_OP_CONFIG_MSI_INTR IORPC_OPCODE(IORPC_FORMAT_KERNEL_INTERRUPT, 0x1421)
+
+#define GXIO_TRIO_OP_SET_MPS_MRS IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x1423)
+#define GXIO_TRIO_OP_FORCE_RC_LINK_UP IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x1424)
+#define GXIO_TRIO_OP_FORCE_EP_LINK_UP IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x1425)
#define GXIO_TRIO_OP_GET_MMIO_BASE IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x8000)
#define GXIO_TRIO_OP_CHECK_MMIO_OFFSET IORPC_OPCODE(IORPC_FORMAT_NONE_NOUSER, 0x8001)
diff --git a/arch/tile/kernel/pci.c b/arch/tile/kernel/pci.c
index 33c1086..7598226 100644
--- a/arch/tile/kernel/pci.c
+++ b/arch/tile/kernel/pci.c
@@ -246,16 +246,13 @@
/* Scan for the smallest maximum payload size. */
while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
- int pcie_caps_offset;
u32 devcap;
int max_payload;
- pcie_caps_offset = pci_find_capability(dev, PCI_CAP_ID_EXP);
- if (pcie_caps_offset == 0)
+ if (!pci_is_pcie(dev))
continue;
- pci_read_config_dword(dev, pcie_caps_offset + PCI_EXP_DEVCAP,
- &devcap);
+ pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &devcap);
max_payload = devcap & PCI_EXP_DEVCAP_PAYLOAD;
if (max_payload < smallest_max_payload)
smallest_max_payload = max_payload;
@@ -263,21 +260,10 @@
/* Now, set the max_payload_size for all devices to that value. */
new_values = (max_read_size << 12) | (smallest_max_payload << 5);
- while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL) {
- int pcie_caps_offset;
- u16 devctl;
-
- pcie_caps_offset = pci_find_capability(dev, PCI_CAP_ID_EXP);
- if (pcie_caps_offset == 0)
- continue;
-
- pci_read_config_word(dev, pcie_caps_offset + PCI_EXP_DEVCTL,
- &devctl);
- devctl &= ~(PCI_EXP_DEVCTL_PAYLOAD | PCI_EXP_DEVCTL_READRQ);
- devctl |= new_values;
- pci_write_config_word(dev, pcie_caps_offset + PCI_EXP_DEVCTL,
- devctl);
- }
+ while ((dev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, dev)) != NULL)
+ pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_PAYLOAD | PCI_EXP_DEVCTL_READRQ,
+ new_values);
}
@@ -404,14 +390,6 @@
}
/*
- * This is called from the generic Linux layer.
- */
-void __devinit pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
-/*
* Enable memory and/or address decoding, as appropriate, for the
* device described by the 'dev' struct.
*
diff --git a/arch/tile/kernel/pci_gx.c b/arch/tile/kernel/pci_gx.c
index 0e213e3..2ba6d05 100644
--- a/arch/tile/kernel/pci_gx.c
+++ b/arch/tile/kernel/pci_gx.c
@@ -1034,14 +1034,6 @@
}
/*
- * This is called from the generic Linux layer.
- */
-void __devinit pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
-/*
* Enable memory address decoding, as appropriate, for the
* device described by the 'dev' struct. The I/O decoding
* is disabled, though the TILE-Gx supports I/O addressing.
diff --git a/arch/um/drivers/mconsole_kern.c b/arch/um/drivers/mconsole_kern.c
index 664a60e..c17de0d 100644
--- a/arch/um/drivers/mconsole_kern.c
+++ b/arch/um/drivers/mconsole_kern.c
@@ -705,6 +705,7 @@
struct task_struct *from = current, *to = arg;
to->thread.saved_task = from;
+ rcu_switch(from, to);
switch_to(from, to, from);
}
diff --git a/arch/um/include/asm/processor-generic.h b/arch/um/include/asm/processor-generic.h
index 69f1c57..33a6a24 100644
--- a/arch/um/include/asm/processor-generic.h
+++ b/arch/um/include/asm/processor-generic.h
@@ -20,14 +20,6 @@
struct thread_struct {
struct task_struct *saved_task;
- /*
- * This flag is set to 1 before calling do_fork (and analyzed in
- * copy_thread) to mark that we are begin called from userspace (fork /
- * vfork / clone), and reset to 0 after. It is left to 0 when called
- * from kernelspace (i.e. kernel_thread() or fork_idle(),
- * as of 2.6.11).
- */
- int forking;
struct pt_regs regs;
int singlestep_syscall;
void *fault_addr;
@@ -58,7 +50,6 @@
#define INIT_THREAD \
{ \
- .forking = 0, \
.regs = EMPTY_REGS, \
.fault_addr = NULL, \
.prev_sched = NULL, \
diff --git a/arch/um/include/shared/common-offsets.h b/arch/um/include/shared/common-offsets.h
index 40db8f7..2df313b 100644
--- a/arch/um/include/shared/common-offsets.h
+++ b/arch/um/include/shared/common-offsets.h
@@ -7,16 +7,6 @@
DEFINE(UM_KERN_PAGE_SHIFT, PAGE_SHIFT);
DEFINE(UM_NSEC_PER_SEC, NSEC_PER_SEC);
-DEFINE_STR(UM_KERN_EMERG, KERN_EMERG);
-DEFINE_STR(UM_KERN_ALERT, KERN_ALERT);
-DEFINE_STR(UM_KERN_CRIT, KERN_CRIT);
-DEFINE_STR(UM_KERN_ERR, KERN_ERR);
-DEFINE_STR(UM_KERN_WARNING, KERN_WARNING);
-DEFINE_STR(UM_KERN_NOTICE, KERN_NOTICE);
-DEFINE_STR(UM_KERN_INFO, KERN_INFO);
-DEFINE_STR(UM_KERN_DEBUG, KERN_DEBUG);
-DEFINE_STR(UM_KERN_CONT, KERN_CONT);
-
DEFINE(UM_ELF_CLASS, ELF_CLASS);
DEFINE(UM_ELFCLASS32, ELFCLASS32);
DEFINE(UM_ELFCLASS64, ELFCLASS64);
diff --git a/arch/um/include/shared/user.h b/arch/um/include/shared/user.h
index 4fa82c0..cef0685 100644
--- a/arch/um/include/shared/user.h
+++ b/arch/um/include/shared/user.h
@@ -26,6 +26,17 @@
extern void panic(const char *fmt, ...)
__attribute__ ((format (printf, 1, 2)));
+/* Requires preincluding include/linux/kern_levels.h */
+#define UM_KERN_EMERG KERN_EMERG
+#define UM_KERN_ALERT KERN_ALERT
+#define UM_KERN_CRIT KERN_CRIT
+#define UM_KERN_ERR KERN_ERR
+#define UM_KERN_WARNING KERN_WARNING
+#define UM_KERN_NOTICE KERN_NOTICE
+#define UM_KERN_INFO KERN_INFO
+#define UM_KERN_DEBUG KERN_DEBUG
+#define UM_KERN_CONT KERN_CONT
+
#ifdef UML_CONFIG_PRINTK
extern int printk(const char *fmt, ...)
__attribute__ ((format (printf, 1, 2)));
diff --git a/arch/um/kernel/exec.c b/arch/um/kernel/exec.c
index 6cade93..8c82786 100644
--- a/arch/um/kernel/exec.c
+++ b/arch/um/kernel/exec.c
@@ -39,34 +39,21 @@
void start_thread(struct pt_regs *regs, unsigned long eip, unsigned long esp)
{
+ get_safe_registers(regs->regs.gp, regs->regs.fp);
PT_REGS_IP(regs) = eip;
PT_REGS_SP(regs) = esp;
+ current->ptrace &= ~PT_DTRACE;
+#ifdef SUBARCH_EXECVE1
+ SUBARCH_EXECVE1(regs->regs);
+#endif
}
EXPORT_SYMBOL(start_thread);
-static long execve1(const char *file,
- const char __user *const __user *argv,
- const char __user *const __user *env)
-{
- long error;
-
- error = do_execve(file, argv, env, ¤t->thread.regs);
- if (error == 0) {
- task_lock(current);
- current->ptrace &= ~PT_DTRACE;
-#ifdef SUBARCH_EXECVE1
- SUBARCH_EXECVE1(¤t->thread.regs.regs);
-#endif
- task_unlock(current);
- }
- return error;
-}
-
long um_execve(const char *file, const char __user *const __user *argv, const char __user *const __user *env)
{
long err;
- err = execve1(file, argv, env);
+ err = do_execve(file, argv, env, ¤t->thread.regs);
if (!err)
UML_LONGJMP(current->thread.exec_buf, 1);
return err;
@@ -81,7 +68,7 @@
filename = getname(file);
error = PTR_ERR(filename);
if (IS_ERR(filename)) goto out;
- error = execve1(filename, argv, env);
+ error = do_execve(filename, argv, env, ¤t->thread.regs);
putname(filename);
out:
return error;
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 57fc702..c5f5afa 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -181,11 +181,12 @@
struct pt_regs *regs)
{
void (*handler)(void);
+ int kthread = current->flags & PF_KTHREAD;
int ret = 0;
p->thread = (struct thread_struct) INIT_THREAD;
- if (current->thread.forking) {
+ if (!kthread) {
memcpy(&p->thread.regs.regs, ®s->regs,
sizeof(p->thread.regs.regs));
PT_REGS_SET_SYSCALL_RETURN(&p->thread.regs, 0);
@@ -195,8 +196,7 @@
handler = fork_handler;
arch_copy_thread(¤t->thread.arch, &p->thread.arch);
- }
- else {
+ } else {
get_safe_registers(p->thread.regs.regs.gp, p->thread.regs.regs.fp);
p->thread.request.u.thread = current->thread.request.u.thread;
handler = new_thread_handler;
@@ -204,7 +204,7 @@
new_thread(task_stack_page(p), &p->thread.switch_buf, handler);
- if (current->thread.forking) {
+ if (!kthread) {
clear_flushed_tls(p);
/*
diff --git a/arch/um/kernel/signal.c b/arch/um/kernel/signal.c
index 7362d58..cc9c235 100644
--- a/arch/um/kernel/signal.c
+++ b/arch/um/kernel/signal.c
@@ -22,9 +22,13 @@
struct k_sigaction *ka, siginfo_t *info)
{
sigset_t *oldset = sigmask_to_save();
+ int singlestep = 0;
unsigned long sp;
int err;
+ if ((current->ptrace & PT_DTRACE) && (current->ptrace & PT_PTRACED))
+ singlestep = 1;
+
/* Did we come from a system call? */
if (PT_REGS_SYSCALL_NR(regs) >= 0) {
/* If so, check system call restarting.. */
@@ -61,7 +65,7 @@
if (err)
force_sigsegv(signr, current);
else
- signal_delivered(signr, info, ka, regs, 0);
+ signal_delivered(signr, info, ka, regs, singlestep);
}
static int kern_do_signal(struct pt_regs *regs)
diff --git a/arch/um/kernel/syscall.c b/arch/um/kernel/syscall.c
index f958cb8..a4c6d8e 100644
--- a/arch/um/kernel/syscall.c
+++ b/arch/um/kernel/syscall.c
@@ -17,25 +17,25 @@
long sys_fork(void)
{
- long ret;
-
- current->thread.forking = 1;
- ret = do_fork(SIGCHLD, UPT_SP(¤t->thread.regs.regs),
+ return do_fork(SIGCHLD, UPT_SP(¤t->thread.regs.regs),
¤t->thread.regs, 0, NULL, NULL);
- current->thread.forking = 0;
- return ret;
}
long sys_vfork(void)
{
- long ret;
-
- current->thread.forking = 1;
- ret = do_fork(CLONE_VFORK | CLONE_VM | SIGCHLD,
+ return do_fork(CLONE_VFORK | CLONE_VM | SIGCHLD,
UPT_SP(¤t->thread.regs.regs),
¤t->thread.regs, 0, NULL, NULL);
- current->thread.forking = 0;
- return ret;
+}
+
+long sys_clone(unsigned long clone_flags, unsigned long newsp,
+ void __user *parent_tid, void __user *child_tid)
+{
+ if (!newsp)
+ newsp = UPT_SP(¤t->thread.regs.regs);
+
+ return do_fork(clone_flags, newsp, ¤t->thread.regs, 0, parent_tid,
+ child_tid);
}
long old_mmap(unsigned long addr, unsigned long len,
diff --git a/arch/um/os-Linux/time.c b/arch/um/os-Linux/time.c
index f602385..0748fe0 100644
--- a/arch/um/os-Linux/time.c
+++ b/arch/um/os-Linux/time.c
@@ -114,7 +114,7 @@
skew += this_tick - last_tick;
while (skew >= one_tick) {
- alarm_handler(SIGVTALRM, NULL);
+ alarm_handler(SIGVTALRM, NULL, NULL);
skew -= one_tick;
}
diff --git a/arch/um/scripts/Makefile.rules b/arch/um/scripts/Makefile.rules
index d50270d..15889df 100644
--- a/arch/um/scripts/Makefile.rules
+++ b/arch/um/scripts/Makefile.rules
@@ -8,7 +8,7 @@
USER_OBJS := $(foreach file,$(USER_OBJS),$(obj)/$(file))
$(USER_OBJS:.o=.%): \
- c_flags = -Wp,-MD,$(depfile) $(USER_CFLAGS) -include user.h $(CFLAGS_$(basetarget).o)
+ c_flags = -Wp,-MD,$(depfile) $(USER_CFLAGS) -include $(srctree)/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
# These are like USER_OBJS but filter USER_CFLAGS through unprofile instead of
# using it directly.
diff --git a/arch/unicore32/kernel/pci.c b/arch/unicore32/kernel/pci.c
index 46cb6c9..b0056f6 100644
--- a/arch/unicore32/kernel/pci.c
+++ b/arch/unicore32/kernel/pci.c
@@ -154,14 +154,6 @@
zhole_size[0] = 0;
}
-void __devinit pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- if (debug_pci)
- printk(KERN_DEBUG "PCI: Assigning IRQ %02d to %s\n",
- irq, pci_name(dev));
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
/*
* If the bus contains any of these devices, then we must not turn on
* parity checking of any kind.
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 8ec3a1a..9436670 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -7,11 +7,13 @@
Say no to build a 32-bit kernel - formerly known as i386
config X86_32
- def_bool !64BIT
+ def_bool y
+ depends on !64BIT
select CLKSRC_I8253
config X86_64
- def_bool 64BIT
+ def_bool y
+ depends on 64BIT
select X86_DEV_DMA_OPS
### Arch settings
@@ -36,6 +38,7 @@
select HAVE_KRETPROBES
select HAVE_OPTPROBES
select HAVE_FTRACE_MCOUNT_RECORD
+ select HAVE_FENTRY if X86_64
select HAVE_C_RECORDMCOUNT
select HAVE_DYNAMIC_FTRACE
select HAVE_FUNCTION_TRACER
@@ -60,6 +63,8 @@
select HAVE_MIXED_BREAKPOINTS_REGS
select PERF_EVENTS
select HAVE_PERF_EVENTS_NMI
+ select HAVE_PERF_REGS
+ select HAVE_PERF_USER_STACK_DUMP
select ANON_INODES
select HAVE_ALIGNED_STRUCT_PAGE if SLUB && !M386
select HAVE_CMPXCHG_LOCAL if !M386
@@ -97,9 +102,12 @@
select KTIME_SCALAR if X86_32
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select HAVE_RCU_USER_QS if X86_64
+ select HAVE_IRQ_TIME_ACCOUNTING
config INSTRUCTION_DECODER
- def_bool (KPROBES || PERF_EVENTS || UPROBES)
+ def_bool y
+ depends on KPROBES || PERF_EVENTS || UPROBES
config OUTPUT_FORMAT
string
@@ -127,13 +135,15 @@
bool
config NEED_DMA_MAP_STATE
- def_bool (X86_64 || INTEL_IOMMU || DMA_API_DEBUG)
+ def_bool y
+ depends on X86_64 || INTEL_IOMMU || DMA_API_DEBUG
config NEED_SG_DMA_LENGTH
def_bool y
config GENERIC_ISA_DMA
- def_bool ISA_DMA_API
+ def_bool y
+ depends on ISA_DMA_API
config GENERIC_BUG
def_bool y
@@ -150,13 +160,16 @@
bool
config ARCH_MAY_HAVE_PC_FDC
- def_bool ISA_DMA_API
+ def_bool y
+ depends on ISA_DMA_API
config RWSEM_GENERIC_SPINLOCK
- def_bool !X86_XADD
+ def_bool y
+ depends on !X86_XADD
config RWSEM_XCHGADD_ALGORITHM
- def_bool X86_XADD
+ def_bool y
+ depends on X86_XADD
config GENERIC_CALIBRATE_DELAY
def_bool y
@@ -746,13 +759,14 @@
def_bool y if X86_64
---help---
Support for software bounce buffers used on x86-64 systems
- which don't have a hardware IOMMU (e.g. the current generation
- of Intel's x86-64 CPUs). Using this PCI devices which can only
- access 32-bits of memory can be used on systems with more than
- 3 GB of memory. If unsure, say Y.
+ which don't have a hardware IOMMU. Using this PCI devices
+ which can only access 32-bits of memory can be used on systems
+ with more than 3 GB of memory.
+ If unsure, say Y.
config IOMMU_HELPER
- def_bool (CALGARY_IOMMU || GART_IOMMU || SWIOTLB || AMD_IOMMU)
+ def_bool y
+ depends on CALGARY_IOMMU || GART_IOMMU || SWIOTLB || AMD_IOMMU
config MAXSMP
bool "Enable Maximum number of SMP Processors and NUMA Nodes"
@@ -796,17 +810,6 @@
making when dealing with multi-core CPU chips at a cost of slightly
increased overhead in some places. If unsure say N here.
-config IRQ_TIME_ACCOUNTING
- bool "Fine granularity task level IRQ time accounting"
- default n
- ---help---
- Select this option to enable fine granularity task irq time
- accounting. This is done by reading a timestamp on each
- transitions between softirq and hardirq state, so there can be a
- small performance impact.
-
- If in doubt, say N here.
-
source "kernel/Kconfig.preempt"
config X86_UP_APIC
@@ -871,6 +874,7 @@
config X86_MCE
bool "Machine Check / overheating reporting"
+ default y
---help---
Machine Check support allows the processor to notify the
kernel if it detects a problem (e.g. overheating, data corruption).
@@ -982,25 +986,25 @@
Say N otherwise.
config MICROCODE
- tristate "/dev/cpu/microcode - microcode support"
+ tristate "CPU microcode loading support"
select FW_LOADER
---help---
+
If you say Y here, you will be able to update the microcode on
certain Intel and AMD processors. The Intel support is for the
- IA32 family, e.g. Pentium Pro, Pentium II, Pentium III,
- Pentium 4, Xeon etc. The AMD support is for family 0x10 and
- 0x11 processors, e.g. Opteron, Phenom and Turion 64 Ultra.
- You will obviously need the actual microcode binary data itself
- which is not shipped with the Linux kernel.
+ IA32 family, e.g. Pentium Pro, Pentium II, Pentium III, Pentium 4,
+ Xeon etc. The AMD support is for families 0x10 and later. You will
+ obviously need the actual microcode binary data itself which is not
+ shipped with the Linux kernel.
This option selects the general module only, you need to select
at least one vendor specific module as well.
- To compile this driver as a module, choose M here: the
- module will be called microcode.
+ To compile this driver as a module, choose M here: the module
+ will be called microcode.
config MICROCODE_INTEL
- bool "Intel microcode patch loading support"
+ bool "Intel microcode loading support"
depends on MICROCODE
default MICROCODE
select FW_LOADER
@@ -1013,7 +1017,7 @@
<http://www.urbanmyth.org/microcode/>.
config MICROCODE_AMD
- bool "AMD microcode patch loading support"
+ bool "AMD microcode loading support"
depends on MICROCODE
select FW_LOADER
---help---
@@ -1159,10 +1163,12 @@
consumes more pagetable space per process.
config ARCH_PHYS_ADDR_T_64BIT
- def_bool X86_64 || X86_PAE
+ def_bool y
+ depends on X86_64 || X86_PAE
config ARCH_DMA_ADDR_T_64BIT
- def_bool X86_64 || HIGHMEM64G
+ def_bool y
+ depends on X86_64 || HIGHMEM64G
config DIRECT_GBPAGES
bool "Enable 1GB pages for kernel pagetables" if EXPERT
@@ -1285,8 +1291,8 @@
depends on ARCH_SPARSEMEM_ENABLE
config ARCH_MEMORY_PROBE
- def_bool X86_64
- depends on MEMORY_HOTPLUG
+ def_bool y
+ depends on X86_64 && MEMORY_HOTPLUG
config ARCH_PROC_KCORE_TEXT
def_bool y
@@ -1975,7 +1981,6 @@
config PCI_CNB20LE_QUIRK
bool "Read CNB20LE Host Bridge Windows" if EXPERT
- default n
depends on PCI && EXPERIMENTAL
help
Read the PCI windows out of the CNB20LE host bridge. This allows
@@ -2186,18 +2191,18 @@
depends on IA32_EMULATION || X86_X32
select ARCH_WANT_OLD_COMPAT_IPC
+if COMPAT
config COMPAT_FOR_U64_ALIGNMENT
- def_bool COMPAT
- depends on X86_64
+ def_bool y
config SYSVIPC_COMPAT
def_bool y
- depends on COMPAT && SYSVIPC
+ depends on SYSVIPC
config KEYS_COMPAT
- bool
- depends on COMPAT && KEYS
- default y
+ def_bool y
+ depends on KEYS
+endif
endmenu
diff --git a/arch/x86/Kconfig.cpu b/arch/x86/Kconfig.cpu
index 706e12e..f3b86d0 100644
--- a/arch/x86/Kconfig.cpu
+++ b/arch/x86/Kconfig.cpu
@@ -306,7 +306,8 @@
default X86_L1_CACHE_SHIFT
config X86_CMPXCHG
- def_bool X86_64 || (X86_32 && !M386)
+ def_bool y
+ depends on X86_64 || (X86_32 && !M386)
config X86_L1_CACHE_SHIFT
int
@@ -317,7 +318,7 @@
config X86_XADD
def_bool y
- depends on X86_64 || !M386
+ depends on !M386
config X86_PPRO_FENCE
bool "PentiumPro memory ordering errata workaround"
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index 682e9c2..474ca35 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -142,7 +142,7 @@
KBUILD_CFLAGS += $(mflags-y)
KBUILD_AFLAGS += $(mflags-y)
-archscripts:
+archscripts: scripts_basic
$(Q)$(MAKE) $(build)=arch/x86/tools relocs
###
diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile
index e398bb5..8a84501 100644
--- a/arch/x86/boot/compressed/Makefile
+++ b/arch/x86/boot/compressed/Makefile
@@ -28,6 +28,9 @@
$(obj)/string.o $(obj)/cmdline.o $(obj)/early_serial_console.o \
$(obj)/piggy.o
+$(obj)/eboot.o: KBUILD_CFLAGS += -fshort-wchar -mno-red-zone
+$(obj)/efi_stub_$(BITS).o: KBUILD_CLFAGS += -fshort-wchar -mno-red-zone
+
ifeq ($(CONFIG_EFI_STUB), y)
VMLINUX_OBJS += $(obj)/eboot.o $(obj)/efi_stub_$(BITS).o
endif
diff --git a/arch/x86/boot/compressed/eboot.c b/arch/x86/boot/compressed/eboot.c
index b3e0227..c760e07 100644
--- a/arch/x86/boot/compressed/eboot.c
+++ b/arch/x86/boot/compressed/eboot.c
@@ -276,8 +276,9 @@
nr_gops = size / sizeof(void *);
for (i = 0; i < nr_gops; i++) {
struct efi_graphics_output_mode_info *info;
- efi_guid_t pciio_proto = EFI_PCI_IO_PROTOCOL_GUID;
- void *pciio;
+ efi_guid_t conout_proto = EFI_CONSOLE_OUT_DEVICE_GUID;
+ bool conout_found = false;
+ void *dummy;
void *h = gop_handle[i];
status = efi_call_phys3(sys_table->boottime->handle_protocol,
@@ -285,19 +286,21 @@
if (status != EFI_SUCCESS)
continue;
- efi_call_phys3(sys_table->boottime->handle_protocol,
- h, &pciio_proto, &pciio);
+ status = efi_call_phys3(sys_table->boottime->handle_protocol,
+ h, &conout_proto, &dummy);
+
+ if (status == EFI_SUCCESS)
+ conout_found = true;
status = efi_call_phys4(gop->query_mode, gop,
gop->mode->mode, &size, &info);
- if (status == EFI_SUCCESS && (!first_gop || pciio)) {
+ if (status == EFI_SUCCESS && (!first_gop || conout_found)) {
/*
- * Apple provide GOPs that are not backed by
- * real hardware (they're used to handle
- * multiple displays). The workaround is to
- * search for a GOP implementing the PCIIO
- * protocol, and if one isn't found, to just
- * fallback to the first GOP.
+ * Systems that use the UEFI Console Splitter may
+ * provide multiple GOP devices, not all of which are
+ * backed by real hardware. The workaround is to search
+ * for a GOP implementing the ConOut protocol, and if
+ * one isn't found, to just fall back to the first GOP.
*/
width = info->horizontal_resolution;
height = info->vertical_resolution;
@@ -308,10 +311,10 @@
pixels_per_scan_line = info->pixels_per_scan_line;
/*
- * Once we've found a GOP supporting PCIIO,
+ * Once we've found a GOP supporting ConOut,
* don't bother looking any further.
*/
- if (pciio)
+ if (conout_found)
break;
first_gop = gop;
@@ -328,7 +331,6 @@
si->lfb_width = width;
si->lfb_height = height;
si->lfb_base = fb_base;
- si->lfb_size = fb_size;
si->pages = 1;
if (pixel_format == PIXEL_RGB_RESERVED_8BIT_PER_COLOR) {
@@ -376,6 +378,10 @@
si->rsvd_pos = 0;
}
+ si->lfb_size = si->lfb_linelength * si->lfb_height;
+
+ si->capabilities |= VIDEO_CAPABILITY_SKIP_QUIRKS;
+
free_handle:
efi_call_phys1(sys_table->boottime->free_pool, gop_handle);
return status;
diff --git a/arch/x86/boot/compressed/eboot.h b/arch/x86/boot/compressed/eboot.h
index 3b6e156..e5b0a8f 100644
--- a/arch/x86/boot/compressed/eboot.h
+++ b/arch/x86/boot/compressed/eboot.h
@@ -14,6 +14,10 @@
#define EFI_PAGE_SIZE (1UL << EFI_PAGE_SHIFT)
#define EFI_READ_CHUNK_SIZE (1024 * 1024)
+#define EFI_CONSOLE_OUT_DEVICE_GUID \
+ EFI_GUID(0xd3b36f2c, 0xd551, 0x11d4, 0x9a, 0x46, 0x0, 0x90, 0x27, \
+ 0x3f, 0xc1, 0x4d)
+
#define PIXEL_RGB_RESERVED_8BIT_PER_COLOR 0
#define PIXEL_BGR_RESERVED_8BIT_PER_COLOR 1
#define PIXEL_BIT_MASK 2
diff --git a/arch/x86/boot/header.S b/arch/x86/boot/header.S
index b4e15dd..2a01744 100644
--- a/arch/x86/boot/header.S
+++ b/arch/x86/boot/header.S
@@ -32,10 +32,6 @@
#define SVGA_MODE ASK_VGA
#endif
-#ifndef RAMDISK
-#define RAMDISK 0
-#endif
-
#ifndef ROOT_RDONLY
#define ROOT_RDONLY 1
#endif
diff --git a/arch/x86/configs/i386_defconfig b/arch/x86/configs/i386_defconfig
index 119db67..5598547 100644
--- a/arch/x86/configs/i386_defconfig
+++ b/arch/x86/configs/i386_defconfig
@@ -8,6 +8,8 @@
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_AUDIT=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_LOG_BUF_SHIFT=18
CONFIG_CGROUPS=y
CONFIG_CGROUP_FREEZER=y
@@ -34,8 +36,6 @@
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
CONFIG_SMP=y
CONFIG_X86_GENERIC=y
CONFIG_HPET_TIMER=y
@@ -144,8 +144,6 @@
CONFIG_DEBUG_DEVRES=y
CONFIG_CONNECTOR=y
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_RAM=y
-CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_SR_VENDOR=y
@@ -231,8 +229,6 @@
CONFIG_SND_HDA_INTEL=y
CONFIG_SND_HDA_HWDEP=y
CONFIG_HIDRAW=y
-CONFIG_HID_PID=y
-CONFIG_USB_HIDDEV=y
CONFIG_HID_GYRATION=y
CONFIG_LOGITECH_FF=y
CONFIG_HID_NTRIG=y
@@ -243,11 +239,11 @@
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
CONFIG_HID_TOPSEED=y
+CONFIG_HID_PID=y
+CONFIG_USB_HIDDEV=y
CONFIG_USB=y
CONFIG_USB_DEBUG=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
-CONFIG_USB_DEVICEFS=y
-# CONFIG_USB_DEVICE_CLASS is not set
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
@@ -262,10 +258,9 @@
CONFIG_DMADEVICES=y
CONFIG_EEEPC_LAPTOP=y
CONFIG_EFI_VARS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-CONFIG_EXT3_FS_POSIX_ACL=y
-CONFIG_EXT3_FS_SECURITY=y
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not set
@@ -280,7 +275,6 @@
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
@@ -299,13 +293,11 @@
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
CONFIG_DEBUG_STACK_USAGE=y
-CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_PROVIDE_OHCI1394_DMA_INIT=y
CONFIG_EARLY_PRINTK_DBGP=y
CONFIG_DEBUG_STACKOVERFLOW=y
# CONFIG_DEBUG_RODATA_TEST is not set
-CONFIG_DEBUG_NX_TEST=m
CONFIG_DEBUG_BOOT_PARAMS=y
CONFIG_OPTIMIZE_INLINING=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
@@ -316,4 +308,3 @@
CONFIG_SECURITY_SELINUX_DISABLE=y
CONFIG_CRYPTO_AES_586=y
# CONFIG_CRYPTO_ANSI_CPRNG is not set
-CONFIG_CRC_T10DIF=y
diff --git a/arch/x86/configs/x86_64_defconfig b/arch/x86/configs/x86_64_defconfig
index 76eb290..671524d 100644
--- a/arch/x86/configs/x86_64_defconfig
+++ b/arch/x86/configs/x86_64_defconfig
@@ -8,6 +8,8 @@
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_AUDIT=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_LOG_BUF_SHIFT=18
CONFIG_CGROUPS=y
CONFIG_CGROUP_FREEZER=y
@@ -34,8 +36,6 @@
CONFIG_SUN_PARTITION=y
CONFIG_KARMA_PARTITION=y
CONFIG_EFI_PARTITION=y
-CONFIG_NO_HZ=y
-CONFIG_HIGH_RES_TIMERS=y
CONFIG_SMP=y
CONFIG_CALGARY_IOMMU=y
CONFIG_NR_CPUS=64
@@ -144,8 +144,6 @@
CONFIG_DEBUG_DEVRES=y
CONFIG_CONNECTOR=y
CONFIG_BLK_DEV_LOOP=y
-CONFIG_BLK_DEV_RAM=y
-CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_SR_VENDOR=y
@@ -227,8 +225,6 @@
CONFIG_SND_HDA_INTEL=y
CONFIG_SND_HDA_HWDEP=y
CONFIG_HIDRAW=y
-CONFIG_HID_PID=y
-CONFIG_USB_HIDDEV=y
CONFIG_HID_GYRATION=y
CONFIG_LOGITECH_FF=y
CONFIG_HID_NTRIG=y
@@ -239,11 +235,11 @@
CONFIG_HID_SONY=y
CONFIG_HID_SUNPLUS=y
CONFIG_HID_TOPSEED=y
+CONFIG_HID_PID=y
+CONFIG_USB_HIDDEV=y
CONFIG_USB=y
CONFIG_USB_DEBUG=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
-CONFIG_USB_DEVICEFS=y
-# CONFIG_USB_DEVICE_CLASS is not set
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
@@ -262,10 +258,9 @@
CONFIG_INTEL_IOMMU=y
# CONFIG_INTEL_IOMMU_DEFAULT_ON is not set
CONFIG_EFI_VARS=y
-CONFIG_EXT3_FS=y
-# CONFIG_EXT3_DEFAULTS_TO_ORDERED is not set
-CONFIG_EXT3_FS_POSIX_ACL=y
-CONFIG_EXT3_FS_SECURITY=y
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not set
@@ -280,7 +275,6 @@
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_NFS_FS=y
-CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_ROOT_NFS=y
@@ -298,13 +292,11 @@
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
CONFIG_DEBUG_STACK_USAGE=y
-CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_PROVIDE_OHCI1394_DMA_INIT=y
CONFIG_EARLY_PRINTK_DBGP=y
CONFIG_DEBUG_STACKOVERFLOW=y
# CONFIG_DEBUG_RODATA_TEST is not set
-CONFIG_DEBUG_NX_TEST=m
CONFIG_DEBUG_BOOT_PARAMS=y
CONFIG_OPTIMIZE_INLINING=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
@@ -314,4 +306,3 @@
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_DISABLE=y
# CONFIG_CRYPTO_ANSI_CPRNG is not set
-CONFIG_CRC_T10DIF=y
diff --git a/arch/x86/ia32/ia32_signal.c b/arch/x86/ia32/ia32_signal.c
index 673ac9b..8c77c64 100644
--- a/arch/x86/ia32/ia32_signal.c
+++ b/arch/x86/ia32/ia32_signal.c
@@ -162,7 +162,8 @@
}
seg = get_fs();
set_fs(KERNEL_DS);
- ret = do_sigaltstack(uss_ptr ? &uss : NULL, &uoss, regs->sp);
+ ret = do_sigaltstack((stack_t __force __user *) (uss_ptr ? &uss : NULL),
+ (stack_t __force __user *) &uoss, regs->sp);
set_fs(seg);
if (ret >= 0 && uoss_ptr) {
if (!access_ok(VERIFY_WRITE, uoss_ptr, sizeof(stack_ia32_t)))
@@ -250,7 +251,7 @@
get_user_ex(tmp, &sc->fpstate);
buf = compat_ptr(tmp);
- err |= restore_i387_xstate_ia32(buf);
+ err |= restore_xstate_sig(buf, 1);
get_user_ex(*pax, &sc->ax);
} get_user_catch(err);
@@ -361,7 +362,7 @@
*/
static void __user *get_sigframe(struct k_sigaction *ka, struct pt_regs *regs,
size_t frame_size,
- void **fpstate)
+ void __user **fpstate)
{
unsigned long sp;
@@ -381,9 +382,12 @@
sp = (unsigned long) ka->sa.sa_restorer;
if (used_math()) {
- sp = sp - sig_xstate_ia32_size;
- *fpstate = (struct _fpstate_ia32 *) sp;
- if (save_i387_xstate_ia32(*fpstate) < 0)
+ unsigned long fx_aligned, math_size;
+
+ sp = alloc_mathframe(sp, 1, &fx_aligned, &math_size);
+ *fpstate = (struct _fpstate_ia32 __user *) sp;
+ if (save_xstate_sig(*fpstate, (void __user *)fx_aligned,
+ math_size) < 0)
return (void __user *) -1L;
}
@@ -448,7 +452,7 @@
* These are actually not used anymore, but left because some
* gdb versions depend on them as a marker.
*/
- put_user_ex(*((u64 *)&code), (u64 *)frame->retcode);
+ put_user_ex(*((u64 *)&code), (u64 __user *)frame->retcode);
} put_user_catch(err);
if (err)
@@ -529,7 +533,7 @@
* Not actually used anymore, but left because some gdb
* versions need it.
*/
- put_user_ex(*((u64 *)&code), (u64 *)frame->retcode);
+ put_user_ex(*((u64 *)&code), (u64 __user *)frame->retcode);
} put_user_catch(err);
if (err)
diff --git a/arch/x86/ia32/sys_ia32.c b/arch/x86/ia32/sys_ia32.c
index 4540bec..c5b938d 100644
--- a/arch/x86/ia32/sys_ia32.c
+++ b/arch/x86/ia32/sys_ia32.c
@@ -287,7 +287,7 @@
return ret;
}
-asmlinkage long sys32_waitpid(compat_pid_t pid, unsigned int *stat_addr,
+asmlinkage long sys32_waitpid(compat_pid_t pid, unsigned int __user *stat_addr,
int options)
{
return compat_sys_wait4(pid, stat_addr, options, NULL);
diff --git a/arch/x86/include/asm/alternative.h b/arch/x86/include/asm/alternative.h
index 7078068..444704c 100644
--- a/arch/x86/include/asm/alternative.h
+++ b/arch/x86/include/asm/alternative.h
@@ -60,7 +60,7 @@
void *locks, void *locks_end,
void *text, void *text_end);
extern void alternatives_smp_module_del(struct module *mod);
-extern void alternatives_smp_switch(int smp);
+extern void alternatives_enable_smp(void);
extern int alternatives_text_reserved(void *start, void *end);
extern bool skip_smp_alternatives;
#else
@@ -68,7 +68,7 @@
void *locks, void *locks_end,
void *text, void *text_end) {}
static inline void alternatives_smp_module_del(struct module *mod) {}
-static inline void alternatives_smp_switch(int smp) {}
+static inline void alternatives_enable_smp(void) {}
static inline int alternatives_text_reserved(void *start, void *end)
{
return 0;
diff --git a/arch/x86/include/asm/bitops.h b/arch/x86/include/asm/bitops.h
index 72f5009..6dfd019 100644
--- a/arch/x86/include/asm/bitops.h
+++ b/arch/x86/include/asm/bitops.h
@@ -355,7 +355,7 @@
*/
static inline unsigned long __ffs(unsigned long word)
{
- asm("bsf %1,%0"
+ asm("rep; bsf %1,%0"
: "=r" (word)
: "rm" (word));
return word;
@@ -369,7 +369,7 @@
*/
static inline unsigned long ffz(unsigned long word)
{
- asm("bsf %1,%0"
+ asm("rep; bsf %1,%0"
: "=r" (word)
: "r" (~word));
return word;
@@ -417,10 +417,9 @@
* We cannot do this on 32 bits because at the very least some
* 486 CPUs did not behave this way.
*/
- long tmp = -1;
asm("bsfl %1,%0"
: "=r" (r)
- : "rm" (x), "0" (tmp));
+ : "rm" (x), "0" (-1));
#elif defined(CONFIG_X86_CMOV)
asm("bsfl %1,%0\n\t"
"cmovzl %2,%0"
@@ -459,10 +458,9 @@
* We cannot do this on 32 bits because at the very least some
* 486 CPUs did not behave this way.
*/
- long tmp = -1;
asm("bsrl %1,%0"
: "=r" (r)
- : "rm" (x), "0" (tmp));
+ : "rm" (x), "0" (-1));
#elif defined(CONFIG_X86_CMOV)
asm("bsrl %1,%0\n\t"
"cmovzl %2,%0"
@@ -490,13 +488,13 @@
#ifdef CONFIG_X86_64
static __always_inline int fls64(__u64 x)
{
- long bitpos = -1;
+ int bitpos = -1;
/*
* AMD64 says BSRQ won't clobber the dest reg if x==0; Intel64 says the
* dest reg is undefined if x==0, but their CPU architect says its
* value is written to set it to the same as before.
*/
- asm("bsrq %1,%0"
+ asm("bsrq %1,%q0"
: "+r" (bitpos)
: "rm" (x));
return bitpos + 1;
diff --git a/arch/x86/include/asm/calling.h b/arch/x86/include/asm/calling.h
index a9e3a74..7f8422a 100644
--- a/arch/x86/include/asm/calling.h
+++ b/arch/x86/include/asm/calling.h
@@ -49,38 +49,36 @@
#include "dwarf2.h"
/*
- * 64-bit system call stack frame layout defines and helpers, for
- * assembly code (note that the seemingly unnecessary parentheses
- * are to prevent cpp from inserting spaces in expressions that get
- * passed to macros):
+ * 64-bit system call stack frame layout defines and helpers,
+ * for assembly code:
*/
-#define R15 (0)
-#define R14 (8)
-#define R13 (16)
-#define R12 (24)
-#define RBP (32)
-#define RBX (40)
+#define R15 0
+#define R14 8
+#define R13 16
+#define R12 24
+#define RBP 32
+#define RBX 40
/* arguments: interrupts/non tracing syscalls only save up to here: */
-#define R11 (48)
-#define R10 (56)
-#define R9 (64)
-#define R8 (72)
-#define RAX (80)
-#define RCX (88)
-#define RDX (96)
-#define RSI (104)
-#define RDI (112)
-#define ORIG_RAX (120) /* + error_code */
+#define R11 48
+#define R10 56
+#define R9 64
+#define R8 72
+#define RAX 80
+#define RCX 88
+#define RDX 96
+#define RSI 104
+#define RDI 112
+#define ORIG_RAX 120 /* + error_code */
/* end of arguments */
/* cpu exception frame or undefined in case of fast syscall: */
-#define RIP (128)
-#define CS (136)
-#define EFLAGS (144)
-#define RSP (152)
-#define SS (160)
+#define RIP 128
+#define CS 136
+#define EFLAGS 144
+#define RSP 152
+#define SS 160
#define ARGOFFSET R11
#define SWFRAME ORIG_RAX
diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h
index 6b7ee5f..16cae42 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -97,6 +97,7 @@
#define X86_FEATURE_EXTD_APICID (3*32+26) /* has extended APICID (8 bits) */
#define X86_FEATURE_AMD_DCM (3*32+27) /* multi-node processor */
#define X86_FEATURE_APERFMPERF (3*32+28) /* APERFMPERF */
+#define X86_FEATURE_EAGER_FPU (3*32+29) /* "eagerfpu" Non lazy FPU restore */
/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
#define X86_FEATURE_XMM3 (4*32+ 0) /* "pni" SSE-3 */
@@ -209,6 +210,7 @@
#define X86_FEATURE_RTM (9*32+11) /* Restricted Transactional Memory */
#define X86_FEATURE_RDSEED (9*32+18) /* The RDSEED instruction */
#define X86_FEATURE_ADX (9*32+19) /* The ADCX and ADOX instructions */
+#define X86_FEATURE_SMAP (9*32+20) /* Supervisor Mode Access Prevention */
#if defined(__KERNEL__) && !defined(__ASSEMBLY__)
@@ -299,12 +301,14 @@
#define cpu_has_xmm4_2 boot_cpu_has(X86_FEATURE_XMM4_2)
#define cpu_has_x2apic boot_cpu_has(X86_FEATURE_X2APIC)
#define cpu_has_xsave boot_cpu_has(X86_FEATURE_XSAVE)
+#define cpu_has_xsaveopt boot_cpu_has(X86_FEATURE_XSAVEOPT)
#define cpu_has_osxsave boot_cpu_has(X86_FEATURE_OSXSAVE)
#define cpu_has_hypervisor boot_cpu_has(X86_FEATURE_HYPERVISOR)
#define cpu_has_pclmulqdq boot_cpu_has(X86_FEATURE_PCLMULQDQ)
#define cpu_has_perfctr_core boot_cpu_has(X86_FEATURE_PERFCTR_CORE)
#define cpu_has_cx8 boot_cpu_has(X86_FEATURE_CX8)
#define cpu_has_cx16 boot_cpu_has(X86_FEATURE_CX16)
+#define cpu_has_eager_fpu boot_cpu_has(X86_FEATURE_EAGER_FPU)
#if defined(CONFIG_X86_INVLPG) || defined(CONFIG_X86_64)
# define cpu_has_invlpg 1
diff --git a/arch/x86/include/asm/fpu-internal.h b/arch/x86/include/asm/fpu-internal.h
index 75f4c6d..92f3c6e 100644
--- a/arch/x86/include/asm/fpu-internal.h
+++ b/arch/x86/include/asm/fpu-internal.h
@@ -12,6 +12,7 @@
#include <linux/kernel_stat.h>
#include <linux/regset.h>
+#include <linux/compat.h>
#include <linux/slab.h>
#include <asm/asm.h>
#include <asm/cpufeature.h>
@@ -21,42 +22,74 @@
#include <asm/uaccess.h>
#include <asm/xsave.h>
-extern unsigned int sig_xstate_size;
+#ifdef CONFIG_X86_64
+# include <asm/sigcontext32.h>
+# include <asm/user32.h>
+int ia32_setup_rt_frame(int sig, struct k_sigaction *ka, siginfo_t *info,
+ compat_sigset_t *set, struct pt_regs *regs);
+int ia32_setup_frame(int sig, struct k_sigaction *ka,
+ compat_sigset_t *set, struct pt_regs *regs);
+#else
+# define user_i387_ia32_struct user_i387_struct
+# define user32_fxsr_struct user_fxsr_struct
+# define ia32_setup_frame __setup_frame
+# define ia32_setup_rt_frame __setup_rt_frame
+#endif
+
+extern unsigned int mxcsr_feature_mask;
extern void fpu_init(void);
+extern void eager_fpu_init(void);
DECLARE_PER_CPU(struct task_struct *, fpu_owner_task);
+extern void convert_from_fxsr(struct user_i387_ia32_struct *env,
+ struct task_struct *tsk);
+extern void convert_to_fxsr(struct task_struct *tsk,
+ const struct user_i387_ia32_struct *env);
+
extern user_regset_active_fn fpregs_active, xfpregs_active;
extern user_regset_get_fn fpregs_get, xfpregs_get, fpregs_soft_get,
xstateregs_get;
extern user_regset_set_fn fpregs_set, xfpregs_set, fpregs_soft_set,
xstateregs_set;
-
/*
* xstateregs_active == fpregs_active. Please refer to the comment
* at the definition of fpregs_active.
*/
#define xstateregs_active fpregs_active
-extern struct _fpx_sw_bytes fx_sw_reserved;
-#ifdef CONFIG_IA32_EMULATION
-extern unsigned int sig_xstate_ia32_size;
-extern struct _fpx_sw_bytes fx_sw_reserved_ia32;
-struct _fpstate_ia32;
-struct _xstate_ia32;
-extern int save_i387_xstate_ia32(void __user *buf);
-extern int restore_i387_xstate_ia32(void __user *buf);
-#endif
-
#ifdef CONFIG_MATH_EMULATION
+# define HAVE_HWFP (boot_cpu_data.hard_math)
extern void finit_soft_fpu(struct i387_soft_struct *soft);
#else
+# define HAVE_HWFP 1
static inline void finit_soft_fpu(struct i387_soft_struct *soft) {}
#endif
+static inline int is_ia32_compat_frame(void)
+{
+ return config_enabled(CONFIG_IA32_EMULATION) &&
+ test_thread_flag(TIF_IA32);
+}
+
+static inline int is_ia32_frame(void)
+{
+ return config_enabled(CONFIG_X86_32) || is_ia32_compat_frame();
+}
+
+static inline int is_x32_frame(void)
+{
+ return config_enabled(CONFIG_X86_X32_ABI) && test_thread_flag(TIF_X32);
+}
+
#define X87_FSW_ES (1 << 7) /* Exception Summary */
+static __always_inline __pure bool use_eager_fpu(void)
+{
+ return static_cpu_has(X86_FEATURE_EAGER_FPU);
+}
+
static __always_inline __pure bool use_xsaveopt(void)
{
return static_cpu_has(X86_FEATURE_XSAVEOPT);
@@ -72,6 +105,13 @@
return static_cpu_has(X86_FEATURE_FXSR);
}
+static inline void fx_finit(struct i387_fxsave_struct *fx)
+{
+ memset(fx, 0, xstate_size);
+ fx->cwd = 0x37f;
+ fx->mxcsr = MXCSR_DEFAULT;
+}
+
extern void __sanitize_i387_state(struct task_struct *);
static inline void sanitize_i387_state(struct task_struct *tsk)
@@ -81,131 +121,88 @@
__sanitize_i387_state(tsk);
}
-#ifdef CONFIG_X86_64
-static inline int fxrstor_checking(struct i387_fxsave_struct *fx)
-{
- int err;
+#define check_insn(insn, output, input...) \
+({ \
+ int err; \
+ asm volatile("1:" #insn "\n\t" \
+ "2:\n" \
+ ".section .fixup,\"ax\"\n" \
+ "3: movl $-1,%[err]\n" \
+ " jmp 2b\n" \
+ ".previous\n" \
+ _ASM_EXTABLE(1b, 3b) \
+ : [err] "=r" (err), output \
+ : "0"(0), input); \
+ err; \
+})
- /* See comment in fxsave() below. */
-#ifdef CONFIG_AS_FXSAVEQ
- asm volatile("1: fxrstorq %[fx]\n\t"
- "2:\n"
- ".section .fixup,\"ax\"\n"
- "3: movl $-1,%[err]\n"
- " jmp 2b\n"
- ".previous\n"
- _ASM_EXTABLE(1b, 3b)
- : [err] "=r" (err)
- : [fx] "m" (*fx), "0" (0));
-#else
- asm volatile("1: rex64/fxrstor (%[fx])\n\t"
- "2:\n"
- ".section .fixup,\"ax\"\n"
- "3: movl $-1,%[err]\n"
- " jmp 2b\n"
- ".previous\n"
- _ASM_EXTABLE(1b, 3b)
- : [err] "=r" (err)
- : [fx] "R" (fx), "m" (*fx), "0" (0));
-#endif
- return err;
+static inline int fsave_user(struct i387_fsave_struct __user *fx)
+{
+ return check_insn(fnsave %[fx]; fwait, [fx] "=m" (*fx), "m" (*fx));
}
static inline int fxsave_user(struct i387_fxsave_struct __user *fx)
{
- int err;
+ if (config_enabled(CONFIG_X86_32))
+ return check_insn(fxsave %[fx], [fx] "=m" (*fx), "m" (*fx));
+ else if (config_enabled(CONFIG_AS_FXSAVEQ))
+ return check_insn(fxsaveq %[fx], [fx] "=m" (*fx), "m" (*fx));
- /*
- * Clear the bytes not touched by the fxsave and reserved
- * for the SW usage.
- */
- err = __clear_user(&fx->sw_reserved,
- sizeof(struct _fpx_sw_bytes));
- if (unlikely(err))
- return -EFAULT;
-
- /* See comment in fxsave() below. */
-#ifdef CONFIG_AS_FXSAVEQ
- asm volatile("1: fxsaveq %[fx]\n\t"
- "2:\n"
- ".section .fixup,\"ax\"\n"
- "3: movl $-1,%[err]\n"
- " jmp 2b\n"
- ".previous\n"
- _ASM_EXTABLE(1b, 3b)
- : [err] "=r" (err), [fx] "=m" (*fx)
- : "0" (0));
-#else
- asm volatile("1: rex64/fxsave (%[fx])\n\t"
- "2:\n"
- ".section .fixup,\"ax\"\n"
- "3: movl $-1,%[err]\n"
- " jmp 2b\n"
- ".previous\n"
- _ASM_EXTABLE(1b, 3b)
- : [err] "=r" (err), "=m" (*fx)
- : [fx] "R" (fx), "0" (0));
-#endif
- if (unlikely(err) &&
- __clear_user(fx, sizeof(struct i387_fxsave_struct)))
- err = -EFAULT;
- /* No need to clear here because the caller clears USED_MATH */
- return err;
+ /* See comment in fpu_fxsave() below. */
+ return check_insn(rex64/fxsave (%[fx]), "=m" (*fx), [fx] "R" (fx));
}
-static inline void fpu_fxsave(struct fpu *fpu)
-{
- /* Using "rex64; fxsave %0" is broken because, if the memory operand
- uses any extended registers for addressing, a second REX prefix
- will be generated (to the assembler, rex64 followed by semicolon
- is a separate instruction), and hence the 64-bitness is lost. */
-
-#ifdef CONFIG_AS_FXSAVEQ
- /* Using "fxsaveq %0" would be the ideal choice, but is only supported
- starting with gas 2.16. */
- __asm__ __volatile__("fxsaveq %0"
- : "=m" (fpu->state->fxsave));
-#else
- /* Using, as a workaround, the properly prefixed form below isn't
- accepted by any binutils version so far released, complaining that
- the same type of prefix is used twice if an extended register is
- needed for addressing (fix submitted to mainline 2005-11-21).
- asm volatile("rex64/fxsave %0"
- : "=m" (fpu->state->fxsave));
- This, however, we can work around by forcing the compiler to select
- an addressing mode that doesn't require extended registers. */
- asm volatile("rex64/fxsave (%[fx])"
- : "=m" (fpu->state->fxsave)
- : [fx] "R" (&fpu->state->fxsave));
-#endif
-}
-
-#else /* CONFIG_X86_32 */
-
-/* perform fxrstor iff the processor has extended states, otherwise frstor */
static inline int fxrstor_checking(struct i387_fxsave_struct *fx)
{
- /*
- * The "nop" is needed to make the instructions the same
- * length.
- */
- alternative_input(
- "nop ; frstor %1",
- "fxrstor %1",
- X86_FEATURE_FXSR,
- "m" (*fx));
+ if (config_enabled(CONFIG_X86_32))
+ return check_insn(fxrstor %[fx], "=m" (*fx), [fx] "m" (*fx));
+ else if (config_enabled(CONFIG_AS_FXSAVEQ))
+ return check_insn(fxrstorq %[fx], "=m" (*fx), [fx] "m" (*fx));
- return 0;
+ /* See comment in fpu_fxsave() below. */
+ return check_insn(rex64/fxrstor (%[fx]), "=m" (*fx), [fx] "R" (fx),
+ "m" (*fx));
+}
+
+static inline int frstor_checking(struct i387_fsave_struct *fx)
+{
+ return check_insn(frstor %[fx], "=m" (*fx), [fx] "m" (*fx));
}
static inline void fpu_fxsave(struct fpu *fpu)
{
- asm volatile("fxsave %[fx]"
- : [fx] "=m" (fpu->state->fxsave));
+ if (config_enabled(CONFIG_X86_32))
+ asm volatile( "fxsave %[fx]" : [fx] "=m" (fpu->state->fxsave));
+ else if (config_enabled(CONFIG_AS_FXSAVEQ))
+ asm volatile("fxsaveq %0" : "=m" (fpu->state->fxsave));
+ else {
+ /* Using "rex64; fxsave %0" is broken because, if the memory
+ * operand uses any extended registers for addressing, a second
+ * REX prefix will be generated (to the assembler, rex64
+ * followed by semicolon is a separate instruction), and hence
+ * the 64-bitness is lost.
+ *
+ * Using "fxsaveq %0" would be the ideal choice, but is only
+ * supported starting with gas 2.16.
+ *
+ * Using, as a workaround, the properly prefixed form below
+ * isn't accepted by any binutils version so far released,
+ * complaining that the same type of prefix is used twice if
+ * an extended register is needed for addressing (fix submitted
+ * to mainline 2005-11-21).
+ *
+ * asm volatile("rex64/fxsave %0" : "=m" (fpu->state->fxsave));
+ *
+ * This, however, we can work around by forcing the compiler to
+ * select an addressing mode that doesn't require extended
+ * registers.
+ */
+ asm volatile( "rex64/fxsave (%[fx])"
+ : "=m" (fpu->state->fxsave)
+ : [fx] "R" (&fpu->state->fxsave));
+ }
}
-#endif /* CONFIG_X86_64 */
-
/*
* These must be called with preempt disabled. Returns
* 'true' if the FPU state is still intact.
@@ -248,17 +245,14 @@
return fpu_save_init(&tsk->thread.fpu);
}
-static inline int fpu_fxrstor_checking(struct fpu *fpu)
-{
- return fxrstor_checking(&fpu->state->fxsave);
-}
-
static inline int fpu_restore_checking(struct fpu *fpu)
{
if (use_xsave())
- return fpu_xrstor_checking(fpu);
+ return fpu_xrstor_checking(&fpu->state->xsave);
+ else if (use_fxsr())
+ return fxrstor_checking(&fpu->state->fxsave);
else
- return fpu_fxrstor_checking(fpu);
+ return frstor_checking(&fpu->state->fsave);
}
static inline int restore_fpu_checking(struct task_struct *tsk)
@@ -310,15 +304,52 @@
static inline void __thread_fpu_end(struct task_struct *tsk)
{
__thread_clear_has_fpu(tsk);
- stts();
+ if (!use_eager_fpu())
+ stts();
}
static inline void __thread_fpu_begin(struct task_struct *tsk)
{
- clts();
+ if (!use_eager_fpu())
+ clts();
__thread_set_has_fpu(tsk);
}
+static inline void __drop_fpu(struct task_struct *tsk)
+{
+ if (__thread_has_fpu(tsk)) {
+ /* Ignore delayed exceptions from user space */
+ asm volatile("1: fwait\n"
+ "2:\n"
+ _ASM_EXTABLE(1b, 2b));
+ __thread_fpu_end(tsk);
+ }
+}
+
+static inline void drop_fpu(struct task_struct *tsk)
+{
+ /*
+ * Forget coprocessor state..
+ */
+ preempt_disable();
+ tsk->fpu_counter = 0;
+ __drop_fpu(tsk);
+ clear_used_math();
+ preempt_enable();
+}
+
+static inline void drop_init_fpu(struct task_struct *tsk)
+{
+ if (!use_eager_fpu())
+ drop_fpu(tsk);
+ else {
+ if (use_xsave())
+ xrstor_state(init_xstate_buf, -1);
+ else
+ fxrstor_checking(&init_xstate_buf->i387);
+ }
+}
+
/*
* FPU state switching for scheduling.
*
@@ -352,7 +383,12 @@
{
fpu_switch_t fpu;
- fpu.preload = tsk_used_math(new) && new->fpu_counter > 5;
+ /*
+ * If the task has used the math, pre-load the FPU on xsave processors
+ * or if the past 5 consecutive context-switches used math.
+ */
+ fpu.preload = tsk_used_math(new) && (use_eager_fpu() ||
+ new->fpu_counter > 5);
if (__thread_has_fpu(old)) {
if (!__save_init_fpu(old))
cpu = ~0;
@@ -364,14 +400,14 @@
new->fpu_counter++;
__thread_set_has_fpu(new);
prefetch(new->thread.fpu.state);
- } else
+ } else if (!use_eager_fpu())
stts();
} else {
old->fpu_counter = 0;
old->thread.fpu.last_cpu = ~0;
if (fpu.preload) {
new->fpu_counter++;
- if (fpu_lazy_restore(new, cpu))
+ if (!use_eager_fpu() && fpu_lazy_restore(new, cpu))
fpu.preload = 0;
else
prefetch(new->thread.fpu.state);
@@ -391,44 +427,40 @@
{
if (fpu.preload) {
if (unlikely(restore_fpu_checking(new)))
- __thread_fpu_end(new);
+ drop_init_fpu(new);
}
}
/*
* Signal frame handlers...
*/
-extern int save_i387_xstate(void __user *buf);
-extern int restore_i387_xstate(void __user *buf);
+extern int save_xstate_sig(void __user *buf, void __user *fx, int size);
+extern int __restore_xstate_sig(void __user *buf, void __user *fx, int size);
-static inline void __clear_fpu(struct task_struct *tsk)
+static inline int xstate_sigframe_size(void)
{
- if (__thread_has_fpu(tsk)) {
- /* Ignore delayed exceptions from user space */
- asm volatile("1: fwait\n"
- "2:\n"
- _ASM_EXTABLE(1b, 2b));
- __thread_fpu_end(tsk);
+ return use_xsave() ? xstate_size + FP_XSTATE_MAGIC2_SIZE : xstate_size;
+}
+
+static inline int restore_xstate_sig(void __user *buf, int ia32_frame)
+{
+ void __user *buf_fx = buf;
+ int size = xstate_sigframe_size();
+
+ if (ia32_frame && use_fxsr()) {
+ buf_fx = buf + sizeof(struct i387_fsave_struct);
+ size += sizeof(struct i387_fsave_struct);
}
+
+ return __restore_xstate_sig(buf, buf_fx, size);
}
/*
- * The actual user_fpu_begin/end() functions
- * need to be preemption-safe.
+ * Need to be preemption-safe.
*
- * NOTE! user_fpu_end() must be used only after you
- * have saved the FP state, and user_fpu_begin() must
- * be used only immediately before restoring it.
- * These functions do not do any save/restore on
- * their own.
+ * NOTE! user_fpu_begin() must be used only immediately before restoring
+ * it. This function does not do any save/restore on their own.
*/
-static inline void user_fpu_end(void)
-{
- preempt_disable();
- __thread_fpu_end(current);
- preempt_enable();
-}
-
static inline void user_fpu_begin(void)
{
preempt_disable();
@@ -437,25 +469,32 @@
preempt_enable();
}
+static inline void __save_fpu(struct task_struct *tsk)
+{
+ if (use_xsave())
+ xsave_state(&tsk->thread.fpu.state->xsave, -1);
+ else
+ fpu_fxsave(&tsk->thread.fpu);
+}
+
/*
* These disable preemption on their own and are safe
*/
static inline void save_init_fpu(struct task_struct *tsk)
{
WARN_ON_ONCE(!__thread_has_fpu(tsk));
+
+ if (use_eager_fpu()) {
+ __save_fpu(tsk);
+ return;
+ }
+
preempt_disable();
__save_init_fpu(tsk);
__thread_fpu_end(tsk);
preempt_enable();
}
-static inline void clear_fpu(struct task_struct *tsk)
-{
- preempt_disable();
- __clear_fpu(tsk);
- preempt_enable();
-}
-
/*
* i387 state interaction
*/
@@ -510,11 +549,34 @@
}
}
-static inline void fpu_copy(struct fpu *dst, struct fpu *src)
+static inline void fpu_copy(struct task_struct *dst, struct task_struct *src)
{
- memcpy(dst->state, src->state, xstate_size);
+ if (use_eager_fpu()) {
+ memset(&dst->thread.fpu.state->xsave, 0, xstate_size);
+ __save_fpu(dst);
+ } else {
+ struct fpu *dfpu = &dst->thread.fpu;
+ struct fpu *sfpu = &src->thread.fpu;
+
+ unlazy_fpu(src);
+ memcpy(dfpu->state, sfpu->state, xstate_size);
+ }
}
-extern void fpu_finit(struct fpu *fpu);
+static inline unsigned long
+alloc_mathframe(unsigned long sp, int ia32_frame, unsigned long *buf_fx,
+ unsigned long *size)
+{
+ unsigned long frame_size = xstate_sigframe_size();
+
+ *buf_fx = sp = round_down(sp - frame_size, 64);
+ if (ia32_frame && use_fxsr()) {
+ frame_size += sizeof(struct i387_fsave_struct);
+ sp -= sizeof(struct i387_fsave_struct);
+ }
+
+ *size = frame_size;
+ return sp;
+}
#endif
diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
index b0767bc..9a25b52 100644
--- a/arch/x86/include/asm/ftrace.h
+++ b/arch/x86/include/asm/ftrace.h
@@ -3,38 +3,54 @@
#ifdef __ASSEMBLY__
- .macro MCOUNT_SAVE_FRAME
- /* taken from glibc */
- subq $0x38, %rsp
- movq %rax, (%rsp)
- movq %rcx, 8(%rsp)
- movq %rdx, 16(%rsp)
- movq %rsi, 24(%rsp)
- movq %rdi, 32(%rsp)
- movq %r8, 40(%rsp)
- movq %r9, 48(%rsp)
+ /* skip is set if the stack was already partially adjusted */
+ .macro MCOUNT_SAVE_FRAME skip=0
+ /*
+ * We add enough stack to save all regs.
+ */
+ subq $(SS+8-\skip), %rsp
+ movq %rax, RAX(%rsp)
+ movq %rcx, RCX(%rsp)
+ movq %rdx, RDX(%rsp)
+ movq %rsi, RSI(%rsp)
+ movq %rdi, RDI(%rsp)
+ movq %r8, R8(%rsp)
+ movq %r9, R9(%rsp)
+ /* Move RIP to its proper location */
+ movq SS+8(%rsp), %rdx
+ movq %rdx, RIP(%rsp)
.endm
- .macro MCOUNT_RESTORE_FRAME
- movq 48(%rsp), %r9
- movq 40(%rsp), %r8
- movq 32(%rsp), %rdi
- movq 24(%rsp), %rsi
- movq 16(%rsp), %rdx
- movq 8(%rsp), %rcx
- movq (%rsp), %rax
- addq $0x38, %rsp
+ .macro MCOUNT_RESTORE_FRAME skip=0
+ movq R9(%rsp), %r9
+ movq R8(%rsp), %r8
+ movq RDI(%rsp), %rdi
+ movq RSI(%rsp), %rsi
+ movq RDX(%rsp), %rdx
+ movq RCX(%rsp), %rcx
+ movq RAX(%rsp), %rax
+ addq $(SS+8-\skip), %rsp
.endm
#endif
#ifdef CONFIG_FUNCTION_TRACER
-#define MCOUNT_ADDR ((long)(mcount))
+#ifdef CC_USING_FENTRY
+# define MCOUNT_ADDR ((long)(__fentry__))
+#else
+# define MCOUNT_ADDR ((long)(mcount))
+#endif
#define MCOUNT_INSN_SIZE 5 /* sizeof mcount call */
+#ifdef CONFIG_DYNAMIC_FTRACE
+#define ARCH_SUPPORTS_FTRACE_OPS 1
+#define ARCH_SUPPORTS_FTRACE_SAVE_REGS
+#endif
+
#ifndef __ASSEMBLY__
extern void mcount(void);
extern atomic_t modifying_ftrace_code;
+extern void __fentry__(void);
static inline unsigned long ftrace_call_adjust(unsigned long addr)
{
diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h
index d3895db..81f04ce 100644
--- a/arch/x86/include/asm/hardirq.h
+++ b/arch/x86/include/asm/hardirq.h
@@ -18,6 +18,10 @@
#ifdef CONFIG_SMP
unsigned int irq_resched_count;
unsigned int irq_call_count;
+ /*
+ * irq_tlb_count is double-counted in irq_call_count, so it must be
+ * subtracted from irq_call_count when displaying irq_call_count
+ */
unsigned int irq_tlb_count;
#endif
#ifdef CONFIG_X86_THERMAL_VECTOR
diff --git a/arch/x86/include/asm/hpet.h b/arch/x86/include/asm/hpet.h
index 2c392d6..434e210 100644
--- a/arch/x86/include/asm/hpet.h
+++ b/arch/x86/include/asm/hpet.h
@@ -35,8 +35,6 @@
#define HPET_ID_NUMBER_SHIFT 8
#define HPET_ID_VENDOR_SHIFT 16
-#define HPET_ID_VENDOR_8086 0x8086
-
#define HPET_CFG_ENABLE 0x001
#define HPET_CFG_LEGACY 0x002
#define HPET_LEGACY_8254 2
diff --git a/arch/x86/include/asm/i387.h b/arch/x86/include/asm/i387.h
index 257d9cc..ed8089d6 100644
--- a/arch/x86/include/asm/i387.h
+++ b/arch/x86/include/asm/i387.h
@@ -19,12 +19,37 @@
struct user_i387_struct;
extern int init_fpu(struct task_struct *child);
+extern void fpu_finit(struct fpu *fpu);
extern int dump_fpu(struct pt_regs *, struct user_i387_struct *);
extern void math_state_restore(void);
extern bool irq_fpu_usable(void);
-extern void kernel_fpu_begin(void);
-extern void kernel_fpu_end(void);
+
+/*
+ * Careful: __kernel_fpu_begin/end() must be called with preempt disabled
+ * and they don't touch the preempt state on their own.
+ * If you enable preemption after __kernel_fpu_begin(), preempt notifier
+ * should call the __kernel_fpu_end() to prevent the kernel/user FPU
+ * state from getting corrupted. KVM for example uses this model.
+ *
+ * All other cases use kernel_fpu_begin/end() which disable preemption
+ * during kernel FPU usage.
+ */
+extern void __kernel_fpu_begin(void);
+extern void __kernel_fpu_end(void);
+
+static inline void kernel_fpu_begin(void)
+{
+ WARN_ON_ONCE(!irq_fpu_usable());
+ preempt_disable();
+ __kernel_fpu_begin();
+}
+
+static inline void kernel_fpu_end(void)
+{
+ __kernel_fpu_end();
+ preempt_enable();
+}
/*
* Some instructions like VIA's padlock instructions generate a spurious
diff --git a/arch/x86/include/asm/iommu_table.h b/arch/x86/include/asm/iommu_table.h
index f229b13..f42a047 100644
--- a/arch/x86/include/asm/iommu_table.h
+++ b/arch/x86/include/asm/iommu_table.h
@@ -48,7 +48,7 @@
#define __IOMMU_INIT(_detect, _depend, _early_init, _late_init, _finish)\
- static const struct iommu_table_entry const \
+ static const struct iommu_table_entry \
__iommu_entry_##_detect __used \
__attribute__ ((unused, __section__(".iommu_table"), \
aligned((sizeof(void *))))) \
@@ -63,10 +63,10 @@
* to stop detecting the other IOMMUs after yours has been detected.
*/
#define IOMMU_INIT_POST(_detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, 0, 0, 0)
+ __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 0)
#define IOMMU_INIT_POST_FINISH(detect) \
- __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, 0, 0, 1)
+ __IOMMU_INIT(_detect, pci_swiotlb_detect_4gb, NULL, NULL, 1)
/*
* A more sophisticated version of IOMMU_INIT. This variant requires:
diff --git a/arch/x86/include/asm/kprobes.h b/arch/x86/include/asm/kprobes.h
index 5478825..d3ddd17 100644
--- a/arch/x86/include/asm/kprobes.h
+++ b/arch/x86/include/asm/kprobes.h
@@ -27,6 +27,7 @@
#include <asm/insn.h>
#define __ARCH_WANT_KPROBES_INSN_SLOT
+#define ARCH_SUPPORTS_KPROBES_ON_FTRACE
struct pt_regs;
struct kprobe;
diff --git a/arch/x86/include/asm/kvm.h b/arch/x86/include/asm/kvm.h
index 246617e..41e08cb 100644
--- a/arch/x86/include/asm/kvm.h
+++ b/arch/x86/include/asm/kvm.h
@@ -9,6 +9,22 @@
#include <linux/types.h>
#include <linux/ioctl.h>
+#define DE_VECTOR 0
+#define DB_VECTOR 1
+#define BP_VECTOR 3
+#define OF_VECTOR 4
+#define BR_VECTOR 5
+#define UD_VECTOR 6
+#define NM_VECTOR 7
+#define DF_VECTOR 8
+#define TS_VECTOR 10
+#define NP_VECTOR 11
+#define SS_VECTOR 12
+#define GP_VECTOR 13
+#define PF_VECTOR 14
+#define MF_VECTOR 16
+#define MC_VECTOR 18
+
/* Select x86 specific features in <linux/kvm.h> */
#define __KVM_HAVE_PIT
#define __KVM_HAVE_IOAPIC
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 09155d6..1eaa6b0 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -75,22 +75,6 @@
#define KVM_HPAGE_MASK(x) (~(KVM_HPAGE_SIZE(x) - 1))
#define KVM_PAGES_PER_HPAGE(x) (KVM_HPAGE_SIZE(x) / PAGE_SIZE)
-#define DE_VECTOR 0
-#define DB_VECTOR 1
-#define BP_VECTOR 3
-#define OF_VECTOR 4
-#define BR_VECTOR 5
-#define UD_VECTOR 6
-#define NM_VECTOR 7
-#define DF_VECTOR 8
-#define TS_VECTOR 10
-#define NP_VECTOR 11
-#define SS_VECTOR 12
-#define GP_VECTOR 13
-#define PF_VECTOR 14
-#define MF_VECTOR 16
-#define MC_VECTOR 18
-
#define SELECTOR_TI_MASK (1 << 2)
#define SELECTOR_RPL_MASK 0x03
diff --git a/arch/x86/include/asm/mce.h b/arch/x86/include/asm/mce.h
index a3ac52b..54d73b1 100644
--- a/arch/x86/include/asm/mce.h
+++ b/arch/x86/include/asm/mce.h
@@ -116,19 +116,9 @@
/* Software defined banks */
#define MCE_EXTENDED_BANK 128
#define MCE_THERMAL_BANK MCE_EXTENDED_BANK + 0
-
-#define K8_MCE_THRESHOLD_BASE (MCE_EXTENDED_BANK + 1) /* MCE_AMD */
-#define K8_MCE_THRESHOLD_BANK_0 (MCE_THRESHOLD_BASE + 0 * 9)
-#define K8_MCE_THRESHOLD_BANK_1 (MCE_THRESHOLD_BASE + 1 * 9)
-#define K8_MCE_THRESHOLD_BANK_2 (MCE_THRESHOLD_BASE + 2 * 9)
-#define K8_MCE_THRESHOLD_BANK_3 (MCE_THRESHOLD_BASE + 3 * 9)
-#define K8_MCE_THRESHOLD_BANK_4 (MCE_THRESHOLD_BASE + 4 * 9)
-#define K8_MCE_THRESHOLD_BANK_5 (MCE_THRESHOLD_BASE + 5 * 9)
-#define K8_MCE_THRESHOLD_DRAM_ECC (MCE_THRESHOLD_BANK_4 + 0)
-
+#define K8_MCE_THRESHOLD_BASE (MCE_EXTENDED_BANK + 1)
#ifdef __KERNEL__
-
extern void mce_register_decode_chain(struct notifier_block *nb);
extern void mce_unregister_decode_chain(struct notifier_block *nb);
@@ -171,6 +161,7 @@
#ifdef CONFIG_X86_MCE_INTEL
extern int mce_cmci_disabled;
extern int mce_ignore_ce;
+extern int mce_bios_cmci_threshold;
void mce_intel_feature_init(struct cpuinfo_x86 *c);
void cmci_clear(void);
void cmci_reenable(void);
diff --git a/arch/x86/include/asm/microcode.h b/arch/x86/include/asm/microcode.h
index 4ebe157..43d921b 100644
--- a/arch/x86/include/asm/microcode.h
+++ b/arch/x86/include/asm/microcode.h
@@ -15,8 +15,8 @@
enum ucode_state (*request_microcode_user) (int cpu,
const void __user *buf, size_t size);
- enum ucode_state (*request_microcode_fw) (int cpu,
- struct device *device);
+ enum ucode_state (*request_microcode_fw) (int cpu, struct device *,
+ bool refresh_fw);
void (*microcode_fini_cpu) (int cpu);
@@ -49,12 +49,6 @@
#ifdef CONFIG_MICROCODE_AMD
extern struct microcode_ops * __init init_amd_microcode(void);
extern void __exit exit_amd_microcode(void);
-
-static inline void get_ucode_data(void *to, const u8 *from, size_t n)
-{
- memcpy(to, from, n);
-}
-
#else
static inline struct microcode_ops * __init init_amd_microcode(void)
{
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index cb4e43b..4fabcdf 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -262,4 +262,6 @@
static inline void amd_pmu_disable_virt(void) { }
#endif
+#define arch_perf_out_copy_user copy_from_user_nmi
+
#endif /* _ASM_X86_PERF_EVENT_H */
diff --git a/arch/x86/include/asm/perf_regs.h b/arch/x86/include/asm/perf_regs.h
new file mode 100644
index 0000000..3f2207b
--- /dev/null
+++ b/arch/x86/include/asm/perf_regs.h
@@ -0,0 +1,33 @@
+#ifndef _ASM_X86_PERF_REGS_H
+#define _ASM_X86_PERF_REGS_H
+
+enum perf_event_x86_regs {
+ PERF_REG_X86_AX,
+ PERF_REG_X86_BX,
+ PERF_REG_X86_CX,
+ PERF_REG_X86_DX,
+ PERF_REG_X86_SI,
+ PERF_REG_X86_DI,
+ PERF_REG_X86_BP,
+ PERF_REG_X86_SP,
+ PERF_REG_X86_IP,
+ PERF_REG_X86_FLAGS,
+ PERF_REG_X86_CS,
+ PERF_REG_X86_SS,
+ PERF_REG_X86_DS,
+ PERF_REG_X86_ES,
+ PERF_REG_X86_FS,
+ PERF_REG_X86_GS,
+ PERF_REG_X86_R8,
+ PERF_REG_X86_R9,
+ PERF_REG_X86_R10,
+ PERF_REG_X86_R11,
+ PERF_REG_X86_R12,
+ PERF_REG_X86_R13,
+ PERF_REG_X86_R14,
+ PERF_REG_X86_R15,
+
+ PERF_REG_X86_32_MAX = PERF_REG_X86_GS + 1,
+ PERF_REG_X86_64_MAX = PERF_REG_X86_R15 + 1,
+};
+#endif /* _ASM_X86_PERF_REGS_H */
diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
index 013286a1..db8fec6 100644
--- a/arch/x86/include/asm/pgtable_types.h
+++ b/arch/x86/include/asm/pgtable_types.h
@@ -303,11 +303,9 @@
extern void native_pagetable_reserve(u64 start, u64 end);
#ifdef CONFIG_X86_32
-extern void native_pagetable_setup_start(pgd_t *base);
-extern void native_pagetable_setup_done(pgd_t *base);
+extern void native_pagetable_init(void);
#else
-#define native_pagetable_setup_start x86_init_pgd_noop
-#define native_pagetable_setup_done x86_init_pgd_noop
+#define native_pagetable_init paging_init
#endif
struct seq_file;
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index d048cad..b98c0d9 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -423,7 +423,6 @@
DECLARE_PER_CPU(char *, irq_stack_ptr);
DECLARE_PER_CPU(unsigned int, irq_count);
-extern unsigned long kernel_eflags;
extern asmlinkage void ignore_sysret(void);
#else /* X86_64 */
#ifdef CONFIG_CC_STACKPROTECTOR
@@ -759,6 +758,8 @@
wrmsrl(MSR_IA32_DEBUGCTLMSR, debugctlmsr);
}
+extern void set_task_blockstep(struct task_struct *task, bool on);
+
/*
* from system description table in BIOS. Mostly for MCA use, but
* others may find it useful:
diff --git a/arch/x86/include/asm/rcu.h b/arch/x86/include/asm/rcu.h
new file mode 100644
index 0000000..d1ac07a
--- /dev/null
+++ b/arch/x86/include/asm/rcu.h
@@ -0,0 +1,32 @@
+#ifndef _ASM_X86_RCU_H
+#define _ASM_X86_RCU_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/rcupdate.h>
+#include <asm/ptrace.h>
+
+static inline void exception_enter(struct pt_regs *regs)
+{
+ rcu_user_exit();
+}
+
+static inline void exception_exit(struct pt_regs *regs)
+{
+#ifdef CONFIG_RCU_USER_QS
+ if (user_mode(regs))
+ rcu_user_enter();
+#endif
+}
+
+#else /* __ASSEMBLY__ */
+
+#ifdef CONFIG_RCU_USER_QS
+# define SCHEDULE_USER call schedule_user
+#else
+# define SCHEDULE_USER call schedule
+#endif
+
+#endif /* !__ASSEMBLY__ */
+
+#endif
diff --git a/arch/x86/include/asm/signal.h b/arch/x86/include/asm/signal.h
index 598457c..323973f 100644
--- a/arch/x86/include/asm/signal.h
+++ b/arch/x86/include/asm/signal.h
@@ -31,6 +31,10 @@
unsigned long sig[_NSIG_WORDS];
} sigset_t;
+#ifndef CONFIG_COMPAT
+typedef sigset_t compat_sigset_t;
+#endif
+
#else
/* Here we must cater to libcs that poke about in kernel headers. */
diff --git a/arch/x86/include/asm/spinlock.h b/arch/x86/include/asm/spinlock.h
index b315a33..33692ea 100644
--- a/arch/x86/include/asm/spinlock.h
+++ b/arch/x86/include/asm/spinlock.h
@@ -12,8 +12,7 @@
* Simple spin lock operations. There are two variants, one clears IRQ's
* on the local processor, one does not.
*
- * These are fair FIFO ticket locks, which are currently limited to 256
- * CPUs.
+ * These are fair FIFO ticket locks, which support up to 2^16 CPUs.
*
* (the type definitions are in asm/spinlock_types.h)
*/
diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index f2b83bc..cdf5674 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -1,6 +1,135 @@
#ifndef __SVM_H
#define __SVM_H
+#define SVM_EXIT_READ_CR0 0x000
+#define SVM_EXIT_READ_CR3 0x003
+#define SVM_EXIT_READ_CR4 0x004
+#define SVM_EXIT_READ_CR8 0x008
+#define SVM_EXIT_WRITE_CR0 0x010
+#define SVM_EXIT_WRITE_CR3 0x013
+#define SVM_EXIT_WRITE_CR4 0x014
+#define SVM_EXIT_WRITE_CR8 0x018
+#define SVM_EXIT_READ_DR0 0x020
+#define SVM_EXIT_READ_DR1 0x021
+#define SVM_EXIT_READ_DR2 0x022
+#define SVM_EXIT_READ_DR3 0x023
+#define SVM_EXIT_READ_DR4 0x024
+#define SVM_EXIT_READ_DR5 0x025
+#define SVM_EXIT_READ_DR6 0x026
+#define SVM_EXIT_READ_DR7 0x027
+#define SVM_EXIT_WRITE_DR0 0x030
+#define SVM_EXIT_WRITE_DR1 0x031
+#define SVM_EXIT_WRITE_DR2 0x032
+#define SVM_EXIT_WRITE_DR3 0x033
+#define SVM_EXIT_WRITE_DR4 0x034
+#define SVM_EXIT_WRITE_DR5 0x035
+#define SVM_EXIT_WRITE_DR6 0x036
+#define SVM_EXIT_WRITE_DR7 0x037
+#define SVM_EXIT_EXCP_BASE 0x040
+#define SVM_EXIT_INTR 0x060
+#define SVM_EXIT_NMI 0x061
+#define SVM_EXIT_SMI 0x062
+#define SVM_EXIT_INIT 0x063
+#define SVM_EXIT_VINTR 0x064
+#define SVM_EXIT_CR0_SEL_WRITE 0x065
+#define SVM_EXIT_IDTR_READ 0x066
+#define SVM_EXIT_GDTR_READ 0x067
+#define SVM_EXIT_LDTR_READ 0x068
+#define SVM_EXIT_TR_READ 0x069
+#define SVM_EXIT_IDTR_WRITE 0x06a
+#define SVM_EXIT_GDTR_WRITE 0x06b
+#define SVM_EXIT_LDTR_WRITE 0x06c
+#define SVM_EXIT_TR_WRITE 0x06d
+#define SVM_EXIT_RDTSC 0x06e
+#define SVM_EXIT_RDPMC 0x06f
+#define SVM_EXIT_PUSHF 0x070
+#define SVM_EXIT_POPF 0x071
+#define SVM_EXIT_CPUID 0x072
+#define SVM_EXIT_RSM 0x073
+#define SVM_EXIT_IRET 0x074
+#define SVM_EXIT_SWINT 0x075
+#define SVM_EXIT_INVD 0x076
+#define SVM_EXIT_PAUSE 0x077
+#define SVM_EXIT_HLT 0x078
+#define SVM_EXIT_INVLPG 0x079
+#define SVM_EXIT_INVLPGA 0x07a
+#define SVM_EXIT_IOIO 0x07b
+#define SVM_EXIT_MSR 0x07c
+#define SVM_EXIT_TASK_SWITCH 0x07d
+#define SVM_EXIT_FERR_FREEZE 0x07e
+#define SVM_EXIT_SHUTDOWN 0x07f
+#define SVM_EXIT_VMRUN 0x080
+#define SVM_EXIT_VMMCALL 0x081
+#define SVM_EXIT_VMLOAD 0x082
+#define SVM_EXIT_VMSAVE 0x083
+#define SVM_EXIT_STGI 0x084
+#define SVM_EXIT_CLGI 0x085
+#define SVM_EXIT_SKINIT 0x086
+#define SVM_EXIT_RDTSCP 0x087
+#define SVM_EXIT_ICEBP 0x088
+#define SVM_EXIT_WBINVD 0x089
+#define SVM_EXIT_MONITOR 0x08a
+#define SVM_EXIT_MWAIT 0x08b
+#define SVM_EXIT_MWAIT_COND 0x08c
+#define SVM_EXIT_XSETBV 0x08d
+#define SVM_EXIT_NPF 0x400
+
+#define SVM_EXIT_ERR -1
+
+#define SVM_EXIT_REASONS \
+ { SVM_EXIT_READ_CR0, "read_cr0" }, \
+ { SVM_EXIT_READ_CR3, "read_cr3" }, \
+ { SVM_EXIT_READ_CR4, "read_cr4" }, \
+ { SVM_EXIT_READ_CR8, "read_cr8" }, \
+ { SVM_EXIT_WRITE_CR0, "write_cr0" }, \
+ { SVM_EXIT_WRITE_CR3, "write_cr3" }, \
+ { SVM_EXIT_WRITE_CR4, "write_cr4" }, \
+ { SVM_EXIT_WRITE_CR8, "write_cr8" }, \
+ { SVM_EXIT_READ_DR0, "read_dr0" }, \
+ { SVM_EXIT_READ_DR1, "read_dr1" }, \
+ { SVM_EXIT_READ_DR2, "read_dr2" }, \
+ { SVM_EXIT_READ_DR3, "read_dr3" }, \
+ { SVM_EXIT_WRITE_DR0, "write_dr0" }, \
+ { SVM_EXIT_WRITE_DR1, "write_dr1" }, \
+ { SVM_EXIT_WRITE_DR2, "write_dr2" }, \
+ { SVM_EXIT_WRITE_DR3, "write_dr3" }, \
+ { SVM_EXIT_WRITE_DR5, "write_dr5" }, \
+ { SVM_EXIT_WRITE_DR7, "write_dr7" }, \
+ { SVM_EXIT_EXCP_BASE + DB_VECTOR, "DB excp" }, \
+ { SVM_EXIT_EXCP_BASE + BP_VECTOR, "BP excp" }, \
+ { SVM_EXIT_EXCP_BASE + UD_VECTOR, "UD excp" }, \
+ { SVM_EXIT_EXCP_BASE + PF_VECTOR, "PF excp" }, \
+ { SVM_EXIT_EXCP_BASE + NM_VECTOR, "NM excp" }, \
+ { SVM_EXIT_EXCP_BASE + MC_VECTOR, "MC excp" }, \
+ { SVM_EXIT_INTR, "interrupt" }, \
+ { SVM_EXIT_NMI, "nmi" }, \
+ { SVM_EXIT_SMI, "smi" }, \
+ { SVM_EXIT_INIT, "init" }, \
+ { SVM_EXIT_VINTR, "vintr" }, \
+ { SVM_EXIT_CPUID, "cpuid" }, \
+ { SVM_EXIT_INVD, "invd" }, \
+ { SVM_EXIT_HLT, "hlt" }, \
+ { SVM_EXIT_INVLPG, "invlpg" }, \
+ { SVM_EXIT_INVLPGA, "invlpga" }, \
+ { SVM_EXIT_IOIO, "io" }, \
+ { SVM_EXIT_MSR, "msr" }, \
+ { SVM_EXIT_TASK_SWITCH, "task_switch" }, \
+ { SVM_EXIT_SHUTDOWN, "shutdown" }, \
+ { SVM_EXIT_VMRUN, "vmrun" }, \
+ { SVM_EXIT_VMMCALL, "hypercall" }, \
+ { SVM_EXIT_VMLOAD, "vmload" }, \
+ { SVM_EXIT_VMSAVE, "vmsave" }, \
+ { SVM_EXIT_STGI, "stgi" }, \
+ { SVM_EXIT_CLGI, "clgi" }, \
+ { SVM_EXIT_SKINIT, "skinit" }, \
+ { SVM_EXIT_WBINVD, "wbinvd" }, \
+ { SVM_EXIT_MONITOR, "monitor" }, \
+ { SVM_EXIT_MWAIT, "mwait" }, \
+ { SVM_EXIT_XSETBV, "xsetbv" }, \
+ { SVM_EXIT_NPF, "npf" }
+
+#ifdef __KERNEL__
+
enum {
INTERCEPT_INTR,
INTERCEPT_NMI,
@@ -264,81 +393,6 @@
#define SVM_EXITINFO_REG_MASK 0x0F
-#define SVM_EXIT_READ_CR0 0x000
-#define SVM_EXIT_READ_CR3 0x003
-#define SVM_EXIT_READ_CR4 0x004
-#define SVM_EXIT_READ_CR8 0x008
-#define SVM_EXIT_WRITE_CR0 0x010
-#define SVM_EXIT_WRITE_CR3 0x013
-#define SVM_EXIT_WRITE_CR4 0x014
-#define SVM_EXIT_WRITE_CR8 0x018
-#define SVM_EXIT_READ_DR0 0x020
-#define SVM_EXIT_READ_DR1 0x021
-#define SVM_EXIT_READ_DR2 0x022
-#define SVM_EXIT_READ_DR3 0x023
-#define SVM_EXIT_READ_DR4 0x024
-#define SVM_EXIT_READ_DR5 0x025
-#define SVM_EXIT_READ_DR6 0x026
-#define SVM_EXIT_READ_DR7 0x027
-#define SVM_EXIT_WRITE_DR0 0x030
-#define SVM_EXIT_WRITE_DR1 0x031
-#define SVM_EXIT_WRITE_DR2 0x032
-#define SVM_EXIT_WRITE_DR3 0x033
-#define SVM_EXIT_WRITE_DR4 0x034
-#define SVM_EXIT_WRITE_DR5 0x035
-#define SVM_EXIT_WRITE_DR6 0x036
-#define SVM_EXIT_WRITE_DR7 0x037
-#define SVM_EXIT_EXCP_BASE 0x040
-#define SVM_EXIT_INTR 0x060
-#define SVM_EXIT_NMI 0x061
-#define SVM_EXIT_SMI 0x062
-#define SVM_EXIT_INIT 0x063
-#define SVM_EXIT_VINTR 0x064
-#define SVM_EXIT_CR0_SEL_WRITE 0x065
-#define SVM_EXIT_IDTR_READ 0x066
-#define SVM_EXIT_GDTR_READ 0x067
-#define SVM_EXIT_LDTR_READ 0x068
-#define SVM_EXIT_TR_READ 0x069
-#define SVM_EXIT_IDTR_WRITE 0x06a
-#define SVM_EXIT_GDTR_WRITE 0x06b
-#define SVM_EXIT_LDTR_WRITE 0x06c
-#define SVM_EXIT_TR_WRITE 0x06d
-#define SVM_EXIT_RDTSC 0x06e
-#define SVM_EXIT_RDPMC 0x06f
-#define SVM_EXIT_PUSHF 0x070
-#define SVM_EXIT_POPF 0x071
-#define SVM_EXIT_CPUID 0x072
-#define SVM_EXIT_RSM 0x073
-#define SVM_EXIT_IRET 0x074
-#define SVM_EXIT_SWINT 0x075
-#define SVM_EXIT_INVD 0x076
-#define SVM_EXIT_PAUSE 0x077
-#define SVM_EXIT_HLT 0x078
-#define SVM_EXIT_INVLPG 0x079
-#define SVM_EXIT_INVLPGA 0x07a
-#define SVM_EXIT_IOIO 0x07b
-#define SVM_EXIT_MSR 0x07c
-#define SVM_EXIT_TASK_SWITCH 0x07d
-#define SVM_EXIT_FERR_FREEZE 0x07e
-#define SVM_EXIT_SHUTDOWN 0x07f
-#define SVM_EXIT_VMRUN 0x080
-#define SVM_EXIT_VMMCALL 0x081
-#define SVM_EXIT_VMLOAD 0x082
-#define SVM_EXIT_VMSAVE 0x083
-#define SVM_EXIT_STGI 0x084
-#define SVM_EXIT_CLGI 0x085
-#define SVM_EXIT_SKINIT 0x086
-#define SVM_EXIT_RDTSCP 0x087
-#define SVM_EXIT_ICEBP 0x088
-#define SVM_EXIT_WBINVD 0x089
-#define SVM_EXIT_MONITOR 0x08a
-#define SVM_EXIT_MWAIT 0x08b
-#define SVM_EXIT_MWAIT_COND 0x08c
-#define SVM_EXIT_XSETBV 0x08d
-#define SVM_EXIT_NPF 0x400
-
-#define SVM_EXIT_ERR -1
-
#define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
#define SVM_VMLOAD ".byte 0x0f, 0x01, 0xda"
@@ -350,3 +404,4 @@
#endif
+#endif
diff --git a/arch/x86/include/asm/sys_ia32.h b/arch/x86/include/asm/sys_ia32.h
index 3fda9db4..4ca1c61 100644
--- a/arch/x86/include/asm/sys_ia32.h
+++ b/arch/x86/include/asm/sys_ia32.h
@@ -40,7 +40,7 @@
struct old_sigaction32 __user *);
asmlinkage long sys32_alarm(unsigned int);
-asmlinkage long sys32_waitpid(compat_pid_t, unsigned int *, int);
+asmlinkage long sys32_waitpid(compat_pid_t, unsigned int __user *, int);
asmlinkage long sys32_sysfs(int, u32, u32);
asmlinkage long sys32_sched_rr_get_interval(compat_pid_t,
diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index 89f794f..c535d847 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -89,6 +89,7 @@
#define TIF_NOTSC 16 /* TSC is not accessible in userland */
#define TIF_IA32 17 /* IA32 compatibility process */
#define TIF_FORK 18 /* ret_from_fork */
+#define TIF_NOHZ 19 /* in adaptive nohz mode */
#define TIF_MEMDIE 20 /* is terminating due to OOM killer */
#define TIF_DEBUG 21 /* uses debug registers */
#define TIF_IO_BITMAP 22 /* uses I/O bitmap */
@@ -114,6 +115,7 @@
#define _TIF_NOTSC (1 << TIF_NOTSC)
#define _TIF_IA32 (1 << TIF_IA32)
#define _TIF_FORK (1 << TIF_FORK)
+#define _TIF_NOHZ (1 << TIF_NOHZ)
#define _TIF_DEBUG (1 << TIF_DEBUG)
#define _TIF_IO_BITMAP (1 << TIF_IO_BITMAP)
#define _TIF_FORCED_TF (1 << TIF_FORCED_TF)
@@ -126,12 +128,13 @@
/* work to do in syscall_trace_enter() */
#define _TIF_WORK_SYSCALL_ENTRY \
(_TIF_SYSCALL_TRACE | _TIF_SYSCALL_EMU | _TIF_SYSCALL_AUDIT | \
- _TIF_SECCOMP | _TIF_SINGLESTEP | _TIF_SYSCALL_TRACEPOINT)
+ _TIF_SECCOMP | _TIF_SINGLESTEP | _TIF_SYSCALL_TRACEPOINT | \
+ _TIF_NOHZ)
/* work to do in syscall_trace_leave() */
#define _TIF_WORK_SYSCALL_EXIT \
(_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | _TIF_SINGLESTEP | \
- _TIF_SYSCALL_TRACEPOINT)
+ _TIF_SYSCALL_TRACEPOINT | _TIF_NOHZ)
/* work to do on interrupt/exception return */
#define _TIF_WORK_MASK \
@@ -141,7 +144,8 @@
/* work to do on any return to user space */
#define _TIF_ALLWORK_MASK \
- ((0x0000FFFF & ~_TIF_SECCOMP) | _TIF_SYSCALL_TRACEPOINT)
+ ((0x0000FFFF & ~_TIF_SECCOMP) | _TIF_SYSCALL_TRACEPOINT | \
+ _TIF_NOHZ)
/* Only used for 64 bit */
#define _TIF_DO_NOTIFY_MASK \
diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index f3971bb..8ff8be7 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -42,10 +42,11 @@
};
struct arch_uprobe_task {
- unsigned long saved_trap_nr;
#ifdef CONFIG_X86_64
unsigned long saved_scratch_register;
#endif
+ unsigned int saved_trap_nr;
+ unsigned int saved_tf;
};
extern int arch_uprobe_analyze_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long addr);
diff --git a/arch/x86/include/asm/vdso.h b/arch/x86/include/asm/vdso.h
index bb05228..fddb53d 100644
--- a/arch/x86/include/asm/vdso.h
+++ b/arch/x86/include/asm/vdso.h
@@ -11,7 +11,8 @@
#define VDSO32_SYMBOL(base, name) \
({ \
extern const char VDSO32_##name[]; \
- (void *)(VDSO32_##name - VDSO32_PRELINK + (unsigned long)(base)); \
+ (void __user *)(VDSO32_##name - VDSO32_PRELINK + \
+ (unsigned long)(base)); \
})
#endif
diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h
index 74fcb96..36ec21c 100644
--- a/arch/x86/include/asm/vmx.h
+++ b/arch/x86/include/asm/vmx.h
@@ -25,6 +25,88 @@
*
*/
+#define VMX_EXIT_REASONS_FAILED_VMENTRY 0x80000000
+
+#define EXIT_REASON_EXCEPTION_NMI 0
+#define EXIT_REASON_EXTERNAL_INTERRUPT 1
+#define EXIT_REASON_TRIPLE_FAULT 2
+
+#define EXIT_REASON_PENDING_INTERRUPT 7
+#define EXIT_REASON_NMI_WINDOW 8
+#define EXIT_REASON_TASK_SWITCH 9
+#define EXIT_REASON_CPUID 10
+#define EXIT_REASON_HLT 12
+#define EXIT_REASON_INVD 13
+#define EXIT_REASON_INVLPG 14
+#define EXIT_REASON_RDPMC 15
+#define EXIT_REASON_RDTSC 16
+#define EXIT_REASON_VMCALL 18
+#define EXIT_REASON_VMCLEAR 19
+#define EXIT_REASON_VMLAUNCH 20
+#define EXIT_REASON_VMPTRLD 21
+#define EXIT_REASON_VMPTRST 22
+#define EXIT_REASON_VMREAD 23
+#define EXIT_REASON_VMRESUME 24
+#define EXIT_REASON_VMWRITE 25
+#define EXIT_REASON_VMOFF 26
+#define EXIT_REASON_VMON 27
+#define EXIT_REASON_CR_ACCESS 28
+#define EXIT_REASON_DR_ACCESS 29
+#define EXIT_REASON_IO_INSTRUCTION 30
+#define EXIT_REASON_MSR_READ 31
+#define EXIT_REASON_MSR_WRITE 32
+#define EXIT_REASON_INVALID_STATE 33
+#define EXIT_REASON_MWAIT_INSTRUCTION 36
+#define EXIT_REASON_MONITOR_INSTRUCTION 39
+#define EXIT_REASON_PAUSE_INSTRUCTION 40
+#define EXIT_REASON_MCE_DURING_VMENTRY 41
+#define EXIT_REASON_TPR_BELOW_THRESHOLD 43
+#define EXIT_REASON_APIC_ACCESS 44
+#define EXIT_REASON_EPT_VIOLATION 48
+#define EXIT_REASON_EPT_MISCONFIG 49
+#define EXIT_REASON_WBINVD 54
+#define EXIT_REASON_XSETBV 55
+#define EXIT_REASON_INVPCID 58
+
+#define VMX_EXIT_REASONS \
+ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
+ { EXIT_REASON_EXTERNAL_INTERRUPT, "EXTERNAL_INTERRUPT" }, \
+ { EXIT_REASON_TRIPLE_FAULT, "TRIPLE_FAULT" }, \
+ { EXIT_REASON_PENDING_INTERRUPT, "PENDING_INTERRUPT" }, \
+ { EXIT_REASON_NMI_WINDOW, "NMI_WINDOW" }, \
+ { EXIT_REASON_TASK_SWITCH, "TASK_SWITCH" }, \
+ { EXIT_REASON_CPUID, "CPUID" }, \
+ { EXIT_REASON_HLT, "HLT" }, \
+ { EXIT_REASON_INVLPG, "INVLPG" }, \
+ { EXIT_REASON_RDPMC, "RDPMC" }, \
+ { EXIT_REASON_RDTSC, "RDTSC" }, \
+ { EXIT_REASON_VMCALL, "VMCALL" }, \
+ { EXIT_REASON_VMCLEAR, "VMCLEAR" }, \
+ { EXIT_REASON_VMLAUNCH, "VMLAUNCH" }, \
+ { EXIT_REASON_VMPTRLD, "VMPTRLD" }, \
+ { EXIT_REASON_VMPTRST, "VMPTRST" }, \
+ { EXIT_REASON_VMREAD, "VMREAD" }, \
+ { EXIT_REASON_VMRESUME, "VMRESUME" }, \
+ { EXIT_REASON_VMWRITE, "VMWRITE" }, \
+ { EXIT_REASON_VMOFF, "VMOFF" }, \
+ { EXIT_REASON_VMON, "VMON" }, \
+ { EXIT_REASON_CR_ACCESS, "CR_ACCESS" }, \
+ { EXIT_REASON_DR_ACCESS, "DR_ACCESS" }, \
+ { EXIT_REASON_IO_INSTRUCTION, "IO_INSTRUCTION" }, \
+ { EXIT_REASON_MSR_READ, "MSR_READ" }, \
+ { EXIT_REASON_MSR_WRITE, "MSR_WRITE" }, \
+ { EXIT_REASON_MWAIT_INSTRUCTION, "MWAIT_INSTRUCTION" }, \
+ { EXIT_REASON_MONITOR_INSTRUCTION, "MONITOR_INSTRUCTION" }, \
+ { EXIT_REASON_PAUSE_INSTRUCTION, "PAUSE_INSTRUCTION" }, \
+ { EXIT_REASON_MCE_DURING_VMENTRY, "MCE_DURING_VMENTRY" }, \
+ { EXIT_REASON_TPR_BELOW_THRESHOLD, "TPR_BELOW_THRESHOLD" }, \
+ { EXIT_REASON_APIC_ACCESS, "APIC_ACCESS" }, \
+ { EXIT_REASON_EPT_VIOLATION, "EPT_VIOLATION" }, \
+ { EXIT_REASON_EPT_MISCONFIG, "EPT_MISCONFIG" }, \
+ { EXIT_REASON_WBINVD, "WBINVD" }
+
+#ifdef __KERNEL__
+
#include <linux/types.h>
/*
@@ -241,49 +323,6 @@
HOST_RIP = 0x00006c16,
};
-#define VMX_EXIT_REASONS_FAILED_VMENTRY 0x80000000
-
-#define EXIT_REASON_EXCEPTION_NMI 0
-#define EXIT_REASON_EXTERNAL_INTERRUPT 1
-#define EXIT_REASON_TRIPLE_FAULT 2
-
-#define EXIT_REASON_PENDING_INTERRUPT 7
-#define EXIT_REASON_NMI_WINDOW 8
-#define EXIT_REASON_TASK_SWITCH 9
-#define EXIT_REASON_CPUID 10
-#define EXIT_REASON_HLT 12
-#define EXIT_REASON_INVD 13
-#define EXIT_REASON_INVLPG 14
-#define EXIT_REASON_RDPMC 15
-#define EXIT_REASON_RDTSC 16
-#define EXIT_REASON_VMCALL 18
-#define EXIT_REASON_VMCLEAR 19
-#define EXIT_REASON_VMLAUNCH 20
-#define EXIT_REASON_VMPTRLD 21
-#define EXIT_REASON_VMPTRST 22
-#define EXIT_REASON_VMREAD 23
-#define EXIT_REASON_VMRESUME 24
-#define EXIT_REASON_VMWRITE 25
-#define EXIT_REASON_VMOFF 26
-#define EXIT_REASON_VMON 27
-#define EXIT_REASON_CR_ACCESS 28
-#define EXIT_REASON_DR_ACCESS 29
-#define EXIT_REASON_IO_INSTRUCTION 30
-#define EXIT_REASON_MSR_READ 31
-#define EXIT_REASON_MSR_WRITE 32
-#define EXIT_REASON_INVALID_STATE 33
-#define EXIT_REASON_MWAIT_INSTRUCTION 36
-#define EXIT_REASON_MONITOR_INSTRUCTION 39
-#define EXIT_REASON_PAUSE_INSTRUCTION 40
-#define EXIT_REASON_MCE_DURING_VMENTRY 41
-#define EXIT_REASON_TPR_BELOW_THRESHOLD 43
-#define EXIT_REASON_APIC_ACCESS 44
-#define EXIT_REASON_EPT_VIOLATION 48
-#define EXIT_REASON_EPT_MISCONFIG 49
-#define EXIT_REASON_WBINVD 54
-#define EXIT_REASON_XSETBV 55
-#define EXIT_REASON_INVPCID 58
-
/*
* Interruption-information format
*/
@@ -488,3 +527,5 @@
};
#endif
+
+#endif
diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h
index 38155f6..5769349 100644
--- a/arch/x86/include/asm/x86_init.h
+++ b/arch/x86/include/asm/x86_init.h
@@ -81,12 +81,13 @@
/**
* struct x86_init_paging - platform specific paging functions
- * @pagetable_setup_start: platform specific pre paging_init() call
- * @pagetable_setup_done: platform specific post paging_init() call
+ * @pagetable_init: platform specific paging initialization call to setup
+ * the kernel pagetables and prepare accessors functions.
+ * Callback must call paging_init(). Called once after the
+ * direct mapping for phys memory is available.
*/
struct x86_init_paging {
- void (*pagetable_setup_start)(pgd_t *base);
- void (*pagetable_setup_done)(pgd_t *base);
+ void (*pagetable_init)(void);
};
/**
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index 93971e8..472b9b7 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -51,7 +51,8 @@
extern int m2p_add_override(unsigned long mfn, struct page *page,
struct gnttab_map_grant_ref *kmap_op);
-extern int m2p_remove_override(struct page *page, bool clear_pte);
+extern int m2p_remove_override(struct page *page,
+ struct gnttab_map_grant_ref *kmap_op);
extern struct page *m2p_find_override(unsigned long mfn);
extern unsigned long m2p_find_override_pfn(unsigned long mfn, unsigned long pfn);
diff --git a/arch/x86/include/asm/xor_32.h b/arch/x86/include/asm/xor_32.h
index 4545708..aabd585 100644
--- a/arch/x86/include/asm/xor_32.h
+++ b/arch/x86/include/asm/xor_32.h
@@ -534,38 +534,6 @@
* Copyright (C) 1999 Zach Brown (with obvious credit due Ingo)
*/
-#define XMMS_SAVE \
-do { \
- preempt_disable(); \
- cr0 = read_cr0(); \
- clts(); \
- asm volatile( \
- "movups %%xmm0,(%0) ;\n\t" \
- "movups %%xmm1,0x10(%0) ;\n\t" \
- "movups %%xmm2,0x20(%0) ;\n\t" \
- "movups %%xmm3,0x30(%0) ;\n\t" \
- : \
- : "r" (xmm_save) \
- : "memory"); \
-} while (0)
-
-#define XMMS_RESTORE \
-do { \
- asm volatile( \
- "sfence ;\n\t" \
- "movups (%0),%%xmm0 ;\n\t" \
- "movups 0x10(%0),%%xmm1 ;\n\t" \
- "movups 0x20(%0),%%xmm2 ;\n\t" \
- "movups 0x30(%0),%%xmm3 ;\n\t" \
- : \
- : "r" (xmm_save) \
- : "memory"); \
- write_cr0(cr0); \
- preempt_enable(); \
-} while (0)
-
-#define ALIGN16 __attribute__((aligned(16)))
-
#define OFFS(x) "16*("#x")"
#define PF_OFFS(x) "256+16*("#x")"
#define PF0(x) " prefetchnta "PF_OFFS(x)"(%1) ;\n"
@@ -587,10 +555,8 @@
xor_sse_2(unsigned long bytes, unsigned long *p1, unsigned long *p2)
{
unsigned long lines = bytes >> 8;
- char xmm_save[16*4] ALIGN16;
- int cr0;
- XMMS_SAVE;
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
@@ -633,7 +599,7 @@
:
: "memory");
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static void
@@ -641,10 +607,8 @@
unsigned long *p3)
{
unsigned long lines = bytes >> 8;
- char xmm_save[16*4] ALIGN16;
- int cr0;
- XMMS_SAVE;
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
@@ -694,7 +658,7 @@
:
: "memory" );
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static void
@@ -702,10 +666,8 @@
unsigned long *p3, unsigned long *p4)
{
unsigned long lines = bytes >> 8;
- char xmm_save[16*4] ALIGN16;
- int cr0;
- XMMS_SAVE;
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
@@ -762,7 +724,7 @@
:
: "memory" );
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static void
@@ -770,10 +732,8 @@
unsigned long *p3, unsigned long *p4, unsigned long *p5)
{
unsigned long lines = bytes >> 8;
- char xmm_save[16*4] ALIGN16;
- int cr0;
- XMMS_SAVE;
+ kernel_fpu_begin();
/* Make sure GCC forgets anything it knows about p4 or p5,
such that it won't pass to the asm volatile below a
@@ -850,7 +810,7 @@
like assuming they have some legal value. */
asm("" : "=r" (p4), "=r" (p5));
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static struct xor_block_template xor_block_pIII_sse = {
diff --git a/arch/x86/include/asm/xor_64.h b/arch/x86/include/asm/xor_64.h
index b9b2323..5fc06d0 100644
--- a/arch/x86/include/asm/xor_64.h
+++ b/arch/x86/include/asm/xor_64.h
@@ -34,41 +34,7 @@
* no advantages to be gotten from x86-64 here anyways.
*/
-typedef struct {
- unsigned long a, b;
-} __attribute__((aligned(16))) xmm_store_t;
-
-/* Doesn't use gcc to save the XMM registers, because there is no easy way to
- tell it to do a clts before the register saving. */
-#define XMMS_SAVE \
-do { \
- preempt_disable(); \
- asm volatile( \
- "movq %%cr0,%0 ;\n\t" \
- "clts ;\n\t" \
- "movups %%xmm0,(%1) ;\n\t" \
- "movups %%xmm1,0x10(%1) ;\n\t" \
- "movups %%xmm2,0x20(%1) ;\n\t" \
- "movups %%xmm3,0x30(%1) ;\n\t" \
- : "=&r" (cr0) \
- : "r" (xmm_save) \
- : "memory"); \
-} while (0)
-
-#define XMMS_RESTORE \
-do { \
- asm volatile( \
- "sfence ;\n\t" \
- "movups (%1),%%xmm0 ;\n\t" \
- "movups 0x10(%1),%%xmm1 ;\n\t" \
- "movups 0x20(%1),%%xmm2 ;\n\t" \
- "movups 0x30(%1),%%xmm3 ;\n\t" \
- "movq %0,%%cr0 ;\n\t" \
- : \
- : "r" (cr0), "r" (xmm_save) \
- : "memory"); \
- preempt_enable(); \
-} while (0)
+#include <asm/i387.h>
#define OFFS(x) "16*("#x")"
#define PF_OFFS(x) "256+16*("#x")"
@@ -91,10 +57,8 @@
xor_sse_2(unsigned long bytes, unsigned long *p1, unsigned long *p2)
{
unsigned int lines = bytes >> 8;
- unsigned long cr0;
- xmm_store_t xmm_save[4];
- XMMS_SAVE;
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
@@ -135,7 +99,7 @@
: [inc] "r" (256UL)
: "memory");
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static void
@@ -143,11 +107,8 @@
unsigned long *p3)
{
unsigned int lines = bytes >> 8;
- xmm_store_t xmm_save[4];
- unsigned long cr0;
- XMMS_SAVE;
-
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
#define BLOCK(i) \
@@ -194,7 +155,7 @@
[p1] "+r" (p1), [p2] "+r" (p2), [p3] "+r" (p3)
: [inc] "r" (256UL)
: "memory");
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static void
@@ -202,10 +163,8 @@
unsigned long *p3, unsigned long *p4)
{
unsigned int lines = bytes >> 8;
- xmm_store_t xmm_save[4];
- unsigned long cr0;
- XMMS_SAVE;
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
@@ -261,7 +220,7 @@
: [inc] "r" (256UL)
: "memory" );
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static void
@@ -269,10 +228,8 @@
unsigned long *p3, unsigned long *p4, unsigned long *p5)
{
unsigned int lines = bytes >> 8;
- xmm_store_t xmm_save[4];
- unsigned long cr0;
- XMMS_SAVE;
+ kernel_fpu_begin();
asm volatile(
#undef BLOCK
@@ -336,7 +293,7 @@
: [inc] "r" (256UL)
: "memory");
- XMMS_RESTORE;
+ kernel_fpu_end();
}
static struct xor_block_template xor_block_sse = {
diff --git a/arch/x86/include/asm/xor_avx.h b/arch/x86/include/asm/xor_avx.h
index 2510d35..7ea79c5 100644
--- a/arch/x86/include/asm/xor_avx.h
+++ b/arch/x86/include/asm/xor_avx.h
@@ -20,32 +20,6 @@
#include <linux/compiler.h>
#include <asm/i387.h>
-#define ALIGN32 __aligned(32)
-
-#define YMM_SAVED_REGS 4
-
-#define YMMS_SAVE \
-do { \
- preempt_disable(); \
- cr0 = read_cr0(); \
- clts(); \
- asm volatile("vmovaps %%ymm0, %0" : "=m" (ymm_save[0]) : : "memory"); \
- asm volatile("vmovaps %%ymm1, %0" : "=m" (ymm_save[32]) : : "memory"); \
- asm volatile("vmovaps %%ymm2, %0" : "=m" (ymm_save[64]) : : "memory"); \
- asm volatile("vmovaps %%ymm3, %0" : "=m" (ymm_save[96]) : : "memory"); \
-} while (0);
-
-#define YMMS_RESTORE \
-do { \
- asm volatile("sfence" : : : "memory"); \
- asm volatile("vmovaps %0, %%ymm3" : : "m" (ymm_save[96])); \
- asm volatile("vmovaps %0, %%ymm2" : : "m" (ymm_save[64])); \
- asm volatile("vmovaps %0, %%ymm1" : : "m" (ymm_save[32])); \
- asm volatile("vmovaps %0, %%ymm0" : : "m" (ymm_save[0])); \
- write_cr0(cr0); \
- preempt_enable(); \
-} while (0);
-
#define BLOCK4(i) \
BLOCK(32 * i, 0) \
BLOCK(32 * (i + 1), 1) \
@@ -60,10 +34,9 @@
static void xor_avx_2(unsigned long bytes, unsigned long *p0, unsigned long *p1)
{
- unsigned long cr0, lines = bytes >> 9;
- char ymm_save[32 * YMM_SAVED_REGS] ALIGN32;
+ unsigned long lines = bytes >> 9;
- YMMS_SAVE
+ kernel_fpu_begin();
while (lines--) {
#undef BLOCK
@@ -82,16 +55,15 @@
p1 = (unsigned long *)((uintptr_t)p1 + 512);
}
- YMMS_RESTORE
+ kernel_fpu_end();
}
static void xor_avx_3(unsigned long bytes, unsigned long *p0, unsigned long *p1,
unsigned long *p2)
{
- unsigned long cr0, lines = bytes >> 9;
- char ymm_save[32 * YMM_SAVED_REGS] ALIGN32;
+ unsigned long lines = bytes >> 9;
- YMMS_SAVE
+ kernel_fpu_begin();
while (lines--) {
#undef BLOCK
@@ -113,16 +85,15 @@
p2 = (unsigned long *)((uintptr_t)p2 + 512);
}
- YMMS_RESTORE
+ kernel_fpu_end();
}
static void xor_avx_4(unsigned long bytes, unsigned long *p0, unsigned long *p1,
unsigned long *p2, unsigned long *p3)
{
- unsigned long cr0, lines = bytes >> 9;
- char ymm_save[32 * YMM_SAVED_REGS] ALIGN32;
+ unsigned long lines = bytes >> 9;
- YMMS_SAVE
+ kernel_fpu_begin();
while (lines--) {
#undef BLOCK
@@ -147,16 +118,15 @@
p3 = (unsigned long *)((uintptr_t)p3 + 512);
}
- YMMS_RESTORE
+ kernel_fpu_end();
}
static void xor_avx_5(unsigned long bytes, unsigned long *p0, unsigned long *p1,
unsigned long *p2, unsigned long *p3, unsigned long *p4)
{
- unsigned long cr0, lines = bytes >> 9;
- char ymm_save[32 * YMM_SAVED_REGS] ALIGN32;
+ unsigned long lines = bytes >> 9;
- YMMS_SAVE
+ kernel_fpu_begin();
while (lines--) {
#undef BLOCK
@@ -184,7 +154,7 @@
p4 = (unsigned long *)((uintptr_t)p4 + 512);
}
- YMMS_RESTORE
+ kernel_fpu_end();
}
static struct xor_block_template xor_block_avx = {
diff --git a/arch/x86/include/asm/xsave.h b/arch/x86/include/asm/xsave.h
index 8a1b6f9..2ddee1b8 100644
--- a/arch/x86/include/asm/xsave.h
+++ b/arch/x86/include/asm/xsave.h
@@ -34,17 +34,14 @@
extern unsigned int xstate_size;
extern u64 pcntxt_mask;
extern u64 xstate_fx_sw_bytes[USER_XSTATE_FX_SW_WORDS];
+extern struct xsave_struct *init_xstate_buf;
extern void xsave_init(void);
extern void update_regset_xstate_info(unsigned int size, u64 xstate_mask);
extern int init_fpu(struct task_struct *child);
-extern int check_for_xstate(struct i387_fxsave_struct __user *buf,
- void __user *fpstate,
- struct _fpx_sw_bytes *sw);
-static inline int fpu_xrstor_checking(struct fpu *fpu)
+static inline int fpu_xrstor_checking(struct xsave_struct *fx)
{
- struct xsave_struct *fx = &fpu->state->xsave;
int err;
asm volatile("1: .byte " REX_PREFIX "0x0f,0xae,0x2f\n\t"
@@ -69,8 +66,7 @@
* Clear the xsave header first, so that reserved fields are
* initialized to zero.
*/
- err = __clear_user(&buf->xsave_hdr,
- sizeof(struct xsave_hdr_struct));
+ err = __clear_user(&buf->xsave_hdr, sizeof(buf->xsave_hdr));
if (unlikely(err))
return -EFAULT;
@@ -84,9 +80,6 @@
: [err] "=r" (err)
: "D" (buf), "a" (-1), "d" (-1), "0" (0)
: "memory");
- if (unlikely(err) && __clear_user(buf, xstate_size))
- err = -EFAULT;
- /* No need to clear here because the caller clears USED_MATH */
return err;
}
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index 8215e56..8d7a619 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -100,6 +100,8 @@
obj-$(CONFIG_OF) += devicetree.o
obj-$(CONFIG_UPROBES) += uprobes.o
+obj-$(CONFIG_PERF_EVENTS) += perf_regs.o
+
###
# 64 bit specific files
ifeq ($(CONFIG_X86_64),y)
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index b2297e5..e651f7a 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -656,7 +656,7 @@
acpi_register_lapic(physid, ACPI_MADT_ENABLED);
/*
- * If mp_register_lapic successfully generates a new logical cpu
+ * If acpi_register_lapic successfully generates a new logical cpu
* number, then the following will get us exactly what was mapped
*/
cpumask_andnot(new_map, cpu_present_mask, tmp_map);
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index afb7ff7..ef5ccca 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -23,19 +23,6 @@
#define MAX_PATCH_LEN (255-1)
-#ifdef CONFIG_HOTPLUG_CPU
-static int smp_alt_once;
-
-static int __init bootonly(char *str)
-{
- smp_alt_once = 1;
- return 1;
-}
-__setup("smp-alt-boot", bootonly);
-#else
-#define smp_alt_once 1
-#endif
-
static int __initdata_or_module debug_alternative;
static int __init debug_alt(char *str)
@@ -165,7 +152,7 @@
#endif
#ifdef P6_NOP1
-static const unsigned char __initconst_or_module p6nops[] =
+static const unsigned char p6nops[] =
{
P6_NOP1,
P6_NOP2,
@@ -317,7 +304,7 @@
/* turn DS segment override prefix into lock prefix */
if (*ptr == 0x3e)
text_poke(ptr, ((unsigned char []){0xf0}), 1);
- };
+ }
mutex_unlock(&text_mutex);
}
@@ -326,9 +313,6 @@
{
const s32 *poff;
- if (noreplace_smp)
- return;
-
mutex_lock(&text_mutex);
for (poff = start; poff < end; poff++) {
u8 *ptr = (u8 *)poff + *poff;
@@ -338,7 +322,7 @@
/* turn lock prefix into DS segment override prefix */
if (*ptr == 0xf0)
text_poke(ptr, ((unsigned char []){0x3E}), 1);
- };
+ }
mutex_unlock(&text_mutex);
}
@@ -359,7 +343,7 @@
};
static LIST_HEAD(smp_alt_modules);
static DEFINE_MUTEX(smp_alt);
-static int smp_mode = 1; /* protected by smp_alt */
+static bool uniproc_patched = false; /* protected by smp_alt */
void __init_or_module alternatives_smp_module_add(struct module *mod,
char *name,
@@ -368,19 +352,18 @@
{
struct smp_alt_module *smp;
- if (noreplace_smp)
- return;
+ mutex_lock(&smp_alt);
+ if (!uniproc_patched)
+ goto unlock;
- if (smp_alt_once) {
- if (boot_cpu_has(X86_FEATURE_UP))
- alternatives_smp_unlock(locks, locks_end,
- text, text_end);
- return;
- }
+ if (num_possible_cpus() == 1)
+ /* Don't bother remembering, we'll never have to undo it. */
+ goto smp_unlock;
smp = kzalloc(sizeof(*smp), GFP_KERNEL);
if (NULL == smp)
- return; /* we'll run the (safe but slow) SMP code then ... */
+ /* we'll run the (safe but slow) SMP code then ... */
+ goto unlock;
smp->mod = mod;
smp->name = name;
@@ -392,11 +375,10 @@
__func__, smp->locks, smp->locks_end,
smp->text, smp->text_end, smp->name);
- mutex_lock(&smp_alt);
list_add_tail(&smp->next, &smp_alt_modules);
- if (boot_cpu_has(X86_FEATURE_UP))
- alternatives_smp_unlock(smp->locks, smp->locks_end,
- smp->text, smp->text_end);
+smp_unlock:
+ alternatives_smp_unlock(locks, locks_end, text, text_end);
+unlock:
mutex_unlock(&smp_alt);
}
@@ -404,24 +386,18 @@
{
struct smp_alt_module *item;
- if (smp_alt_once || noreplace_smp)
- return;
-
mutex_lock(&smp_alt);
list_for_each_entry(item, &smp_alt_modules, next) {
if (mod != item->mod)
continue;
list_del(&item->next);
- mutex_unlock(&smp_alt);
- DPRINTK("%s: %s\n", __func__, item->name);
kfree(item);
- return;
+ break;
}
mutex_unlock(&smp_alt);
}
-bool skip_smp_alternatives;
-void alternatives_smp_switch(int smp)
+void alternatives_enable_smp(void)
{
struct smp_alt_module *mod;
@@ -436,34 +412,21 @@
pr_info("lockdep: fixing up alternatives\n");
#endif
- if (noreplace_smp || smp_alt_once || skip_smp_alternatives)
- return;
- BUG_ON(!smp && (num_online_cpus() > 1));
+ /* Why bother if there are no other CPUs? */
+ BUG_ON(num_possible_cpus() == 1);
mutex_lock(&smp_alt);
- /*
- * Avoid unnecessary switches because it forces JIT based VMs to
- * throw away all cached translations, which can be quite costly.
- */
- if (smp == smp_mode) {
- /* nothing */
- } else if (smp) {
+ if (uniproc_patched) {
pr_info("switching to SMP code\n");
+ BUG_ON(num_online_cpus() != 1);
clear_cpu_cap(&boot_cpu_data, X86_FEATURE_UP);
clear_cpu_cap(&cpu_data(0), X86_FEATURE_UP);
list_for_each_entry(mod, &smp_alt_modules, next)
alternatives_smp_lock(mod->locks, mod->locks_end,
mod->text, mod->text_end);
- } else {
- pr_info("switching to UP code\n");
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_UP);
- set_cpu_cap(&cpu_data(0), X86_FEATURE_UP);
- list_for_each_entry(mod, &smp_alt_modules, next)
- alternatives_smp_unlock(mod->locks, mod->locks_end,
- mod->text, mod->text_end);
+ uniproc_patched = false;
}
- smp_mode = smp;
mutex_unlock(&smp_alt);
}
@@ -540,40 +503,22 @@
apply_alternatives(__alt_instructions, __alt_instructions_end);
- /* switch to patch-once-at-boottime-only mode and free the
- * tables in case we know the number of CPUs will never ever
- * change */
-#ifdef CONFIG_HOTPLUG_CPU
- if (num_possible_cpus() < 2)
- smp_alt_once = 1;
-#endif
-
#ifdef CONFIG_SMP
- if (smp_alt_once) {
- if (1 == num_possible_cpus()) {
- pr_info("switching to UP code\n");
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_UP);
- set_cpu_cap(&cpu_data(0), X86_FEATURE_UP);
-
- alternatives_smp_unlock(__smp_locks, __smp_locks_end,
- _text, _etext);
- }
- } else {
+ /* Patch to UP if other cpus not imminent. */
+ if (!noreplace_smp && (num_present_cpus() == 1 || setup_max_cpus <= 1)) {
+ uniproc_patched = true;
alternatives_smp_module_add(NULL, "core kernel",
__smp_locks, __smp_locks_end,
_text, _etext);
-
- /* Only switch to UP mode if we don't immediately boot others */
- if (num_present_cpus() == 1 || setup_max_cpus <= 1)
- alternatives_smp_switch(0);
}
-#endif
- apply_paravirt(__parainstructions, __parainstructions_end);
- if (smp_alt_once)
+ if (!uniproc_patched || num_possible_cpus() == 1)
free_init_pages("SMP alternatives",
(unsigned long)__smp_locks,
(unsigned long)__smp_locks_end);
+#endif
+
+ apply_paravirt(__parainstructions, __parainstructions_end);
restart_nmi();
}
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 24deb30..b17416e 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1934,7 +1934,7 @@
apic_printk(APIC_DEBUG, KERN_CONT " : %s", error_interrupt_reason[i]);
i++;
v1 >>= 1;
- };
+ }
apic_printk(APIC_DEBUG, KERN_CONT "\n");
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 9d92e19..f7e98a2 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -737,6 +737,72 @@
}
#endif
+static void __cpuinit cpu_set_tlb_flushall_shift(struct cpuinfo_x86 *c)
+{
+ if (!cpu_has_invlpg)
+ return;
+
+ tlb_flushall_shift = 5;
+
+ if (c->x86 <= 0x11)
+ tlb_flushall_shift = 4;
+}
+
+static void __cpuinit cpu_detect_tlb_amd(struct cpuinfo_x86 *c)
+{
+ u32 ebx, eax, ecx, edx;
+ u16 mask = 0xfff;
+
+ if (c->x86 < 0xf)
+ return;
+
+ if (c->extended_cpuid_level < 0x80000006)
+ return;
+
+ cpuid(0x80000006, &eax, &ebx, &ecx, &edx);
+
+ tlb_lld_4k[ENTRIES] = (ebx >> 16) & mask;
+ tlb_lli_4k[ENTRIES] = ebx & mask;
+
+ /*
+ * K8 doesn't have 2M/4M entries in the L2 TLB so read out the L1 TLB
+ * characteristics from the CPUID function 0x80000005 instead.
+ */
+ if (c->x86 == 0xf) {
+ cpuid(0x80000005, &eax, &ebx, &ecx, &edx);
+ mask = 0xff;
+ }
+
+ /* Handle DTLB 2M and 4M sizes, fall back to L1 if L2 is disabled */
+ if (!((eax >> 16) & mask)) {
+ u32 a, b, c, d;
+
+ cpuid(0x80000005, &a, &b, &c, &d);
+ tlb_lld_2m[ENTRIES] = (a >> 16) & 0xff;
+ } else {
+ tlb_lld_2m[ENTRIES] = (eax >> 16) & mask;
+ }
+
+ /* a 4M entry uses two 2M entries */
+ tlb_lld_4m[ENTRIES] = tlb_lld_2m[ENTRIES] >> 1;
+
+ /* Handle ITLB 2M and 4M sizes, fall back to L1 if L2 is disabled */
+ if (!(eax & mask)) {
+ /* Erratum 658 */
+ if (c->x86 == 0x15 && c->x86_model <= 0x1f) {
+ tlb_lli_2m[ENTRIES] = 1024;
+ } else {
+ cpuid(0x80000005, &eax, &ebx, &ecx, &edx);
+ tlb_lli_2m[ENTRIES] = eax & 0xff;
+ }
+ } else
+ tlb_lli_2m[ENTRIES] = eax & mask;
+
+ tlb_lli_4m[ENTRIES] = tlb_lli_2m[ENTRIES] >> 1;
+
+ cpu_set_tlb_flushall_shift(c);
+}
+
static const struct cpu_dev __cpuinitconst amd_cpu_dev = {
.c_vendor = "AMD",
.c_ident = { "AuthenticAMD" },
@@ -756,6 +822,7 @@
.c_size_cache = amd_size_cache,
#endif
.c_early_init = early_init_amd,
+ .c_detect_tlb = cpu_detect_tlb_amd,
.c_bsp_init = bsp_init_amd,
.c_init = init_amd,
.c_x86_vendor = X86_VENDOR_AMD,
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index c97bb7b..d0e910d 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -165,10 +165,15 @@
print_cpu_info(&boot_cpu_data);
#endif
check_config();
- check_fpu();
check_hlt();
check_popad();
init_utsname()->machine[1] =
'0' + (boot_cpu_data.x86 > 6 ? 6 : boot_cpu_data.x86);
alternative_instructions();
+
+ /*
+ * kernel_fpu_begin/end() in check_fpu() relies on the patched
+ * alternative instructions.
+ */
+ check_fpu();
}
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index a5fbc3c..532691b 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -476,7 +476,7 @@
printk(KERN_INFO "Last level iTLB entries: 4KB %d, 2MB %d, 4MB %d\n" \
"Last level dTLB entries: 4KB %d, 2MB %d, 4MB %d\n" \
- "tlb_flushall_shift is 0x%x\n",
+ "tlb_flushall_shift: %d\n",
tlb_lli_4k[ENTRIES], tlb_lli_2m[ENTRIES],
tlb_lli_4m[ENTRIES], tlb_lld_4k[ENTRIES],
tlb_lld_2m[ENTRIES], tlb_lld_4m[ENTRIES],
@@ -942,8 +942,7 @@
#else
vgetcpu_set_mode();
#endif
- if (boot_cpu_data.cpuid_level >= 2)
- cpu_detect_tlb(&boot_cpu_data);
+ cpu_detect_tlb(&boot_cpu_data);
}
void __cpuinit identify_secondary_cpu(struct cpuinfo_x86 *c)
@@ -1023,14 +1022,16 @@
printk(KERN_CONT "%s ", vendor);
if (c->x86_model_id[0])
- printk(KERN_CONT "%s", c->x86_model_id);
+ printk(KERN_CONT "%s", strim(c->x86_model_id));
else
printk(KERN_CONT "%d86", c->x86);
+ printk(KERN_CONT " (fam: %02x, model: %02x", c->x86, c->x86_model);
+
if (c->x86_mask || c->cpuid_level >= 0)
- printk(KERN_CONT " stepping %02x\n", c->x86_mask);
+ printk(KERN_CONT ", stepping: %02x)\n", c->x86_mask);
else
- printk(KERN_CONT "\n");
+ printk(KERN_CONT ")\n");
print_cpu_msr(c);
}
@@ -1116,8 +1117,6 @@
X86_EFLAGS_TF|X86_EFLAGS_DF|X86_EFLAGS_IF|X86_EFLAGS_IOPL);
}
-unsigned long kernel_eflags;
-
/*
* Copies of the original ist values from the tss are only accessed during
* debugging, no special alignment required.
@@ -1297,9 +1296,6 @@
dbg_restore_debug_regs();
fpu_init();
- xsave_init();
-
- raw_local_save_flags(kernel_eflags);
if (is_uv_system())
uv_cpu_init();
@@ -1352,6 +1348,5 @@
dbg_restore_debug_regs();
fpu_init();
- xsave_init();
}
#endif
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 0a4ce29..198e019 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -648,6 +648,10 @@
int i, j, n;
unsigned int regs[4];
unsigned char *desc = (unsigned char *)regs;
+
+ if (c->cpuid_level < 2)
+ return;
+
/* Number of times to iterate */
n = cpuid_eax(2) & 0xFF;
diff --git a/arch/x86/kernel/cpu/mcheck/mce-inject.c b/arch/x86/kernel/cpu/mcheck/mce-inject.c
index fc4beb3..ddc72f8 100644
--- a/arch/x86/kernel/cpu/mcheck/mce-inject.c
+++ b/arch/x86/kernel/cpu/mcheck/mce-inject.c
@@ -78,6 +78,7 @@
}
static cpumask_var_t mce_inject_cpumask;
+static DEFINE_MUTEX(mce_inject_mutex);
static int mce_raise_notify(unsigned int cmd, struct pt_regs *regs)
{
@@ -194,7 +195,11 @@
put_online_cpus();
} else
#endif
+ {
+ preempt_disable();
raise_local();
+ preempt_enable();
+ }
}
/* Error injection interface */
@@ -225,7 +230,10 @@
* so do it a jiffie or two later everywhere.
*/
schedule_timeout(2);
+
+ mutex_lock(&mce_inject_mutex);
raise_mce(&m);
+ mutex_unlock(&mce_inject_mutex);
return usize;
}
diff --git a/arch/x86/kernel/cpu/mcheck/mce-internal.h b/arch/x86/kernel/cpu/mcheck/mce-internal.h
index ed44c8a..6a05c1d 100644
--- a/arch/x86/kernel/cpu/mcheck/mce-internal.h
+++ b/arch/x86/kernel/cpu/mcheck/mce-internal.h
@@ -28,6 +28,18 @@
extern struct mce_bank *mce_banks;
+#ifdef CONFIG_X86_MCE_INTEL
+unsigned long mce_intel_adjust_timer(unsigned long interval);
+void mce_intel_cmci_poll(void);
+void mce_intel_hcpu_update(unsigned long cpu);
+#else
+# define mce_intel_adjust_timer mce_adjust_timer_default
+static inline void mce_intel_cmci_poll(void) { }
+static inline void mce_intel_hcpu_update(unsigned long cpu) { }
+#endif
+
+void mce_timer_kick(unsigned long interval);
+
#ifdef CONFIG_ACPI_APEI
int apei_write_mce(struct mce *m);
ssize_t apei_read_mce(struct mce *m, u64 *record_id);
diff --git a/arch/x86/kernel/cpu/mcheck/mce.c b/arch/x86/kernel/cpu/mcheck/mce.c
index 292d025..29e87d3 100644
--- a/arch/x86/kernel/cpu/mcheck/mce.c
+++ b/arch/x86/kernel/cpu/mcheck/mce.c
@@ -83,6 +83,7 @@
int mce_cmci_disabled __read_mostly;
int mce_ignore_ce __read_mostly;
int mce_ser __read_mostly;
+int mce_bios_cmci_threshold __read_mostly;
struct mce_bank *mce_banks __read_mostly;
@@ -1266,6 +1267,14 @@
static DEFINE_PER_CPU(unsigned long, mce_next_interval); /* in jiffies */
static DEFINE_PER_CPU(struct timer_list, mce_timer);
+static unsigned long mce_adjust_timer_default(unsigned long interval)
+{
+ return interval;
+}
+
+static unsigned long (*mce_adjust_timer)(unsigned long interval) =
+ mce_adjust_timer_default;
+
static void mce_timer_fn(unsigned long data)
{
struct timer_list *t = &__get_cpu_var(mce_timer);
@@ -1276,6 +1285,7 @@
if (mce_available(__this_cpu_ptr(&cpu_info))) {
machine_check_poll(MCP_TIMESTAMP,
&__get_cpu_var(mce_poll_banks));
+ mce_intel_cmci_poll();
}
/*
@@ -1283,14 +1293,38 @@
* polling interval, otherwise increase the polling interval.
*/
iv = __this_cpu_read(mce_next_interval);
- if (mce_notify_irq())
+ if (mce_notify_irq()) {
iv = max(iv / 2, (unsigned long) HZ/100);
- else
+ } else {
iv = min(iv * 2, round_jiffies_relative(check_interval * HZ));
+ iv = mce_adjust_timer(iv);
+ }
__this_cpu_write(mce_next_interval, iv);
+ /* Might have become 0 after CMCI storm subsided */
+ if (iv) {
+ t->expires = jiffies + iv;
+ add_timer_on(t, smp_processor_id());
+ }
+}
- t->expires = jiffies + iv;
- add_timer_on(t, smp_processor_id());
+/*
+ * Ensure that the timer is firing in @interval from now.
+ */
+void mce_timer_kick(unsigned long interval)
+{
+ struct timer_list *t = &__get_cpu_var(mce_timer);
+ unsigned long when = jiffies + interval;
+ unsigned long iv = __this_cpu_read(mce_next_interval);
+
+ if (timer_pending(t)) {
+ if (time_before(when, t->expires))
+ mod_timer_pinned(t, when);
+ } else {
+ t->expires = round_jiffies(when);
+ add_timer_on(t, smp_processor_id());
+ }
+ if (interval < iv)
+ __this_cpu_write(mce_next_interval, interval);
}
/* Must not be called in IRQ context where del_timer_sync() can deadlock */
@@ -1585,6 +1619,7 @@
switch (c->x86_vendor) {
case X86_VENDOR_INTEL:
mce_intel_feature_init(c);
+ mce_adjust_timer = mce_intel_adjust_timer;
break;
case X86_VENDOR_AMD:
mce_amd_feature_init(c);
@@ -1594,21 +1629,26 @@
}
}
+static void mce_start_timer(unsigned int cpu, struct timer_list *t)
+{
+ unsigned long iv = mce_adjust_timer(check_interval * HZ);
+
+ __this_cpu_write(mce_next_interval, iv);
+
+ if (mce_ignore_ce || !iv)
+ return;
+
+ t->expires = round_jiffies(jiffies + iv);
+ add_timer_on(t, smp_processor_id());
+}
+
static void __mcheck_cpu_init_timer(void)
{
struct timer_list *t = &__get_cpu_var(mce_timer);
- unsigned long iv = check_interval * HZ;
+ unsigned int cpu = smp_processor_id();
- setup_timer(t, mce_timer_fn, smp_processor_id());
-
- if (mce_ignore_ce)
- return;
-
- __this_cpu_write(mce_next_interval, iv);
- if (!iv)
- return;
- t->expires = round_jiffies(jiffies + iv);
- add_timer_on(t, smp_processor_id());
+ setup_timer(t, mce_timer_fn, cpu);
+ mce_start_timer(cpu, t);
}
/* Handle unconfigured int18 (should never happen) */
@@ -1907,6 +1947,7 @@
* check, or 0 to not wait
* mce=bootlog Log MCEs from before booting. Disabled by default on AMD.
* mce=nobootlog Don't log MCEs from before booting.
+ * mce=bios_cmci_threshold Don't program the CMCI threshold
*/
static int __init mcheck_enable(char *str)
{
@@ -1926,6 +1967,8 @@
mce_ignore_ce = 1;
else if (!strcmp(str, "bootlog") || !strcmp(str, "nobootlog"))
mce_bootlog = (str[0] == 'b');
+ else if (!strcmp(str, "bios_cmci_threshold"))
+ mce_bios_cmci_threshold = 1;
else if (isdigit(str[0])) {
get_option(&str, &tolerant);
if (*str == ',') {
@@ -2166,6 +2209,11 @@
&mce_cmci_disabled
};
+static struct dev_ext_attribute dev_attr_bios_cmci_threshold = {
+ __ATTR(bios_cmci_threshold, 0444, device_show_int, NULL),
+ &mce_bios_cmci_threshold
+};
+
static struct device_attribute *mce_device_attrs[] = {
&dev_attr_tolerant.attr,
&dev_attr_check_interval.attr,
@@ -2174,6 +2222,7 @@
&dev_attr_dont_log_ce.attr,
&dev_attr_ignore_ce.attr,
&dev_attr_cmci_disabled.attr,
+ &dev_attr_bios_cmci_threshold.attr,
NULL
};
@@ -2294,38 +2343,33 @@
unsigned int cpu = (unsigned long)hcpu;
struct timer_list *t = &per_cpu(mce_timer, cpu);
- switch (action) {
+ switch (action & ~CPU_TASKS_FROZEN) {
case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
mce_device_create(cpu);
if (threshold_cpu_callback)
threshold_cpu_callback(action, cpu);
break;
case CPU_DEAD:
- case CPU_DEAD_FROZEN:
if (threshold_cpu_callback)
threshold_cpu_callback(action, cpu);
mce_device_remove(cpu);
+ mce_intel_hcpu_update(cpu);
break;
case CPU_DOWN_PREPARE:
- case CPU_DOWN_PREPARE_FROZEN:
- del_timer_sync(t);
smp_call_function_single(cpu, mce_disable_cpu, &action, 1);
+ del_timer_sync(t);
break;
case CPU_DOWN_FAILED:
- case CPU_DOWN_FAILED_FROZEN:
- if (!mce_ignore_ce && check_interval) {
- t->expires = round_jiffies(jiffies +
- per_cpu(mce_next_interval, cpu));
- add_timer_on(t, cpu);
- }
smp_call_function_single(cpu, mce_reenable_cpu, &action, 1);
- break;
- case CPU_POST_DEAD:
- /* intentionally ignoring frozen here */
- cmci_rediscover(cpu);
+ mce_start_timer(cpu, t);
break;
}
+
+ if (action == CPU_POST_DEAD) {
+ /* intentionally ignoring frozen here */
+ cmci_rediscover(cpu);
+ }
+
return NOTIFY_OK;
}
diff --git a/arch/x86/kernel/cpu/mcheck/mce_intel.c b/arch/x86/kernel/cpu/mcheck/mce_intel.c
index 38e49bc9..5f88abf 100644
--- a/arch/x86/kernel/cpu/mcheck/mce_intel.c
+++ b/arch/x86/kernel/cpu/mcheck/mce_intel.c
@@ -15,6 +15,8 @@
#include <asm/msr.h>
#include <asm/mce.h>
+#include "mce-internal.h"
+
/*
* Support for Intel Correct Machine Check Interrupts. This allows
* the CPU to raise an interrupt when a corrected machine check happened.
@@ -30,7 +32,22 @@
*/
static DEFINE_RAW_SPINLOCK(cmci_discover_lock);
-#define CMCI_THRESHOLD 1
+#define CMCI_THRESHOLD 1
+#define CMCI_POLL_INTERVAL (30 * HZ)
+#define CMCI_STORM_INTERVAL (1 * HZ)
+#define CMCI_STORM_THRESHOLD 15
+
+static DEFINE_PER_CPU(unsigned long, cmci_time_stamp);
+static DEFINE_PER_CPU(unsigned int, cmci_storm_cnt);
+static DEFINE_PER_CPU(unsigned int, cmci_storm_state);
+
+enum {
+ CMCI_STORM_NONE,
+ CMCI_STORM_ACTIVE,
+ CMCI_STORM_SUBSIDED,
+};
+
+static atomic_t cmci_storm_on_cpus;
static int cmci_supported(int *banks)
{
@@ -53,6 +70,93 @@
return !!(cap & MCG_CMCI_P);
}
+void mce_intel_cmci_poll(void)
+{
+ if (__this_cpu_read(cmci_storm_state) == CMCI_STORM_NONE)
+ return;
+ machine_check_poll(MCP_TIMESTAMP, &__get_cpu_var(mce_banks_owned));
+}
+
+void mce_intel_hcpu_update(unsigned long cpu)
+{
+ if (per_cpu(cmci_storm_state, cpu) == CMCI_STORM_ACTIVE)
+ atomic_dec(&cmci_storm_on_cpus);
+
+ per_cpu(cmci_storm_state, cpu) = CMCI_STORM_NONE;
+}
+
+unsigned long mce_intel_adjust_timer(unsigned long interval)
+{
+ int r;
+
+ if (interval < CMCI_POLL_INTERVAL)
+ return interval;
+
+ switch (__this_cpu_read(cmci_storm_state)) {
+ case CMCI_STORM_ACTIVE:
+ /*
+ * We switch back to interrupt mode once the poll timer has
+ * silenced itself. That means no events recorded and the
+ * timer interval is back to our poll interval.
+ */
+ __this_cpu_write(cmci_storm_state, CMCI_STORM_SUBSIDED);
+ r = atomic_sub_return(1, &cmci_storm_on_cpus);
+ if (r == 0)
+ pr_notice("CMCI storm subsided: switching to interrupt mode\n");
+ /* FALLTHROUGH */
+
+ case CMCI_STORM_SUBSIDED:
+ /*
+ * We wait for all cpus to go back to SUBSIDED
+ * state. When that happens we switch back to
+ * interrupt mode.
+ */
+ if (!atomic_read(&cmci_storm_on_cpus)) {
+ __this_cpu_write(cmci_storm_state, CMCI_STORM_NONE);
+ cmci_reenable();
+ cmci_recheck();
+ }
+ return CMCI_POLL_INTERVAL;
+ default:
+ /*
+ * We have shiny weather. Let the poll do whatever it
+ * thinks.
+ */
+ return interval;
+ }
+}
+
+static bool cmci_storm_detect(void)
+{
+ unsigned int cnt = __this_cpu_read(cmci_storm_cnt);
+ unsigned long ts = __this_cpu_read(cmci_time_stamp);
+ unsigned long now = jiffies;
+ int r;
+
+ if (__this_cpu_read(cmci_storm_state) != CMCI_STORM_NONE)
+ return true;
+
+ if (time_before_eq(now, ts + CMCI_STORM_INTERVAL)) {
+ cnt++;
+ } else {
+ cnt = 1;
+ __this_cpu_write(cmci_time_stamp, now);
+ }
+ __this_cpu_write(cmci_storm_cnt, cnt);
+
+ if (cnt <= CMCI_STORM_THRESHOLD)
+ return false;
+
+ cmci_clear();
+ __this_cpu_write(cmci_storm_state, CMCI_STORM_ACTIVE);
+ r = atomic_add_return(1, &cmci_storm_on_cpus);
+ mce_timer_kick(CMCI_POLL_INTERVAL);
+
+ if (r == 1)
+ pr_notice("CMCI storm detected: switching to poll mode\n");
+ return true;
+}
+
/*
* The interrupt handler. This is called on every event.
* Just call the poller directly to log any events.
@@ -61,33 +165,28 @@
*/
static void intel_threshold_interrupt(void)
{
+ if (cmci_storm_detect())
+ return;
machine_check_poll(MCP_TIMESTAMP, &__get_cpu_var(mce_banks_owned));
mce_notify_irq();
}
-static void print_update(char *type, int *hdr, int num)
-{
- if (*hdr == 0)
- printk(KERN_INFO "CPU %d MCA banks", smp_processor_id());
- *hdr = 1;
- printk(KERN_CONT " %s:%d", type, num);
-}
-
/*
* Enable CMCI (Corrected Machine Check Interrupt) for available MCE banks
* on this CPU. Use the algorithm recommended in the SDM to discover shared
* banks.
*/
-static void cmci_discover(int banks, int boot)
+static void cmci_discover(int banks)
{
unsigned long *owned = (void *)&__get_cpu_var(mce_banks_owned);
unsigned long flags;
- int hdr = 0;
int i;
+ int bios_wrong_thresh = 0;
raw_spin_lock_irqsave(&cmci_discover_lock, flags);
for (i = 0; i < banks; i++) {
u64 val;
+ int bios_zero_thresh = 0;
if (test_bit(i, owned))
continue;
@@ -96,29 +195,52 @@
/* Already owned by someone else? */
if (val & MCI_CTL2_CMCI_EN) {
- if (test_and_clear_bit(i, owned) && !boot)
- print_update("SHD", &hdr, i);
+ clear_bit(i, owned);
__clear_bit(i, __get_cpu_var(mce_poll_banks));
continue;
}
- val &= ~MCI_CTL2_CMCI_THRESHOLD_MASK;
- val |= MCI_CTL2_CMCI_EN | CMCI_THRESHOLD;
+ if (!mce_bios_cmci_threshold) {
+ val &= ~MCI_CTL2_CMCI_THRESHOLD_MASK;
+ val |= CMCI_THRESHOLD;
+ } else if (!(val & MCI_CTL2_CMCI_THRESHOLD_MASK)) {
+ /*
+ * If bios_cmci_threshold boot option was specified
+ * but the threshold is zero, we'll try to initialize
+ * it to 1.
+ */
+ bios_zero_thresh = 1;
+ val |= CMCI_THRESHOLD;
+ }
+
+ val |= MCI_CTL2_CMCI_EN;
wrmsrl(MSR_IA32_MCx_CTL2(i), val);
rdmsrl(MSR_IA32_MCx_CTL2(i), val);
/* Did the enable bit stick? -- the bank supports CMCI */
if (val & MCI_CTL2_CMCI_EN) {
- if (!test_and_set_bit(i, owned) && !boot)
- print_update("CMCI", &hdr, i);
+ set_bit(i, owned);
__clear_bit(i, __get_cpu_var(mce_poll_banks));
+ /*
+ * We are able to set thresholds for some banks that
+ * had a threshold of 0. This means the BIOS has not
+ * set the thresholds properly or does not work with
+ * this boot option. Note down now and report later.
+ */
+ if (mce_bios_cmci_threshold && bios_zero_thresh &&
+ (val & MCI_CTL2_CMCI_THRESHOLD_MASK))
+ bios_wrong_thresh = 1;
} else {
WARN_ON(!test_bit(i, __get_cpu_var(mce_poll_banks)));
}
}
raw_spin_unlock_irqrestore(&cmci_discover_lock, flags);
- if (hdr)
- printk(KERN_CONT "\n");
+ if (mce_bios_cmci_threshold && bios_wrong_thresh) {
+ pr_info_once(
+ "bios_cmci_threshold: Some banks do not have valid thresholds set\n");
+ pr_info_once(
+ "bios_cmci_threshold: Make sure your BIOS supports this boot option\n");
+ }
}
/*
@@ -156,7 +278,7 @@
continue;
/* Disable CMCI */
rdmsrl(MSR_IA32_MCx_CTL2(i), val);
- val &= ~(MCI_CTL2_CMCI_EN|MCI_CTL2_CMCI_THRESHOLD_MASK);
+ val &= ~MCI_CTL2_CMCI_EN;
wrmsrl(MSR_IA32_MCx_CTL2(i), val);
__clear_bit(i, __get_cpu_var(mce_banks_owned));
}
@@ -186,7 +308,7 @@
continue;
/* Recheck banks in case CPUs don't all have the same */
if (cmci_supported(&banks))
- cmci_discover(banks, 0);
+ cmci_discover(banks);
}
set_cpus_allowed_ptr(current, old);
@@ -200,7 +322,7 @@
{
int banks;
if (cmci_supported(&banks))
- cmci_discover(banks, 0);
+ cmci_discover(banks);
}
static void intel_init_cmci(void)
@@ -211,7 +333,7 @@
return;
mce_threshold_vector = intel_threshold_interrupt;
- cmci_discover(banks, 1);
+ cmci_discover(banks);
/*
* For CPU #0 this runs with still disabled APIC, but that's
* ok because only the vector is set up. We still do another
diff --git a/arch/x86/kernel/cpu/perf_event.h b/arch/x86/kernel/cpu/perf_event.h
index 6605a81..8b6defe 100644
--- a/arch/x86/kernel/cpu/perf_event.h
+++ b/arch/x86/kernel/cpu/perf_event.h
@@ -586,6 +586,8 @@
extern struct event_constraint intel_snb_pebs_event_constraints[];
+extern struct event_constraint intel_ivb_pebs_event_constraints[];
+
struct event_constraint *intel_pebs_constraints(struct perf_event *event);
void intel_pmu_pebs_enable(struct perf_event *event);
diff --git a/arch/x86/kernel/cpu/perf_event_amd_ibs.c b/arch/x86/kernel/cpu/perf_event_amd_ibs.c
index 7bfb5be..eebd5ff 100644
--- a/arch/x86/kernel/cpu/perf_event_amd_ibs.c
+++ b/arch/x86/kernel/cpu/perf_event_amd_ibs.c
@@ -209,6 +209,15 @@
return -EOPNOTSUPP;
}
+static const struct perf_event_attr ibs_notsupp = {
+ .exclude_user = 1,
+ .exclude_kernel = 1,
+ .exclude_hv = 1,
+ .exclude_idle = 1,
+ .exclude_host = 1,
+ .exclude_guest = 1,
+};
+
static int perf_ibs_init(struct perf_event *event)
{
struct hw_perf_event *hwc = &event->hw;
@@ -229,6 +238,9 @@
if (event->pmu != &perf_ibs->pmu)
return -ENOENT;
+ if (perf_flags(&event->attr) & perf_flags(&ibs_notsupp))
+ return -EINVAL;
+
if (config & ~perf_ibs->config_mask)
return -EINVAL;
diff --git a/arch/x86/kernel/cpu/perf_event_intel.c b/arch/x86/kernel/cpu/perf_event_intel.c
index 7f2739e..6bca492 100644
--- a/arch/x86/kernel/cpu/perf_event_intel.c
+++ b/arch/x86/kernel/cpu/perf_event_intel.c
@@ -2008,6 +2008,7 @@
break;
case 28: /* Atom */
+ case 54: /* Cedariew */
memcpy(hw_cache_event_ids, atom_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
@@ -2047,7 +2048,6 @@
case 42: /* SandyBridge */
case 45: /* SandyBridge, "Romely-EP" */
x86_add_quirk(intel_sandybridge_quirk);
- case 58: /* IvyBridge */
memcpy(hw_cache_event_ids, snb_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
memcpy(hw_cache_extra_regs, snb_hw_cache_extra_regs,
@@ -2072,6 +2072,29 @@
pr_cont("SandyBridge events, ");
break;
+ case 58: /* IvyBridge */
+ memcpy(hw_cache_event_ids, snb_hw_cache_event_ids,
+ sizeof(hw_cache_event_ids));
+ memcpy(hw_cache_extra_regs, snb_hw_cache_extra_regs,
+ sizeof(hw_cache_extra_regs));
+
+ intel_pmu_lbr_init_snb();
+
+ x86_pmu.event_constraints = intel_snb_event_constraints;
+ x86_pmu.pebs_constraints = intel_ivb_pebs_event_constraints;
+ x86_pmu.pebs_aliases = intel_pebs_aliases_snb;
+ x86_pmu.extra_regs = intel_snb_extra_regs;
+ /* all extra regs are per-cpu when HT is on */
+ x86_pmu.er_flags |= ERF_HAS_RSP_1;
+ x86_pmu.er_flags |= ERF_NO_HT_SHARING;
+
+ /* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
+ intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
+ X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
+
+ pr_cont("IvyBridge events, ");
+ break;
+
default:
switch (x86_pmu.version) {
diff --git a/arch/x86/kernel/cpu/perf_event_intel_ds.c b/arch/x86/kernel/cpu/perf_event_intel_ds.c
index e38d97b..826054a 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_ds.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_ds.c
@@ -407,6 +407,20 @@
EVENT_CONSTRAINT_END
};
+struct event_constraint intel_ivb_pebs_event_constraints[] = {
+ INTEL_UEVENT_CONSTRAINT(0x01c0, 0x2), /* INST_RETIRED.PRECDIST */
+ INTEL_UEVENT_CONSTRAINT(0x01c2, 0xf), /* UOPS_RETIRED.ALL */
+ INTEL_UEVENT_CONSTRAINT(0x02c2, 0xf), /* UOPS_RETIRED.RETIRE_SLOTS */
+ INTEL_EVENT_CONSTRAINT(0xc4, 0xf), /* BR_INST_RETIRED.* */
+ INTEL_EVENT_CONSTRAINT(0xc5, 0xf), /* BR_MISP_RETIRED.* */
+ INTEL_EVENT_CONSTRAINT(0xcd, 0x8), /* MEM_TRANS_RETIRED.* */
+ INTEL_EVENT_CONSTRAINT(0xd0, 0xf), /* MEM_UOP_RETIRED.* */
+ INTEL_EVENT_CONSTRAINT(0xd1, 0xf), /* MEM_LOAD_UOPS_RETIRED.* */
+ INTEL_EVENT_CONSTRAINT(0xd2, 0xf), /* MEM_LOAD_UOPS_LLC_HIT_RETIRED.* */
+ INTEL_EVENT_CONSTRAINT(0xd3, 0xf), /* MEM_LOAD_UOPS_LLC_MISS_RETIRED.* */
+ EVENT_CONSTRAINT_END
+};
+
struct event_constraint *intel_pebs_constraints(struct perf_event *event)
{
struct event_constraint *c;
diff --git a/arch/x86/kernel/cpu/perf_event_intel_lbr.c b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
index 520b426..da02e9c 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_lbr.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_lbr.c
@@ -686,7 +686,8 @@
* to have an operational LBR which can freeze
* on PMU interrupt
*/
- if (boot_cpu_data.x86_mask < 10) {
+ if (boot_cpu_data.x86_model == 28
+ && boot_cpu_data.x86_mask < 10) {
pr_cont("LBR disabled due to erratum");
return;
}
diff --git a/arch/x86/kernel/cpu/perf_event_intel_uncore.c b/arch/x86/kernel/cpu/perf_event_intel_uncore.c
index 0a55710..99d96a4 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_uncore.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_uncore.c
@@ -661,6 +661,11 @@
}
}
+static struct uncore_event_desc snb_uncore_events[] = {
+ INTEL_UNCORE_EVENT_DESC(clockticks, "event=0xff,umask=0x00"),
+ { /* end: all zeroes */ },
+};
+
static struct attribute *snb_uncore_formats_attr[] = {
&format_attr_event.attr,
&format_attr_umask.attr,
@@ -704,6 +709,7 @@
.constraints = snb_uncore_cbox_constraints,
.ops = &snb_uncore_msr_ops,
.format_group = &snb_uncore_format_group,
+ .event_descs = snb_uncore_events,
};
static struct intel_uncore_type *snb_msr_uncores[] = {
@@ -1944,7 +1950,7 @@
static struct intel_uncore_box *
uncore_pmu_to_box(struct intel_uncore_pmu *pmu, int cpu)
{
- static struct intel_uncore_box *box;
+ struct intel_uncore_box *box;
box = *per_cpu_ptr(pmu->box, cpu);
if (box)
@@ -2341,6 +2347,27 @@
return ret;
}
+static ssize_t uncore_get_attr_cpumask(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int n = cpulist_scnprintf(buf, PAGE_SIZE - 2, &uncore_cpu_mask);
+
+ buf[n++] = '\n';
+ buf[n] = '\0';
+ return n;
+}
+
+static DEVICE_ATTR(cpumask, S_IRUGO, uncore_get_attr_cpumask, NULL);
+
+static struct attribute *uncore_pmu_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static struct attribute_group uncore_pmu_attr_group = {
+ .attrs = uncore_pmu_attrs,
+};
+
static int __init uncore_pmu_register(struct intel_uncore_pmu *pmu)
{
int ret;
@@ -2378,8 +2405,8 @@
free_percpu(type->pmus[i].box);
kfree(type->pmus);
type->pmus = NULL;
- kfree(type->attr_groups[1]);
- type->attr_groups[1] = NULL;
+ kfree(type->events_group);
+ type->events_group = NULL;
}
static void __init uncore_types_exit(struct intel_uncore_type **types)
@@ -2431,9 +2458,10 @@
for (j = 0; j < i; j++)
attrs[j] = &type->event_descs[j].attr.attr;
- type->attr_groups[1] = events_group;
+ type->events_group = events_group;
}
+ type->pmu_group = &uncore_pmu_attr_group;
type->pmus = pmus;
return 0;
fail:
diff --git a/arch/x86/kernel/cpu/perf_event_intel_uncore.h b/arch/x86/kernel/cpu/perf_event_intel_uncore.h
index 5b81c18..e68a455 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_uncore.h
+++ b/arch/x86/kernel/cpu/perf_event_intel_uncore.h
@@ -369,10 +369,12 @@
struct intel_uncore_pmu *pmus;
struct intel_uncore_ops *ops;
struct uncore_event_desc *event_descs;
- const struct attribute_group *attr_groups[3];
+ const struct attribute_group *attr_groups[4];
};
-#define format_group attr_groups[0]
+#define pmu_group attr_groups[0]
+#define format_group attr_groups[1]
+#define events_group attr_groups[2]
struct intel_uncore_ops {
void (*init_box)(struct intel_uncore_box *);
diff --git a/arch/x86/kernel/cpu/proc.c b/arch/x86/kernel/cpu/proc.c
index 8022c66..fbd8955 100644
--- a/arch/x86/kernel/cpu/proc.c
+++ b/arch/x86/kernel/cpu/proc.c
@@ -140,10 +140,7 @@
static void *c_start(struct seq_file *m, loff_t *pos)
{
- if (*pos == 0) /* just in case, cpu 0 is not the first */
- *pos = cpumask_first(cpu_online_mask);
- else
- *pos = cpumask_next(*pos - 1, cpu_online_mask);
+ *pos = cpumask_next(*pos - 1, cpu_online_mask);
if ((*pos) < nr_cpu_ids)
return &cpu_data(*pos);
return NULL;
diff --git a/arch/x86/kernel/cpuid.c b/arch/x86/kernel/cpuid.c
index 39472dd..60c7891 100644
--- a/arch/x86/kernel/cpuid.c
+++ b/arch/x86/kernel/cpuid.c
@@ -199,12 +199,14 @@
goto out_chrdev;
}
cpuid_class->devnode = cpuid_devnode;
+ get_online_cpus();
for_each_online_cpu(i) {
err = cpuid_device_create(i);
if (err != 0)
goto out_class;
}
register_hotcpu_notifier(&cpuid_class_cpu_notifier);
+ put_online_cpus();
err = 0;
goto out;
@@ -214,6 +216,7 @@
for_each_online_cpu(i) {
cpuid_device_destroy(i);
}
+ put_online_cpus();
class_destroy(cpuid_class);
out_chrdev:
__unregister_chrdev(CPUID_MAJOR, 0, NR_CPUS, "cpu/cpuid");
@@ -225,11 +228,13 @@
{
int cpu = 0;
+ get_online_cpus();
for_each_online_cpu(cpu)
cpuid_device_destroy(cpu);
class_destroy(cpuid_class);
__unregister_chrdev(CPUID_MAJOR, 0, NR_CPUS, "cpu/cpuid");
unregister_hotcpu_notifier(&cpuid_class_cpu_notifier);
+ put_online_cpus();
}
module_init(cpuid_init);
diff --git a/arch/x86/kernel/devicetree.c b/arch/x86/kernel/devicetree.c
index 3ae2ced..b158152 100644
--- a/arch/x86/kernel/devicetree.c
+++ b/arch/x86/kernel/devicetree.c
@@ -342,6 +342,47 @@
.xlate = ioapic_xlate,
};
+static void dt_add_ioapic_domain(unsigned int ioapic_num,
+ struct device_node *np)
+{
+ struct irq_domain *id;
+ struct mp_ioapic_gsi *gsi_cfg;
+ int ret;
+ int num;
+
+ gsi_cfg = mp_ioapic_gsi_routing(ioapic_num);
+ num = gsi_cfg->gsi_end - gsi_cfg->gsi_base + 1;
+
+ id = irq_domain_add_linear(np, num, &ioapic_irq_domain_ops,
+ (void *)ioapic_num);
+ BUG_ON(!id);
+ if (gsi_cfg->gsi_base == 0) {
+ /*
+ * The first NR_IRQS_LEGACY irq descs are allocated in
+ * early_irq_init() and need just a mapping. The
+ * remaining irqs need both. All of them are preallocated
+ * and assigned so we can keep the 1:1 mapping which the ioapic
+ * is having.
+ */
+ ret = irq_domain_associate_many(id, 0, 0, NR_IRQS_LEGACY);
+ if (ret)
+ pr_err("Error mapping legacy IRQs: %d\n", ret);
+
+ if (num > NR_IRQS_LEGACY) {
+ ret = irq_create_strict_mappings(id, NR_IRQS_LEGACY,
+ NR_IRQS_LEGACY, num - NR_IRQS_LEGACY);
+ if (ret)
+ pr_err("Error creating mapping for the "
+ "remaining IRQs: %d\n", ret);
+ }
+ irq_set_default_host(id);
+ } else {
+ ret = irq_create_strict_mappings(id, gsi_cfg->gsi_base, 0, num);
+ if (ret)
+ pr_err("Error creating IRQ mapping: %d\n", ret);
+ }
+}
+
static void __init ioapic_add_ofnode(struct device_node *np)
{
struct resource r;
@@ -356,15 +397,7 @@
for (i = 0; i < nr_ioapics; i++) {
if (r.start == mpc_ioapic_addr(i)) {
- struct irq_domain *id;
- struct mp_ioapic_gsi *gsi_cfg;
-
- gsi_cfg = mp_ioapic_gsi_routing(i);
-
- id = irq_domain_add_legacy(np, 32, gsi_cfg->gsi_base, 0,
- &ioapic_irq_domain_ops,
- (void*)i);
- BUG_ON(!id);
+ dt_add_ioapic_domain(i, np);
return;
}
}
diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
index 623f288..f438a44 100644
--- a/arch/x86/kernel/entry_32.S
+++ b/arch/x86/kernel/entry_32.S
@@ -1109,17 +1109,21 @@
pushl %eax
pushl %ecx
pushl %edx
- movl 0xc(%esp), %eax
+ pushl $0 /* Pass NULL as regs pointer */
+ movl 4*4(%esp), %eax
movl 0x4(%ebp), %edx
+ leal function_trace_op, %ecx
subl $MCOUNT_INSN_SIZE, %eax
.globl ftrace_call
ftrace_call:
call ftrace_stub
+ addl $4,%esp /* skip NULL pointer */
popl %edx
popl %ecx
popl %eax
+ftrace_ret:
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
.globl ftrace_graph_call
ftrace_graph_call:
@@ -1131,6 +1135,71 @@
ret
END(ftrace_caller)
+ENTRY(ftrace_regs_caller)
+ pushf /* push flags before compare (in cs location) */
+ cmpl $0, function_trace_stop
+ jne ftrace_restore_flags
+
+ /*
+ * i386 does not save SS and ESP when coming from kernel.
+ * Instead, to get sp, ®s->sp is used (see ptrace.h).
+ * Unfortunately, that means eflags must be at the same location
+ * as the current return ip is. We move the return ip into the
+ * ip location, and move flags into the return ip location.
+ */
+ pushl 4(%esp) /* save return ip into ip slot */
+
+ pushl $0 /* Load 0 into orig_ax */
+ pushl %gs
+ pushl %fs
+ pushl %es
+ pushl %ds
+ pushl %eax
+ pushl %ebp
+ pushl %edi
+ pushl %esi
+ pushl %edx
+ pushl %ecx
+ pushl %ebx
+
+ movl 13*4(%esp), %eax /* Get the saved flags */
+ movl %eax, 14*4(%esp) /* Move saved flags into regs->flags location */
+ /* clobbering return ip */
+ movl $__KERNEL_CS,13*4(%esp)
+
+ movl 12*4(%esp), %eax /* Load ip (1st parameter) */
+ subl $MCOUNT_INSN_SIZE, %eax /* Adjust ip */
+ movl 0x4(%ebp), %edx /* Load parent ip (2nd parameter) */
+ leal function_trace_op, %ecx /* Save ftrace_pos in 3rd parameter */
+ pushl %esp /* Save pt_regs as 4th parameter */
+
+GLOBAL(ftrace_regs_call)
+ call ftrace_stub
+
+ addl $4, %esp /* Skip pt_regs */
+ movl 14*4(%esp), %eax /* Move flags back into cs */
+ movl %eax, 13*4(%esp) /* Needed to keep addl from modifying flags */
+ movl 12*4(%esp), %eax /* Get return ip from regs->ip */
+ movl %eax, 14*4(%esp) /* Put return ip back for ret */
+
+ popl %ebx
+ popl %ecx
+ popl %edx
+ popl %esi
+ popl %edi
+ popl %ebp
+ popl %eax
+ popl %ds
+ popl %es
+ popl %fs
+ popl %gs
+ addl $8, %esp /* Skip orig_ax and ip */
+ popf /* Pop flags at end (no addl to corrupt flags) */
+ jmp ftrace_ret
+
+ftrace_restore_flags:
+ popf
+ jmp ftrace_stub
#else /* ! CONFIG_DYNAMIC_FTRACE */
ENTRY(mcount)
@@ -1171,9 +1240,6 @@
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
ENTRY(ftrace_graph_caller)
- cmpl $0, function_trace_stop
- jne ftrace_stub
-
pushl %eax
pushl %ecx
pushl %edx
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
index 69babd8..066334b 100644
--- a/arch/x86/kernel/entry_64.S
+++ b/arch/x86/kernel/entry_64.S
@@ -56,6 +56,7 @@
#include <asm/ftrace.h>
#include <asm/percpu.h>
#include <asm/asm.h>
+#include <asm/rcu.h>
#include <linux/err.h>
/* Avoid __ASSEMBLER__'ifying <linux/audit.h> just for this. */
@@ -68,25 +69,51 @@
.section .entry.text, "ax"
#ifdef CONFIG_FUNCTION_TRACER
+
+#ifdef CC_USING_FENTRY
+# define function_hook __fentry__
+#else
+# define function_hook mcount
+#endif
+
#ifdef CONFIG_DYNAMIC_FTRACE
-ENTRY(mcount)
+
+ENTRY(function_hook)
retq
-END(mcount)
+END(function_hook)
+
+/* skip is set if stack has been adjusted */
+.macro ftrace_caller_setup skip=0
+ MCOUNT_SAVE_FRAME \skip
+
+ /* Load the ftrace_ops into the 3rd parameter */
+ leaq function_trace_op, %rdx
+
+ /* Load ip into the first parameter */
+ movq RIP(%rsp), %rdi
+ subq $MCOUNT_INSN_SIZE, %rdi
+ /* Load the parent_ip into the second parameter */
+#ifdef CC_USING_FENTRY
+ movq SS+16(%rsp), %rsi
+#else
+ movq 8(%rbp), %rsi
+#endif
+.endm
ENTRY(ftrace_caller)
+ /* Check if tracing was disabled (quick check) */
cmpl $0, function_trace_stop
jne ftrace_stub
- MCOUNT_SAVE_FRAME
-
- movq 0x38(%rsp), %rdi
- movq 8(%rbp), %rsi
- subq $MCOUNT_INSN_SIZE, %rdi
+ ftrace_caller_setup
+ /* regs go into 4th parameter (but make it NULL) */
+ movq $0, %rcx
GLOBAL(ftrace_call)
call ftrace_stub
MCOUNT_RESTORE_FRAME
+ftrace_return:
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
GLOBAL(ftrace_graph_call)
@@ -97,8 +124,78 @@
retq
END(ftrace_caller)
+ENTRY(ftrace_regs_caller)
+ /* Save the current flags before compare (in SS location)*/
+ pushfq
+
+ /* Check if tracing was disabled (quick check) */
+ cmpl $0, function_trace_stop
+ jne ftrace_restore_flags
+
+ /* skip=8 to skip flags saved in SS */
+ ftrace_caller_setup 8
+
+ /* Save the rest of pt_regs */
+ movq %r15, R15(%rsp)
+ movq %r14, R14(%rsp)
+ movq %r13, R13(%rsp)
+ movq %r12, R12(%rsp)
+ movq %r11, R11(%rsp)
+ movq %r10, R10(%rsp)
+ movq %rbp, RBP(%rsp)
+ movq %rbx, RBX(%rsp)
+ /* Copy saved flags */
+ movq SS(%rsp), %rcx
+ movq %rcx, EFLAGS(%rsp)
+ /* Kernel segments */
+ movq $__KERNEL_DS, %rcx
+ movq %rcx, SS(%rsp)
+ movq $__KERNEL_CS, %rcx
+ movq %rcx, CS(%rsp)
+ /* Stack - skipping return address */
+ leaq SS+16(%rsp), %rcx
+ movq %rcx, RSP(%rsp)
+
+ /* regs go into 4th parameter */
+ leaq (%rsp), %rcx
+
+GLOBAL(ftrace_regs_call)
+ call ftrace_stub
+
+ /* Copy flags back to SS, to restore them */
+ movq EFLAGS(%rsp), %rax
+ movq %rax, SS(%rsp)
+
+ /* Handlers can change the RIP */
+ movq RIP(%rsp), %rax
+ movq %rax, SS+8(%rsp)
+
+ /* restore the rest of pt_regs */
+ movq R15(%rsp), %r15
+ movq R14(%rsp), %r14
+ movq R13(%rsp), %r13
+ movq R12(%rsp), %r12
+ movq R10(%rsp), %r10
+ movq RBP(%rsp), %rbp
+ movq RBX(%rsp), %rbx
+
+ /* skip=8 to skip flags saved in SS */
+ MCOUNT_RESTORE_FRAME 8
+
+ /* Restore flags */
+ popfq
+
+ jmp ftrace_return
+ftrace_restore_flags:
+ popfq
+ jmp ftrace_stub
+
+END(ftrace_regs_caller)
+
+
#else /* ! CONFIG_DYNAMIC_FTRACE */
-ENTRY(mcount)
+
+ENTRY(function_hook)
cmpl $0, function_trace_stop
jne ftrace_stub
@@ -119,8 +216,12 @@
trace:
MCOUNT_SAVE_FRAME
- movq 0x38(%rsp), %rdi
+ movq RIP(%rsp), %rdi
+#ifdef CC_USING_FENTRY
+ movq SS+16(%rsp), %rsi
+#else
movq 8(%rbp), %rsi
+#endif
subq $MCOUNT_INSN_SIZE, %rdi
call *ftrace_trace_function
@@ -128,20 +229,22 @@
MCOUNT_RESTORE_FRAME
jmp ftrace_stub
-END(mcount)
+END(function_hook)
#endif /* CONFIG_DYNAMIC_FTRACE */
#endif /* CONFIG_FUNCTION_TRACER */
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
ENTRY(ftrace_graph_caller)
- cmpl $0, function_trace_stop
- jne ftrace_stub
-
MCOUNT_SAVE_FRAME
+#ifdef CC_USING_FENTRY
+ leaq SS+16(%rsp), %rdi
+ movq $0, %rdx /* No framepointers needed */
+#else
leaq 8(%rbp), %rdi
- movq 0x38(%rsp), %rsi
movq (%rbp), %rdx
+#endif
+ movq RIP(%rsp), %rsi
subq $MCOUNT_INSN_SIZE, %rsi
call prepare_ftrace_return
@@ -342,15 +445,15 @@
.macro SAVE_ARGS_IRQ
cld
/* start from rbp in pt_regs and jump over */
- movq_cfi rdi, RDI-RBP
- movq_cfi rsi, RSI-RBP
- movq_cfi rdx, RDX-RBP
- movq_cfi rcx, RCX-RBP
- movq_cfi rax, RAX-RBP
- movq_cfi r8, R8-RBP
- movq_cfi r9, R9-RBP
- movq_cfi r10, R10-RBP
- movq_cfi r11, R11-RBP
+ movq_cfi rdi, (RDI-RBP)
+ movq_cfi rsi, (RSI-RBP)
+ movq_cfi rdx, (RDX-RBP)
+ movq_cfi rcx, (RCX-RBP)
+ movq_cfi rax, (RAX-RBP)
+ movq_cfi r8, (R8-RBP)
+ movq_cfi r9, (R9-RBP)
+ movq_cfi r10, (R10-RBP)
+ movq_cfi r11, (R11-RBP)
/* Save rbp so that we can unwind from get_irq_regs() */
movq_cfi rbp, 0
@@ -384,7 +487,7 @@
.endm
ENTRY(save_rest)
- PARTIAL_FRAME 1 REST_SKIP+8
+ PARTIAL_FRAME 1 (REST_SKIP+8)
movq 5*8+16(%rsp), %r11 /* save return address */
movq_cfi rbx, RBX+16
movq_cfi rbp, RBP+16
@@ -440,7 +543,7 @@
LOCK ; btr $TIF_FORK,TI_flags(%r8)
- pushq_cfi kernel_eflags(%rip)
+ pushq_cfi $0x0002
popfq_cfi # reset kernel eflags
call schedule_tail # rdi: 'prev' task parameter
@@ -565,7 +668,7 @@
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_NONE)
pushq_cfi %rdi
- call schedule
+ SCHEDULE_USER
popq_cfi %rdi
jmp sysret_check
@@ -678,7 +781,7 @@
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_NONE)
pushq_cfi %rdi
- call schedule
+ SCHEDULE_USER
popq_cfi %rdi
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
@@ -974,7 +1077,7 @@
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_NONE)
pushq_cfi %rdi
- call schedule
+ SCHEDULE_USER
popq_cfi %rdi
GET_THREAD_INFO(%rcx)
DISABLE_INTERRUPTS(CLBR_NONE)
@@ -1449,7 +1552,7 @@
paranoid_schedule:
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_ANY)
- call schedule
+ SCHEDULE_USER
DISABLE_INTERRUPTS(CLBR_ANY)
TRACE_IRQS_OFF
jmp paranoid_userspace
diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c
index c3a7cb4..1d41402 100644
--- a/arch/x86/kernel/ftrace.c
+++ b/arch/x86/kernel/ftrace.c
@@ -206,6 +206,21 @@
ftrace_modify_code(unsigned long ip, unsigned const char *old_code,
unsigned const char *new_code);
+/*
+ * Should never be called:
+ * As it is only called by __ftrace_replace_code() which is called by
+ * ftrace_replace_code() that x86 overrides, and by ftrace_update_code()
+ * which is called to turn mcount into nops or nops into function calls
+ * but not to convert a function from not using regs to one that uses
+ * regs, which ftrace_modify_call() is for.
+ */
+int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+ unsigned long addr)
+{
+ WARN_ON(1);
+ return -EINVAL;
+}
+
int ftrace_update_ftrace_func(ftrace_func_t func)
{
unsigned long ip = (unsigned long)(&ftrace_call);
@@ -220,6 +235,14 @@
ret = ftrace_modify_code(ip, old, new);
+ /* Also update the regs callback function */
+ if (!ret) {
+ ip = (unsigned long)(&ftrace_regs_call);
+ memcpy(old, &ftrace_regs_call, MCOUNT_INSN_SIZE);
+ new = ftrace_call_replace(ip, (unsigned long)func);
+ ret = ftrace_modify_code(ip, old, new);
+ }
+
atomic_dec(&modifying_ftrace_code);
return ret;
@@ -299,6 +322,32 @@
return add_break(rec->ip, old);
}
+/*
+ * If the record has the FTRACE_FL_REGS set, that means that it
+ * wants to convert to a callback that saves all regs. If FTRACE_FL_REGS
+ * is not not set, then it wants to convert to the normal callback.
+ */
+static unsigned long get_ftrace_addr(struct dyn_ftrace *rec)
+{
+ if (rec->flags & FTRACE_FL_REGS)
+ return (unsigned long)FTRACE_REGS_ADDR;
+ else
+ return (unsigned long)FTRACE_ADDR;
+}
+
+/*
+ * The FTRACE_FL_REGS_EN is set when the record already points to
+ * a function that saves all the regs. Basically the '_EN' version
+ * represents the current state of the function.
+ */
+static unsigned long get_ftrace_old_addr(struct dyn_ftrace *rec)
+{
+ if (rec->flags & FTRACE_FL_REGS_EN)
+ return (unsigned long)FTRACE_REGS_ADDR;
+ else
+ return (unsigned long)FTRACE_ADDR;
+}
+
static int add_breakpoints(struct dyn_ftrace *rec, int enable)
{
unsigned long ftrace_addr;
@@ -306,7 +355,7 @@
ret = ftrace_test_record(rec, enable);
- ftrace_addr = (unsigned long)FTRACE_ADDR;
+ ftrace_addr = get_ftrace_addr(rec);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
@@ -316,6 +365,10 @@
/* converting nop to call */
return add_brk_on_nop(rec);
+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
+ case FTRACE_UPDATE_MODIFY_CALL:
+ ftrace_addr = get_ftrace_old_addr(rec);
+ /* fall through */
case FTRACE_UPDATE_MAKE_NOP:
/* converting a call to a nop */
return add_brk_on_call(rec, ftrace_addr);
@@ -360,13 +413,21 @@
* If not, don't touch the breakpoint, we make just create
* a disaster.
*/
- ftrace_addr = (unsigned long)FTRACE_ADDR;
+ ftrace_addr = get_ftrace_addr(rec);
+ nop = ftrace_call_replace(ip, ftrace_addr);
+
+ if (memcmp(&ins[1], &nop[1], MCOUNT_INSN_SIZE - 1) == 0)
+ goto update;
+
+ /* Check both ftrace_addr and ftrace_old_addr */
+ ftrace_addr = get_ftrace_old_addr(rec);
nop = ftrace_call_replace(ip, ftrace_addr);
if (memcmp(&ins[1], &nop[1], MCOUNT_INSN_SIZE - 1) != 0)
return -EINVAL;
}
+ update:
return probe_kernel_write((void *)ip, &nop[0], 1);
}
@@ -405,12 +466,14 @@
ret = ftrace_test_record(rec, enable);
- ftrace_addr = (unsigned long)FTRACE_ADDR;
+ ftrace_addr = get_ftrace_addr(rec);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
return 0;
+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
+ case FTRACE_UPDATE_MODIFY_CALL:
case FTRACE_UPDATE_MAKE_CALL:
/* converting nop to call */
return add_update_call(rec, ftrace_addr);
@@ -455,12 +518,14 @@
ret = ftrace_update_record(rec, enable);
- ftrace_addr = (unsigned long)FTRACE_ADDR;
+ ftrace_addr = get_ftrace_addr(rec);
switch (ret) {
case FTRACE_UPDATE_IGNORE:
return 0;
+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
+ case FTRACE_UPDATE_MODIFY_CALL:
case FTRACE_UPDATE_MAKE_CALL:
/* converting nop to call */
return finish_update_call(rec, ftrace_addr);
diff --git a/arch/x86/kernel/i387.c b/arch/x86/kernel/i387.c
index f250431..675a050 100644
--- a/arch/x86/kernel/i387.c
+++ b/arch/x86/kernel/i387.c
@@ -19,24 +19,17 @@
#include <asm/fpu-internal.h>
#include <asm/user.h>
-#ifdef CONFIG_X86_64
-# include <asm/sigcontext32.h>
-# include <asm/user32.h>
-#else
-# define save_i387_xstate_ia32 save_i387_xstate
-# define restore_i387_xstate_ia32 restore_i387_xstate
-# define _fpstate_ia32 _fpstate
-# define _xstate_ia32 _xstate
-# define sig_xstate_ia32_size sig_xstate_size
-# define fx_sw_reserved_ia32 fx_sw_reserved
-# define user_i387_ia32_struct user_i387_struct
-# define user32_fxsr_struct user_fxsr_struct
-#endif
-
/*
* Were we in an interrupt that interrupted kernel mode?
*
- * We can do a kernel_fpu_begin/end() pair *ONLY* if that
+ * For now, with eagerfpu we will return interrupted kernel FPU
+ * state as not-idle. TBD: Ideally we can change the return value
+ * to something like __thread_has_fpu(current). But we need to
+ * be careful of doing __thread_clear_has_fpu() before saving
+ * the FPU etc for supporting nested uses etc. For now, take
+ * the simple route!
+ *
+ * On others, we can do a kernel_fpu_begin/end() pair *ONLY* if that
* pair does nothing at all: the thread must not have fpu (so
* that we don't try to save the FPU state), and TS must
* be set (so that the clts/stts pair does nothing that is
@@ -44,6 +37,9 @@
*/
static inline bool interrupted_kernel_fpu_idle(void)
{
+ if (use_eager_fpu())
+ return 0;
+
return !__thread_has_fpu(current) &&
(read_cr0() & X86_CR0_TS);
}
@@ -77,29 +73,29 @@
}
EXPORT_SYMBOL(irq_fpu_usable);
-void kernel_fpu_begin(void)
+void __kernel_fpu_begin(void)
{
struct task_struct *me = current;
- WARN_ON_ONCE(!irq_fpu_usable());
- preempt_disable();
if (__thread_has_fpu(me)) {
__save_init_fpu(me);
__thread_clear_has_fpu(me);
- /* We do 'stts()' in kernel_fpu_end() */
- } else {
+ /* We do 'stts()' in __kernel_fpu_end() */
+ } else if (!use_eager_fpu()) {
this_cpu_write(fpu_owner_task, NULL);
clts();
}
}
-EXPORT_SYMBOL(kernel_fpu_begin);
+EXPORT_SYMBOL(__kernel_fpu_begin);
-void kernel_fpu_end(void)
+void __kernel_fpu_end(void)
{
- stts();
- preempt_enable();
+ if (use_eager_fpu())
+ math_state_restore();
+ else
+ stts();
}
-EXPORT_SYMBOL(kernel_fpu_end);
+EXPORT_SYMBOL(__kernel_fpu_end);
void unlazy_fpu(struct task_struct *tsk)
{
@@ -113,23 +109,15 @@
}
EXPORT_SYMBOL(unlazy_fpu);
-#ifdef CONFIG_MATH_EMULATION
-# define HAVE_HWFP (boot_cpu_data.hard_math)
-#else
-# define HAVE_HWFP 1
-#endif
-
-static unsigned int mxcsr_feature_mask __read_mostly = 0xffffffffu;
+unsigned int mxcsr_feature_mask __read_mostly = 0xffffffffu;
unsigned int xstate_size;
EXPORT_SYMBOL_GPL(xstate_size);
-unsigned int sig_xstate_ia32_size = sizeof(struct _fpstate_ia32);
static struct i387_fxsave_struct fx_scratch __cpuinitdata;
static void __cpuinit mxcsr_feature_mask_init(void)
{
unsigned long mask = 0;
- clts();
if (cpu_has_fxsr) {
memset(&fx_scratch, 0, sizeof(struct i387_fxsave_struct));
asm volatile("fxsave %0" : : "m" (fx_scratch));
@@ -138,7 +126,6 @@
mask = 0x0000ffbf;
}
mxcsr_feature_mask &= mask;
- stts();
}
static void __cpuinit init_thread_xstate(void)
@@ -192,9 +179,8 @@
init_thread_xstate();
mxcsr_feature_mask_init();
- /* clean state in init */
- current_thread_info()->status = 0;
- clear_used_math();
+ xsave_init();
+ eager_fpu_init();
}
void fpu_finit(struct fpu *fpu)
@@ -205,12 +191,7 @@
}
if (cpu_has_fxsr) {
- struct i387_fxsave_struct *fx = &fpu->state->fxsave;
-
- memset(fx, 0, xstate_size);
- fx->cwd = 0x37f;
- if (cpu_has_xmm)
- fx->mxcsr = MXCSR_DEFAULT;
+ fx_finit(&fpu->state->fxsave);
} else {
struct i387_fsave_struct *fp = &fpu->state->fsave;
memset(fp, 0, xstate_size);
@@ -454,7 +435,7 @@
* FXSR floating point environment conversions.
*/
-static void
+void
convert_from_fxsr(struct user_i387_ia32_struct *env, struct task_struct *tsk)
{
struct i387_fxsave_struct *fxsave = &tsk->thread.fpu.state->fxsave;
@@ -491,8 +472,8 @@
memcpy(&to[i], &from[i], sizeof(to[0]));
}
-static void convert_to_fxsr(struct task_struct *tsk,
- const struct user_i387_ia32_struct *env)
+void convert_to_fxsr(struct task_struct *tsk,
+ const struct user_i387_ia32_struct *env)
{
struct i387_fxsave_struct *fxsave = &tsk->thread.fpu.state->fxsave;
@@ -589,223 +570,6 @@
}
/*
- * Signal frame handlers.
- */
-
-static inline int save_i387_fsave(struct _fpstate_ia32 __user *buf)
-{
- struct task_struct *tsk = current;
- struct i387_fsave_struct *fp = &tsk->thread.fpu.state->fsave;
-
- fp->status = fp->swd;
- if (__copy_to_user(buf, fp, sizeof(struct i387_fsave_struct)))
- return -1;
- return 1;
-}
-
-static int save_i387_fxsave(struct _fpstate_ia32 __user *buf)
-{
- struct task_struct *tsk = current;
- struct i387_fxsave_struct *fx = &tsk->thread.fpu.state->fxsave;
- struct user_i387_ia32_struct env;
- int err = 0;
-
- convert_from_fxsr(&env, tsk);
- if (__copy_to_user(buf, &env, sizeof(env)))
- return -1;
-
- err |= __put_user(fx->swd, &buf->status);
- err |= __put_user(X86_FXSR_MAGIC, &buf->magic);
- if (err)
- return -1;
-
- if (__copy_to_user(&buf->_fxsr_env[0], fx, xstate_size))
- return -1;
- return 1;
-}
-
-static int save_i387_xsave(void __user *buf)
-{
- struct task_struct *tsk = current;
- struct _fpstate_ia32 __user *fx = buf;
- int err = 0;
-
-
- sanitize_i387_state(tsk);
-
- /*
- * For legacy compatible, we always set FP/SSE bits in the bit
- * vector while saving the state to the user context.
- * This will enable us capturing any changes(during sigreturn) to
- * the FP/SSE bits by the legacy applications which don't touch
- * xstate_bv in the xsave header.
- *
- * xsave aware applications can change the xstate_bv in the xsave
- * header as well as change any contents in the memory layout.
- * xrestore as part of sigreturn will capture all the changes.
- */
- tsk->thread.fpu.state->xsave.xsave_hdr.xstate_bv |= XSTATE_FPSSE;
-
- if (save_i387_fxsave(fx) < 0)
- return -1;
-
- err = __copy_to_user(&fx->sw_reserved, &fx_sw_reserved_ia32,
- sizeof(struct _fpx_sw_bytes));
- err |= __put_user(FP_XSTATE_MAGIC2,
- (__u32 __user *) (buf + sig_xstate_ia32_size
- - FP_XSTATE_MAGIC2_SIZE));
- if (err)
- return -1;
-
- return 1;
-}
-
-int save_i387_xstate_ia32(void __user *buf)
-{
- struct _fpstate_ia32 __user *fp = (struct _fpstate_ia32 __user *) buf;
- struct task_struct *tsk = current;
-
- if (!used_math())
- return 0;
-
- if (!access_ok(VERIFY_WRITE, buf, sig_xstate_ia32_size))
- return -EACCES;
- /*
- * This will cause a "finit" to be triggered by the next
- * attempted FPU operation by the 'current' process.
- */
- clear_used_math();
-
- if (!HAVE_HWFP) {
- return fpregs_soft_get(current, NULL,
- 0, sizeof(struct user_i387_ia32_struct),
- NULL, fp) ? -1 : 1;
- }
-
- unlazy_fpu(tsk);
-
- if (cpu_has_xsave)
- return save_i387_xsave(fp);
- if (cpu_has_fxsr)
- return save_i387_fxsave(fp);
- else
- return save_i387_fsave(fp);
-}
-
-static inline int restore_i387_fsave(struct _fpstate_ia32 __user *buf)
-{
- struct task_struct *tsk = current;
-
- return __copy_from_user(&tsk->thread.fpu.state->fsave, buf,
- sizeof(struct i387_fsave_struct));
-}
-
-static int restore_i387_fxsave(struct _fpstate_ia32 __user *buf,
- unsigned int size)
-{
- struct task_struct *tsk = current;
- struct user_i387_ia32_struct env;
- int err;
-
- err = __copy_from_user(&tsk->thread.fpu.state->fxsave, &buf->_fxsr_env[0],
- size);
- /* mxcsr reserved bits must be masked to zero for security reasons */
- tsk->thread.fpu.state->fxsave.mxcsr &= mxcsr_feature_mask;
- if (err || __copy_from_user(&env, buf, sizeof(env)))
- return 1;
- convert_to_fxsr(tsk, &env);
-
- return 0;
-}
-
-static int restore_i387_xsave(void __user *buf)
-{
- struct _fpx_sw_bytes fx_sw_user;
- struct _fpstate_ia32 __user *fx_user =
- ((struct _fpstate_ia32 __user *) buf);
- struct i387_fxsave_struct __user *fx =
- (struct i387_fxsave_struct __user *) &fx_user->_fxsr_env[0];
- struct xsave_hdr_struct *xsave_hdr =
- ¤t->thread.fpu.state->xsave.xsave_hdr;
- u64 mask;
- int err;
-
- if (check_for_xstate(fx, buf, &fx_sw_user))
- goto fx_only;
-
- mask = fx_sw_user.xstate_bv;
-
- err = restore_i387_fxsave(buf, fx_sw_user.xstate_size);
-
- xsave_hdr->xstate_bv &= pcntxt_mask;
- /*
- * These bits must be zero.
- */
- xsave_hdr->reserved1[0] = xsave_hdr->reserved1[1] = 0;
-
- /*
- * Init the state that is not present in the memory layout
- * and enabled by the OS.
- */
- mask = ~(pcntxt_mask & ~mask);
- xsave_hdr->xstate_bv &= mask;
-
- return err;
-fx_only:
- /*
- * Couldn't find the extended state information in the memory
- * layout. Restore the FP/SSE and init the other extended state
- * enabled by the OS.
- */
- xsave_hdr->xstate_bv = XSTATE_FPSSE;
- return restore_i387_fxsave(buf, sizeof(struct i387_fxsave_struct));
-}
-
-int restore_i387_xstate_ia32(void __user *buf)
-{
- int err;
- struct task_struct *tsk = current;
- struct _fpstate_ia32 __user *fp = (struct _fpstate_ia32 __user *) buf;
-
- if (HAVE_HWFP)
- clear_fpu(tsk);
-
- if (!buf) {
- if (used_math()) {
- clear_fpu(tsk);
- clear_used_math();
- }
-
- return 0;
- } else
- if (!access_ok(VERIFY_READ, buf, sig_xstate_ia32_size))
- return -EACCES;
-
- if (!used_math()) {
- err = init_fpu(tsk);
- if (err)
- return err;
- }
-
- if (HAVE_HWFP) {
- if (cpu_has_xsave)
- err = restore_i387_xsave(buf);
- else if (cpu_has_fxsr)
- err = restore_i387_fxsave(fp, sizeof(struct
- i387_fxsave_struct));
- else
- err = restore_i387_fsave(fp);
- } else {
- err = fpregs_soft_set(current, NULL,
- 0, sizeof(struct user_i387_ia32_struct),
- NULL, fp) != 0;
- }
- set_used_math();
-
- return err;
-}
-
-/*
* FPU state for core dumps.
* This is only used for a.out dumps now.
* It is declared generically using elf_fpregset_t (which is
diff --git a/arch/x86/kernel/i8259.c b/arch/x86/kernel/i8259.c
index 36d1853..9a5c460 100644
--- a/arch/x86/kernel/i8259.c
+++ b/arch/x86/kernel/i8259.c
@@ -263,7 +263,7 @@
* out of.
*/
outb(0xff, PIC_MASTER_IMR); /* mask all of 8259A-1 */
- outb(0xff, PIC_SLAVE_IMR); /* mask all of 8259A-1 */
+ outb(0xff, PIC_SLAVE_IMR); /* mask all of 8259A-2 */
}
static struct syscore_ops i8259_syscore_ops = {
diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c
index 7ad683d..e4595f1 100644
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -92,7 +92,8 @@
seq_printf(p, " Rescheduling interrupts\n");
seq_printf(p, "%*s: ", prec, "CAL");
for_each_online_cpu(j)
- seq_printf(p, "%10u ", irq_stats(j)->irq_call_count);
+ seq_printf(p, "%10u ", irq_stats(j)->irq_call_count -
+ irq_stats(j)->irq_tlb_count);
seq_printf(p, " Function call interrupts\n");
seq_printf(p, "%*s: ", prec, "TLB");
for_each_online_cpu(j)
@@ -147,7 +148,6 @@
#ifdef CONFIG_SMP
sum += irq_stats(cpu)->irq_resched_count;
sum += irq_stats(cpu)->irq_call_count;
- sum += irq_stats(cpu)->irq_tlb_count;
#endif
#ifdef CONFIG_X86_THERMAL_VECTOR
sum += irq_stats(cpu)->irq_thermal_count;
@@ -270,7 +270,7 @@
if (cpumask_any_and(affinity, cpu_online_mask) >= nr_cpu_ids) {
break_affinity = 1;
- affinity = cpu_all_mask;
+ affinity = cpu_online_mask;
}
chip = irq_data_get_irq_chip(data);
diff --git a/arch/x86/kernel/kprobes.c b/arch/x86/kernel/kprobes.c
index e2f751e..57916c0 100644
--- a/arch/x86/kernel/kprobes.c
+++ b/arch/x86/kernel/kprobes.c
@@ -541,6 +541,23 @@
return 1;
}
+#ifdef KPROBES_CAN_USE_FTRACE
+static void __kprobes skip_singlestep(struct kprobe *p, struct pt_regs *regs,
+ struct kprobe_ctlblk *kcb)
+{
+ /*
+ * Emulate singlestep (and also recover regs->ip)
+ * as if there is a 5byte nop
+ */
+ regs->ip = (unsigned long)p->addr + MCOUNT_INSN_SIZE;
+ if (unlikely(p->post_handler)) {
+ kcb->kprobe_status = KPROBE_HIT_SSDONE;
+ p->post_handler(p, regs, 0);
+ }
+ __this_cpu_write(current_kprobe, NULL);
+}
+#endif
+
/*
* Interrupts are disabled on entry as trap3 is an interrupt gate and they
* remain disabled throughout this function.
@@ -599,6 +616,12 @@
} else if (kprobe_running()) {
p = __this_cpu_read(current_kprobe);
if (p->break_handler && p->break_handler(p, regs)) {
+#ifdef KPROBES_CAN_USE_FTRACE
+ if (kprobe_ftrace(p)) {
+ skip_singlestep(p, regs, kcb);
+ return 1;
+ }
+#endif
setup_singlestep(p, regs, kcb, 0);
return 1;
}
@@ -1052,6 +1075,50 @@
return 0;
}
+#ifdef KPROBES_CAN_USE_FTRACE
+/* Ftrace callback handler for kprobes */
+void __kprobes kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *ops, struct pt_regs *regs)
+{
+ struct kprobe *p;
+ struct kprobe_ctlblk *kcb;
+ unsigned long flags;
+
+ /* Disable irq for emulating a breakpoint and avoiding preempt */
+ local_irq_save(flags);
+
+ p = get_kprobe((kprobe_opcode_t *)ip);
+ if (unlikely(!p) || kprobe_disabled(p))
+ goto end;
+
+ kcb = get_kprobe_ctlblk();
+ if (kprobe_running()) {
+ kprobes_inc_nmissed_count(p);
+ } else {
+ /* Kprobe handler expects regs->ip = ip + 1 as breakpoint hit */
+ regs->ip = ip + sizeof(kprobe_opcode_t);
+
+ __this_cpu_write(current_kprobe, p);
+ kcb->kprobe_status = KPROBE_HIT_ACTIVE;
+ if (!p->pre_handler || !p->pre_handler(p, regs))
+ skip_singlestep(p, regs, kcb);
+ /*
+ * If pre_handler returns !0, it sets regs->ip and
+ * resets current kprobe.
+ */
+ }
+end:
+ local_irq_restore(flags);
+}
+
+int __kprobes arch_prepare_kprobe_ftrace(struct kprobe *p)
+{
+ p->ainsn.insn = NULL;
+ p->ainsn.boostable = -1;
+ return 0;
+}
+#endif
+
int __init arch_init_kprobes(void)
{
return arch_init_optprobes();
diff --git a/arch/x86/kernel/microcode_amd.c b/arch/x86/kernel/microcode_amd.c
index 8a2ce8f..7720ff5 100644
--- a/arch/x86/kernel/microcode_amd.c
+++ b/arch/x86/kernel/microcode_amd.c
@@ -75,20 +75,113 @@
static struct equiv_cpu_entry *equiv_cpu_table;
-/* page-sized ucode patch buffer */
-void *patch;
+struct ucode_patch {
+ struct list_head plist;
+ void *data;
+ u32 patch_id;
+ u16 equiv_cpu;
+};
+
+static LIST_HEAD(pcache);
+
+static u16 find_equiv_id(unsigned int cpu)
+{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ int i = 0;
+
+ if (!equiv_cpu_table)
+ return 0;
+
+ while (equiv_cpu_table[i].installed_cpu != 0) {
+ if (uci->cpu_sig.sig == equiv_cpu_table[i].installed_cpu)
+ return equiv_cpu_table[i].equiv_cpu;
+
+ i++;
+ }
+ return 0;
+}
+
+static u32 find_cpu_family_by_equiv_cpu(u16 equiv_cpu)
+{
+ int i = 0;
+
+ BUG_ON(!equiv_cpu_table);
+
+ while (equiv_cpu_table[i].equiv_cpu != 0) {
+ if (equiv_cpu == equiv_cpu_table[i].equiv_cpu)
+ return equiv_cpu_table[i].installed_cpu;
+ i++;
+ }
+ return 0;
+}
+
+/*
+ * a small, trivial cache of per-family ucode patches
+ */
+static struct ucode_patch *cache_find_patch(u16 equiv_cpu)
+{
+ struct ucode_patch *p;
+
+ list_for_each_entry(p, &pcache, plist)
+ if (p->equiv_cpu == equiv_cpu)
+ return p;
+ return NULL;
+}
+
+static void update_cache(struct ucode_patch *new_patch)
+{
+ struct ucode_patch *p;
+
+ list_for_each_entry(p, &pcache, plist) {
+ if (p->equiv_cpu == new_patch->equiv_cpu) {
+ if (p->patch_id >= new_patch->patch_id)
+ /* we already have the latest patch */
+ return;
+
+ list_replace(&p->plist, &new_patch->plist);
+ kfree(p->data);
+ kfree(p);
+ return;
+ }
+ }
+ /* no patch found, add it */
+ list_add_tail(&new_patch->plist, &pcache);
+}
+
+static void free_cache(void)
+{
+ struct ucode_patch *p, *tmp;
+
+ list_for_each_entry_safe(p, tmp, &pcache, plist) {
+ __list_del(p->plist.prev, p->plist.next);
+ kfree(p->data);
+ kfree(p);
+ }
+}
+
+static struct ucode_patch *find_patch(unsigned int cpu)
+{
+ u16 equiv_id;
+
+ equiv_id = find_equiv_id(cpu);
+ if (!equiv_id)
+ return NULL;
+
+ return cache_find_patch(equiv_id);
+}
static int collect_cpu_info_amd(int cpu, struct cpu_signature *csig)
{
struct cpuinfo_x86 *c = &cpu_data(cpu);
+ csig->sig = cpuid_eax(0x00000001);
csig->rev = c->microcode;
pr_info("CPU%d: patch_level=0x%08x\n", cpu, csig->rev);
return 0;
}
-static unsigned int verify_ucode_size(int cpu, u32 patch_size,
+static unsigned int verify_patch_size(int cpu, u32 patch_size,
unsigned int size)
{
struct cpuinfo_x86 *c = &cpu_data(cpu);
@@ -118,94 +211,37 @@
return patch_size;
}
-static u16 find_equiv_id(void)
-{
- unsigned int current_cpu_id, i = 0;
-
- BUG_ON(equiv_cpu_table == NULL);
-
- current_cpu_id = cpuid_eax(0x00000001);
-
- while (equiv_cpu_table[i].installed_cpu != 0) {
- if (current_cpu_id == equiv_cpu_table[i].installed_cpu)
- return equiv_cpu_table[i].equiv_cpu;
-
- i++;
- }
- return 0;
-}
-
-/*
- * we signal a good patch is found by returning its size > 0
- */
-static int get_matching_microcode(int cpu, const u8 *ucode_ptr,
- unsigned int leftover_size, int rev,
- unsigned int *current_size)
-{
- struct microcode_header_amd *mc_hdr;
- unsigned int actual_size;
- u16 equiv_cpu_id;
-
- /* size of the current patch we're staring at */
- *current_size = *(u32 *)(ucode_ptr + 4) + SECTION_HDR_SIZE;
-
- equiv_cpu_id = find_equiv_id();
- if (!equiv_cpu_id)
- return 0;
-
- /*
- * let's look at the patch header itself now
- */
- mc_hdr = (struct microcode_header_amd *)(ucode_ptr + SECTION_HDR_SIZE);
-
- if (mc_hdr->processor_rev_id != equiv_cpu_id)
- return 0;
-
- /* ucode might be chipset specific -- currently we don't support this */
- if (mc_hdr->nb_dev_id || mc_hdr->sb_dev_id) {
- pr_err("CPU%d: chipset specific code not yet supported\n",
- cpu);
- return 0;
- }
-
- if (mc_hdr->patch_id <= rev)
- return 0;
-
- /*
- * now that the header looks sane, verify its size
- */
- actual_size = verify_ucode_size(cpu, *current_size, leftover_size);
- if (!actual_size)
- return 0;
-
- /* clear the patch buffer */
- memset(patch, 0, PAGE_SIZE);
-
- /* all looks ok, get the binary patch */
- get_ucode_data(patch, ucode_ptr + SECTION_HDR_SIZE, actual_size);
-
- return actual_size;
-}
-
static int apply_microcode_amd(int cpu)
{
- u32 rev, dummy;
- int cpu_num = raw_smp_processor_id();
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu_num;
- struct microcode_amd *mc_amd = uci->mc;
struct cpuinfo_x86 *c = &cpu_data(cpu);
+ struct microcode_amd *mc_amd;
+ struct ucode_cpu_info *uci;
+ struct ucode_patch *p;
+ u32 rev, dummy;
- /* We should bind the task to the CPU */
- BUG_ON(cpu_num != cpu);
+ BUG_ON(raw_smp_processor_id() != cpu);
- if (mc_amd == NULL)
+ uci = ucode_cpu_info + cpu;
+
+ p = find_patch(cpu);
+ if (!p)
return 0;
- wrmsrl(MSR_AMD64_PATCH_LOADER, (u64)(long)&mc_amd->hdr.data_code);
- /* get patch id after patching */
+ mc_amd = p->data;
+ uci->mc = p->data;
+
rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
- /* check current patch id and patch's id for match */
+ /* need to apply patch? */
+ if (rev >= mc_amd->hdr.patch_id) {
+ c->microcode = rev;
+ return 0;
+ }
+
+ wrmsrl(MSR_AMD64_PATCH_LOADER, (u64)(long)&mc_amd->hdr.data_code);
+
+ /* verify patch application was successful */
+ rdmsr(MSR_AMD64_PATCH_LEVEL, rev, dummy);
if (rev != mc_amd->hdr.patch_id) {
pr_err("CPU%d: update failed for patch_level=0x%08x\n",
cpu, mc_amd->hdr.patch_id);
@@ -237,7 +273,7 @@
return -ENOMEM;
}
- get_ucode_data(equiv_cpu_table, buf + CONTAINER_HDR_SZ, size);
+ memcpy(equiv_cpu_table, buf + CONTAINER_HDR_SZ, size);
/* add header length */
return size + CONTAINER_HDR_SZ;
@@ -249,61 +285,113 @@
equiv_cpu_table = NULL;
}
-static enum ucode_state
-generic_load_microcode(int cpu, const u8 *data, size_t size)
+static void cleanup(void)
{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- struct microcode_header_amd *mc_hdr = NULL;
- unsigned int mc_size, leftover, current_size = 0;
- int offset;
- const u8 *ucode_ptr = data;
- void *new_mc = NULL;
- unsigned int new_rev = uci->cpu_sig.rev;
- enum ucode_state state = UCODE_ERROR;
+ free_equiv_cpu_table();
+ free_cache();
+}
- offset = install_equiv_cpu_table(ucode_ptr);
+/*
+ * We return the current size even if some of the checks failed so that
+ * we can skip over the next patch. If we return a negative value, we
+ * signal a grave error like a memory allocation has failed and the
+ * driver cannot continue functioning normally. In such cases, we tear
+ * down everything we've used up so far and exit.
+ */
+static int verify_and_add_patch(unsigned int cpu, u8 *fw, unsigned int leftover)
+{
+ struct cpuinfo_x86 *c = &cpu_data(cpu);
+ struct microcode_header_amd *mc_hdr;
+ struct ucode_patch *patch;
+ unsigned int patch_size, crnt_size, ret;
+ u32 proc_fam;
+ u16 proc_id;
+
+ patch_size = *(u32 *)(fw + 4);
+ crnt_size = patch_size + SECTION_HDR_SIZE;
+ mc_hdr = (struct microcode_header_amd *)(fw + SECTION_HDR_SIZE);
+ proc_id = mc_hdr->processor_rev_id;
+
+ proc_fam = find_cpu_family_by_equiv_cpu(proc_id);
+ if (!proc_fam) {
+ pr_err("No patch family for equiv ID: 0x%04x\n", proc_id);
+ return crnt_size;
+ }
+
+ /* check if patch is for the current family */
+ proc_fam = ((proc_fam >> 8) & 0xf) + ((proc_fam >> 20) & 0xff);
+ if (proc_fam != c->x86)
+ return crnt_size;
+
+ if (mc_hdr->nb_dev_id || mc_hdr->sb_dev_id) {
+ pr_err("Patch-ID 0x%08x: chipset-specific code unsupported.\n",
+ mc_hdr->patch_id);
+ return crnt_size;
+ }
+
+ ret = verify_patch_size(cpu, patch_size, leftover);
+ if (!ret) {
+ pr_err("Patch-ID 0x%08x: size mismatch.\n", mc_hdr->patch_id);
+ return crnt_size;
+ }
+
+ patch = kzalloc(sizeof(*patch), GFP_KERNEL);
+ if (!patch) {
+ pr_err("Patch allocation failure.\n");
+ return -EINVAL;
+ }
+
+ patch->data = kzalloc(patch_size, GFP_KERNEL);
+ if (!patch->data) {
+ pr_err("Patch data allocation failure.\n");
+ kfree(patch);
+ return -EINVAL;
+ }
+
+ /* All looks ok, copy patch... */
+ memcpy(patch->data, fw + SECTION_HDR_SIZE, patch_size);
+ INIT_LIST_HEAD(&patch->plist);
+ patch->patch_id = mc_hdr->patch_id;
+ patch->equiv_cpu = proc_id;
+
+ /* ... and add to cache. */
+ update_cache(patch);
+
+ return crnt_size;
+}
+
+static enum ucode_state load_microcode_amd(int cpu, const u8 *data, size_t size)
+{
+ enum ucode_state ret = UCODE_ERROR;
+ unsigned int leftover;
+ u8 *fw = (u8 *)data;
+ int crnt_size = 0;
+ int offset;
+
+ offset = install_equiv_cpu_table(data);
if (offset < 0) {
pr_err("failed to create equivalent cpu table\n");
- goto out;
+ return ret;
}
- ucode_ptr += offset;
+ fw += offset;
leftover = size - offset;
- if (*(u32 *)ucode_ptr != UCODE_UCODE_TYPE) {
+ if (*(u32 *)fw != UCODE_UCODE_TYPE) {
pr_err("invalid type field in container file section header\n");
- goto free_table;
+ free_equiv_cpu_table();
+ return ret;
}
while (leftover) {
- mc_size = get_matching_microcode(cpu, ucode_ptr, leftover,
- new_rev, ¤t_size);
- if (mc_size) {
- mc_hdr = patch;
- new_mc = patch;
- new_rev = mc_hdr->patch_id;
- goto out_ok;
- }
+ crnt_size = verify_and_add_patch(cpu, fw, leftover);
+ if (crnt_size < 0)
+ return ret;
- ucode_ptr += current_size;
- leftover -= current_size;
+ fw += crnt_size;
+ leftover -= crnt_size;
}
- if (!new_mc) {
- state = UCODE_NFOUND;
- goto free_table;
- }
-
-out_ok:
- uci->mc = new_mc;
- state = UCODE_OK;
- pr_debug("CPU%d update ucode (0x%08x -> 0x%08x)\n",
- cpu, uci->cpu_sig.rev, new_rev);
-
-free_table:
- free_equiv_cpu_table();
-
-out:
- return state;
+ return UCODE_OK;
}
/*
@@ -314,7 +402,7 @@
*
* This legacy file is always smaller than 2K in size.
*
- * Starting at family 15h they are in family specific firmware files:
+ * Beginning with family 15h, they are in family-specific firmware files:
*
* amd-ucode/microcode_amd_fam15h.bin
* amd-ucode/microcode_amd_fam16h.bin
@@ -322,12 +410,17 @@
*
* These might be larger than 2K.
*/
-static enum ucode_state request_microcode_amd(int cpu, struct device *device)
+static enum ucode_state request_microcode_amd(int cpu, struct device *device,
+ bool refresh_fw)
{
char fw_name[36] = "amd-ucode/microcode_amd.bin";
- const struct firmware *fw;
- enum ucode_state ret = UCODE_NFOUND;
struct cpuinfo_x86 *c = &cpu_data(cpu);
+ enum ucode_state ret = UCODE_NFOUND;
+ const struct firmware *fw;
+
+ /* reload ucode container only on the boot cpu */
+ if (!refresh_fw || c->cpu_index != boot_cpu_data.cpu_index)
+ return UCODE_OK;
if (c->x86 >= 0x15)
snprintf(fw_name, sizeof(fw_name), "amd-ucode/microcode_amd_fam%.2xh.bin", c->x86);
@@ -343,12 +436,17 @@
goto fw_release;
}
- ret = generic_load_microcode(cpu, fw->data, fw->size);
+ /* free old equiv table */
+ free_equiv_cpu_table();
-fw_release:
+ ret = load_microcode_amd(cpu, fw->data, fw->size);
+ if (ret != UCODE_OK)
+ cleanup();
+
+ fw_release:
release_firmware(fw);
-out:
+ out:
return ret;
}
@@ -382,14 +480,10 @@
return NULL;
}
- patch = (void *)get_zeroed_page(GFP_KERNEL);
- if (!patch)
- return NULL;
-
return µcode_amd_ops;
}
void __exit exit_amd_microcode(void)
{
- free_page((unsigned long)patch);
+ cleanup();
}
diff --git a/arch/x86/kernel/microcode_core.c b/arch/x86/kernel/microcode_core.c
index 4873e62..3a04b22 100644
--- a/arch/x86/kernel/microcode_core.c
+++ b/arch/x86/kernel/microcode_core.c
@@ -225,6 +225,9 @@
if (do_microcode_update(buf, len) == 0)
ret = (ssize_t)len;
+ if (ret > 0)
+ perf_check_microcode();
+
mutex_unlock(µcode_mutex);
put_online_cpus();
@@ -276,19 +279,18 @@
static int reload_for_cpu(int cpu)
{
struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ enum ucode_state ustate;
int err = 0;
- if (uci->valid) {
- enum ucode_state ustate;
+ if (!uci->valid)
+ return err;
- ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev);
- if (ustate == UCODE_OK)
- apply_microcode_on_target(cpu);
- else
- if (ustate == UCODE_ERROR)
- err = -EINVAL;
- }
-
+ ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev, true);
+ if (ustate == UCODE_OK)
+ apply_microcode_on_target(cpu);
+ else
+ if (ustate == UCODE_ERROR)
+ err = -EINVAL;
return err;
}
@@ -370,18 +372,15 @@
static enum ucode_state microcode_resume_cpu(int cpu)
{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-
- if (!uci->mc)
- return UCODE_NFOUND;
-
pr_debug("CPU%d updated upon resume\n", cpu);
- apply_microcode_on_target(cpu);
+
+ if (apply_microcode_on_target(cpu))
+ return UCODE_ERROR;
return UCODE_OK;
}
-static enum ucode_state microcode_init_cpu(int cpu)
+static enum ucode_state microcode_init_cpu(int cpu, bool refresh_fw)
{
enum ucode_state ustate;
@@ -392,7 +391,8 @@
if (system_state != SYSTEM_RUNNING)
return UCODE_NFOUND;
- ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev);
+ ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev,
+ refresh_fw);
if (ustate == UCODE_OK) {
pr_debug("CPU%d updated upon init\n", cpu);
@@ -405,14 +405,11 @@
static enum ucode_state microcode_update_cpu(int cpu)
{
struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- enum ucode_state ustate;
if (uci->valid)
- ustate = microcode_resume_cpu(cpu);
- else
- ustate = microcode_init_cpu(cpu);
+ return microcode_resume_cpu(cpu);
- return ustate;
+ return microcode_init_cpu(cpu, false);
}
static int mc_device_add(struct device *dev, struct subsys_interface *sif)
@@ -428,7 +425,7 @@
if (err)
return err;
- if (microcode_init_cpu(cpu) == UCODE_ERROR)
+ if (microcode_init_cpu(cpu, true) == UCODE_ERROR)
return -EINVAL;
return err;
@@ -477,34 +474,41 @@
struct device *dev;
dev = get_cpu_device(cpu);
- switch (action) {
+
+ switch (action & ~CPU_TASKS_FROZEN) {
case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
microcode_update_cpu(cpu);
- case CPU_DOWN_FAILED:
- case CPU_DOWN_FAILED_FROZEN:
pr_debug("CPU%d added\n", cpu);
+ /*
+ * "break" is missing on purpose here because we want to fall
+ * through in order to create the sysfs group.
+ */
+
+ case CPU_DOWN_FAILED:
if (sysfs_create_group(&dev->kobj, &mc_attr_group))
pr_err("Failed to create group for CPU%d\n", cpu);
break;
+
case CPU_DOWN_PREPARE:
- case CPU_DOWN_PREPARE_FROZEN:
/* Suspend is in progress, only remove the interface */
sysfs_remove_group(&dev->kobj, &mc_attr_group);
pr_debug("CPU%d removed\n", cpu);
break;
/*
+ * case CPU_DEAD:
+ *
* When a CPU goes offline, don't free up or invalidate the copy of
* the microcode in kernel memory, so that we can reuse it when the
* CPU comes back online without unnecessarily requesting the userspace
* for it again.
*/
- case CPU_UP_CANCELED_FROZEN:
- /* The CPU refused to come up during a system resume */
- microcode_fini_cpu(cpu);
- break;
}
+
+ /* The CPU refused to come up during a system resume */
+ if (action == CPU_UP_CANCELED_FROZEN)
+ microcode_fini_cpu(cpu);
+
return NOTIFY_OK;
}
diff --git a/arch/x86/kernel/microcode_intel.c b/arch/x86/kernel/microcode_intel.c
index 0327e2b..3544aed 100644
--- a/arch/x86/kernel/microcode_intel.c
+++ b/arch/x86/kernel/microcode_intel.c
@@ -405,7 +405,8 @@
return 0;
}
-static enum ucode_state request_microcode_fw(int cpu, struct device *device)
+static enum ucode_state request_microcode_fw(int cpu, struct device *device,
+ bool refresh_fw)
{
char name[30];
struct cpuinfo_x86 *c = &cpu_data(cpu);
diff --git a/arch/x86/kernel/msr.c b/arch/x86/kernel/msr.c
index eb11369..a7c5661 100644
--- a/arch/x86/kernel/msr.c
+++ b/arch/x86/kernel/msr.c
@@ -257,12 +257,14 @@
goto out_chrdev;
}
msr_class->devnode = msr_devnode;
+ get_online_cpus();
for_each_online_cpu(i) {
err = msr_device_create(i);
if (err != 0)
goto out_class;
}
register_hotcpu_notifier(&msr_class_cpu_notifier);
+ put_online_cpus();
err = 0;
goto out;
@@ -271,6 +273,7 @@
i = 0;
for_each_online_cpu(i)
msr_device_destroy(i);
+ put_online_cpus();
class_destroy(msr_class);
out_chrdev:
__unregister_chrdev(MSR_MAJOR, 0, NR_CPUS, "cpu/msr");
@@ -281,11 +284,13 @@
static void __exit msr_exit(void)
{
int cpu = 0;
+ get_online_cpus();
for_each_online_cpu(cpu)
msr_device_destroy(cpu);
class_destroy(msr_class);
__unregister_chrdev(MSR_MAJOR, 0, NR_CPUS, "cpu/msr");
unregister_hotcpu_notifier(&msr_class_cpu_notifier);
+ put_online_cpus();
}
module_init(msr_init);
diff --git a/arch/x86/kernel/perf_regs.c b/arch/x86/kernel/perf_regs.c
new file mode 100644
index 0000000..e309cc5
--- /dev/null
+++ b/arch/x86/kernel/perf_regs.c
@@ -0,0 +1,105 @@
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/perf_event.h>
+#include <linux/bug.h>
+#include <linux/stddef.h>
+#include <asm/perf_regs.h>
+#include <asm/ptrace.h>
+
+#ifdef CONFIG_X86_32
+#define PERF_REG_X86_MAX PERF_REG_X86_32_MAX
+#else
+#define PERF_REG_X86_MAX PERF_REG_X86_64_MAX
+#endif
+
+#define PT_REGS_OFFSET(id, r) [id] = offsetof(struct pt_regs, r)
+
+static unsigned int pt_regs_offset[PERF_REG_X86_MAX] = {
+ PT_REGS_OFFSET(PERF_REG_X86_AX, ax),
+ PT_REGS_OFFSET(PERF_REG_X86_BX, bx),
+ PT_REGS_OFFSET(PERF_REG_X86_CX, cx),
+ PT_REGS_OFFSET(PERF_REG_X86_DX, dx),
+ PT_REGS_OFFSET(PERF_REG_X86_SI, si),
+ PT_REGS_OFFSET(PERF_REG_X86_DI, di),
+ PT_REGS_OFFSET(PERF_REG_X86_BP, bp),
+ PT_REGS_OFFSET(PERF_REG_X86_SP, sp),
+ PT_REGS_OFFSET(PERF_REG_X86_IP, ip),
+ PT_REGS_OFFSET(PERF_REG_X86_FLAGS, flags),
+ PT_REGS_OFFSET(PERF_REG_X86_CS, cs),
+ PT_REGS_OFFSET(PERF_REG_X86_SS, ss),
+#ifdef CONFIG_X86_32
+ PT_REGS_OFFSET(PERF_REG_X86_DS, ds),
+ PT_REGS_OFFSET(PERF_REG_X86_ES, es),
+ PT_REGS_OFFSET(PERF_REG_X86_FS, fs),
+ PT_REGS_OFFSET(PERF_REG_X86_GS, gs),
+#else
+ /*
+ * The pt_regs struct does not store
+ * ds, es, fs, gs in 64 bit mode.
+ */
+ (unsigned int) -1,
+ (unsigned int) -1,
+ (unsigned int) -1,
+ (unsigned int) -1,
+#endif
+#ifdef CONFIG_X86_64
+ PT_REGS_OFFSET(PERF_REG_X86_R8, r8),
+ PT_REGS_OFFSET(PERF_REG_X86_R9, r9),
+ PT_REGS_OFFSET(PERF_REG_X86_R10, r10),
+ PT_REGS_OFFSET(PERF_REG_X86_R11, r11),
+ PT_REGS_OFFSET(PERF_REG_X86_R12, r12),
+ PT_REGS_OFFSET(PERF_REG_X86_R13, r13),
+ PT_REGS_OFFSET(PERF_REG_X86_R14, r14),
+ PT_REGS_OFFSET(PERF_REG_X86_R15, r15),
+#endif
+};
+
+u64 perf_reg_value(struct pt_regs *regs, int idx)
+{
+ if (WARN_ON_ONCE(idx >= ARRAY_SIZE(pt_regs_offset)))
+ return 0;
+
+ return regs_get_register(regs, pt_regs_offset[idx]);
+}
+
+#define REG_RESERVED (~((1ULL << PERF_REG_X86_MAX) - 1ULL))
+
+#ifdef CONFIG_X86_32
+int perf_reg_validate(u64 mask)
+{
+ if (!mask || mask & REG_RESERVED)
+ return -EINVAL;
+
+ return 0;
+}
+
+u64 perf_reg_abi(struct task_struct *task)
+{
+ return PERF_SAMPLE_REGS_ABI_32;
+}
+#else /* CONFIG_X86_64 */
+#define REG_NOSUPPORT ((1ULL << PERF_REG_X86_DS) | \
+ (1ULL << PERF_REG_X86_ES) | \
+ (1ULL << PERF_REG_X86_FS) | \
+ (1ULL << PERF_REG_X86_GS))
+
+int perf_reg_validate(u64 mask)
+{
+ if (!mask || mask & REG_RESERVED)
+ return -EINVAL;
+
+ if (mask & REG_NOSUPPORT)
+ return -EINVAL;
+
+ return 0;
+}
+
+u64 perf_reg_abi(struct task_struct *task)
+{
+ if (test_tsk_thread_flag(task, TIF_IA32))
+ return PERF_SAMPLE_REGS_ABI_32;
+ else
+ return PERF_SAMPLE_REGS_ABI_64;
+}
+#endif /* CONFIG_X86_32 */
diff --git a/arch/x86/kernel/probe_roms.c b/arch/x86/kernel/probe_roms.c
index 0bc72e2..d5f15c3 100644
--- a/arch/x86/kernel/probe_roms.c
+++ b/arch/x86/kernel/probe_roms.c
@@ -150,7 +150,7 @@
return oprom;
}
-void *pci_map_biosrom(struct pci_dev *pdev)
+void __iomem *pci_map_biosrom(struct pci_dev *pdev)
{
struct resource *oprom = find_oprom(pdev);
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index ef6a845..dc3567e 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -66,15 +66,13 @@
{
int ret;
- unlazy_fpu(src);
-
*dst = *src;
if (fpu_allocated(&src->thread.fpu)) {
memset(&dst->thread.fpu, 0, sizeof(dst->thread.fpu));
ret = fpu_alloc(&dst->thread.fpu);
if (ret)
return ret;
- fpu_copy(&dst->thread.fpu, &src->thread.fpu);
+ fpu_copy(dst, src);
}
return 0;
}
@@ -97,16 +95,6 @@
SLAB_PANIC | SLAB_NOTRACK, NULL);
}
-static inline void drop_fpu(struct task_struct *tsk)
-{
- /*
- * Forget coprocessor state..
- */
- tsk->fpu_counter = 0;
- clear_fpu(tsk);
- clear_used_math();
-}
-
/*
* Free current thread data structures etc..
*/
@@ -163,7 +151,13 @@
flush_ptrace_hw_breakpoint(tsk);
memset(tsk->thread.tls_array, 0, sizeof(tsk->thread.tls_array));
- drop_fpu(tsk);
+ drop_init_fpu(tsk);
+ /*
+ * Free the FPU state for non xsave platforms. They get reallocated
+ * lazily at the first use.
+ */
+ if (!use_eager_fpu())
+ free_thread_xstate(tsk);
}
static void hard_disable_TSC(void)
diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c
index 516fa18..b9ff83c 100644
--- a/arch/x86/kernel/process_32.c
+++ b/arch/x86/kernel/process_32.c
@@ -190,10 +190,6 @@
regs->cs = __USER_CS;
regs->ip = new_ip;
regs->sp = new_sp;
- /*
- * Free the old FP and other extended state
- */
- free_thread_xstate(current);
}
EXPORT_SYMBOL_GPL(start_thread);
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 0a980c9..8a6d20c 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -232,10 +232,6 @@
regs->cs = _cs;
regs->ss = _ss;
regs->flags = X86_EFLAGS_IF;
- /*
- * Free the old FP and other extended state
- */
- free_thread_xstate(current);
}
void
diff --git a/arch/x86/kernel/ptrace.c b/arch/x86/kernel/ptrace.c
index c4c6a5c..b00b33a 100644
--- a/arch/x86/kernel/ptrace.c
+++ b/arch/x86/kernel/ptrace.c
@@ -21,6 +21,7 @@
#include <linux/signal.h>
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
+#include <linux/rcupdate.h>
#include <asm/uaccess.h>
#include <asm/pgtable.h>
@@ -1332,9 +1333,6 @@
#define genregs32_get genregs_get
#define genregs32_set genregs_set
-#define user_i387_ia32_struct user_i387_struct
-#define user32_fxsr_struct user_fxsr_struct
-
#endif /* CONFIG_X86_64 */
#if defined CONFIG_X86_32 || defined CONFIG_IA32_EMULATION
@@ -1463,6 +1461,8 @@
{
long ret = 0;
+ rcu_user_exit();
+
/*
* If we stepped into a sysenter/syscall insn, it trapped in
* kernel mode; do_debug() cleared TF and set TIF_SINGLESTEP.
@@ -1526,4 +1526,6 @@
!test_thread_flag(TIF_SYSCALL_EMU);
if (step || test_thread_flag(TIF_SYSCALL_TRACE))
tracehook_report_syscall_exit(regs, step);
+
+ rcu_user_enter();
}
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index f4b9b80..4f16547 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -961,9 +961,7 @@
kvmclock_init();
#endif
- x86_init.paging.pagetable_setup_start(swapper_pg_dir);
- paging_init();
- x86_init.paging.pagetable_setup_done(swapper_pg_dir);
+ x86_init.paging.pagetable_init();
if (boot_cpu_data.cpuid_level >= 0) {
/* A CPU has %cr4 if and only if it has CPUID */
diff --git a/arch/x86/kernel/signal.c b/arch/x86/kernel/signal.c
index b280908..3160c26 100644
--- a/arch/x86/kernel/signal.c
+++ b/arch/x86/kernel/signal.c
@@ -114,7 +114,7 @@
regs->orig_ax = -1; /* disable syscall checks */
get_user_ex(buf, &sc->fpstate);
- err |= restore_i387_xstate(buf);
+ err |= restore_xstate_sig(buf, config_enabled(CONFIG_X86_32));
get_user_ex(*pax, &sc->ax);
} get_user_catch(err);
@@ -206,35 +206,32 @@
void __user **fpstate)
{
/* Default to using normal stack */
+ unsigned long math_size = 0;
unsigned long sp = regs->sp;
+ unsigned long buf_fx = 0;
int onsigstack = on_sig_stack(sp);
-#ifdef CONFIG_X86_64
/* redzone */
- sp -= 128;
-#endif /* CONFIG_X86_64 */
+ if (config_enabled(CONFIG_X86_64))
+ sp -= 128;
if (!onsigstack) {
/* This is the X/Open sanctioned signal stack switching. */
if (ka->sa.sa_flags & SA_ONSTACK) {
if (current->sas_ss_size)
sp = current->sas_ss_sp + current->sas_ss_size;
- } else {
-#ifdef CONFIG_X86_32
- /* This is the legacy signal stack switching. */
- if ((regs->ss & 0xffff) != __USER_DS &&
- !(ka->sa.sa_flags & SA_RESTORER) &&
- ka->sa.sa_restorer)
+ } else if (config_enabled(CONFIG_X86_32) &&
+ (regs->ss & 0xffff) != __USER_DS &&
+ !(ka->sa.sa_flags & SA_RESTORER) &&
+ ka->sa.sa_restorer) {
+ /* This is the legacy signal stack switching. */
sp = (unsigned long) ka->sa.sa_restorer;
-#endif /* CONFIG_X86_32 */
}
}
if (used_math()) {
- sp -= sig_xstate_size;
-#ifdef CONFIG_X86_64
- sp = round_down(sp, 64);
-#endif /* CONFIG_X86_64 */
+ sp = alloc_mathframe(sp, config_enabled(CONFIG_X86_32),
+ &buf_fx, &math_size);
*fpstate = (void __user *)sp;
}
@@ -247,8 +244,9 @@
if (onsigstack && !likely(on_sig_stack(sp)))
return (void __user *)-1L;
- /* save i387 state */
- if (used_math() && save_i387_xstate(*fpstate) < 0)
+ /* save i387 and extended state */
+ if (used_math() &&
+ save_xstate_sig(*fpstate, (void __user *)buf_fx, math_size) < 0)
return (void __user *)-1L;
return (void __user *)sp;
@@ -474,6 +472,74 @@
}
#endif /* CONFIG_X86_32 */
+static int x32_setup_rt_frame(int sig, struct k_sigaction *ka,
+ siginfo_t *info, compat_sigset_t *set,
+ struct pt_regs *regs)
+{
+#ifdef CONFIG_X86_X32_ABI
+ struct rt_sigframe_x32 __user *frame;
+ void __user *restorer;
+ int err = 0;
+ void __user *fpstate = NULL;
+
+ frame = get_sigframe(ka, regs, sizeof(*frame), &fpstate);
+
+ if (!access_ok(VERIFY_WRITE, frame, sizeof(*frame)))
+ return -EFAULT;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (copy_siginfo_to_user32(&frame->info, info))
+ return -EFAULT;
+ }
+
+ put_user_try {
+ /* Create the ucontext. */
+ if (cpu_has_xsave)
+ put_user_ex(UC_FP_XSTATE, &frame->uc.uc_flags);
+ else
+ put_user_ex(0, &frame->uc.uc_flags);
+ put_user_ex(0, &frame->uc.uc_link);
+ put_user_ex(current->sas_ss_sp, &frame->uc.uc_stack.ss_sp);
+ put_user_ex(sas_ss_flags(regs->sp),
+ &frame->uc.uc_stack.ss_flags);
+ put_user_ex(current->sas_ss_size, &frame->uc.uc_stack.ss_size);
+ put_user_ex(0, &frame->uc.uc__pad0);
+ err |= setup_sigcontext(&frame->uc.uc_mcontext, fpstate,
+ regs, set->sig[0]);
+ err |= __copy_to_user(&frame->uc.uc_sigmask, set, sizeof(*set));
+
+ if (ka->sa.sa_flags & SA_RESTORER) {
+ restorer = ka->sa.sa_restorer;
+ } else {
+ /* could use a vstub here */
+ restorer = NULL;
+ err |= -EFAULT;
+ }
+ put_user_ex(restorer, &frame->pretcode);
+ } put_user_catch(err);
+
+ if (err)
+ return -EFAULT;
+
+ /* Set up registers for signal handler */
+ regs->sp = (unsigned long) frame;
+ regs->ip = (unsigned long) ka->sa.sa_handler;
+
+ /* We use the x32 calling convention here... */
+ regs->di = sig;
+ regs->si = (unsigned long) &frame->info;
+ regs->dx = (unsigned long) &frame->uc;
+
+ loadsegment(ds, __USER_DS);
+ loadsegment(es, __USER_DS);
+
+ regs->cs = __USER_CS;
+ regs->ss = __USER_DS;
+#endif /* CONFIG_X86_X32_ABI */
+
+ return 0;
+}
+
#ifdef CONFIG_X86_32
/*
* Atomically swap in the new signal mask, and wait for a signal.
@@ -612,55 +678,22 @@
return sig;
}
-#ifdef CONFIG_X86_32
-
-#define is_ia32 1
-#define ia32_setup_frame __setup_frame
-#define ia32_setup_rt_frame __setup_rt_frame
-
-#else /* !CONFIG_X86_32 */
-
-#ifdef CONFIG_IA32_EMULATION
-#define is_ia32 test_thread_flag(TIF_IA32)
-#else /* !CONFIG_IA32_EMULATION */
-#define is_ia32 0
-#endif /* CONFIG_IA32_EMULATION */
-
-#ifdef CONFIG_X86_X32_ABI
-#define is_x32 test_thread_flag(TIF_X32)
-
-static int x32_setup_rt_frame(int sig, struct k_sigaction *ka,
- siginfo_t *info, compat_sigset_t *set,
- struct pt_regs *regs);
-#else /* !CONFIG_X86_X32_ABI */
-#define is_x32 0
-#endif /* CONFIG_X86_X32_ABI */
-
-int ia32_setup_rt_frame(int sig, struct k_sigaction *ka, siginfo_t *info,
- sigset_t *set, struct pt_regs *regs);
-int ia32_setup_frame(int sig, struct k_sigaction *ka,
- sigset_t *set, struct pt_regs *regs);
-
-#endif /* CONFIG_X86_32 */
-
static int
setup_rt_frame(int sig, struct k_sigaction *ka, siginfo_t *info,
struct pt_regs *regs)
{
int usig = signr_convert(sig);
sigset_t *set = sigmask_to_save();
+ compat_sigset_t *cset = (compat_sigset_t *) set;
/* Set up the stack frame */
- if (is_ia32) {
+ if (is_ia32_frame()) {
if (ka->sa.sa_flags & SA_SIGINFO)
- return ia32_setup_rt_frame(usig, ka, info, set, regs);
+ return ia32_setup_rt_frame(usig, ka, info, cset, regs);
else
- return ia32_setup_frame(usig, ka, set, regs);
-#ifdef CONFIG_X86_X32_ABI
- } else if (is_x32) {
- return x32_setup_rt_frame(usig, ka, info,
- (compat_sigset_t *)set, regs);
-#endif
+ return ia32_setup_frame(usig, ka, cset, regs);
+ } else if (is_x32_frame()) {
+ return x32_setup_rt_frame(usig, ka, info, cset, regs);
} else {
return __setup_rt_frame(sig, ka, info, set, regs);
}
@@ -779,6 +812,8 @@
void
do_notify_resume(struct pt_regs *regs, void *unused, __u32 thread_info_flags)
{
+ rcu_user_exit();
+
#ifdef CONFIG_X86_MCE
/* notify userspace of pending MCEs */
if (thread_info_flags & _TIF_MCE_NOTIFY)
@@ -804,6 +839,8 @@
#ifdef CONFIG_X86_32
clear_thread_flag(TIF_IRET);
#endif /* CONFIG_X86_32 */
+
+ rcu_user_enter();
}
void signal_fault(struct pt_regs *regs, void __user *frame, char *where)
@@ -824,72 +861,6 @@
}
#ifdef CONFIG_X86_X32_ABI
-static int x32_setup_rt_frame(int sig, struct k_sigaction *ka,
- siginfo_t *info, compat_sigset_t *set,
- struct pt_regs *regs)
-{
- struct rt_sigframe_x32 __user *frame;
- void __user *restorer;
- int err = 0;
- void __user *fpstate = NULL;
-
- frame = get_sigframe(ka, regs, sizeof(*frame), &fpstate);
-
- if (!access_ok(VERIFY_WRITE, frame, sizeof(*frame)))
- return -EFAULT;
-
- if (ka->sa.sa_flags & SA_SIGINFO) {
- if (copy_siginfo_to_user32(&frame->info, info))
- return -EFAULT;
- }
-
- put_user_try {
- /* Create the ucontext. */
- if (cpu_has_xsave)
- put_user_ex(UC_FP_XSTATE, &frame->uc.uc_flags);
- else
- put_user_ex(0, &frame->uc.uc_flags);
- put_user_ex(0, &frame->uc.uc_link);
- put_user_ex(current->sas_ss_sp, &frame->uc.uc_stack.ss_sp);
- put_user_ex(sas_ss_flags(regs->sp),
- &frame->uc.uc_stack.ss_flags);
- put_user_ex(current->sas_ss_size, &frame->uc.uc_stack.ss_size);
- put_user_ex(0, &frame->uc.uc__pad0);
- err |= setup_sigcontext(&frame->uc.uc_mcontext, fpstate,
- regs, set->sig[0]);
- err |= __copy_to_user(&frame->uc.uc_sigmask, set, sizeof(*set));
-
- if (ka->sa.sa_flags & SA_RESTORER) {
- restorer = ka->sa.sa_restorer;
- } else {
- /* could use a vstub here */
- restorer = NULL;
- err |= -EFAULT;
- }
- put_user_ex(restorer, &frame->pretcode);
- } put_user_catch(err);
-
- if (err)
- return -EFAULT;
-
- /* Set up registers for signal handler */
- regs->sp = (unsigned long) frame;
- regs->ip = (unsigned long) ka->sa.sa_handler;
-
- /* We use the x32 calling convention here... */
- regs->di = sig;
- regs->si = (unsigned long) &frame->info;
- regs->dx = (unsigned long) &frame->uc;
-
- loadsegment(ds, __USER_DS);
- loadsegment(es, __USER_DS);
-
- regs->cs = __USER_CS;
- regs->ss = __USER_DS;
-
- return 0;
-}
-
asmlinkage long sys32_x32_rt_sigreturn(struct pt_regs *regs)
{
struct rt_sigframe_x32 __user *frame;
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 7c5a8c3..c80a33b 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -665,7 +665,8 @@
unsigned long boot_error = 0;
int timeout;
- alternatives_smp_switch(1);
+ /* Just in case we booted with a single CPU. */
+ alternatives_enable_smp();
idle->thread.sp = (unsigned long) (((struct pt_regs *)
(THREAD_SIZE + task_stack_page(idle))) - 1);
@@ -1053,20 +1054,6 @@
preempt_enable();
}
-void arch_disable_nonboot_cpus_begin(void)
-{
- /*
- * Avoid the smp alternatives switch during the disable_nonboot_cpus().
- * In the suspend path, we will be back in the SMP mode shortly anyways.
- */
- skip_smp_alternatives = true;
-}
-
-void arch_disable_nonboot_cpus_end(void)
-{
- skip_smp_alternatives = false;
-}
-
void arch_enable_nonboot_cpus_begin(void)
{
set_mtrr_aps_delayed_init();
@@ -1256,9 +1243,6 @@
if (per_cpu(cpu_state, cpu) == CPU_DEAD) {
if (system_state == SYSTEM_RUNNING)
pr_info("CPU %u is now offline\n", cpu);
-
- if (1 == num_online_cpus())
- alternatives_smp_switch(0);
return;
}
msleep(100);
diff --git a/arch/x86/kernel/step.c b/arch/x86/kernel/step.c
index c346d11..cd3b243 100644
--- a/arch/x86/kernel/step.c
+++ b/arch/x86/kernel/step.c
@@ -157,6 +157,33 @@
return 1;
}
+void set_task_blockstep(struct task_struct *task, bool on)
+{
+ unsigned long debugctl;
+
+ /*
+ * Ensure irq/preemption can't change debugctl in between.
+ * Note also that both TIF_BLOCKSTEP and debugctl should
+ * be changed atomically wrt preemption.
+ * FIXME: this means that set/clear TIF_BLOCKSTEP is simply
+ * wrong if task != current, SIGKILL can wakeup the stopped
+ * tracee and set/clear can play with the running task, this
+ * can confuse the next __switch_to_xtra().
+ */
+ local_irq_disable();
+ debugctl = get_debugctlmsr();
+ if (on) {
+ debugctl |= DEBUGCTLMSR_BTF;
+ set_tsk_thread_flag(task, TIF_BLOCKSTEP);
+ } else {
+ debugctl &= ~DEBUGCTLMSR_BTF;
+ clear_tsk_thread_flag(task, TIF_BLOCKSTEP);
+ }
+ if (task == current)
+ update_debugctlmsr(debugctl);
+ local_irq_enable();
+}
+
/*
* Enable single or block step.
*/
@@ -169,19 +196,10 @@
* So no one should try to use debugger block stepping in a program
* that uses user-mode single stepping itself.
*/
- if (enable_single_step(child) && block) {
- unsigned long debugctl = get_debugctlmsr();
-
- debugctl |= DEBUGCTLMSR_BTF;
- update_debugctlmsr(debugctl);
- set_tsk_thread_flag(child, TIF_BLOCKSTEP);
- } else if (test_tsk_thread_flag(child, TIF_BLOCKSTEP)) {
- unsigned long debugctl = get_debugctlmsr();
-
- debugctl &= ~DEBUGCTLMSR_BTF;
- update_debugctlmsr(debugctl);
- clear_tsk_thread_flag(child, TIF_BLOCKSTEP);
- }
+ if (enable_single_step(child) && block)
+ set_task_blockstep(child, true);
+ else if (test_tsk_thread_flag(child, TIF_BLOCKSTEP))
+ set_task_blockstep(child, false);
}
void user_enable_single_step(struct task_struct *child)
@@ -199,13 +217,8 @@
/*
* Make sure block stepping (BTF) is disabled.
*/
- if (test_tsk_thread_flag(child, TIF_BLOCKSTEP)) {
- unsigned long debugctl = get_debugctlmsr();
-
- debugctl &= ~DEBUGCTLMSR_BTF;
- update_debugctlmsr(debugctl);
- clear_tsk_thread_flag(child, TIF_BLOCKSTEP);
- }
+ if (test_tsk_thread_flag(child, TIF_BLOCKSTEP))
+ set_task_blockstep(child, false);
/* Always clear TIF_SINGLESTEP... */
clear_tsk_thread_flag(child, TIF_SINGLESTEP);
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index b481341..8276dc6 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -55,6 +55,7 @@
#include <asm/i387.h>
#include <asm/fpu-internal.h>
#include <asm/mce.h>
+#include <asm/rcu.h>
#include <asm/mach_traps.h>
@@ -107,30 +108,45 @@
dec_preempt_count();
}
+static int __kprobes
+do_trap_no_signal(struct task_struct *tsk, int trapnr, char *str,
+ struct pt_regs *regs, long error_code)
+{
+#ifdef CONFIG_X86_32
+ if (regs->flags & X86_VM_MASK) {
+ /*
+ * Traps 0, 1, 3, 4, and 5 should be forwarded to vm86.
+ * On nmi (interrupt 2), do_trap should not be called.
+ */
+ if (trapnr < X86_TRAP_UD) {
+ if (!handle_vm86_trap((struct kernel_vm86_regs *) regs,
+ error_code, trapnr))
+ return 0;
+ }
+ return -1;
+ }
+#endif
+ if (!user_mode(regs)) {
+ if (!fixup_exception(regs)) {
+ tsk->thread.error_code = error_code;
+ tsk->thread.trap_nr = trapnr;
+ die(str, regs, error_code);
+ }
+ return 0;
+ }
+
+ return -1;
+}
+
static void __kprobes
do_trap(int trapnr, int signr, char *str, struct pt_regs *regs,
long error_code, siginfo_t *info)
{
struct task_struct *tsk = current;
-#ifdef CONFIG_X86_32
- if (regs->flags & X86_VM_MASK) {
- /*
- * traps 0, 1, 3, 4, and 5 should be forwarded to vm86.
- * On nmi (interrupt 2), do_trap should not be called.
- */
- if (trapnr < X86_TRAP_UD)
- goto vm86_trap;
- goto trap_signal;
- }
-#endif
- if (!user_mode(regs))
- goto kernel_trap;
-
-#ifdef CONFIG_X86_32
-trap_signal:
-#endif
+ if (!do_trap_no_signal(tsk, trapnr, str, regs, error_code))
+ return;
/*
* We want error_code and trap_nr set for userspace faults and
* kernelspace faults which result in die(), but not
@@ -158,33 +174,20 @@
force_sig_info(signr, info, tsk);
else
force_sig(signr, tsk);
- return;
-
-kernel_trap:
- if (!fixup_exception(regs)) {
- tsk->thread.error_code = error_code;
- tsk->thread.trap_nr = trapnr;
- die(str, regs, error_code);
- }
- return;
-
-#ifdef CONFIG_X86_32
-vm86_trap:
- if (handle_vm86_trap((struct kernel_vm86_regs *) regs,
- error_code, trapnr))
- goto trap_signal;
- return;
-#endif
}
#define DO_ERROR(trapnr, signr, str, name) \
dotraplinkage void do_##name(struct pt_regs *regs, long error_code) \
{ \
- if (notify_die(DIE_TRAP, str, regs, error_code, trapnr, signr) \
- == NOTIFY_STOP) \
+ exception_enter(regs); \
+ if (notify_die(DIE_TRAP, str, regs, error_code, \
+ trapnr, signr) == NOTIFY_STOP) { \
+ exception_exit(regs); \
return; \
+ } \
conditional_sti(regs); \
do_trap(trapnr, signr, str, regs, error_code, NULL); \
+ exception_exit(regs); \
}
#define DO_ERROR_INFO(trapnr, signr, str, name, sicode, siaddr) \
@@ -195,11 +198,15 @@
info.si_errno = 0; \
info.si_code = sicode; \
info.si_addr = (void __user *)siaddr; \
- if (notify_die(DIE_TRAP, str, regs, error_code, trapnr, signr) \
- == NOTIFY_STOP) \
+ exception_enter(regs); \
+ if (notify_die(DIE_TRAP, str, regs, error_code, \
+ trapnr, signr) == NOTIFY_STOP) { \
+ exception_exit(regs); \
return; \
+ } \
conditional_sti(regs); \
do_trap(trapnr, signr, str, regs, error_code, &info); \
+ exception_exit(regs); \
}
DO_ERROR_INFO(X86_TRAP_DE, SIGFPE, "divide error", divide_error, FPE_INTDIV,
@@ -222,12 +229,14 @@
/* Runs on IST stack */
dotraplinkage void do_stack_segment(struct pt_regs *regs, long error_code)
{
+ exception_enter(regs);
if (notify_die(DIE_TRAP, "stack segment", regs, error_code,
- X86_TRAP_SS, SIGBUS) == NOTIFY_STOP)
- return;
- preempt_conditional_sti(regs);
- do_trap(X86_TRAP_SS, SIGBUS, "stack segment", regs, error_code, NULL);
- preempt_conditional_cli(regs);
+ X86_TRAP_SS, SIGBUS) != NOTIFY_STOP) {
+ preempt_conditional_sti(regs);
+ do_trap(X86_TRAP_SS, SIGBUS, "stack segment", regs, error_code, NULL);
+ preempt_conditional_cli(regs);
+ }
+ exception_exit(regs);
}
dotraplinkage void do_double_fault(struct pt_regs *regs, long error_code)
@@ -235,6 +244,7 @@
static const char str[] = "double fault";
struct task_struct *tsk = current;
+ exception_enter(regs);
/* Return not checked because double check cannot be ignored */
notify_die(DIE_TRAP, str, regs, error_code, X86_TRAP_DF, SIGSEGV);
@@ -255,16 +265,29 @@
{
struct task_struct *tsk;
+ exception_enter(regs);
conditional_sti(regs);
#ifdef CONFIG_X86_32
- if (regs->flags & X86_VM_MASK)
- goto gp_in_vm86;
+ if (regs->flags & X86_VM_MASK) {
+ local_irq_enable();
+ handle_vm86_fault((struct kernel_vm86_regs *) regs, error_code);
+ goto exit;
+ }
#endif
tsk = current;
- if (!user_mode(regs))
- goto gp_in_kernel;
+ if (!user_mode(regs)) {
+ if (fixup_exception(regs))
+ goto exit;
+
+ tsk->thread.error_code = error_code;
+ tsk->thread.trap_nr = X86_TRAP_GP;
+ if (notify_die(DIE_GPF, "general protection fault", regs, error_code,
+ X86_TRAP_GP, SIGSEGV) != NOTIFY_STOP)
+ die("general protection fault", regs, error_code);
+ goto exit;
+ }
tsk->thread.error_code = error_code;
tsk->thread.trap_nr = X86_TRAP_GP;
@@ -279,25 +302,8 @@
}
force_sig(SIGSEGV, tsk);
- return;
-
-#ifdef CONFIG_X86_32
-gp_in_vm86:
- local_irq_enable();
- handle_vm86_fault((struct kernel_vm86_regs *) regs, error_code);
- return;
-#endif
-
-gp_in_kernel:
- if (fixup_exception(regs))
- return;
-
- tsk->thread.error_code = error_code;
- tsk->thread.trap_nr = X86_TRAP_GP;
- if (notify_die(DIE_GPF, "general protection fault", regs, error_code,
- X86_TRAP_GP, SIGSEGV) == NOTIFY_STOP)
- return;
- die("general protection fault", regs, error_code);
+exit:
+ exception_exit(regs);
}
/* May run on IST stack. */
@@ -312,15 +318,16 @@
ftrace_int3_handler(regs))
return;
#endif
+ exception_enter(regs);
#ifdef CONFIG_KGDB_LOW_LEVEL_TRAP
if (kgdb_ll_trap(DIE_INT3, "int3", regs, error_code, X86_TRAP_BP,
SIGTRAP) == NOTIFY_STOP)
- return;
+ goto exit;
#endif /* CONFIG_KGDB_LOW_LEVEL_TRAP */
if (notify_die(DIE_INT3, "int3", regs, error_code, X86_TRAP_BP,
SIGTRAP) == NOTIFY_STOP)
- return;
+ goto exit;
/*
* Let others (NMI) know that the debug stack is in use
@@ -331,6 +338,8 @@
do_trap(X86_TRAP_BP, SIGTRAP, "int3", regs, error_code, NULL);
preempt_conditional_cli(regs);
debug_stack_usage_dec();
+exit:
+ exception_exit(regs);
}
#ifdef CONFIG_X86_64
@@ -391,6 +400,8 @@
unsigned long dr6;
int si_code;
+ exception_enter(regs);
+
get_debugreg(dr6, 6);
/* Filter out all the reserved bits which are preset to 1 */
@@ -406,7 +417,7 @@
/* Catch kmemcheck conditions first of all! */
if ((dr6 & DR_STEP) && kmemcheck_trap(regs))
- return;
+ goto exit;
/* DR6 may or may not be cleared by the CPU */
set_debugreg(0, 6);
@@ -421,7 +432,7 @@
if (notify_die(DIE_DEBUG, "debug", regs, PTR_ERR(&dr6), error_code,
SIGTRAP) == NOTIFY_STOP)
- return;
+ goto exit;
/*
* Let others (NMI) know that the debug stack is in use
@@ -437,7 +448,7 @@
X86_TRAP_DB);
preempt_conditional_cli(regs);
debug_stack_usage_dec();
- return;
+ goto exit;
}
/*
@@ -458,7 +469,8 @@
preempt_conditional_cli(regs);
debug_stack_usage_dec();
- return;
+exit:
+ exception_exit(regs);
}
/*
@@ -555,14 +567,17 @@
#ifdef CONFIG_X86_32
ignore_fpu_irq = 1;
#endif
-
+ exception_enter(regs);
math_error(regs, error_code, X86_TRAP_MF);
+ exception_exit(regs);
}
dotraplinkage void
do_simd_coprocessor_error(struct pt_regs *regs, long error_code)
{
+ exception_enter(regs);
math_error(regs, error_code, X86_TRAP_XF);
+ exception_exit(regs);
}
dotraplinkage void
@@ -613,11 +628,12 @@
}
__thread_fpu_begin(tsk);
+
/*
* Paranoid restore. send a SIGSEGV if we fail to restore the state.
*/
if (unlikely(restore_fpu_checking(tsk))) {
- __thread_fpu_end(tsk);
+ drop_init_fpu(tsk);
force_sig(SIGSEGV, tsk);
return;
}
@@ -629,6 +645,9 @@
dotraplinkage void __kprobes
do_device_not_available(struct pt_regs *regs, long error_code)
{
+ exception_enter(regs);
+ BUG_ON(use_eager_fpu());
+
#ifdef CONFIG_MATH_EMULATION
if (read_cr0() & X86_CR0_EM) {
struct math_emu_info info = { };
@@ -637,6 +656,7 @@
info.regs = regs;
math_emulate(&info);
+ exception_exit(regs);
return;
}
#endif
@@ -644,12 +664,15 @@
#ifdef CONFIG_X86_32
conditional_sti(regs);
#endif
+ exception_exit(regs);
}
#ifdef CONFIG_X86_32
dotraplinkage void do_iret_error(struct pt_regs *regs, long error_code)
{
siginfo_t info;
+
+ exception_enter(regs);
local_irq_enable();
info.si_signo = SIGILL;
@@ -657,10 +680,11 @@
info.si_code = ILL_BADSTK;
info.si_addr = NULL;
if (notify_die(DIE_TRAP, "iret exception", regs, error_code,
- X86_TRAP_IRET, SIGILL) == NOTIFY_STOP)
- return;
- do_trap(X86_TRAP_IRET, SIGILL, "iret exception", regs, error_code,
- &info);
+ X86_TRAP_IRET, SIGILL) != NOTIFY_STOP) {
+ do_trap(X86_TRAP_IRET, SIGILL, "iret exception", regs, error_code,
+ &info);
+ }
+ exception_exit(regs);
}
#endif
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 36fd420..9538f00 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -41,6 +41,9 @@
/* Adjust the return address of a call insn */
#define UPROBE_FIX_CALL 0x2
+/* Instruction will modify TF, don't change it */
+#define UPROBE_FIX_SETF 0x4
+
#define UPROBE_FIX_RIP_AX 0x8000
#define UPROBE_FIX_RIP_CX 0x4000
@@ -239,6 +242,10 @@
insn_get_opcode(insn); /* should be a nop */
switch (OPCODE1(insn)) {
+ case 0x9d:
+ /* popf */
+ auprobe->fixups |= UPROBE_FIX_SETF;
+ break;
case 0xc3: /* ret/lret */
case 0xcb:
case 0xc2:
@@ -646,7 +653,7 @@
* Skip these instructions as per the currently known x86 ISA.
* 0x66* { 0x90 | 0x0f 0x1f | 0x0f 0x19 | 0x87 0xc0 }
*/
-bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
+static bool __skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
{
int i;
@@ -673,3 +680,46 @@
}
return false;
}
+
+bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
+{
+ bool ret = __skip_sstep(auprobe, regs);
+ if (ret && (regs->flags & X86_EFLAGS_TF))
+ send_sig(SIGTRAP, current, 0);
+ return ret;
+}
+
+void arch_uprobe_enable_step(struct arch_uprobe *auprobe)
+{
+ struct task_struct *task = current;
+ struct arch_uprobe_task *autask = &task->utask->autask;
+ struct pt_regs *regs = task_pt_regs(task);
+
+ autask->saved_tf = !!(regs->flags & X86_EFLAGS_TF);
+
+ regs->flags |= X86_EFLAGS_TF;
+ if (test_tsk_thread_flag(task, TIF_BLOCKSTEP))
+ set_task_blockstep(task, false);
+}
+
+void arch_uprobe_disable_step(struct arch_uprobe *auprobe)
+{
+ struct task_struct *task = current;
+ struct arch_uprobe_task *autask = &task->utask->autask;
+ bool trapped = (task->utask->state == UTASK_SSTEP_TRAPPED);
+ struct pt_regs *regs = task_pt_regs(task);
+ /*
+ * The state of TIF_BLOCKSTEP was not saved so we can get an extra
+ * SIGTRAP if we do not clear TF. We need to examine the opcode to
+ * make it right.
+ */
+ if (unlikely(trapped)) {
+ if (!autask->saved_tf)
+ regs->flags &= ~X86_EFLAGS_TF;
+ } else {
+ if (autask->saved_tf)
+ send_sig(SIGTRAP, task, 0);
+ else if (!(auprobe->fixups & UPROBE_FIX_SETF))
+ regs->flags &= ~X86_EFLAGS_TF;
+ }
+}
diff --git a/arch/x86/kernel/x8664_ksyms_64.c b/arch/x86/kernel/x8664_ksyms_64.c
index 6020f6f..1330dd1 100644
--- a/arch/x86/kernel/x8664_ksyms_64.c
+++ b/arch/x86/kernel/x8664_ksyms_64.c
@@ -13,9 +13,13 @@
#include <asm/ftrace.h>
#ifdef CONFIG_FUNCTION_TRACER
-/* mcount is defined in assembly */
+/* mcount and __fentry__ are defined in assembly */
+#ifdef CC_USING_FENTRY
+EXPORT_SYMBOL(__fentry__);
+#else
EXPORT_SYMBOL(mcount);
#endif
+#endif
EXPORT_SYMBOL(__get_user_1);
EXPORT_SYMBOL(__get_user_2);
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 9f3167e..7a3d075 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -26,7 +26,6 @@
void __cpuinit x86_init_noop(void) { }
void __init x86_init_uint_noop(unsigned int unused) { }
-void __init x86_init_pgd_noop(pgd_t *unused) { }
int __init iommu_init_noop(void) { return 0; }
void iommu_shutdown_noop(void) { }
@@ -68,8 +67,7 @@
},
.paging = {
- .pagetable_setup_start = native_pagetable_setup_start,
- .pagetable_setup_done = native_pagetable_setup_done,
+ .pagetable_init = native_pagetable_init,
},
.timers = {
diff --git a/arch/x86/kernel/xsave.c b/arch/x86/kernel/xsave.c
index 3d3e207..4e89b3d 100644
--- a/arch/x86/kernel/xsave.c
+++ b/arch/x86/kernel/xsave.c
@@ -10,9 +10,7 @@
#include <linux/compat.h>
#include <asm/i387.h>
#include <asm/fpu-internal.h>
-#ifdef CONFIG_IA32_EMULATION
-#include <asm/sigcontext32.h>
-#endif
+#include <asm/sigframe.h>
#include <asm/xcr.h>
/*
@@ -23,13 +21,9 @@
/*
* Represents init state for the supported extended state.
*/
-static struct xsave_struct *init_xstate_buf;
+struct xsave_struct *init_xstate_buf;
-struct _fpx_sw_bytes fx_sw_reserved;
-#ifdef CONFIG_IA32_EMULATION
-struct _fpx_sw_bytes fx_sw_reserved_ia32;
-#endif
-
+static struct _fpx_sw_bytes fx_sw_reserved, fx_sw_reserved_ia32;
static unsigned int *xstate_offsets, *xstate_sizes, xstate_features;
/*
@@ -44,9 +38,9 @@
*/
void __sanitize_i387_state(struct task_struct *tsk)
{
- u64 xstate_bv;
- int feature_bit = 0x2;
struct i387_fxsave_struct *fx = &tsk->thread.fpu.state->fxsave;
+ int feature_bit = 0x2;
+ u64 xstate_bv;
if (!fx)
return;
@@ -104,213 +98,326 @@
* Check for the presence of extended state information in the
* user fpstate pointer in the sigcontext.
*/
-int check_for_xstate(struct i387_fxsave_struct __user *buf,
- void __user *fpstate,
- struct _fpx_sw_bytes *fx_sw_user)
+static inline int check_for_xstate(struct i387_fxsave_struct __user *buf,
+ void __user *fpstate,
+ struct _fpx_sw_bytes *fx_sw)
{
int min_xstate_size = sizeof(struct i387_fxsave_struct) +
sizeof(struct xsave_hdr_struct);
unsigned int magic2;
- int err;
- err = __copy_from_user(fx_sw_user, &buf->sw_reserved[0],
- sizeof(struct _fpx_sw_bytes));
- if (err)
- return -EFAULT;
+ if (__copy_from_user(fx_sw, &buf->sw_reserved[0], sizeof(*fx_sw)))
+ return -1;
- /*
- * First Magic check failed.
- */
- if (fx_sw_user->magic1 != FP_XSTATE_MAGIC1)
- return -EINVAL;
+ /* Check for the first magic field and other error scenarios. */
+ if (fx_sw->magic1 != FP_XSTATE_MAGIC1 ||
+ fx_sw->xstate_size < min_xstate_size ||
+ fx_sw->xstate_size > xstate_size ||
+ fx_sw->xstate_size > fx_sw->extended_size)
+ return -1;
/*
- * Check for error scenarios.
- */
- if (fx_sw_user->xstate_size < min_xstate_size ||
- fx_sw_user->xstate_size > xstate_size ||
- fx_sw_user->xstate_size > fx_sw_user->extended_size)
- return -EINVAL;
-
- err = __get_user(magic2, (__u32 *) (((void *)fpstate) +
- fx_sw_user->extended_size -
- FP_XSTATE_MAGIC2_SIZE));
- if (err)
- return err;
- /*
* Check for the presence of second magic word at the end of memory
* layout. This detects the case where the user just copied the legacy
* fpstate layout with out copying the extended state information
* in the memory layout.
*/
- if (magic2 != FP_XSTATE_MAGIC2)
- return -EFAULT;
+ if (__get_user(magic2, (__u32 __user *)(fpstate + fx_sw->xstate_size))
+ || magic2 != FP_XSTATE_MAGIC2)
+ return -1;
return 0;
}
-#ifdef CONFIG_X86_64
/*
* Signal frame handlers.
*/
-
-int save_i387_xstate(void __user *buf)
+static inline int save_fsave_header(struct task_struct *tsk, void __user *buf)
{
- struct task_struct *tsk = current;
- int err = 0;
+ if (use_fxsr()) {
+ struct xsave_struct *xsave = &tsk->thread.fpu.state->xsave;
+ struct user_i387_ia32_struct env;
+ struct _fpstate_ia32 __user *fp = buf;
- if (!access_ok(VERIFY_WRITE, buf, sig_xstate_size))
- return -EACCES;
+ convert_from_fxsr(&env, tsk);
- BUG_ON(sig_xstate_size < xstate_size);
-
- if ((unsigned long)buf % 64)
- pr_err("%s: bad fpstate %p\n", __func__, buf);
-
- if (!used_math())
- return 0;
-
- if (user_has_fpu()) {
- if (use_xsave())
- err = xsave_user(buf);
- else
- err = fxsave_user(buf);
-
- if (err)
- return err;
- user_fpu_end();
+ if (__copy_to_user(buf, &env, sizeof(env)) ||
+ __put_user(xsave->i387.swd, &fp->status) ||
+ __put_user(X86_FXSR_MAGIC, &fp->magic))
+ return -1;
} else {
- sanitize_i387_state(tsk);
- if (__copy_to_user(buf, &tsk->thread.fpu.state->fxsave,
- xstate_size))
+ struct i387_fsave_struct __user *fp = buf;
+ u32 swd;
+ if (__get_user(swd, &fp->swd) || __put_user(swd, &fp->status))
return -1;
}
- clear_used_math(); /* trigger finit */
-
- if (use_xsave()) {
- struct _fpstate __user *fx = buf;
- struct _xstate __user *x = buf;
- u64 xstate_bv;
-
- err = __copy_to_user(&fx->sw_reserved, &fx_sw_reserved,
- sizeof(struct _fpx_sw_bytes));
-
- err |= __put_user(FP_XSTATE_MAGIC2,
- (__u32 __user *) (buf + sig_xstate_size
- - FP_XSTATE_MAGIC2_SIZE));
-
- /*
- * Read the xstate_bv which we copied (directly from the cpu or
- * from the state in task struct) to the user buffers and
- * set the FP/SSE bits.
- */
- err |= __get_user(xstate_bv, &x->xstate_hdr.xstate_bv);
-
- /*
- * For legacy compatible, we always set FP/SSE bits in the bit
- * vector while saving the state to the user context. This will
- * enable us capturing any changes(during sigreturn) to
- * the FP/SSE bits by the legacy applications which don't touch
- * xstate_bv in the xsave header.
- *
- * xsave aware apps can change the xstate_bv in the xsave
- * header as well as change any contents in the memory layout.
- * xrestore as part of sigreturn will capture all the changes.
- */
- xstate_bv |= XSTATE_FPSSE;
-
- err |= __put_user(xstate_bv, &x->xstate_hdr.xstate_bv);
-
- if (err)
- return err;
- }
-
- return 1;
+ return 0;
}
-/*
- * Restore the extended state if present. Otherwise, restore the FP/SSE
- * state.
- */
-static int restore_user_xstate(void __user *buf)
+static inline int save_xstate_epilog(void __user *buf, int ia32_frame)
{
- struct _fpx_sw_bytes fx_sw_user;
- u64 mask;
+ struct xsave_struct __user *x = buf;
+ struct _fpx_sw_bytes *sw_bytes;
+ u32 xstate_bv;
int err;
- if (((unsigned long)buf % 64) ||
- check_for_xstate(buf, buf, &fx_sw_user))
- goto fx_only;
+ /* Setup the bytes not touched by the [f]xsave and reserved for SW. */
+ sw_bytes = ia32_frame ? &fx_sw_reserved_ia32 : &fx_sw_reserved;
+ err = __copy_to_user(&x->i387.sw_reserved, sw_bytes, sizeof(*sw_bytes));
- mask = fx_sw_user.xstate_bv;
-
- /*
- * restore the state passed by the user.
- */
- err = xrestore_user(buf, mask);
- if (err)
+ if (!use_xsave())
return err;
- /*
- * init the state skipped by the user.
- */
- mask = pcntxt_mask & ~mask;
- if (unlikely(mask))
- xrstor_state(init_xstate_buf, mask);
+ err |= __put_user(FP_XSTATE_MAGIC2, (__u32 *)(buf + xstate_size));
- return 0;
-
-fx_only:
/*
- * couldn't find the extended state information in the
- * memory layout. Restore just the FP/SSE and init all
- * the other extended state.
+ * Read the xstate_bv which we copied (directly from the cpu or
+ * from the state in task struct) to the user buffers.
*/
- xrstor_state(init_xstate_buf, pcntxt_mask & ~XSTATE_FPSSE);
- return fxrstor_checking((__force struct i387_fxsave_struct *)buf);
+ err |= __get_user(xstate_bv, (__u32 *)&x->xsave_hdr.xstate_bv);
+
+ /*
+ * For legacy compatible, we always set FP/SSE bits in the bit
+ * vector while saving the state to the user context. This will
+ * enable us capturing any changes(during sigreturn) to
+ * the FP/SSE bits by the legacy applications which don't touch
+ * xstate_bv in the xsave header.
+ *
+ * xsave aware apps can change the xstate_bv in the xsave
+ * header as well as change any contents in the memory layout.
+ * xrestore as part of sigreturn will capture all the changes.
+ */
+ xstate_bv |= XSTATE_FPSSE;
+
+ err |= __put_user(xstate_bv, (__u32 *)&x->xsave_hdr.xstate_bv);
+
+ return err;
+}
+
+static inline int save_user_xstate(struct xsave_struct __user *buf)
+{
+ int err;
+
+ if (use_xsave())
+ err = xsave_user(buf);
+ else if (use_fxsr())
+ err = fxsave_user((struct i387_fxsave_struct __user *) buf);
+ else
+ err = fsave_user((struct i387_fsave_struct __user *) buf);
+
+ if (unlikely(err) && __clear_user(buf, xstate_size))
+ err = -EFAULT;
+ return err;
}
/*
- * This restores directly out of user space. Exceptions are handled.
+ * Save the fpu, extended register state to the user signal frame.
+ *
+ * 'buf_fx' is the 64-byte aligned pointer at which the [f|fx|x]save
+ * state is copied.
+ * 'buf' points to the 'buf_fx' or to the fsave header followed by 'buf_fx'.
+ *
+ * buf == buf_fx for 64-bit frames and 32-bit fsave frame.
+ * buf != buf_fx for 32-bit frames with fxstate.
+ *
+ * If the fpu, extended register state is live, save the state directly
+ * to the user frame pointed by the aligned pointer 'buf_fx'. Otherwise,
+ * copy the thread's fpu state to the user frame starting at 'buf_fx'.
+ *
+ * If this is a 32-bit frame with fxstate, put a fsave header before
+ * the aligned state at 'buf_fx'.
+ *
+ * For [f]xsave state, update the SW reserved fields in the [f]xsave frame
+ * indicating the absence/presence of the extended state to the user.
*/
-int restore_i387_xstate(void __user *buf)
+int save_xstate_sig(void __user *buf, void __user *buf_fx, int size)
{
+ struct xsave_struct *xsave = ¤t->thread.fpu.state->xsave;
struct task_struct *tsk = current;
- int err = 0;
+ int ia32_fxstate = (buf != buf_fx);
+
+ ia32_fxstate &= (config_enabled(CONFIG_X86_32) ||
+ config_enabled(CONFIG_IA32_EMULATION));
+
+ if (!access_ok(VERIFY_WRITE, buf, size))
+ return -EACCES;
+
+ if (!HAVE_HWFP)
+ return fpregs_soft_get(current, NULL, 0,
+ sizeof(struct user_i387_ia32_struct), NULL,
+ (struct _fpstate_ia32 __user *) buf) ? -1 : 1;
+
+ if (user_has_fpu()) {
+ /* Save the live register state to the user directly. */
+ if (save_user_xstate(buf_fx))
+ return -1;
+ /* Update the thread's fxstate to save the fsave header. */
+ if (ia32_fxstate)
+ fpu_fxsave(&tsk->thread.fpu);
+ } else {
+ sanitize_i387_state(tsk);
+ if (__copy_to_user(buf_fx, xsave, xstate_size))
+ return -1;
+ }
+
+ /* Save the fsave header for the 32-bit frames. */
+ if ((ia32_fxstate || !use_fxsr()) && save_fsave_header(tsk, buf))
+ return -1;
+
+ if (use_fxsr() && save_xstate_epilog(buf_fx, ia32_fxstate))
+ return -1;
+
+ drop_init_fpu(tsk); /* trigger finit */
+
+ return 0;
+}
+
+static inline void
+sanitize_restored_xstate(struct task_struct *tsk,
+ struct user_i387_ia32_struct *ia32_env,
+ u64 xstate_bv, int fx_only)
+{
+ struct xsave_struct *xsave = &tsk->thread.fpu.state->xsave;
+ struct xsave_hdr_struct *xsave_hdr = &xsave->xsave_hdr;
+
+ if (use_xsave()) {
+ /* These bits must be zero. */
+ xsave_hdr->reserved1[0] = xsave_hdr->reserved1[1] = 0;
+
+ /*
+ * Init the state that is not present in the memory
+ * layout and not enabled by the OS.
+ */
+ if (fx_only)
+ xsave_hdr->xstate_bv = XSTATE_FPSSE;
+ else
+ xsave_hdr->xstate_bv &= (pcntxt_mask & xstate_bv);
+ }
+
+ if (use_fxsr()) {
+ /*
+ * mscsr reserved bits must be masked to zero for security
+ * reasons.
+ */
+ xsave->i387.mxcsr &= mxcsr_feature_mask;
+
+ convert_to_fxsr(tsk, ia32_env);
+ }
+}
+
+/*
+ * Restore the extended state if present. Otherwise, restore the FP/SSE state.
+ */
+static inline int restore_user_xstate(void __user *buf, u64 xbv, int fx_only)
+{
+ if (use_xsave()) {
+ if ((unsigned long)buf % 64 || fx_only) {
+ u64 init_bv = pcntxt_mask & ~XSTATE_FPSSE;
+ xrstor_state(init_xstate_buf, init_bv);
+ return fxrstor_checking((__force void *) buf);
+ } else {
+ u64 init_bv = pcntxt_mask & ~xbv;
+ if (unlikely(init_bv))
+ xrstor_state(init_xstate_buf, init_bv);
+ return xrestore_user(buf, xbv);
+ }
+ } else if (use_fxsr()) {
+ return fxrstor_checking((__force void *) buf);
+ } else
+ return frstor_checking((__force void *) buf);
+}
+
+int __restore_xstate_sig(void __user *buf, void __user *buf_fx, int size)
+{
+ int ia32_fxstate = (buf != buf_fx);
+ struct task_struct *tsk = current;
+ int state_size = xstate_size;
+ u64 xstate_bv = 0;
+ int fx_only = 0;
+
+ ia32_fxstate &= (config_enabled(CONFIG_X86_32) ||
+ config_enabled(CONFIG_IA32_EMULATION));
if (!buf) {
- if (used_math())
- goto clear;
+ drop_init_fpu(tsk);
return 0;
- } else
- if (!access_ok(VERIFY_READ, buf, sig_xstate_size))
- return -EACCES;
-
- if (!used_math()) {
- err = init_fpu(tsk);
- if (err)
- return err;
}
- user_fpu_begin();
- if (use_xsave())
- err = restore_user_xstate(buf);
- else
- err = fxrstor_checking((__force struct i387_fxsave_struct *)
- buf);
- if (unlikely(err)) {
+ if (!access_ok(VERIFY_READ, buf, size))
+ return -EACCES;
+
+ if (!used_math() && init_fpu(tsk))
+ return -1;
+
+ if (!HAVE_HWFP) {
+ return fpregs_soft_set(current, NULL,
+ 0, sizeof(struct user_i387_ia32_struct),
+ NULL, buf) != 0;
+ }
+
+ if (use_xsave()) {
+ struct _fpx_sw_bytes fx_sw_user;
+ if (unlikely(check_for_xstate(buf_fx, buf_fx, &fx_sw_user))) {
+ /*
+ * Couldn't find the extended state information in the
+ * memory layout. Restore just the FP/SSE and init all
+ * the other extended state.
+ */
+ state_size = sizeof(struct i387_fxsave_struct);
+ fx_only = 1;
+ } else {
+ state_size = fx_sw_user.xstate_size;
+ xstate_bv = fx_sw_user.xstate_bv;
+ }
+ }
+
+ if (ia32_fxstate) {
/*
- * Encountered an error while doing the restore from the
- * user buffer, clear the fpu state.
+ * For 32-bit frames with fxstate, copy the user state to the
+ * thread's fpu state, reconstruct fxstate from the fsave
+ * header. Sanitize the copied state etc.
*/
-clear:
- clear_fpu(tsk);
- clear_used_math();
+ struct xsave_struct *xsave = &tsk->thread.fpu.state->xsave;
+ struct user_i387_ia32_struct env;
+ int err = 0;
+
+ /*
+ * Drop the current fpu which clears used_math(). This ensures
+ * that any context-switch during the copy of the new state,
+ * avoids the intermediate state from getting restored/saved.
+ * Thus avoiding the new restored state from getting corrupted.
+ * We will be ready to restore/save the state only after
+ * set_used_math() is again set.
+ */
+ drop_fpu(tsk);
+
+ if (__copy_from_user(xsave, buf_fx, state_size) ||
+ __copy_from_user(&env, buf, sizeof(env))) {
+ err = -1;
+ } else {
+ sanitize_restored_xstate(tsk, &env, xstate_bv, fx_only);
+ set_used_math();
+ }
+
+ if (use_eager_fpu())
+ math_state_restore();
+
+ return err;
+ } else {
+ /*
+ * For 64-bit frames and 32-bit fsave frames, restore the user
+ * state to the registers directly (with exceptions handled).
+ */
+ user_fpu_begin();
+ if (restore_user_xstate(buf_fx, xstate_bv, fx_only)) {
+ drop_init_fpu(tsk);
+ return -1;
+ }
}
- return err;
+
+ return 0;
}
-#endif
/*
* Prepare the SW reserved portion of the fxsave memory layout, indicating
@@ -321,31 +428,22 @@
*/
static void prepare_fx_sw_frame(void)
{
- int size_extended = (xstate_size - sizeof(struct i387_fxsave_struct)) +
- FP_XSTATE_MAGIC2_SIZE;
+ int fsave_header_size = sizeof(struct i387_fsave_struct);
+ int size = xstate_size + FP_XSTATE_MAGIC2_SIZE;
- sig_xstate_size = sizeof(struct _fpstate) + size_extended;
-
-#ifdef CONFIG_IA32_EMULATION
- sig_xstate_ia32_size = sizeof(struct _fpstate_ia32) + size_extended;
-#endif
-
- memset(&fx_sw_reserved, 0, sizeof(fx_sw_reserved));
+ if (config_enabled(CONFIG_X86_32))
+ size += fsave_header_size;
fx_sw_reserved.magic1 = FP_XSTATE_MAGIC1;
- fx_sw_reserved.extended_size = sig_xstate_size;
+ fx_sw_reserved.extended_size = size;
fx_sw_reserved.xstate_bv = pcntxt_mask;
fx_sw_reserved.xstate_size = xstate_size;
-#ifdef CONFIG_IA32_EMULATION
- memcpy(&fx_sw_reserved_ia32, &fx_sw_reserved,
- sizeof(struct _fpx_sw_bytes));
- fx_sw_reserved_ia32.extended_size = sig_xstate_ia32_size;
-#endif
-}
-#ifdef CONFIG_X86_64
-unsigned int sig_xstate_size = sizeof(struct _fpstate);
-#endif
+ if (config_enabled(CONFIG_IA32_EMULATION)) {
+ fx_sw_reserved_ia32 = fx_sw_reserved;
+ fx_sw_reserved_ia32.extended_size += fsave_header_size;
+ }
+}
/*
* Enable the extended processor state save/restore feature
@@ -384,19 +482,21 @@
/*
* setup the xstate image representing the init state
*/
-static void __init setup_xstate_init(void)
+static void __init setup_init_fpu_buf(void)
{
- setup_xstate_features();
-
/*
* Setup init_xstate_buf to represent the init state of
* all the features managed by the xsave
*/
init_xstate_buf = alloc_bootmem_align(xstate_size,
__alignof__(struct xsave_struct));
- init_xstate_buf->i387.mxcsr = MXCSR_DEFAULT;
+ fx_finit(&init_xstate_buf->i387);
- clts();
+ if (!cpu_has_xsave)
+ return;
+
+ setup_xstate_features();
+
/*
* Init all the features state with header_bv being 0x0
*/
@@ -406,9 +506,21 @@
* of any feature which is not represented by all zero's.
*/
xsave_state(init_xstate_buf, -1);
- stts();
}
+static enum { AUTO, ENABLE, DISABLE } eagerfpu = AUTO;
+static int __init eager_fpu_setup(char *s)
+{
+ if (!strcmp(s, "on"))
+ eagerfpu = ENABLE;
+ else if (!strcmp(s, "off"))
+ eagerfpu = DISABLE;
+ else if (!strcmp(s, "auto"))
+ eagerfpu = AUTO;
+ return 1;
+}
+__setup("eagerfpu=", eager_fpu_setup);
+
/*
* Enable and initialize the xsave feature.
*/
@@ -445,8 +557,11 @@
update_regset_xstate_info(xstate_size, pcntxt_mask);
prepare_fx_sw_frame();
+ setup_init_fpu_buf();
- setup_xstate_init();
+ /* Auto enable eagerfpu for xsaveopt */
+ if (cpu_has_xsaveopt && eagerfpu != DISABLE)
+ eagerfpu = ENABLE;
pr_info("enabled xstate_bv 0x%llx, cntxt size 0x%x\n",
pcntxt_mask, xstate_size);
@@ -471,3 +586,43 @@
next_func = xstate_enable;
this_func();
}
+
+static inline void __init eager_fpu_init_bp(void)
+{
+ current->thread.fpu.state =
+ alloc_bootmem_align(xstate_size, __alignof__(struct xsave_struct));
+ if (!init_xstate_buf)
+ setup_init_fpu_buf();
+}
+
+void __cpuinit eager_fpu_init(void)
+{
+ static __refdata void (*boot_func)(void) = eager_fpu_init_bp;
+
+ clear_used_math();
+ current_thread_info()->status = 0;
+
+ if (eagerfpu == ENABLE)
+ setup_force_cpu_cap(X86_FEATURE_EAGER_FPU);
+
+ if (!cpu_has_eager_fpu) {
+ stts();
+ return;
+ }
+
+ if (boot_func) {
+ boot_func();
+ boot_func = NULL;
+ }
+
+ /*
+ * This is same as math_state_restore(). But use_xsave() is
+ * not yet patched to use math_state_restore().
+ */
+ init_fpu(current);
+ __thread_fpu_begin(current);
+ if (cpu_has_xsave)
+ xrstor_state(init_xstate_buf, -1);
+ else
+ fxrstor_checking(&init_xstate_buf->i387);
+}
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 97d9a99..a3b57a2 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -475,13 +475,26 @@
return address_mask(ctxt, reg);
}
+static void masked_increment(ulong *reg, ulong mask, int inc)
+{
+ assign_masked(reg, *reg + inc, mask);
+}
+
static inline void
register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, int inc)
{
+ ulong mask;
+
if (ctxt->ad_bytes == sizeof(unsigned long))
- *reg += inc;
+ mask = ~0UL;
else
- *reg = (*reg & ~ad_mask(ctxt)) | ((*reg + inc) & ad_mask(ctxt));
+ mask = ad_mask(ctxt);
+ masked_increment(reg, mask, inc);
+}
+
+static void rsp_increment(struct x86_emulate_ctxt *ctxt, int inc)
+{
+ masked_increment(&ctxt->regs[VCPU_REGS_RSP], stack_mask(ctxt), inc);
}
static inline void jmp_rel(struct x86_emulate_ctxt *ctxt, int rel)
@@ -1522,8 +1535,8 @@
{
struct segmented_address addr;
- register_address_increment(ctxt, &ctxt->regs[VCPU_REGS_RSP], -bytes);
- addr.ea = register_address(ctxt, ctxt->regs[VCPU_REGS_RSP]);
+ rsp_increment(ctxt, -bytes);
+ addr.ea = ctxt->regs[VCPU_REGS_RSP] & stack_mask(ctxt);
addr.seg = VCPU_SREG_SS;
return segmented_write(ctxt, addr, data, bytes);
@@ -1542,13 +1555,13 @@
int rc;
struct segmented_address addr;
- addr.ea = register_address(ctxt, ctxt->regs[VCPU_REGS_RSP]);
+ addr.ea = ctxt->regs[VCPU_REGS_RSP] & stack_mask(ctxt);
addr.seg = VCPU_SREG_SS;
rc = segmented_read(ctxt, addr, dest, len);
if (rc != X86EMUL_CONTINUE)
return rc;
- register_address_increment(ctxt, &ctxt->regs[VCPU_REGS_RSP], len);
+ rsp_increment(ctxt, len);
return rc;
}
@@ -1688,8 +1701,7 @@
while (reg >= VCPU_REGS_RAX) {
if (reg == VCPU_REGS_RSP) {
- register_address_increment(ctxt, &ctxt->regs[VCPU_REGS_RSP],
- ctxt->op_bytes);
+ rsp_increment(ctxt, ctxt->op_bytes);
--reg;
}
@@ -2825,7 +2837,7 @@
rc = emulate_pop(ctxt, &ctxt->dst.val, ctxt->op_bytes);
if (rc != X86EMUL_CONTINUE)
return rc;
- register_address_increment(ctxt, &ctxt->regs[VCPU_REGS_RSP], ctxt->src.val);
+ rsp_increment(ctxt, ctxt->src.val);
return X86EMUL_CONTINUE;
}
diff --git a/arch/x86/kvm/i8259.c b/arch/x86/kvm/i8259.c
index e498b18..9fc9aa7 100644
--- a/arch/x86/kvm/i8259.c
+++ b/arch/x86/kvm/i8259.c
@@ -318,7 +318,7 @@
if (val & 0x10) {
u8 edge_irr = s->irr & ~s->elcr;
int i;
- bool found;
+ bool found = false;
struct kvm_vcpu *vcpu;
s->init4 = val & 1;
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 01ca004..7fbd0d2 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -4113,16 +4113,21 @@
LIST_HEAD(invalid_list);
/*
+ * Never scan more than sc->nr_to_scan VM instances.
+ * Will not hit this condition practically since we do not try
+ * to shrink more than one VM and it is very unlikely to see
+ * !n_used_mmu_pages so many times.
+ */
+ if (!nr_to_scan--)
+ break;
+ /*
* n_used_mmu_pages is accessed without holding kvm->mmu_lock
* here. We may skip a VM instance errorneosly, but we do not
* want to shrink a VM that only started to populate its MMU
* anyway.
*/
- if (kvm->arch.n_used_mmu_pages > 0) {
- if (!nr_to_scan--)
- break;
+ if (!kvm->arch.n_used_mmu_pages)
continue;
- }
idx = srcu_read_lock(&kvm->srcu);
spin_lock(&kvm->mmu_lock);
diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index a71faf7..bca63f0 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -183,95 +183,6 @@
#define KVM_ISA_VMX 1
#define KVM_ISA_SVM 2
-#define VMX_EXIT_REASONS \
- { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
- { EXIT_REASON_EXTERNAL_INTERRUPT, "EXTERNAL_INTERRUPT" }, \
- { EXIT_REASON_TRIPLE_FAULT, "TRIPLE_FAULT" }, \
- { EXIT_REASON_PENDING_INTERRUPT, "PENDING_INTERRUPT" }, \
- { EXIT_REASON_NMI_WINDOW, "NMI_WINDOW" }, \
- { EXIT_REASON_TASK_SWITCH, "TASK_SWITCH" }, \
- { EXIT_REASON_CPUID, "CPUID" }, \
- { EXIT_REASON_HLT, "HLT" }, \
- { EXIT_REASON_INVLPG, "INVLPG" }, \
- { EXIT_REASON_RDPMC, "RDPMC" }, \
- { EXIT_REASON_RDTSC, "RDTSC" }, \
- { EXIT_REASON_VMCALL, "VMCALL" }, \
- { EXIT_REASON_VMCLEAR, "VMCLEAR" }, \
- { EXIT_REASON_VMLAUNCH, "VMLAUNCH" }, \
- { EXIT_REASON_VMPTRLD, "VMPTRLD" }, \
- { EXIT_REASON_VMPTRST, "VMPTRST" }, \
- { EXIT_REASON_VMREAD, "VMREAD" }, \
- { EXIT_REASON_VMRESUME, "VMRESUME" }, \
- { EXIT_REASON_VMWRITE, "VMWRITE" }, \
- { EXIT_REASON_VMOFF, "VMOFF" }, \
- { EXIT_REASON_VMON, "VMON" }, \
- { EXIT_REASON_CR_ACCESS, "CR_ACCESS" }, \
- { EXIT_REASON_DR_ACCESS, "DR_ACCESS" }, \
- { EXIT_REASON_IO_INSTRUCTION, "IO_INSTRUCTION" }, \
- { EXIT_REASON_MSR_READ, "MSR_READ" }, \
- { EXIT_REASON_MSR_WRITE, "MSR_WRITE" }, \
- { EXIT_REASON_MWAIT_INSTRUCTION, "MWAIT_INSTRUCTION" }, \
- { EXIT_REASON_MONITOR_INSTRUCTION, "MONITOR_INSTRUCTION" }, \
- { EXIT_REASON_PAUSE_INSTRUCTION, "PAUSE_INSTRUCTION" }, \
- { EXIT_REASON_MCE_DURING_VMENTRY, "MCE_DURING_VMENTRY" }, \
- { EXIT_REASON_TPR_BELOW_THRESHOLD, "TPR_BELOW_THRESHOLD" }, \
- { EXIT_REASON_APIC_ACCESS, "APIC_ACCESS" }, \
- { EXIT_REASON_EPT_VIOLATION, "EPT_VIOLATION" }, \
- { EXIT_REASON_EPT_MISCONFIG, "EPT_MISCONFIG" }, \
- { EXIT_REASON_WBINVD, "WBINVD" }
-
-#define SVM_EXIT_REASONS \
- { SVM_EXIT_READ_CR0, "read_cr0" }, \
- { SVM_EXIT_READ_CR3, "read_cr3" }, \
- { SVM_EXIT_READ_CR4, "read_cr4" }, \
- { SVM_EXIT_READ_CR8, "read_cr8" }, \
- { SVM_EXIT_WRITE_CR0, "write_cr0" }, \
- { SVM_EXIT_WRITE_CR3, "write_cr3" }, \
- { SVM_EXIT_WRITE_CR4, "write_cr4" }, \
- { SVM_EXIT_WRITE_CR8, "write_cr8" }, \
- { SVM_EXIT_READ_DR0, "read_dr0" }, \
- { SVM_EXIT_READ_DR1, "read_dr1" }, \
- { SVM_EXIT_READ_DR2, "read_dr2" }, \
- { SVM_EXIT_READ_DR3, "read_dr3" }, \
- { SVM_EXIT_WRITE_DR0, "write_dr0" }, \
- { SVM_EXIT_WRITE_DR1, "write_dr1" }, \
- { SVM_EXIT_WRITE_DR2, "write_dr2" }, \
- { SVM_EXIT_WRITE_DR3, "write_dr3" }, \
- { SVM_EXIT_WRITE_DR5, "write_dr5" }, \
- { SVM_EXIT_WRITE_DR7, "write_dr7" }, \
- { SVM_EXIT_EXCP_BASE + DB_VECTOR, "DB excp" }, \
- { SVM_EXIT_EXCP_BASE + BP_VECTOR, "BP excp" }, \
- { SVM_EXIT_EXCP_BASE + UD_VECTOR, "UD excp" }, \
- { SVM_EXIT_EXCP_BASE + PF_VECTOR, "PF excp" }, \
- { SVM_EXIT_EXCP_BASE + NM_VECTOR, "NM excp" }, \
- { SVM_EXIT_EXCP_BASE + MC_VECTOR, "MC excp" }, \
- { SVM_EXIT_INTR, "interrupt" }, \
- { SVM_EXIT_NMI, "nmi" }, \
- { SVM_EXIT_SMI, "smi" }, \
- { SVM_EXIT_INIT, "init" }, \
- { SVM_EXIT_VINTR, "vintr" }, \
- { SVM_EXIT_CPUID, "cpuid" }, \
- { SVM_EXIT_INVD, "invd" }, \
- { SVM_EXIT_HLT, "hlt" }, \
- { SVM_EXIT_INVLPG, "invlpg" }, \
- { SVM_EXIT_INVLPGA, "invlpga" }, \
- { SVM_EXIT_IOIO, "io" }, \
- { SVM_EXIT_MSR, "msr" }, \
- { SVM_EXIT_TASK_SWITCH, "task_switch" }, \
- { SVM_EXIT_SHUTDOWN, "shutdown" }, \
- { SVM_EXIT_VMRUN, "vmrun" }, \
- { SVM_EXIT_VMMCALL, "hypercall" }, \
- { SVM_EXIT_VMLOAD, "vmload" }, \
- { SVM_EXIT_VMSAVE, "vmsave" }, \
- { SVM_EXIT_STGI, "stgi" }, \
- { SVM_EXIT_CLGI, "clgi" }, \
- { SVM_EXIT_SKINIT, "skinit" }, \
- { SVM_EXIT_WBINVD, "wbinvd" }, \
- { SVM_EXIT_MONITOR, "monitor" }, \
- { SVM_EXIT_MWAIT, "mwait" }, \
- { SVM_EXIT_XSETBV, "xsetbv" }, \
- { SVM_EXIT_NPF, "npf" }
-
/*
* Tracepoint for kvm guest exit:
*/
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index c00f03d..851aa7c 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -1493,8 +1493,12 @@
#ifdef CONFIG_X86_64
wrmsrl(MSR_KERNEL_GS_BASE, vmx->msr_host_kernel_gs_base);
#endif
- if (user_has_fpu())
- clts();
+ /*
+ * If the FPU is not active (through the host task or
+ * the guest vcpu), then restore the cr0.TS bit.
+ */
+ if (!user_has_fpu() && !vmx->vcpu.guest_fpu_loaded)
+ stts();
load_gdt(&__get_cpu_var(host_gdt));
}
@@ -3619,6 +3623,7 @@
static int alloc_apic_access_page(struct kvm *kvm)
{
+ struct page *page;
struct kvm_userspace_memory_region kvm_userspace_mem;
int r = 0;
@@ -3633,7 +3638,13 @@
if (r)
goto out;
- kvm->arch.apic_access_page = gfn_to_page(kvm, 0xfee00);
+ page = gfn_to_page(kvm, 0xfee00);
+ if (is_error_page(page)) {
+ r = -EFAULT;
+ goto out;
+ }
+
+ kvm->arch.apic_access_page = page;
out:
mutex_unlock(&kvm->slots_lock);
return r;
@@ -3641,6 +3652,7 @@
static int alloc_identity_pagetable(struct kvm *kvm)
{
+ struct page *page;
struct kvm_userspace_memory_region kvm_userspace_mem;
int r = 0;
@@ -3656,8 +3668,13 @@
if (r)
goto out;
- kvm->arch.ept_identity_pagetable = gfn_to_page(kvm,
- kvm->arch.ept_identity_map_addr >> PAGE_SHIFT);
+ page = gfn_to_page(kvm, kvm->arch.ept_identity_map_addr >> PAGE_SHIFT);
+ if (is_error_page(page)) {
+ r = -EFAULT;
+ goto out;
+ }
+
+ kvm->arch.ept_identity_pagetable = page;
out:
mutex_unlock(&kvm->slots_lock);
return r;
@@ -3730,7 +3747,7 @@
unsigned long tmpl;
struct desc_ptr dt;
- vmcs_writel(HOST_CR0, read_cr0() | X86_CR0_TS); /* 22.2.3 */
+ vmcs_writel(HOST_CR0, read_cr0() & ~X86_CR0_TS); /* 22.2.3 */
vmcs_writel(HOST_CR4, read_cr4()); /* 22.2.3, 22.2.5 */
vmcs_writel(HOST_CR3, read_cr3()); /* 22.2.3 FIXME: shadow tables */
@@ -4530,7 +4547,7 @@
vcpu->run->exit_reason = KVM_EXIT_SET_TPR;
return 0;
}
- };
+ }
break;
case 2: /* clts */
handle_clts(vcpu);
@@ -6575,7 +6592,7 @@
/* Exposing INVPCID only when PCID is exposed */
best = kvm_find_cpuid_entry(vcpu, 0x7, 0);
if (vmx_invpcid_supported() &&
- best && (best->ecx & bit(X86_FEATURE_INVPCID)) &&
+ best && (best->ebx & bit(X86_FEATURE_INVPCID)) &&
guest_cpuid_has_pcid(vcpu)) {
exec_control |= SECONDARY_EXEC_ENABLE_INVPCID;
vmcs_write32(SECONDARY_VM_EXEC_CONTROL,
@@ -6585,7 +6602,7 @@
vmcs_write32(SECONDARY_VM_EXEC_CONTROL,
exec_control);
if (best)
- best->ecx &= ~bit(X86_FEATURE_INVPCID);
+ best->ebx &= ~bit(X86_FEATURE_INVPCID);
}
}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 42bce48..1f09552 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -806,7 +806,7 @@
* kvm-specific. Those are put in the beginning of the list.
*/
-#define KVM_SAVE_MSRS_BEGIN 9
+#define KVM_SAVE_MSRS_BEGIN 10
static u32 msrs_to_save[] = {
MSR_KVM_SYSTEM_TIME, MSR_KVM_WALL_CLOCK,
MSR_KVM_SYSTEM_TIME_NEW, MSR_KVM_WALL_CLOCK_NEW,
@@ -2000,6 +2000,9 @@
case MSR_KVM_STEAL_TIME:
data = vcpu->arch.st.msr_val;
break;
+ case MSR_KVM_PV_EOI_EN:
+ data = vcpu->arch.pv_eoi.msr_val;
+ break;
case MSR_IA32_P5_MC_ADDR:
case MSR_IA32_P5_MC_TYPE:
case MSR_IA32_MCG_CAP:
@@ -5110,17 +5113,20 @@
!kvm_event_needs_reinjection(vcpu);
}
-static void vapic_enter(struct kvm_vcpu *vcpu)
+static int vapic_enter(struct kvm_vcpu *vcpu)
{
struct kvm_lapic *apic = vcpu->arch.apic;
struct page *page;
if (!apic || !apic->vapic_addr)
- return;
+ return 0;
page = gfn_to_page(vcpu->kvm, apic->vapic_addr >> PAGE_SHIFT);
+ if (is_error_page(page))
+ return -EFAULT;
vcpu->arch.apic->vapic_page = page;
+ return 0;
}
static void vapic_exit(struct kvm_vcpu *vcpu)
@@ -5427,7 +5433,11 @@
}
vcpu->srcu_idx = srcu_read_lock(&kvm->srcu);
- vapic_enter(vcpu);
+ r = vapic_enter(vcpu);
+ if (r) {
+ srcu_read_unlock(&kvm->srcu, vcpu->srcu_idx);
+ return r;
+ }
r = 1;
while (r > 0) {
@@ -5969,7 +5979,7 @@
*/
kvm_put_guest_xcr0(vcpu);
vcpu->guest_fpu_loaded = 1;
- unlazy_fpu(current);
+ __kernel_fpu_begin();
fpu_restore_checking(&vcpu->arch.guest_fpu);
trace_kvm_fpu(1);
}
@@ -5983,6 +5993,7 @@
vcpu->guest_fpu_loaded = 0;
fpu_save_init(&vcpu->arch.guest_fpu);
+ __kernel_fpu_end();
++vcpu->stat.fpu_reload;
kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
trace_kvm_fpu(0);
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 76dcd9d..7dde46d 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -18,6 +18,7 @@
#include <asm/pgalloc.h> /* pgd_*(), ... */
#include <asm/kmemcheck.h> /* kmemcheck_*(), ... */
#include <asm/fixmap.h> /* VSYSCALL_START */
+#include <asm/rcu.h> /* exception_enter(), ... */
/*
* Page fault error code bits:
@@ -1000,8 +1001,8 @@
* and the problem, and then passes it off to one of the appropriate
* routines.
*/
-dotraplinkage void __kprobes
-do_page_fault(struct pt_regs *regs, unsigned long error_code)
+static void __kprobes
+__do_page_fault(struct pt_regs *regs, unsigned long error_code)
{
struct vm_area_struct *vma;
struct task_struct *tsk;
@@ -1209,3 +1210,11 @@
up_read(&mm->mmap_sem);
}
+
+dotraplinkage void __kprobes
+do_page_fault(struct pt_regs *regs, unsigned long error_code)
+{
+ exception_enter(regs);
+ __do_page_fault(regs, error_code);
+ exception_exit(regs);
+}
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index e0e6990..ab1f6a9 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -319,7 +319,7 @@
*/
int devmem_is_allowed(unsigned long pagenr)
{
- if (pagenr <= 256)
+ if (pagenr < 256)
return 1;
if (iomem_is_exclusive(pagenr << PAGE_SHIFT))
return 0;
diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
index 575d86f..4f04db1 100644
--- a/arch/x86/mm/init_32.c
+++ b/arch/x86/mm/init_32.c
@@ -445,10 +445,10 @@
}
#endif /* CONFIG_HIGHMEM */
-void __init native_pagetable_setup_start(pgd_t *base)
+void __init native_pagetable_init(void)
{
unsigned long pfn, va;
- pgd_t *pgd;
+ pgd_t *pgd, *base = swapper_pg_dir;
pud_t *pud;
pmd_t *pmd;
pte_t *pte;
@@ -475,10 +475,7 @@
pte_clear(NULL, va, pte);
}
paravirt_alloc_pmd(&init_mm, __pa(base) >> PAGE_SHIFT);
-}
-
-void __init native_pagetable_setup_done(pgd_t *base)
-{
+ paging_init();
}
/*
@@ -493,7 +490,7 @@
* If we're booting paravirtualized under a hypervisor, then there are
* more options: we may already be running PAE, and the pagetable may
* or may not be based in swapper_pg_dir. In any case,
- * paravirt_pagetable_setup_start() will set up swapper_pg_dir
+ * paravirt_pagetable_init() will set up swapper_pg_dir
* appropriately for the rest of the initialization to work.
*
* In general, pagetable_init() assumes that the pagetable may already
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 613cd83..0777f04 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -98,6 +98,8 @@
{
struct flush_tlb_info *f = info;
+ inc_irq_stat(irq_tlb_count);
+
if (f->flush_mm != this_cpu_read(cpu_tlbstate.active_mm))
return;
@@ -320,7 +322,7 @@
if (kstrtos8(buf, 0, &shift))
return -EINVAL;
- if (shift > 64)
+ if (shift < -1 || shift >= BITS_PER_LONG)
return -EINVAL;
tlb_flushall_shift = shift;
diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
index 505acdd..192397c 100644
--- a/arch/x86/pci/acpi.c
+++ b/arch/x86/pci/acpi.c
@@ -305,7 +305,6 @@
res->flags = flags;
res->start = start;
res->end = end;
- res->child = NULL;
if (!pci_use_crs) {
dev_printk(KERN_DEBUG, &info->bridge->dev,
@@ -434,7 +433,7 @@
size = sizeof(*info->res) * info->res_num;
info->res_num = 0;
- info->res = kmalloc(size, GFP_KERNEL);
+ info->res = kzalloc(size, GFP_KERNEL);
if (!info->res)
return;
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 937bcec..704b9ec 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -585,7 +585,7 @@
while (i >= sizeof(struct acpi_mcfg_allocation)) {
entries++;
i -= sizeof(struct acpi_mcfg_allocation);
- };
+ }
if (entries == 0) {
pr_err(PREFIX "MMCONFIG has no entries\n");
return -ENODEV;
diff --git a/arch/x86/pci/visws.c b/arch/x86/pci/visws.c
index 6f2f8ee..3e6d2a6 100644
--- a/arch/x86/pci/visws.c
+++ b/arch/x86/pci/visws.c
@@ -62,11 +62,6 @@
return irq;
}
-void __init pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
int __init pci_visws_init(void)
{
pcibios_enable_irq = &pci_visws_enable_irq;
diff --git a/arch/x86/platform/efi/Makefile b/arch/x86/platform/efi/Makefile
index 73b8be0..6db1cc4 100644
--- a/arch/x86/platform/efi/Makefile
+++ b/arch/x86/platform/efi/Makefile
@@ -1 +1,2 @@
obj-$(CONFIG_EFI) += efi.o efi_$(BITS).o efi_stub_$(BITS).o
+obj-$(CONFIG_ACPI_BGRT) += efi-bgrt.o
diff --git a/arch/x86/platform/efi/efi-bgrt.c b/arch/x86/platform/efi/efi-bgrt.c
new file mode 100644
index 0000000..f6a0c1b
--- /dev/null
+++ b/arch/x86/platform/efi/efi-bgrt.c
@@ -0,0 +1,76 @@
+/*
+ * Copyright 2012 Intel Corporation
+ * Author: Josh Triplett <josh@joshtriplett.org>
+ *
+ * Based on the bgrt driver:
+ * Copyright 2012 Red Hat, Inc <mjg@redhat.com>
+ * Author: Matthew Garrett
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/kernel.h>
+#include <linux/acpi.h>
+#include <linux/efi.h>
+#include <linux/efi-bgrt.h>
+
+struct acpi_table_bgrt *bgrt_tab;
+void *bgrt_image;
+size_t bgrt_image_size;
+
+struct bmp_header {
+ u16 id;
+ u32 size;
+} __packed;
+
+void efi_bgrt_init(void)
+{
+ acpi_status status;
+ void __iomem *image;
+ bool ioremapped = false;
+ struct bmp_header bmp_header;
+
+ if (acpi_disabled)
+ return;
+
+ status = acpi_get_table("BGRT", 0,
+ (struct acpi_table_header **)&bgrt_tab);
+ if (ACPI_FAILURE(status))
+ return;
+
+ if (bgrt_tab->version != 1)
+ return;
+ if (bgrt_tab->image_type != 0 || !bgrt_tab->image_address)
+ return;
+
+ image = efi_lookup_mapped_addr(bgrt_tab->image_address);
+ if (!image) {
+ image = ioremap(bgrt_tab->image_address, sizeof(bmp_header));
+ ioremapped = true;
+ if (!image)
+ return;
+ }
+
+ memcpy_fromio(&bmp_header, image, sizeof(bmp_header));
+ if (ioremapped)
+ iounmap(image);
+ bgrt_image_size = bmp_header.size;
+
+ bgrt_image = kmalloc(bgrt_image_size, GFP_KERNEL);
+ if (!bgrt_image)
+ return;
+
+ if (ioremapped) {
+ image = ioremap(bgrt_tab->image_address, bmp_header.size);
+ if (!image) {
+ kfree(bgrt_image);
+ bgrt_image = NULL;
+ return;
+ }
+ }
+
+ memcpy_fromio(bgrt_image, image, bgrt_image_size);
+ if (ioremapped)
+ iounmap(image);
+}
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 92660eda..aded2a9 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -31,6 +31,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/efi.h>
+#include <linux/efi-bgrt.h>
#include <linux/export.h>
#include <linux/bootmem.h>
#include <linux/memblock.h>
@@ -419,10 +420,21 @@
}
}
-static void __init efi_free_boot_services(void)
+static void __init efi_unmap_memmap(void)
+{
+ if (memmap.map) {
+ early_iounmap(memmap.map, memmap.nr_map * memmap.desc_size);
+ memmap.map = NULL;
+ }
+}
+
+void __init efi_free_boot_services(void)
{
void *p;
+ if (!efi_native)
+ return;
+
for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
efi_memory_desc_t *md = p;
unsigned long long start = md->phys_addr;
@@ -438,6 +450,8 @@
free_bootmem_late(start, size);
}
+
+ efi_unmap_memmap();
}
static int __init efi_systab_init(void *phys)
@@ -732,6 +746,11 @@
#endif
}
+void __init efi_late_init(void)
+{
+ efi_bgrt_init();
+}
+
void __init efi_set_executable(efi_memory_desc_t *md, bool executable)
{
u64 addr, npages;
@@ -764,6 +783,34 @@
}
/*
+ * We can't ioremap data in EFI boot services RAM, because we've already mapped
+ * it as RAM. So, look it up in the existing EFI memory map instead. Only
+ * callable after efi_enter_virtual_mode and before efi_free_boot_services.
+ */
+void __iomem *efi_lookup_mapped_addr(u64 phys_addr)
+{
+ void *p;
+ if (WARN_ON(!memmap.map))
+ return NULL;
+ for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
+ efi_memory_desc_t *md = p;
+ u64 size = md->num_pages << EFI_PAGE_SHIFT;
+ u64 end = md->phys_addr + size;
+ if (!(md->attribute & EFI_MEMORY_RUNTIME) &&
+ md->type != EFI_BOOT_SERVICES_CODE &&
+ md->type != EFI_BOOT_SERVICES_DATA)
+ continue;
+ if (!md->virt_addr)
+ continue;
+ if (phys_addr >= md->phys_addr && phys_addr < end) {
+ phys_addr += md->virt_addr - md->phys_addr;
+ return (__force void __iomem *)(unsigned long)phys_addr;
+ }
+ }
+ return NULL;
+}
+
+/*
* This function will switch the EFI runtime services to virtual mode.
* Essentially, look through the EFI memmap and map every region that
* has the runtime attribute bit set in its memory descriptor and update
@@ -787,8 +834,10 @@
* non-native EFI
*/
- if (!efi_native)
- goto out;
+ if (!efi_native) {
+ efi_unmap_memmap();
+ return;
+ }
/* Merge contiguous regions of the same type and attribute */
for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) {
@@ -878,18 +927,12 @@
}
/*
- * Thankfully, it does seem that no runtime services other than
- * SetVirtualAddressMap() will touch boot services code, so we can
- * get rid of it all at this point
- */
- efi_free_boot_services();
-
- /*
* Now that EFI is in virtual mode, update the function
* pointers in the runtime service table to the new virtual addresses.
*
* Call EFI services through wrapper functions.
*/
+ efi.runtime_version = efi_systab.fw_revision;
efi.get_time = virt_efi_get_time;
efi.set_time = virt_efi_set_time;
efi.get_wakeup_time = virt_efi_get_wakeup_time;
@@ -906,9 +949,6 @@
if (__supported_pte_mask & _PAGE_NX)
runtime_code_page_mkexec();
-out:
- early_iounmap(memmap.map, memmap.nr_map * memmap.desc_size);
- memmap.map = NULL;
kfree(new_memmap);
}
diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig
index 9926e11..aeaff8b 100644
--- a/arch/x86/um/Kconfig
+++ b/arch/x86/um/Kconfig
@@ -21,6 +21,7 @@
config X86_32
def_bool !64BIT
select HAVE_AOUT
+ select ARCH_WANT_IPC_PARSE_VERSION
config X86_64
def_bool 64BIT
diff --git a/arch/x86/um/shared/sysdep/kernel-offsets.h b/arch/x86/um/shared/sysdep/kernel-offsets.h
index 5868526..46a9df9 100644
--- a/arch/x86/um/shared/sysdep/kernel-offsets.h
+++ b/arch/x86/um/shared/sysdep/kernel-offsets.h
@@ -7,9 +7,6 @@
#define DEFINE(sym, val) \
asm volatile("\n->" #sym " %0 " #val : : "i" (val))
-#define STR(x) #x
-#define DEFINE_STR(sym, val) asm volatile("\n->" #sym " " STR(val) " " #val: : )
-
#define BLANK() asm volatile("\n->" : : )
#define OFFSET(sym, str, mem) \
diff --git a/arch/x86/um/shared/sysdep/syscalls.h b/arch/x86/um/shared/sysdep/syscalls.h
index bd9a89b..ca255a8 100644
--- a/arch/x86/um/shared/sysdep/syscalls.h
+++ b/arch/x86/um/shared/sysdep/syscalls.h
@@ -1,3 +1,5 @@
+extern long sys_clone(unsigned long clone_flags, unsigned long newsp,
+ void __user *parent_tid, void __user *child_tid);
#ifdef __i386__
#include "syscalls_32.h"
#else
diff --git a/arch/x86/um/signal.c b/arch/x86/um/signal.c
index a508cea1..ba7363e 100644
--- a/arch/x86/um/signal.c
+++ b/arch/x86/um/signal.c
@@ -416,9 +416,6 @@
PT_REGS_AX(regs) = (unsigned long) sig;
PT_REGS_DX(regs) = (unsigned long) 0;
PT_REGS_CX(regs) = (unsigned long) 0;
-
- if ((current->ptrace & PT_DTRACE) && (current->ptrace & PT_PTRACED))
- ptrace_notify(SIGTRAP);
return 0;
}
@@ -466,9 +463,6 @@
PT_REGS_AX(regs) = (unsigned long) sig;
PT_REGS_DX(regs) = (unsigned long) &frame->info;
PT_REGS_CX(regs) = (unsigned long) &frame->uc;
-
- if ((current->ptrace & PT_DTRACE) && (current->ptrace & PT_PTRACED))
- ptrace_notify(SIGTRAP);
return 0;
}
diff --git a/arch/x86/um/sys_call_table_32.c b/arch/x86/um/sys_call_table_32.c
index 68d1dc9..b5408ce 100644
--- a/arch/x86/um/sys_call_table_32.c
+++ b/arch/x86/um/sys_call_table_32.c
@@ -28,7 +28,7 @@
#define ptregs_execve sys_execve
#define ptregs_iopl sys_iopl
#define ptregs_vm86old sys_vm86old
-#define ptregs_clone sys_clone
+#define ptregs_clone i386_clone
#define ptregs_vm86 sys_vm86
#define ptregs_sigaltstack sys_sigaltstack
#define ptregs_vfork sys_vfork
diff --git a/arch/x86/um/syscalls_32.c b/arch/x86/um/syscalls_32.c
index b853e86..db444c7 100644
--- a/arch/x86/um/syscalls_32.c
+++ b/arch/x86/um/syscalls_32.c
@@ -3,37 +3,24 @@
* Licensed under the GPL
*/
-#include "linux/sched.h"
-#include "linux/shm.h"
-#include "linux/ipc.h"
-#include "linux/syscalls.h"
-#include "asm/mman.h"
-#include "asm/uaccess.h"
-#include "asm/unistd.h"
+#include <linux/syscalls.h>
+#include <sysdep/syscalls.h>
/*
* The prototype on i386 is:
*
- * int clone(int flags, void * child_stack, int * parent_tidptr, struct user_desc * newtls, int * child_tidptr)
+ * int clone(int flags, void * child_stack, int * parent_tidptr, struct user_desc * newtls
*
* and the "newtls" arg. on i386 is read by copy_thread directly from the
* register saved on the stack.
*/
-long sys_clone(unsigned long clone_flags, unsigned long newsp,
- int __user *parent_tid, void *newtls, int __user *child_tid)
+long i386_clone(unsigned long clone_flags, unsigned long newsp,
+ int __user *parent_tid, void *newtls, int __user *child_tid)
{
- long ret;
-
- if (!newsp)
- newsp = UPT_SP(¤t->thread.regs.regs);
-
- current->thread.forking = 1;
- ret = do_fork(clone_flags, newsp, ¤t->thread.regs, 0, parent_tid,
- child_tid);
- current->thread.forking = 0;
- return ret;
+ return sys_clone(clone_flags, newsp, parent_tid, child_tid);
}
+
long sys_sigaction(int sig, const struct old_sigaction __user *act,
struct old_sigaction __user *oact)
{
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index f3d82bb..adb08eb 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -5,12 +5,9 @@
* Licensed under the GPL
*/
-#include "linux/linkage.h"
-#include "linux/personality.h"
-#include "linux/utsname.h"
-#include "asm/prctl.h" /* XXX This should get the constants from libc */
-#include "asm/uaccess.h"
-#include "os.h"
+#include <linux/sched.h>
+#include <asm/prctl.h> /* XXX This should get the constants from libc */
+#include <os.h>
long arch_prctl(struct task_struct *task, int code, unsigned long __user *addr)
{
@@ -79,20 +76,6 @@
return arch_prctl(current, code, (unsigned long __user *) addr);
}
-long sys_clone(unsigned long clone_flags, unsigned long newsp,
- void __user *parent_tid, void __user *child_tid)
-{
- long ret;
-
- if (!newsp)
- newsp = UPT_SP(¤t->thread.regs.regs);
- current->thread.forking = 1;
- ret = do_fork(clone_flags, newsp, ¤t->thread.regs, 0, parent_tid,
- child_tid);
- current->thread.forking = 0;
- return ret;
-}
-
void arch_switch_to(struct task_struct *to)
{
if ((to->thread.arch.fs == 0) || (to->mm == NULL))
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index bf4bda6..1fbe75a 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -31,7 +31,6 @@
#include <linux/pci.h>
#include <linux/gfp.h>
#include <linux/memblock.h>
-#include <linux/syscore_ops.h>
#include <xen/xen.h>
#include <xen/interface/xen.h>
@@ -1453,6 +1452,10 @@
pci_request_acs();
xen_acpi_sleep_register();
+
+ /* Avoid searching for BIOS MP tables */
+ x86_init.mpparse.find_smp_config = x86_init_noop;
+ x86_init.mpparse.get_smp_config = x86_init_uint_noop;
}
#ifdef CONFIG_PCI
/* PCI BIOS service won't work from a PV guest. */
@@ -1470,130 +1473,38 @@
#endif
}
-#ifdef CONFIG_XEN_PVHVM
-/*
- * The pfn containing the shared_info is located somewhere in RAM. This
- * will cause trouble if the current kernel is doing a kexec boot into a
- * new kernel. The new kernel (and its startup code) can not know where
- * the pfn is, so it can not reserve the page. The hypervisor will
- * continue to update the pfn, and as a result memory corruption occours
- * in the new kernel.
- *
- * One way to work around this issue is to allocate a page in the
- * xen-platform pci device's BAR memory range. But pci init is done very
- * late and the shared_info page is already in use very early to read
- * the pvclock. So moving the pfn from RAM to MMIO is racy because some
- * code paths on other vcpus could access the pfn during the small
- * window when the old pfn is moved to the new pfn. There is even a
- * small window were the old pfn is not backed by a mfn, and during that
- * time all reads return -1.
- *
- * Because it is not known upfront where the MMIO region is located it
- * can not be used right from the start in xen_hvm_init_shared_info.
- *
- * To minimise trouble the move of the pfn is done shortly before kexec.
- * This does not eliminate the race because all vcpus are still online
- * when the syscore_ops will be called. But hopefully there is no work
- * pending at this point in time. Also the syscore_op is run last which
- * reduces the risk further.
- */
-
-static struct shared_info *xen_hvm_shared_info;
-
-static void xen_hvm_connect_shared_info(unsigned long pfn)
+void __ref xen_hvm_init_shared_info(void)
{
+ int cpu;
struct xen_add_to_physmap xatp;
+ static struct shared_info *shared_info_page = 0;
+ if (!shared_info_page)
+ shared_info_page = (struct shared_info *)
+ extend_brk(PAGE_SIZE, PAGE_SIZE);
xatp.domid = DOMID_SELF;
xatp.idx = 0;
xatp.space = XENMAPSPACE_shared_info;
- xatp.gpfn = pfn;
+ xatp.gpfn = __pa(shared_info_page) >> PAGE_SHIFT;
if (HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp))
BUG();
-}
-static void xen_hvm_set_shared_info(struct shared_info *sip)
-{
- int cpu;
-
- HYPERVISOR_shared_info = sip;
+ HYPERVISOR_shared_info = (struct shared_info *)shared_info_page;
/* xen_vcpu is a pointer to the vcpu_info struct in the shared_info
* page, we use it in the event channel upcall and in some pvclock
* related functions. We don't need the vcpu_info placement
* optimizations because we don't use any pv_mmu or pv_irq op on
* HVM.
- * When xen_hvm_set_shared_info is run at boot time only vcpu 0 is
- * online but xen_hvm_set_shared_info is run at resume time too and
+ * When xen_hvm_init_shared_info is run at boot time only vcpu 0 is
+ * online but xen_hvm_init_shared_info is run at resume time too and
* in that case multiple vcpus might be online. */
for_each_online_cpu(cpu) {
per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu];
}
}
-/* Reconnect the shared_info pfn to a mfn */
-void xen_hvm_resume_shared_info(void)
-{
- xen_hvm_connect_shared_info(__pa(xen_hvm_shared_info) >> PAGE_SHIFT);
-}
-
-#ifdef CONFIG_KEXEC
-static struct shared_info *xen_hvm_shared_info_kexec;
-static unsigned long xen_hvm_shared_info_pfn_kexec;
-
-/* Remember a pfn in MMIO space for kexec reboot */
-void __devinit xen_hvm_prepare_kexec(struct shared_info *sip, unsigned long pfn)
-{
- xen_hvm_shared_info_kexec = sip;
- xen_hvm_shared_info_pfn_kexec = pfn;
-}
-
-static void xen_hvm_syscore_shutdown(void)
-{
- struct xen_memory_reservation reservation = {
- .domid = DOMID_SELF,
- .nr_extents = 1,
- };
- unsigned long prev_pfn;
- int rc;
-
- if (!xen_hvm_shared_info_kexec)
- return;
-
- prev_pfn = __pa(xen_hvm_shared_info) >> PAGE_SHIFT;
- set_xen_guest_handle(reservation.extent_start, &prev_pfn);
-
- /* Move pfn to MMIO, disconnects previous pfn from mfn */
- xen_hvm_connect_shared_info(xen_hvm_shared_info_pfn_kexec);
-
- /* Update pointers, following hypercall is also a memory barrier */
- xen_hvm_set_shared_info(xen_hvm_shared_info_kexec);
-
- /* Allocate new mfn for previous pfn */
- do {
- rc = HYPERVISOR_memory_op(XENMEM_populate_physmap, &reservation);
- if (rc == 0)
- msleep(123);
- } while (rc == 0);
-
- /* Make sure the previous pfn is really connected to a (new) mfn */
- BUG_ON(rc != 1);
-}
-
-static struct syscore_ops xen_hvm_syscore_ops = {
- .shutdown = xen_hvm_syscore_shutdown,
-};
-#endif
-
-/* Use a pfn in RAM, may move to MMIO before kexec. */
-static void __init xen_hvm_init_shared_info(void)
-{
- /* Remember pointer for resume */
- xen_hvm_shared_info = extend_brk(PAGE_SIZE, PAGE_SIZE);
- xen_hvm_connect_shared_info(__pa(xen_hvm_shared_info) >> PAGE_SHIFT);
- xen_hvm_set_shared_info(xen_hvm_shared_info);
-}
-
+#ifdef CONFIG_XEN_PVHVM
static void __init init_hvm_pv_info(void)
{
int major, minor;
@@ -1644,9 +1555,6 @@
init_hvm_pv_info();
xen_hvm_init_shared_info();
-#ifdef CONFIG_KEXEC
- register_syscore_ops(&xen_hvm_syscore_ops);
-#endif
if (xen_feature(XENFEAT_hvm_callback_vector))
xen_have_vector_callback = 1;
diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index b65a761..7a769b7 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -1174,8 +1174,13 @@
spin_unlock(&mm->page_table_lock);
}
-static void __init xen_pagetable_setup_start(pgd_t *base)
+static void xen_post_allocator_init(void);
+
+static void __init xen_pagetable_init(void)
{
+ paging_init();
+ xen_setup_shared_info();
+ xen_post_allocator_init();
}
static __init void xen_mapping_pagetable_reserve(u64 start, u64 end)
@@ -1192,14 +1197,6 @@
}
}
-static void xen_post_allocator_init(void);
-
-static void __init xen_pagetable_setup_done(pgd_t *base)
-{
- xen_setup_shared_info();
- xen_post_allocator_init();
-}
-
static void xen_write_cr2(unsigned long cr2)
{
this_cpu_read(xen_vcpu)->arch.cr2 = cr2;
@@ -1283,7 +1280,7 @@
cpumask_clear_cpu(smp_processor_id(), to_cpumask(args->mask));
args->op.cmd = MMUEXT_TLB_FLUSH_MULTI;
- if (start != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
+ if (end != TLB_FLUSH_ALL && (end - start) <= PAGE_SIZE) {
args->op.cmd = MMUEXT_INVLPG_MULTI;
args->op.arg1.linear_addr = start;
}
@@ -2068,8 +2065,7 @@
void __init xen_init_mmu_ops(void)
{
x86_init.mapping.pagetable_reserve = xen_mapping_pagetable_reserve;
- x86_init.paging.pagetable_setup_start = xen_pagetable_setup_start;
- x86_init.paging.pagetable_setup_done = xen_pagetable_setup_done;
+ x86_init.paging.pagetable_init = xen_pagetable_init;
pv_mmu_ops = xen_mmu_ops;
memset(dummy_mapping, 0xff, PAGE_SIZE);
diff --git a/arch/x86/xen/p2m.c b/arch/x86/xen/p2m.c
index b2e91d4..72213da 100644
--- a/arch/x86/xen/p2m.c
+++ b/arch/x86/xen/p2m.c
@@ -196,9 +196,11 @@
/* When we populate back during bootup, the amount of pages can vary. The
* max we have is seen is 395979, but that does not mean it can't be more.
- * But some machines can have 3GB I/O holes even. So lets reserve enough
- * for 4GB of I/O and E820 holes. */
-RESERVE_BRK(p2m_populated, PMD_SIZE * 4);
+ * Some machines can have 3GB I/O holes even. With early_can_reuse_p2m_middle
+ * it can re-use Xen provided mfn_list array, so we only need to allocate at
+ * most three P2M top nodes. */
+RESERVE_BRK(p2m_populated, PAGE_SIZE * 3);
+
static inline unsigned p2m_top_index(unsigned long pfn)
{
BUG_ON(pfn >= MAX_P2M_PFN);
@@ -575,12 +577,99 @@
}
return true;
}
+
+/*
+ * Skim over the P2M tree looking at pages that are either filled with
+ * INVALID_P2M_ENTRY or with 1:1 PFNs. If found, re-use that page and
+ * replace the P2M leaf with a p2m_missing or p2m_identity.
+ * Stick the old page in the new P2M tree location.
+ */
+bool __init early_can_reuse_p2m_middle(unsigned long set_pfn, unsigned long set_mfn)
+{
+ unsigned topidx;
+ unsigned mididx;
+ unsigned ident_pfns;
+ unsigned inv_pfns;
+ unsigned long *p2m;
+ unsigned long *mid_mfn_p;
+ unsigned idx;
+ unsigned long pfn;
+
+ /* We only look when this entails a P2M middle layer */
+ if (p2m_index(set_pfn))
+ return false;
+
+ for (pfn = 0; pfn < MAX_DOMAIN_PAGES; pfn += P2M_PER_PAGE) {
+ topidx = p2m_top_index(pfn);
+
+ if (!p2m_top[topidx])
+ continue;
+
+ if (p2m_top[topidx] == p2m_mid_missing)
+ continue;
+
+ mididx = p2m_mid_index(pfn);
+ p2m = p2m_top[topidx][mididx];
+ if (!p2m)
+ continue;
+
+ if ((p2m == p2m_missing) || (p2m == p2m_identity))
+ continue;
+
+ if ((unsigned long)p2m == INVALID_P2M_ENTRY)
+ continue;
+
+ ident_pfns = 0;
+ inv_pfns = 0;
+ for (idx = 0; idx < P2M_PER_PAGE; idx++) {
+ /* IDENTITY_PFNs are 1:1 */
+ if (p2m[idx] == IDENTITY_FRAME(pfn + idx))
+ ident_pfns++;
+ else if (p2m[idx] == INVALID_P2M_ENTRY)
+ inv_pfns++;
+ else
+ break;
+ }
+ if ((ident_pfns == P2M_PER_PAGE) || (inv_pfns == P2M_PER_PAGE))
+ goto found;
+ }
+ return false;
+found:
+ /* Found one, replace old with p2m_identity or p2m_missing */
+ p2m_top[topidx][mididx] = (ident_pfns ? p2m_identity : p2m_missing);
+ /* And the other for save/restore.. */
+ mid_mfn_p = p2m_top_mfn_p[topidx];
+ /* NOTE: Even if it is a p2m_identity it should still be point to
+ * a page filled with INVALID_P2M_ENTRY entries. */
+ mid_mfn_p[mididx] = virt_to_mfn(p2m_missing);
+
+ /* Reset where we want to stick the old page in. */
+ topidx = p2m_top_index(set_pfn);
+ mididx = p2m_mid_index(set_pfn);
+
+ /* This shouldn't happen */
+ if (WARN_ON(p2m_top[topidx] == p2m_mid_missing))
+ early_alloc_p2m(set_pfn);
+
+ if (WARN_ON(p2m_top[topidx][mididx] != p2m_missing))
+ return false;
+
+ p2m_init(p2m);
+ p2m_top[topidx][mididx] = p2m;
+ mid_mfn_p = p2m_top_mfn_p[topidx];
+ mid_mfn_p[mididx] = virt_to_mfn(p2m);
+
+ return true;
+}
bool __init early_set_phys_to_machine(unsigned long pfn, unsigned long mfn)
{
if (unlikely(!__set_phys_to_machine(pfn, mfn))) {
if (!early_alloc_p2m(pfn))
return false;
+ if (early_can_reuse_p2m_middle(pfn, mfn))
+ return __set_phys_to_machine(pfn, mfn);
+
if (!early_alloc_p2m_middle(pfn, false /* boundary crossover OK!*/))
return false;
@@ -739,9 +828,6 @@
xen_mc_issue(PARAVIRT_LAZY_MMU);
}
- /* let's use dev_bus_addr to record the old mfn instead */
- kmap_op->dev_bus_addr = page->index;
- page->index = (unsigned long) kmap_op;
}
spin_lock_irqsave(&m2p_override_lock, flags);
list_add(&page->lru, &m2p_overrides[mfn_hash(mfn)]);
@@ -768,7 +854,8 @@
return 0;
}
EXPORT_SYMBOL_GPL(m2p_add_override);
-int m2p_remove_override(struct page *page, bool clear_pte)
+int m2p_remove_override(struct page *page,
+ struct gnttab_map_grant_ref *kmap_op)
{
unsigned long flags;
unsigned long mfn;
@@ -798,10 +885,8 @@
WARN_ON(!PagePrivate(page));
ClearPagePrivate(page);
- if (clear_pte) {
- struct gnttab_map_grant_ref *map_op =
- (struct gnttab_map_grant_ref *) page->index;
- set_phys_to_machine(pfn, map_op->dev_bus_addr);
+ set_phys_to_machine(pfn, page->index);
+ if (kmap_op != NULL) {
if (!PageHighMem(page)) {
struct multicall_space mcs;
struct gnttab_unmap_grant_ref *unmap_op;
@@ -813,13 +898,13 @@
* issued. In this case handle is going to -1 because
* it hasn't been modified yet.
*/
- if (map_op->handle == -1)
+ if (kmap_op->handle == -1)
xen_mc_flush();
/*
- * Now if map_op->handle is negative it means that the
+ * Now if kmap_op->handle is negative it means that the
* hypercall actually returned an error.
*/
- if (map_op->handle == GNTST_general_error) {
+ if (kmap_op->handle == GNTST_general_error) {
printk(KERN_WARNING "m2p_remove_override: "
"pfn %lx mfn %lx, failed to modify kernel mappings",
pfn, mfn);
@@ -829,8 +914,8 @@
mcs = xen_mc_entry(
sizeof(struct gnttab_unmap_grant_ref));
unmap_op = mcs.args;
- unmap_op->host_addr = map_op->host_addr;
- unmap_op->handle = map_op->handle;
+ unmap_op->host_addr = kmap_op->host_addr;
+ unmap_op->handle = kmap_op->handle;
unmap_op->dev_bus_addr = 0;
MULTI_grant_table_op(mcs.mc,
@@ -841,10 +926,9 @@
set_pte_at(&init_mm, address, ptep,
pfn_pte(pfn, PAGE_KERNEL));
__flush_tlb_single(address);
- map_op->host_addr = 0;
+ kmap_op->host_addr = 0;
}
- } else
- set_phys_to_machine(pfn, page->index);
+ }
/* p2m(m2p(mfn)) == FOREIGN_FRAME(mfn): the mfn is already present
* somewhere in this domain, even before being added to the
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index ead8557..e2d62d6 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -17,6 +17,7 @@
#include <asm/e820.h>
#include <asm/setup.h>
#include <asm/acpi.h>
+#include <asm/numa.h>
#include <asm/xen/hypervisor.h>
#include <asm/xen/hypercall.h>
@@ -78,9 +79,16 @@
memblock_reserve(start, size);
xen_max_p2m_pfn = PFN_DOWN(start + size);
+ for (pfn = PFN_DOWN(start); pfn < xen_max_p2m_pfn; pfn++) {
+ unsigned long mfn = pfn_to_mfn(pfn);
- for (pfn = PFN_DOWN(start); pfn <= xen_max_p2m_pfn; pfn++)
+ if (WARN(mfn == pfn, "Trying to over-write 1-1 mapping (pfn: %lx)\n", pfn))
+ continue;
+ WARN(mfn != INVALID_P2M_ENTRY, "Trying to remove %lx which has %lx mfn!\n",
+ pfn, mfn);
+
__set_phys_to_machine(pfn, INVALID_P2M_ENTRY);
+ }
}
static unsigned long __init xen_do_chunk(unsigned long start,
@@ -537,4 +545,7 @@
disable_cpufreq();
WARN_ON(set_pm_idle_to_default());
fiddle_vdso();
+#ifdef CONFIG_NUMA
+ numa_off = 1;
+#endif
}
diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index f58dca7..353c50f 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -377,7 +377,8 @@
return rc;
if (num_online_cpus() == 1)
- alternatives_smp_switch(1);
+ /* Just in case we booted with a single CPU. */
+ alternatives_enable_smp();
rc = xen_smp_intr_init(cpu);
if (rc)
@@ -424,9 +425,6 @@
unbind_from_irqhandler(per_cpu(xen_irq_work, cpu), NULL);
xen_uninit_lock_cpu(cpu);
xen_teardown_timer(cpu);
-
- if (num_online_cpus() == 1)
- alternatives_smp_switch(0);
}
static void __cpuinit xen_play_dead(void) /* used only with HOTPLUG_CPU */
diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c
index ae8a00c..45329c8 100644
--- a/arch/x86/xen/suspend.c
+++ b/arch/x86/xen/suspend.c
@@ -30,7 +30,7 @@
{
#ifdef CONFIG_XEN_PVHVM
int cpu;
- xen_hvm_resume_shared_info();
+ xen_hvm_init_shared_info();
xen_callback_vector();
xen_unplug_emulated_devices();
if (xen_feature(XENFEAT_hvm_safe_pvclock)) {
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 1e4329e..202d4c1 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -41,7 +41,7 @@
void xen_vcpu_restore(void);
void xen_callback_vector(void);
-void xen_hvm_resume_shared_info(void);
+void xen_hvm_init_shared_info(void);
void xen_unplug_emulated_devices(void);
void __init xen_build_dynamic_phys_to_machine(void);
diff --git a/arch/xtensa/kernel/pci.c b/arch/xtensa/kernel/pci.c
index 69759e9..54354de 100644
--- a/arch/xtensa/kernel/pci.c
+++ b/arch/xtensa/kernel/pci.c
@@ -210,14 +210,6 @@
/* No special bus mastering setup handling */
}
-/* the next one is stolen from the alpha port... */
-
-void __init
-pcibios_update_irq(struct pci_dev *dev, int irq)
-{
- pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
-}
-
int pcibios_enable_device(struct pci_dev *dev, int mask)
{
u16 cmd, old_cmd;
diff --git a/arch/xtensa/kernel/process.c b/arch/xtensa/kernel/process.c
index 2c8d6a3..bc44311 100644
--- a/arch/xtensa/kernel/process.c
+++ b/arch/xtensa/kernel/process.c
@@ -31,6 +31,7 @@
#include <linux/mqueue.h>
#include <linux/fs.h>
#include <linux/slab.h>
+#include <linux/rcupdate.h>
#include <asm/pgtable.h>
#include <asm/uaccess.h>
@@ -110,8 +111,10 @@
/* endless idle loop with no priority at all */
while (1) {
+ rcu_idle_enter();
while (!need_resched())
platform_idle();
+ rcu_idle_exit();
schedule_preempt_disabled();
}
}
diff --git a/block/blk-core.c b/block/blk-core.c
index 4b4dbdf..ee3cb3a 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -2254,9 +2254,11 @@
error_type = "I/O";
break;
}
- printk(KERN_ERR "end_request: %s error, dev %s, sector %llu\n",
- error_type, req->rq_disk ? req->rq_disk->disk_name : "?",
- (unsigned long long)blk_rq_pos(req));
+ printk_ratelimited(KERN_ERR "end_request: %s error, dev %s, sector %llu\n",
+ error_type, req->rq_disk ?
+ req->rq_disk->disk_name : "?",
+ (unsigned long long)blk_rq_pos(req));
+
}
blk_account_io_completion(req, nr_bytes);
diff --git a/block/blk-lib.c b/block/blk-lib.c
index 2b461b4..19cc761 100644
--- a/block/blk-lib.c
+++ b/block/blk-lib.c
@@ -44,6 +44,7 @@
struct request_queue *q = bdev_get_queue(bdev);
int type = REQ_WRITE | REQ_DISCARD;
unsigned int max_discard_sectors;
+ unsigned int granularity, alignment, mask;
struct bio_batch bb;
struct bio *bio;
int ret = 0;
@@ -54,18 +55,20 @@
if (!blk_queue_discard(q))
return -EOPNOTSUPP;
+ /* Zero-sector (unknown) and one-sector granularities are the same. */
+ granularity = max(q->limits.discard_granularity >> 9, 1U);
+ mask = granularity - 1;
+ alignment = (bdev_discard_alignment(bdev) >> 9) & mask;
+
/*
* Ensure that max_discard_sectors is of the proper
- * granularity
+ * granularity, so that requests stay aligned after a split.
*/
max_discard_sectors = min(q->limits.max_discard_sectors, UINT_MAX >> 9);
+ max_discard_sectors = round_down(max_discard_sectors, granularity);
if (unlikely(!max_discard_sectors)) {
/* Avoid infinite loop below. Being cautious never hurts. */
return -EOPNOTSUPP;
- } else if (q->limits.discard_granularity) {
- unsigned int disc_sects = q->limits.discard_granularity >> 9;
-
- max_discard_sectors &= ~(disc_sects - 1);
}
if (flags & BLKDEV_DISCARD_SECURE) {
@@ -79,25 +82,37 @@
bb.wait = &wait;
while (nr_sects) {
+ unsigned int req_sects;
+ sector_t end_sect;
+
bio = bio_alloc(gfp_mask, 1);
if (!bio) {
ret = -ENOMEM;
break;
}
+ req_sects = min_t(sector_t, nr_sects, max_discard_sectors);
+
+ /*
+ * If splitting a request, and the next starting sector would be
+ * misaligned, stop the discard at the previous aligned sector.
+ */
+ end_sect = sector + req_sects;
+ if (req_sects < nr_sects && (end_sect & mask) != alignment) {
+ end_sect =
+ round_down(end_sect - alignment, granularity)
+ + alignment;
+ req_sects = end_sect - sector;
+ }
+
bio->bi_sector = sector;
bio->bi_end_io = bio_batch_end_io;
bio->bi_bdev = bdev;
bio->bi_private = &bb;
- if (nr_sects > max_discard_sectors) {
- bio->bi_size = max_discard_sectors << 9;
- nr_sects -= max_discard_sectors;
- sector += max_discard_sectors;
- } else {
- bio->bi_size = nr_sects << 9;
- nr_sects = 0;
- }
+ bio->bi_size = req_sects << 9;
+ nr_sects -= req_sects;
+ sector = end_sect;
atomic_inc(&bb.done);
submit_bio(type, bio);
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 160035f..e76279e 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -110,6 +110,49 @@
return 0;
}
+static void
+__blk_segment_map_sg(struct request_queue *q, struct bio_vec *bvec,
+ struct scatterlist *sglist, struct bio_vec **bvprv,
+ struct scatterlist **sg, int *nsegs, int *cluster)
+{
+
+ int nbytes = bvec->bv_len;
+
+ if (*bvprv && *cluster) {
+ if ((*sg)->length + nbytes > queue_max_segment_size(q))
+ goto new_segment;
+
+ if (!BIOVEC_PHYS_MERGEABLE(*bvprv, bvec))
+ goto new_segment;
+ if (!BIOVEC_SEG_BOUNDARY(q, *bvprv, bvec))
+ goto new_segment;
+
+ (*sg)->length += nbytes;
+ } else {
+new_segment:
+ if (!*sg)
+ *sg = sglist;
+ else {
+ /*
+ * If the driver previously mapped a shorter
+ * list, we could see a termination bit
+ * prematurely unless it fully inits the sg
+ * table on each mapping. We KNOW that there
+ * must be more entries here or the driver
+ * would be buggy, so force clear the
+ * termination bit to avoid doing a full
+ * sg_init_table() in drivers for each command.
+ */
+ (*sg)->page_link &= ~0x02;
+ *sg = sg_next(*sg);
+ }
+
+ sg_set_page(*sg, bvec->bv_page, nbytes, bvec->bv_offset);
+ (*nsegs)++;
+ }
+ *bvprv = bvec;
+}
+
/*
* map a request to scatterlist, return number of sg entries setup. Caller
* must make sure sg can hold rq->nr_phys_segments entries
@@ -131,41 +174,8 @@
bvprv = NULL;
sg = NULL;
rq_for_each_segment(bvec, rq, iter) {
- int nbytes = bvec->bv_len;
-
- if (bvprv && cluster) {
- if (sg->length + nbytes > queue_max_segment_size(q))
- goto new_segment;
-
- if (!BIOVEC_PHYS_MERGEABLE(bvprv, bvec))
- goto new_segment;
- if (!BIOVEC_SEG_BOUNDARY(q, bvprv, bvec))
- goto new_segment;
-
- sg->length += nbytes;
- } else {
-new_segment:
- if (!sg)
- sg = sglist;
- else {
- /*
- * If the driver previously mapped a shorter
- * list, we could see a termination bit
- * prematurely unless it fully inits the sg
- * table on each mapping. We KNOW that there
- * must be more entries here or the driver
- * would be buggy, so force clear the
- * termination bit to avoid doing a full
- * sg_init_table() in drivers for each command.
- */
- sg->page_link &= ~0x02;
- sg = sg_next(sg);
- }
-
- sg_set_page(sg, bvec->bv_page, nbytes, bvec->bv_offset);
- nsegs++;
- }
- bvprv = bvec;
+ __blk_segment_map_sg(q, bvec, sglist, &bvprv, &sg,
+ &nsegs, &cluster);
} /* segments in rq */
@@ -199,6 +209,43 @@
}
EXPORT_SYMBOL(blk_rq_map_sg);
+/**
+ * blk_bio_map_sg - map a bio to a scatterlist
+ * @q: request_queue in question
+ * @bio: bio being mapped
+ * @sglist: scatterlist being mapped
+ *
+ * Note:
+ * Caller must make sure sg can hold bio->bi_phys_segments entries
+ *
+ * Will return the number of sg entries setup
+ */
+int blk_bio_map_sg(struct request_queue *q, struct bio *bio,
+ struct scatterlist *sglist)
+{
+ struct bio_vec *bvec, *bvprv;
+ struct scatterlist *sg;
+ int nsegs, cluster;
+ unsigned long i;
+
+ nsegs = 0;
+ cluster = blk_queue_cluster(q);
+
+ bvprv = NULL;
+ sg = NULL;
+ bio_for_each_segment(bvec, bio, i) {
+ __blk_segment_map_sg(q, bvec, sglist, &bvprv, &sg,
+ &nsegs, &cluster);
+ } /* segments in bio */
+
+ if (sg)
+ sg_mark_end(sg);
+
+ BUG_ON(bio->bi_phys_segments && nsegs > bio->bi_phys_segments);
+ return nsegs;
+}
+EXPORT_SYMBOL(blk_bio_map_sg);
+
static inline int ll_new_hw_segment(struct request_queue *q,
struct request *req,
struct bio *bio)
diff --git a/block/genhd.c b/block/genhd.c
index cac7366..d839723 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -835,7 +835,7 @@
static void *show_partition_start(struct seq_file *seqf, loff_t *pos)
{
- static void *p;
+ void *p;
p = disk_seqf_start(seqf, pos);
if (!IS_ERR_OR_NULL(p) && !*pos)
diff --git a/block/ioctl.c b/block/ioctl.c
index 4476e0e8..4a85096 100644
--- a/block/ioctl.c
+++ b/block/ioctl.c
@@ -41,7 +41,7 @@
sizeof(long long) > sizeof(long)) {
long pstart = start, plength = length;
if (pstart != start || plength != length
- || pstart < 0 || plength < 0)
+ || pstart < 0 || plength < 0 || partno > 65535)
return -EINVAL;
}
diff --git a/block/partitions/ibm.c b/block/partitions/ibm.c
index 1104aca..47a6147 100644
--- a/block/partitions/ibm.c
+++ b/block/partitions/ibm.c
@@ -1,9 +1,8 @@
/*
- * File...........: linux/fs/partitions/ibm.c
* Author(s)......: Holger Smolinski <Holger.Smolinski@de.ibm.com>
* Volker Sameske <sameske@de.ibm.com>
* Bugreports.to..: <Linux390@de.ibm.com>
- * (C) IBM Corporation, IBM Deutschland Entwicklung GmbH, 1999,2000
+ * Copyright IBM Corp. 1999, 2012
*/
#include <linux/buffer_head.h>
@@ -17,17 +16,23 @@
#include "check.h"
#include "ibm.h"
+
+union label_t {
+ struct vtoc_volume_label_cdl vol;
+ struct vtoc_volume_label_ldl lnx;
+ struct vtoc_cms_label cms;
+};
+
/*
* compute the block number from a
* cyl-cyl-head-head structure
*/
-static sector_t
-cchh2blk (struct vtoc_cchh *ptr, struct hd_geometry *geo) {
-
+static sector_t cchh2blk(struct vtoc_cchh *ptr, struct hd_geometry *geo)
+{
sector_t cyl;
__u16 head;
- /*decode cylinder and heads for large volumes */
+ /* decode cylinder and heads for large volumes */
cyl = ptr->hh & 0xFFF0;
cyl <<= 12;
cyl |= ptr->cc;
@@ -40,13 +45,12 @@
* compute the block number from a
* cyl-cyl-head-head-block structure
*/
-static sector_t
-cchhb2blk (struct vtoc_cchhb *ptr, struct hd_geometry *geo) {
-
+static sector_t cchhb2blk(struct vtoc_cchhb *ptr, struct hd_geometry *geo)
+{
sector_t cyl;
__u16 head;
- /*decode cylinder and heads for large volumes */
+ /* decode cylinder and heads for large volumes */
cyl = ptr->hh & 0xFFF0;
cyl <<= 12;
cyl |= ptr->cc;
@@ -56,26 +60,243 @@
ptr->b;
}
+static int find_label(struct parsed_partitions *state,
+ dasd_information2_t *info,
+ struct hd_geometry *geo,
+ int blocksize,
+ sector_t *labelsect,
+ char name[],
+ char type[],
+ union label_t *label)
+{
+ Sector sect;
+ unsigned char *data;
+ sector_t testsect[3];
+ unsigned char temp[5];
+ int found = 0;
+ int i, testcount;
+
+ /* There a three places where we may find a valid label:
+ * - on an ECKD disk it's block 2
+ * - on an FBA disk it's block 1
+ * - on an CMS formatted FBA disk it is sector 1, even if the block size
+ * is larger than 512 bytes (possible if the DIAG discipline is used)
+ * If we have a valid info structure, then we know exactly which case we
+ * have, otherwise we just search through all possebilities.
+ */
+ if (info) {
+ if ((info->cu_type == 0x6310 && info->dev_type == 0x9336) ||
+ (info->cu_type == 0x3880 && info->dev_type == 0x3370))
+ testsect[0] = info->label_block;
+ else
+ testsect[0] = info->label_block * (blocksize >> 9);
+ testcount = 1;
+ } else {
+ testsect[0] = 1;
+ testsect[1] = (blocksize >> 9);
+ testsect[2] = 2 * (blocksize >> 9);
+ testcount = 3;
+ }
+ for (i = 0; i < testcount; ++i) {
+ data = read_part_sector(state, testsect[i], §);
+ if (data == NULL)
+ continue;
+ memcpy(label, data, sizeof(*label));
+ memcpy(temp, data, 4);
+ temp[4] = 0;
+ EBCASC(temp, 4);
+ put_dev_sector(sect);
+ if (!strcmp(temp, "VOL1") ||
+ !strcmp(temp, "LNX1") ||
+ !strcmp(temp, "CMS1")) {
+ if (!strcmp(temp, "VOL1")) {
+ strncpy(type, label->vol.vollbl, 4);
+ strncpy(name, label->vol.volid, 6);
+ } else {
+ strncpy(type, label->lnx.vollbl, 4);
+ strncpy(name, label->lnx.volid, 6);
+ }
+ EBCASC(type, 4);
+ EBCASC(name, 6);
+ *labelsect = testsect[i];
+ found = 1;
+ break;
+ }
+ }
+ if (!found)
+ memset(label, 0, sizeof(*label));
+
+ return found;
+}
+
+static int find_vol1_partitions(struct parsed_partitions *state,
+ struct hd_geometry *geo,
+ int blocksize,
+ char name[],
+ union label_t *label)
+{
+ sector_t blk;
+ int counter;
+ char tmp[64];
+ Sector sect;
+ unsigned char *data;
+ loff_t offset, size;
+ struct vtoc_format1_label f1;
+ int secperblk;
+
+ snprintf(tmp, sizeof(tmp), "VOL1/%8s:", name);
+ strlcat(state->pp_buf, tmp, PAGE_SIZE);
+ /*
+ * get start of VTOC from the disk label and then search for format1
+ * and format8 labels
+ */
+ secperblk = blocksize >> 9;
+ blk = cchhb2blk(&label->vol.vtoc, geo) + 1;
+ counter = 0;
+ data = read_part_sector(state, blk * secperblk, §);
+ while (data != NULL) {
+ memcpy(&f1, data, sizeof(struct vtoc_format1_label));
+ put_dev_sector(sect);
+ /* skip FMT4 / FMT5 / FMT7 labels */
+ if (f1.DS1FMTID == _ascebc['4']
+ || f1.DS1FMTID == _ascebc['5']
+ || f1.DS1FMTID == _ascebc['7']
+ || f1.DS1FMTID == _ascebc['9']) {
+ blk++;
+ data = read_part_sector(state, blk * secperblk, §);
+ continue;
+ }
+ /* only FMT1 and 8 labels valid at this point */
+ if (f1.DS1FMTID != _ascebc['1'] &&
+ f1.DS1FMTID != _ascebc['8'])
+ break;
+ /* OK, we got valid partition data */
+ offset = cchh2blk(&f1.DS1EXT1.llimit, geo);
+ size = cchh2blk(&f1.DS1EXT1.ulimit, geo) -
+ offset + geo->sectors;
+ offset *= secperblk;
+ size *= secperblk;
+ if (counter >= state->limit)
+ break;
+ put_partition(state, counter + 1, offset, size);
+ counter++;
+ blk++;
+ data = read_part_sector(state, blk * secperblk, §);
+ }
+ strlcat(state->pp_buf, "\n", PAGE_SIZE);
+
+ if (!data)
+ return -1;
+
+ return 1;
+}
+
+static int find_lnx1_partitions(struct parsed_partitions *state,
+ struct hd_geometry *geo,
+ int blocksize,
+ char name[],
+ union label_t *label,
+ sector_t labelsect,
+ loff_t i_size,
+ dasd_information2_t *info)
+{
+ loff_t offset, geo_size, size;
+ char tmp[64];
+ int secperblk;
+
+ snprintf(tmp, sizeof(tmp), "LNX1/%8s:", name);
+ strlcat(state->pp_buf, tmp, PAGE_SIZE);
+ secperblk = blocksize >> 9;
+ if (label->lnx.ldl_version == 0xf2) {
+ size = label->lnx.formatted_blocks * secperblk;
+ } else {
+ /*
+ * Formated w/o large volume support. If the sanity check
+ * 'size based on geo == size based on i_size' is true, then
+ * we can safely assume that we know the formatted size of
+ * the disk, otherwise we need additional information
+ * that we can only get from a real DASD device.
+ */
+ geo_size = geo->cylinders * geo->heads
+ * geo->sectors * secperblk;
+ size = i_size >> 9;
+ if (size != geo_size) {
+ if (!info) {
+ strlcat(state->pp_buf, "\n", PAGE_SIZE);
+ return 1;
+ }
+ if (!strcmp(info->type, "ECKD"))
+ if (geo_size < size)
+ size = geo_size;
+ /* else keep size based on i_size */
+ }
+ }
+ /* first and only partition starts in the first block after the label */
+ offset = labelsect + secperblk;
+ put_partition(state, 1, offset, size - offset);
+ strlcat(state->pp_buf, "\n", PAGE_SIZE);
+ return 1;
+}
+
+static int find_cms1_partitions(struct parsed_partitions *state,
+ struct hd_geometry *geo,
+ int blocksize,
+ char name[],
+ union label_t *label,
+ sector_t labelsect)
+{
+ loff_t offset, size;
+ char tmp[64];
+ int secperblk;
+
+ /*
+ * VM style CMS1 labeled disk
+ */
+ blocksize = label->cms.block_size;
+ secperblk = blocksize >> 9;
+ if (label->cms.disk_offset != 0) {
+ snprintf(tmp, sizeof(tmp), "CMS1/%8s(MDSK):", name);
+ strlcat(state->pp_buf, tmp, PAGE_SIZE);
+ /* disk is reserved minidisk */
+ offset = label->cms.disk_offset * secperblk;
+ size = (label->cms.block_count - 1) * secperblk;
+ } else {
+ snprintf(tmp, sizeof(tmp), "CMS1/%8s:", name);
+ strlcat(state->pp_buf, tmp, PAGE_SIZE);
+ /*
+ * Special case for FBA devices:
+ * If an FBA device is CMS formatted with blocksize > 512 byte
+ * and the DIAG discipline is used, then the CMS label is found
+ * in sector 1 instead of block 1. However, the partition is
+ * still supposed to start in block 2.
+ */
+ if (labelsect == 1)
+ offset = 2 * secperblk;
+ else
+ offset = labelsect + secperblk;
+ size = label->cms.block_count * secperblk;
+ }
+
+ put_partition(state, 1, offset, size-offset);
+ strlcat(state->pp_buf, "\n", PAGE_SIZE);
+ return 1;
+}
+
+
/*
+ * This is the main function, called by check.c
*/
int ibm_partition(struct parsed_partitions *state)
{
struct block_device *bdev = state->bdev;
int blocksize, res;
- loff_t i_size, offset, size, fmt_size;
+ loff_t i_size, offset, size;
dasd_information2_t *info;
struct hd_geometry *geo;
char type[5] = {0,};
char name[7] = {0,};
- union label_t {
- struct vtoc_volume_label_cdl vol;
- struct vtoc_volume_label_ldl lnx;
- struct vtoc_cms_label cms;
- } *label;
- unsigned char *data;
- Sector sect;
sector_t labelsect;
- char tmp[64];
+ union label_t *label;
res = 0;
blocksize = bdev_logical_block_size(bdev);
@@ -84,7 +305,6 @@
i_size = i_size_read(bdev->bd_inode);
if (i_size == 0)
goto out_exit;
-
info = kmalloc(sizeof(dasd_information2_t), GFP_KERNEL);
if (info == NULL)
goto out_exit;
@@ -94,176 +314,45 @@
label = kmalloc(sizeof(union label_t), GFP_KERNEL);
if (label == NULL)
goto out_nolab;
-
- if (ioctl_by_bdev(bdev, BIODASDINFO2, (unsigned long)info) != 0 ||
- ioctl_by_bdev(bdev, HDIO_GETGEO, (unsigned long)geo) != 0)
+ if (ioctl_by_bdev(bdev, HDIO_GETGEO, (unsigned long)geo) != 0)
goto out_freeall;
-
- /*
- * Special case for FBA disks: label sector does not depend on
- * blocksize.
- */
- if ((info->cu_type == 0x6310 && info->dev_type == 0x9336) ||
- (info->cu_type == 0x3880 && info->dev_type == 0x3370))
- labelsect = info->label_block;
- else
- labelsect = info->label_block * (blocksize >> 9);
-
- /*
- * Get volume label, extract name and type.
- */
- data = read_part_sector(state, labelsect, §);
- if (data == NULL)
- goto out_readerr;
-
- memcpy(label, data, sizeof(union label_t));
- put_dev_sector(sect);
-
- if ((!info->FBA_layout) && (!strcmp(info->type, "ECKD"))) {
- strncpy(type, label->vol.vollbl, 4);
- strncpy(name, label->vol.volid, 6);
- } else {
- strncpy(type, label->lnx.vollbl, 4);
- strncpy(name, label->lnx.volid, 6);
+ if (ioctl_by_bdev(bdev, BIODASDINFO2, (unsigned long)info) != 0) {
+ kfree(info);
+ info = NULL;
}
- EBCASC(type, 4);
- EBCASC(name, 6);
- res = 1;
-
- /*
- * Three different formats: LDL, CDL and unformated disk
- *
- * identified by info->format
- *
- * unformated disks we do not have to care about
- */
- if (info->format == DASD_FORMAT_LDL) {
- if (strncmp(type, "CMS1", 4) == 0) {
- /*
- * VM style CMS1 labeled disk
- */
- blocksize = label->cms.block_size;
- if (label->cms.disk_offset != 0) {
- snprintf(tmp, sizeof(tmp), "CMS1/%8s(MDSK):", name);
- strlcat(state->pp_buf, tmp, PAGE_SIZE);
- /* disk is reserved minidisk */
- offset = label->cms.disk_offset;
- size = (label->cms.block_count - 1)
- * (blocksize >> 9);
- } else {
- snprintf(tmp, sizeof(tmp), "CMS1/%8s:", name);
- strlcat(state->pp_buf, tmp, PAGE_SIZE);
- offset = (info->label_block + 1);
- size = label->cms.block_count
- * (blocksize >> 9);
- }
- put_partition(state, 1, offset*(blocksize >> 9),
- size-offset*(blocksize >> 9));
- } else {
- if (strncmp(type, "LNX1", 4) == 0) {
- snprintf(tmp, sizeof(tmp), "LNX1/%8s:", name);
- strlcat(state->pp_buf, tmp, PAGE_SIZE);
- if (label->lnx.ldl_version == 0xf2) {
- fmt_size = label->lnx.formatted_blocks
- * (blocksize >> 9);
- } else if (!strcmp(info->type, "ECKD")) {
- /* formated w/o large volume support */
- fmt_size = geo->cylinders * geo->heads
- * geo->sectors * (blocksize >> 9);
- } else {
- /* old label and no usable disk geometry
- * (e.g. DIAG) */
- fmt_size = i_size >> 9;
- }
- size = i_size >> 9;
- if (fmt_size < size)
- size = fmt_size;
- offset = (info->label_block + 1);
- } else {
- /* unlabeled disk */
- strlcat(state->pp_buf, "(nonl)", PAGE_SIZE);
- size = i_size >> 9;
- offset = (info->label_block + 1);
- }
- put_partition(state, 1, offset*(blocksize >> 9),
- size-offset*(blocksize >> 9));
+ if (find_label(state, info, geo, blocksize, &labelsect, name, type,
+ label)) {
+ if (!strncmp(type, "VOL1", 4)) {
+ res = find_vol1_partitions(state, geo, blocksize, name,
+ label);
+ } else if (!strncmp(type, "LNX1", 4)) {
+ res = find_lnx1_partitions(state, geo, blocksize, name,
+ label, labelsect, i_size,
+ info);
+ } else if (!strncmp(type, "CMS1", 4)) {
+ res = find_cms1_partitions(state, geo, blocksize, name,
+ label, labelsect);
}
- } else if (info->format == DASD_FORMAT_CDL) {
+ } else if (info) {
/*
- * New style CDL formatted disk
+ * ugly but needed for backward compatibility:
+ * If the block device is a DASD (i.e. BIODASDINFO2 works),
+ * then we claim it in any case, even though it has no valid
+ * label. If it has the LDL format, then we simply define a
+ * partition as if it had an LNX1 label.
*/
- sector_t blk;
- int counter;
+ res = 1;
+ if (info->format == DASD_FORMAT_LDL) {
+ strlcat(state->pp_buf, "(nonl)", PAGE_SIZE);
+ size = i_size >> 9;
+ offset = (info->label_block + 1) * (blocksize >> 9);
+ put_partition(state, 1, offset, size-offset);
+ strlcat(state->pp_buf, "\n", PAGE_SIZE);
+ }
+ } else
+ res = 0;
- /*
- * check if VOL1 label is available
- * if not, something is wrong, skipping partition detection
- */
- if (strncmp(type, "VOL1", 4) == 0) {
- snprintf(tmp, sizeof(tmp), "VOL1/%8s:", name);
- strlcat(state->pp_buf, tmp, PAGE_SIZE);
- /*
- * get block number and read then go through format1
- * labels
- */
- blk = cchhb2blk(&label->vol.vtoc, geo) + 1;
- counter = 0;
- data = read_part_sector(state, blk * (blocksize/512),
- §);
- while (data != NULL) {
- struct vtoc_format1_label f1;
-
- memcpy(&f1, data,
- sizeof(struct vtoc_format1_label));
- put_dev_sector(sect);
-
- /* skip FMT4 / FMT5 / FMT7 labels */
- if (f1.DS1FMTID == _ascebc['4']
- || f1.DS1FMTID == _ascebc['5']
- || f1.DS1FMTID == _ascebc['7']
- || f1.DS1FMTID == _ascebc['9']) {
- blk++;
- data = read_part_sector(state,
- blk * (blocksize/512), §);
- continue;
- }
-
- /* only FMT1 and 8 labels valid at this point */
- if (f1.DS1FMTID != _ascebc['1'] &&
- f1.DS1FMTID != _ascebc['8'])
- break;
-
- /* OK, we got valid partition data */
- offset = cchh2blk(&f1.DS1EXT1.llimit, geo);
- size = cchh2blk(&f1.DS1EXT1.ulimit, geo) -
- offset + geo->sectors;
- if (counter >= state->limit)
- break;
- put_partition(state, counter + 1,
- offset * (blocksize >> 9),
- size * (blocksize >> 9));
- counter++;
- blk++;
- data = read_part_sector(state,
- blk * (blocksize/512), §);
- }
-
- if (!data)
- /* Are we not supposed to report this ? */
- goto out_readerr;
- } else
- printk(KERN_INFO "Expected Label VOL1 not "
- "found, treating as CDL formated Disk");
-
- }
-
- strlcat(state->pp_buf, "\n", PAGE_SIZE);
- goto out_freeall;
-
-
-out_readerr:
- res = -1;
out_freeall:
kfree(label);
out_nolab:
diff --git a/crypto/authenc.c b/crypto/authenc.c
index 5ef7ba6..d0583a4 100644
--- a/crypto/authenc.c
+++ b/crypto/authenc.c
@@ -336,7 +336,7 @@
cryptlen += ivsize;
}
- if (sg_is_last(assoc)) {
+ if (req->assoclen && sg_is_last(assoc)) {
authenc_ahash_fn = crypto_authenc_ahash;
sg_init_table(asg, 2);
sg_set_page(asg, sg_page(assoc), assoc->length, assoc->offset);
@@ -490,7 +490,7 @@
cryptlen += ivsize;
}
- if (sg_is_last(assoc)) {
+ if (req->assoclen && sg_is_last(assoc)) {
authenc_ahash_fn = crypto_authenc_ahash;
sg_init_table(asg, 2);
sg_set_page(asg, sg_page(assoc), assoc->length, assoc->offset);
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 8099895..119d58d 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -385,8 +385,8 @@
to override that restriction).
config ACPI_BGRT
- tristate "Boottime Graphics Resource Table support"
- default n
+ bool "Boottime Graphics Resource Table support"
+ depends on EFI
help
This driver adds support for exposing the ACPI Boottime Graphics
Resource Table, which allows the operating system to obtain
diff --git a/drivers/acpi/bgrt.c b/drivers/acpi/bgrt.c
index 6680df3..be60399 100644
--- a/drivers/acpi/bgrt.c
+++ b/drivers/acpi/bgrt.c
@@ -1,5 +1,6 @@
/*
* Copyright 2012 Red Hat, Inc <mjg@redhat.com>
+ * Copyright 2012 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
@@ -11,20 +12,10 @@
#include <linux/init.h>
#include <linux/device.h>
#include <linux/sysfs.h>
-#include <linux/io.h>
-#include <acpi/acpi.h>
-#include <acpi/acpi_bus.h>
+#include <linux/efi-bgrt.h>
-static struct acpi_table_bgrt *bgrt_tab;
static struct kobject *bgrt_kobj;
-struct bmp_header {
- u16 id;
- u32 size;
-} __attribute ((packed));
-
-static struct bmp_header bmp_header;
-
static ssize_t show_version(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -63,18 +54,7 @@
static ssize_t show_image(struct file *file, struct kobject *kobj,
struct bin_attribute *attr, char *buf, loff_t off, size_t count)
{
- int size = attr->size;
- void __iomem *image = attr->private;
-
- if (off >= size) {
- count = 0;
- } else {
- if (off + count > size)
- count = size - off;
-
- memcpy_fromio(buf, image+off, count);
- }
-
+ memcpy(buf, attr->private + off, count);
return count;
}
@@ -101,45 +81,18 @@
static int __init bgrt_init(void)
{
- acpi_status status;
int ret;
- void __iomem *bgrt;
- if (acpi_disabled)
- return -ENODEV;
-
- status = acpi_get_table("BGRT", 0,
- (struct acpi_table_header **)&bgrt_tab);
-
- if (ACPI_FAILURE(status))
+ if (!bgrt_image)
return -ENODEV;
sysfs_bin_attr_init(&image_attr);
-
- bgrt = ioremap(bgrt_tab->image_address, sizeof(struct bmp_header));
-
- if (!bgrt) {
- ret = -EINVAL;
- goto out_err;
- }
-
- memcpy_fromio(&bmp_header, bgrt, sizeof(bmp_header));
- image_attr.size = bmp_header.size;
- iounmap(bgrt);
-
- image_attr.private = ioremap(bgrt_tab->image_address, image_attr.size);
-
- if (!image_attr.private) {
- ret = -EINVAL;
- goto out_err;
- }
-
+ image_attr.private = bgrt_image;
+ image_attr.size = bgrt_image_size;
bgrt_kobj = kobject_create_and_add("bgrt", acpi_kobj);
- if (!bgrt_kobj) {
- ret = -EINVAL;
- goto out_iounmap;
- }
+ if (!bgrt_kobj)
+ return -EINVAL;
ret = sysfs_create_group(bgrt_kobj, &bgrt_attribute_group);
if (ret)
@@ -155,22 +108,11 @@
sysfs_remove_group(bgrt_kobj, &bgrt_attribute_group);
out_kobject:
kobject_put(bgrt_kobj);
-out_iounmap:
- iounmap(image_attr.private);
-out_err:
return ret;
}
-static void __exit bgrt_exit(void)
-{
- iounmap(image_attr.private);
- sysfs_remove_group(bgrt_kobj, &bgrt_attribute_group);
- sysfs_remove_bin_file(bgrt_kobj, &image_attr);
-}
-
module_init(bgrt_init);
-module_exit(bgrt_exit);
-MODULE_AUTHOR("Matthew Garrett");
+MODULE_AUTHOR("Matthew Garrett, Josh Triplett <josh@joshtriplett.org>");
MODULE_DESCRIPTION("BGRT boot graphic support");
MODULE_LICENSE("GPL");
diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
index 9628652..e059695 100644
--- a/drivers/acpi/bus.c
+++ b/drivers/acpi/bus.c
@@ -237,6 +237,16 @@
} else if (result == ACPI_STATE_D3_HOT) {
result = ACPI_STATE_D3;
}
+
+ /*
+ * If we were unsure about the device parent's power state up to this
+ * point, the fact that the device is in D0 implies that the parent has
+ * to be in D0 too.
+ */
+ if (device->parent && device->parent->power.state == ACPI_STATE_UNKNOWN
+ && result == ACPI_STATE_D0)
+ device->parent->power.state = ACPI_STATE_D0;
+
*state = result;
out:
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index 72a2c98..bce469c 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -27,7 +27,7 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
-#include <linux/spinlock.h>
+#include <linux/mutex.h>
#include <linux/pm.h>
#include <linux/pm_runtime.h>
#include <linux/pci.h>
@@ -71,9 +71,11 @@
},
};
+/* Lock to protect both acpi_pci_roots and acpi_pci_drivers lists */
+static DEFINE_MUTEX(acpi_pci_root_lock);
static LIST_HEAD(acpi_pci_roots);
+static LIST_HEAD(acpi_pci_drivers);
-static struct acpi_pci_driver *sub_driver;
static DEFINE_MUTEX(osc_lock);
int acpi_pci_register_driver(struct acpi_pci_driver *driver)
@@ -81,55 +83,46 @@
int n = 0;
struct acpi_pci_root *root;
- struct acpi_pci_driver **pptr = &sub_driver;
- while (*pptr)
- pptr = &(*pptr)->next;
- *pptr = driver;
-
- if (!driver->add)
- return 0;
-
- list_for_each_entry(root, &acpi_pci_roots, node) {
- driver->add(root->device->handle);
- n++;
- }
+ mutex_lock(&acpi_pci_root_lock);
+ list_add_tail(&driver->node, &acpi_pci_drivers);
+ if (driver->add)
+ list_for_each_entry(root, &acpi_pci_roots, node) {
+ driver->add(root);
+ n++;
+ }
+ mutex_unlock(&acpi_pci_root_lock);
return n;
}
-
EXPORT_SYMBOL(acpi_pci_register_driver);
void acpi_pci_unregister_driver(struct acpi_pci_driver *driver)
{
struct acpi_pci_root *root;
- struct acpi_pci_driver **pptr = &sub_driver;
- while (*pptr) {
- if (*pptr == driver)
- break;
- pptr = &(*pptr)->next;
- }
- BUG_ON(!*pptr);
- *pptr = (*pptr)->next;
-
- if (!driver->remove)
- return;
-
- list_for_each_entry(root, &acpi_pci_roots, node)
- driver->remove(root->device->handle);
+ mutex_lock(&acpi_pci_root_lock);
+ list_del(&driver->node);
+ if (driver->remove)
+ list_for_each_entry(root, &acpi_pci_roots, node)
+ driver->remove(root);
+ mutex_unlock(&acpi_pci_root_lock);
}
-
EXPORT_SYMBOL(acpi_pci_unregister_driver);
acpi_handle acpi_get_pci_rootbridge_handle(unsigned int seg, unsigned int bus)
{
struct acpi_pci_root *root;
+ acpi_handle handle = NULL;
+ mutex_lock(&acpi_pci_root_lock);
list_for_each_entry(root, &acpi_pci_roots, node)
if ((root->segment == (u16) seg) &&
- (root->secondary.start == (u16) bus))
- return root->device->handle;
- return NULL;
+ (root->secondary.start == (u16) bus)) {
+ handle = root->device->handle;
+ break;
+ }
+ mutex_unlock(&acpi_pci_root_lock);
+ return handle;
}
EXPORT_SYMBOL_GPL(acpi_get_pci_rootbridge_handle);
@@ -277,12 +270,15 @@
struct acpi_pci_root *acpi_pci_find_root(acpi_handle handle)
{
struct acpi_pci_root *root;
+ struct acpi_device *device;
- list_for_each_entry(root, &acpi_pci_roots, node) {
- if (root->device->handle == handle)
- return root;
- }
- return NULL;
+ if (acpi_bus_get_device(handle, &device) ||
+ acpi_match_device_ids(device, root_device_ids))
+ return NULL;
+
+ root = acpi_driver_data(device);
+
+ return root;
}
EXPORT_SYMBOL_GPL(acpi_pci_find_root);
@@ -518,8 +514,9 @@
* TBD: Need PCI interface for enumeration/configuration of roots.
*/
- /* TBD: Locking */
+ mutex_lock(&acpi_pci_root_lock);
list_add_tail(&root->node, &acpi_pci_roots);
+ mutex_unlock(&acpi_pci_root_lock);
printk(KERN_INFO PREFIX "%s [%s] (domain %04x %pR)\n",
acpi_device_name(device), acpi_device_bid(device),
@@ -538,7 +535,7 @@
"Bus %04x:%02x not present in PCI namespace\n",
root->segment, (unsigned int)root->secondary.start);
result = -ENODEV;
- goto end;
+ goto out_del_root;
}
/*
@@ -548,7 +545,7 @@
*/
result = acpi_pci_bind_root(device);
if (result)
- goto end;
+ goto out_del_root;
/*
* PCI Routing Table
@@ -633,9 +630,11 @@
return 0;
+out_del_root:
+ mutex_lock(&acpi_pci_root_lock);
+ list_del(&root->node);
+ mutex_unlock(&acpi_pci_root_lock);
end:
- if (!list_empty(&root->node))
- list_del(&root->node);
kfree(root);
return result;
}
@@ -643,18 +642,34 @@
static int acpi_pci_root_start(struct acpi_device *device)
{
struct acpi_pci_root *root = acpi_driver_data(device);
+ struct acpi_pci_driver *driver;
+
+ mutex_lock(&acpi_pci_root_lock);
+ list_for_each_entry(driver, &acpi_pci_drivers, node)
+ if (driver->add)
+ driver->add(root);
+ mutex_unlock(&acpi_pci_root_lock);
pci_bus_add_devices(root->bus);
+
return 0;
}
static int acpi_pci_root_remove(struct acpi_device *device, int type)
{
struct acpi_pci_root *root = acpi_driver_data(device);
+ struct acpi_pci_driver *driver;
+
+ mutex_lock(&acpi_pci_root_lock);
+ list_for_each_entry(driver, &acpi_pci_drivers, node)
+ if (driver->remove)
+ driver->remove(root);
device_set_run_wake(root->bus->bridge, false);
pci_acpi_remove_bus_pm_notifier(device);
+ list_del(&root->node);
+ mutex_unlock(&acpi_pci_root_lock);
kfree(root);
return 0;
}
diff --git a/drivers/acpi/pci_slot.c b/drivers/acpi/pci_slot.c
index e50e31a..d22585f 100644
--- a/drivers/acpi/pci_slot.c
+++ b/drivers/acpi/pci_slot.c
@@ -67,8 +67,8 @@
struct list_head list; /* node in the list of slots */
};
-static int acpi_pci_slot_add(acpi_handle handle);
-static void acpi_pci_slot_remove(acpi_handle handle);
+static int acpi_pci_slot_add(struct acpi_pci_root *root);
+static void acpi_pci_slot_remove(struct acpi_pci_root *root);
static LIST_HEAD(slot_list);
static DEFINE_MUTEX(slot_list_lock);
@@ -233,45 +233,20 @@
/*
* walk_root_bridge - generic root bridge walker
- * @handle: points to an acpi_pci_root
+ * @root: poiner of an acpi_pci_root
* @user_function: user callback for slot objects
*
* Call user_function for all objects underneath this root bridge.
* Walk p2p bridges underneath us and call user_function on those too.
*/
static int
-walk_root_bridge(acpi_handle handle, acpi_walk_callback user_function)
+walk_root_bridge(struct acpi_pci_root *root, acpi_walk_callback user_function)
{
- int seg, bus;
- unsigned long long tmp;
acpi_status status;
- acpi_handle dummy_handle;
- struct pci_bus *pci_bus;
+ acpi_handle handle = root->device->handle;
+ struct pci_bus *pci_bus = root->bus;
struct callback_args context;
- /* If the bridge doesn't have _STA, we assume it is always there */
- status = acpi_get_handle(handle, "_STA", &dummy_handle);
- if (ACPI_SUCCESS(status)) {
- status = acpi_evaluate_integer(handle, "_STA", NULL, &tmp);
- if (ACPI_FAILURE(status)) {
- info("%s: _STA evaluation failure\n", __func__);
- return 0;
- }
- if ((tmp & ACPI_STA_DEVICE_FUNCTIONING) == 0)
- /* don't register this object */
- return 0;
- }
-
- status = acpi_evaluate_integer(handle, "_SEG", NULL, &tmp);
- seg = ACPI_SUCCESS(status) ? tmp : 0;
-
- status = acpi_evaluate_integer(handle, "_BBN", NULL, &tmp);
- bus = ACPI_SUCCESS(status) ? tmp : 0;
-
- pci_bus = pci_find_bus(seg, bus);
- if (!pci_bus)
- return 0;
-
context.pci_bus = pci_bus;
context.user_function = user_function;
context.root_handle = handle;
@@ -295,11 +270,11 @@
* @handle: points to an acpi_pci_root
*/
static int
-acpi_pci_slot_add(acpi_handle handle)
+acpi_pci_slot_add(struct acpi_pci_root *root)
{
acpi_status status;
- status = walk_root_bridge(handle, register_slot);
+ status = walk_root_bridge(root, register_slot);
if (ACPI_FAILURE(status))
err("%s: register_slot failure - %d\n", __func__, status);
@@ -311,10 +286,11 @@
* @handle: points to an acpi_pci_root
*/
static void
-acpi_pci_slot_remove(acpi_handle handle)
+acpi_pci_slot_remove(struct acpi_pci_root *root)
{
struct acpi_pci_slot *slot, *tmp;
struct pci_bus *pbus;
+ acpi_handle handle = root->device->handle;
mutex_lock(&slot_list_lock);
list_for_each_entry_safe(slot, tmp, &slot_list, list) {
diff --git a/drivers/acpi/power.c b/drivers/acpi/power.c
index fc18034..40e38a0 100644
--- a/drivers/acpi/power.c
+++ b/drivers/acpi/power.c
@@ -107,6 +107,7 @@
/* List of devices relying on this power resource */
struct acpi_power_resource_device *devices;
+ struct mutex devices_lock;
};
static struct list_head acpi_power_resource_list;
@@ -225,7 +226,6 @@
static int __acpi_power_on(struct acpi_power_resource *resource)
{
- struct acpi_power_resource_device *device_list = resource->devices;
acpi_status status = AE_OK;
status = acpi_evaluate_object(resource->device->handle, "_ON", NULL, NULL);
@@ -238,19 +238,15 @@
ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Power resource [%s] turned on\n",
resource->name));
- while (device_list) {
- acpi_power_on_device(device_list->device);
-
- device_list = device_list->next;
- }
-
return 0;
}
static int acpi_power_on(acpi_handle handle)
{
int result = 0;
+ bool resume_device = false;
struct acpi_power_resource *resource = NULL;
+ struct acpi_power_resource_device *device_list;
result = acpi_power_get_context(handle, &resource);
if (result)
@@ -266,10 +262,25 @@
result = __acpi_power_on(resource);
if (result)
resource->ref_count--;
+ else
+ resume_device = true;
}
mutex_unlock(&resource->resource_lock);
+ if (!resume_device)
+ return result;
+
+ mutex_lock(&resource->devices_lock);
+
+ device_list = resource->devices;
+ while (device_list) {
+ acpi_power_on_device(device_list->device);
+ device_list = device_list->next;
+ }
+
+ mutex_unlock(&resource->devices_lock);
+
return result;
}
@@ -355,7 +366,7 @@
if (acpi_power_get_context(res_handle, &resource))
return;
- mutex_lock(&resource->resource_lock);
+ mutex_lock(&resource->devices_lock);
prev = NULL;
curr = resource->devices;
while (curr) {
@@ -372,7 +383,7 @@
prev = curr;
curr = curr->next;
}
- mutex_unlock(&resource->resource_lock);
+ mutex_unlock(&resource->devices_lock);
}
/* Unlink dev from all power resources in _PR0 */
@@ -414,10 +425,10 @@
power_resource_device->device = powered_device;
- mutex_lock(&resource->resource_lock);
+ mutex_lock(&resource->devices_lock);
power_resource_device->next = resource->devices;
resource->devices = power_resource_device;
- mutex_unlock(&resource->resource_lock);
+ mutex_unlock(&resource->devices_lock);
return 0;
}
@@ -462,7 +473,7 @@
return ret;
no_power_resource:
- printk(KERN_WARNING PREFIX "Invalid Power Resource to register!");
+ printk(KERN_DEBUG PREFIX "Invalid Power Resource to register!");
return -ENODEV;
}
EXPORT_SYMBOL_GPL(acpi_power_resource_register_device);
@@ -721,6 +732,7 @@
resource->device = device;
mutex_init(&resource->resource_lock);
+ mutex_init(&resource->devices_lock);
strcpy(resource->name, device->pnp.bus_id);
strcpy(acpi_device_name(device), ACPI_POWER_DEVICE_NAME);
strcpy(acpi_device_class(device), ACPI_POWER_CLASS);
diff --git a/drivers/ata/Kconfig b/drivers/ata/Kconfig
index 2be8ef1..27cecd3 100644
--- a/drivers/ata/Kconfig
+++ b/drivers/ata/Kconfig
@@ -115,7 +115,7 @@
If unsure, say N.
config ATA_SFF
- bool "ATA SFF support"
+ bool "ATA SFF support (for legacy IDE and PATA)"
default y
help
This option adds support for ATA controllers with SFF
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 062e6a1..7862d17 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -256,10 +256,21 @@
{ PCI_VDEVICE(INTEL, 0x8c07), board_ahci }, /* Lynx Point RAID */
{ PCI_VDEVICE(INTEL, 0x8c0e), board_ahci }, /* Lynx Point RAID */
{ PCI_VDEVICE(INTEL, 0x8c0f), board_ahci }, /* Lynx Point RAID */
+ { PCI_VDEVICE(INTEL, 0x9c02), board_ahci }, /* Lynx Point-LP AHCI */
+ { PCI_VDEVICE(INTEL, 0x9c03), board_ahci }, /* Lynx Point-LP AHCI */
+ { PCI_VDEVICE(INTEL, 0x9c04), board_ahci }, /* Lynx Point-LP RAID */
+ { PCI_VDEVICE(INTEL, 0x9c05), board_ahci }, /* Lynx Point-LP RAID */
+ { PCI_VDEVICE(INTEL, 0x9c06), board_ahci }, /* Lynx Point-LP RAID */
+ { PCI_VDEVICE(INTEL, 0x9c07), board_ahci }, /* Lynx Point-LP RAID */
+ { PCI_VDEVICE(INTEL, 0x9c0e), board_ahci }, /* Lynx Point-LP RAID */
+ { PCI_VDEVICE(INTEL, 0x9c0f), board_ahci }, /* Lynx Point-LP RAID */
/* JMicron 360/1/3/5/6, match class to avoid IDE function */
{ PCI_VENDOR_ID_JMICRON, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID,
PCI_CLASS_STORAGE_SATA_AHCI, 0xffffff, board_ahci_ign_iferr },
+ /* JMicron 362B and 362C have an AHCI function with IDE class code */
+ { PCI_VDEVICE(JMICRON, 0x2362), board_ahci_ign_iferr },
+ { PCI_VDEVICE(JMICRON, 0x236f), board_ahci_ign_iferr },
/* ATI */
{ PCI_VDEVICE(ATI, 0x4380), board_ahci_sb600 }, /* ATI SB600 */
@@ -385,6 +396,8 @@
.driver_data = board_ahci_yes_fbs }, /* 88se9125 */
{ PCI_DEVICE(0x1b4b, 0x917a),
.driver_data = board_ahci_yes_fbs }, /* 88se9172 */
+ { PCI_DEVICE(0x1b4b, 0x9192),
+ .driver_data = board_ahci_yes_fbs }, /* 88se9172 on some Gigabyte */
{ PCI_DEVICE(0x1b4b, 0x91a3),
.driver_data = board_ahci_yes_fbs },
@@ -392,7 +405,10 @@
{ PCI_VDEVICE(PROMISE, 0x3f20), board_ahci }, /* PDC42819 */
/* Asmedia */
- { PCI_VDEVICE(ASMEDIA, 0x0612), board_ahci }, /* ASM1061 */
+ { PCI_VDEVICE(ASMEDIA, 0x0601), board_ahci }, /* ASM1060 */
+ { PCI_VDEVICE(ASMEDIA, 0x0602), board_ahci }, /* ASM1060 */
+ { PCI_VDEVICE(ASMEDIA, 0x0611), board_ahci }, /* ASM1061 */
+ { PCI_VDEVICE(ASMEDIA, 0x0612), board_ahci }, /* ASM1062 */
/* Generic, PCI class code for AHCI */
{ PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID,
diff --git a/drivers/ata/ahci.h b/drivers/ata/ahci.h
index c2594dd..57eb1c2 100644
--- a/drivers/ata/ahci.h
+++ b/drivers/ata/ahci.h
@@ -320,6 +320,7 @@
extern struct ata_port_operations ahci_ops;
extern struct ata_port_operations ahci_pmp_retry_srst_ops;
+unsigned int ahci_dev_classify(struct ata_port *ap);
void ahci_fill_cmd_slot(struct ahci_port_priv *pp, unsigned int tag,
u32 opts);
void ahci_save_initial_config(struct device *dev,
diff --git a/drivers/ata/ata_piix.c b/drivers/ata/ata_piix.c
index 3c809bf..ef773e1 100644
--- a/drivers/ata/ata_piix.c
+++ b/drivers/ata/ata_piix.c
@@ -329,6 +329,14 @@
{ 0x8086, 0x8c08, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
/* SATA Controller IDE (Lynx Point) */
{ 0x8086, 0x8c09, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
+ /* SATA Controller IDE (Lynx Point-LP) */
+ { 0x8086, 0x9c00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
+ /* SATA Controller IDE (Lynx Point-LP) */
+ { 0x8086, 0x9c01, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_sata_snb },
+ /* SATA Controller IDE (Lynx Point-LP) */
+ { 0x8086, 0x9c08, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
+ /* SATA Controller IDE (Lynx Point-LP) */
+ { 0x8086, 0x9c09, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
/* SATA Controller IDE (DH89xxCC) */
{ 0x8086, 0x2326, PCI_ANY_ID, PCI_ANY_ID, 0, 0, ich8_2port_sata },
{ } /* terminate list */
diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index f9eaa82..555c07a 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -1139,7 +1139,7 @@
}
}
-static unsigned int ahci_dev_classify(struct ata_port *ap)
+unsigned int ahci_dev_classify(struct ata_port *ap)
{
void __iomem *port_mmio = ahci_port_base(ap);
struct ata_taskfile tf;
@@ -1153,6 +1153,7 @@
return ata_dev_classify(&tf);
}
+EXPORT_SYMBOL_GPL(ahci_dev_classify);
void ahci_fill_cmd_slot(struct ahci_port_priv *pp, unsigned int tag,
u32 opts)
diff --git a/drivers/ata/libata-acpi.c b/drivers/ata/libata-acpi.c
index 902b5a4..fd9ecf7 100644
--- a/drivers/ata/libata-acpi.c
+++ b/drivers/ata/libata-acpi.c
@@ -60,17 +60,7 @@
if (ap->flags & ATA_FLAG_ACPI_SATA)
return NULL;
- /*
- * If acpi bind operation has already happened, we can get the handle
- * for the port by checking the corresponding scsi_host device's
- * firmware node, otherwise we will need to find out the handle from
- * its parent's acpi node.
- */
- if (ap->scsi_host)
- return DEVICE_ACPI_HANDLE(&ap->scsi_host->shost_gendev);
- else
- return acpi_get_child(DEVICE_ACPI_HANDLE(ap->host->dev),
- ap->port_no);
+ return acpi_get_child(DEVICE_ACPI_HANDLE(ap->host->dev), ap->port_no);
}
EXPORT_SYMBOL(ata_ap_acpi_handle);
@@ -1101,6 +1091,9 @@
if (!*handle)
return -ENODEV;
+ if (ata_acpi_gtm(ap, &ap->__acpi_init_gtm) == 0)
+ ap->pflags |= ATA_PFLAG_INIT_GTM_VALID;
+
return 0;
}
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index fadd586..8e1039c 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4062,7 +4062,7 @@
{ "_NEC DV5800A", NULL, ATA_HORKAGE_NODMA },
{ "SAMSUNG CD-ROM SN-124", "N001", ATA_HORKAGE_NODMA },
{ "Seagate STT20000A", NULL, ATA_HORKAGE_NODMA },
- { "2GB ATA Flash Disk", "ADMA428M", ATA_HORKAGE_NODMA },
+ { " 2GB ATA Flash Disk", "ADMA428M", ATA_HORKAGE_NODMA },
/* Odd clown on sil3726/4726 PMPs */
{ "Config Disk", NULL, ATA_HORKAGE_DISABLE },
@@ -4128,6 +4128,7 @@
/* Devices that do not need bridging limits applied */
{ "MTRON MSP-SATA*", NULL, ATA_HORKAGE_BRIDGE_OK, },
+ { "BUFFALO HD-QSU2/R5", NULL, ATA_HORKAGE_BRIDGE_OK, },
/* Devices which aren't very happy with higher link speeds */
{ "WD My Book", NULL, ATA_HORKAGE_1_5_GBPS, },
diff --git a/drivers/ata/pata_atiixp.c b/drivers/ata/pata_atiixp.c
index 361c75c..24e5105 100644
--- a/drivers/ata/pata_atiixp.c
+++ b/drivers/ata/pata_atiixp.c
@@ -20,6 +20,7 @@
#include <linux/delay.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
+#include <linux/dmi.h>
#define DRV_NAME "pata_atiixp"
#define DRV_VERSION "0.4.6"
@@ -33,11 +34,26 @@
ATIIXP_IDE_UDMA_MODE = 0x56
};
+static const struct dmi_system_id attixp_cable_override_dmi_table[] = {
+ {
+ /* Board has onboard PATA<->SATA converters */
+ .ident = "MSI E350DM-E33",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "MSI"),
+ DMI_MATCH(DMI_BOARD_NAME, "E350DM-E33(MS-7720)"),
+ },
+ },
+ { }
+};
+
static int atiixp_cable_detect(struct ata_port *ap)
{
struct pci_dev *pdev = to_pci_dev(ap->host->dev);
u8 udma;
+ if (dmi_check_system(attixp_cable_override_dmi_table))
+ return ATA_CBL_PATA40_SHORT;
+
/* Hack from drivers/ide/pci. Really we want to know how to do the
raw detection not play follow the bios mode guess */
pci_read_config_byte(pdev, ATIIXP_IDE_UDMA_MODE + ap->port_no, &udma);
diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c
index 78efb03..34d94c7 100644
--- a/drivers/base/dma-contiguous.c
+++ b/drivers/base/dma-contiguous.c
@@ -250,7 +250,7 @@
return -EINVAL;
/* Sanitise input arguments */
- alignment = PAGE_SIZE << max(MAX_ORDER, pageblock_order);
+ alignment = PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order);
base = ALIGN(base, alignment);
size = ALIGN(size, alignment);
limit &= ~(alignment - 1);
diff --git a/drivers/base/regmap/regmap-irq.c b/drivers/base/regmap/regmap-irq.c
index a897346..5b6b1d8e 100644
--- a/drivers/base/regmap/regmap-irq.c
+++ b/drivers/base/regmap/regmap-irq.c
@@ -16,12 +16,14 @@
#include <linux/irq.h>
#include <linux/interrupt.h>
#include <linux/irqdomain.h>
+#include <linux/pm_runtime.h>
#include <linux/slab.h>
#include "internal.h"
struct regmap_irq_chip_data {
struct mutex lock;
+ struct irq_chip irq_chip;
struct regmap *map;
const struct regmap_irq_chip *chip;
@@ -59,6 +61,14 @@
struct regmap_irq_chip_data *d = irq_data_get_irq_chip_data(data);
struct regmap *map = d->map;
int i, ret;
+ u32 reg;
+
+ if (d->chip->runtime_pm) {
+ ret = pm_runtime_get_sync(map->dev);
+ if (ret < 0)
+ dev_err(map->dev, "IRQ sync failed to resume: %d\n",
+ ret);
+ }
/*
* If there's been a change in the mask write it back to the
@@ -66,15 +76,22 @@
* suppress pointless writes.
*/
for (i = 0; i < d->chip->num_regs; i++) {
- ret = regmap_update_bits(d->map, d->chip->mask_base +
- (i * map->reg_stride *
- d->irq_reg_stride),
+ reg = d->chip->mask_base +
+ (i * map->reg_stride * d->irq_reg_stride);
+ if (d->chip->mask_invert)
+ ret = regmap_update_bits(d->map, reg,
+ d->mask_buf_def[i], ~d->mask_buf[i]);
+ else
+ ret = regmap_update_bits(d->map, reg,
d->mask_buf_def[i], d->mask_buf[i]);
if (ret != 0)
dev_err(d->map->dev, "Failed to sync masks in %x\n",
- d->chip->mask_base + (i * map->reg_stride));
+ reg);
}
+ if (d->chip->runtime_pm)
+ pm_runtime_put(map->dev);
+
/* If we've changed our wakeup count propagate it to the parent */
if (d->wake_count < 0)
for (i = d->wake_count; i < 0; i++)
@@ -128,8 +145,7 @@
return 0;
}
-static struct irq_chip regmap_irq_chip = {
- .name = "regmap",
+static const struct irq_chip regmap_irq_chip = {
.irq_bus_lock = regmap_irq_lock,
.irq_bus_sync_unlock = regmap_irq_sync_unlock,
.irq_disable = regmap_irq_disable,
@@ -144,6 +160,16 @@
struct regmap *map = data->map;
int ret, i;
bool handled = false;
+ u32 reg;
+
+ if (chip->runtime_pm) {
+ ret = pm_runtime_get_sync(map->dev);
+ if (ret < 0) {
+ dev_err(map->dev, "IRQ thread failed to resume: %d\n",
+ ret);
+ return IRQ_NONE;
+ }
+ }
/*
* Ignore masked IRQs and ack if we need to; we ack early so
@@ -160,20 +186,20 @@
if (ret != 0) {
dev_err(map->dev, "Failed to read IRQ status: %d\n",
ret);
+ if (chip->runtime_pm)
+ pm_runtime_put(map->dev);
return IRQ_NONE;
}
data->status_buf[i] &= ~data->mask_buf[i];
if (data->status_buf[i] && chip->ack_base) {
- ret = regmap_write(map, chip->ack_base +
- (i * map->reg_stride *
- data->irq_reg_stride),
- data->status_buf[i]);
+ reg = chip->ack_base +
+ (i * map->reg_stride * data->irq_reg_stride);
+ ret = regmap_write(map, reg, data->status_buf[i]);
if (ret != 0)
dev_err(map->dev, "Failed to ack 0x%x: %d\n",
- chip->ack_base + (i * map->reg_stride),
- ret);
+ reg, ret);
}
}
@@ -185,6 +211,9 @@
}
}
+ if (chip->runtime_pm)
+ pm_runtime_put(map->dev);
+
if (handled)
return IRQ_HANDLED;
else
@@ -197,7 +226,7 @@
struct regmap_irq_chip_data *data = h->host_data;
irq_set_chip_data(virq, data);
- irq_set_chip_and_handler(virq, ®map_irq_chip, handle_edge_irq);
+ irq_set_chip(virq, &data->irq_chip);
irq_set_nested_thread(virq, 1);
/* ARM needs us to explicitly flag the IRQ as valid
@@ -238,6 +267,7 @@
struct regmap_irq_chip_data *d;
int i;
int ret = -ENOMEM;
+ u32 reg;
for (i = 0; i < chip->num_irqs; i++) {
if (chip->irqs[i].reg_offset % map->reg_stride)
@@ -284,6 +314,13 @@
goto err_alloc;
}
+ d->irq_chip = regmap_irq_chip;
+ d->irq_chip.name = chip->name;
+ if (!chip->wake_base) {
+ d->irq_chip.irq_set_wake = NULL;
+ d->irq_chip.flags |= IRQCHIP_MASK_ON_SUSPEND |
+ IRQCHIP_SKIP_SET_WAKE;
+ }
d->irq = irq;
d->map = map;
d->chip = chip;
@@ -303,16 +340,37 @@
/* Mask all the interrupts by default */
for (i = 0; i < chip->num_regs; i++) {
d->mask_buf[i] = d->mask_buf_def[i];
- ret = regmap_write(map, chip->mask_base + (i * map->reg_stride
- * d->irq_reg_stride),
- d->mask_buf[i]);
+ reg = chip->mask_base +
+ (i * map->reg_stride * d->irq_reg_stride);
+ if (chip->mask_invert)
+ ret = regmap_update_bits(map, reg,
+ d->mask_buf[i], ~d->mask_buf[i]);
+ else
+ ret = regmap_update_bits(map, reg,
+ d->mask_buf[i], d->mask_buf[i]);
if (ret != 0) {
dev_err(map->dev, "Failed to set masks in 0x%x: %d\n",
- chip->mask_base + (i * map->reg_stride), ret);
+ reg, ret);
goto err_alloc;
}
}
+ /* Wake is disabled by default */
+ if (d->wake_buf) {
+ for (i = 0; i < chip->num_regs; i++) {
+ d->wake_buf[i] = d->mask_buf_def[i];
+ reg = chip->wake_base +
+ (i * map->reg_stride * d->irq_reg_stride);
+ ret = regmap_update_bits(map, reg, d->wake_buf[i],
+ d->wake_buf[i]);
+ if (ret != 0) {
+ dev_err(map->dev, "Failed to set masks in 0x%x: %d\n",
+ reg, ret);
+ goto err_alloc;
+ }
+ }
+ }
+
if (irq_base)
d->domain = irq_domain_add_legacy(map->dev->of_node,
chip->num_irqs, irq_base, 0,
diff --git a/drivers/base/regmap/regmap.c b/drivers/base/regmap/regmap.c
index c241ae2..52069d2 100644
--- a/drivers/base/regmap/regmap.c
+++ b/drivers/base/regmap/regmap.c
@@ -659,13 +659,12 @@
* new cache. This can be used to restore the cache to defaults or to
* update the cache configuration to reflect runtime discovery of the
* hardware.
+ *
+ * No explicit locking is done here, the user needs to ensure that
+ * this function will not race with other calls to regmap.
*/
int regmap_reinit_cache(struct regmap *map, const struct regmap_config *config)
{
- int ret;
-
- map->lock(map);
-
regcache_exit(map);
regmap_debugfs_exit(map);
@@ -681,11 +680,7 @@
map->cache_bypass = false;
map->cache_only = false;
- ret = regcache_init(map, config);
-
- map->unlock(map);
-
- return ret;
+ return regcache_init(map, config);
}
EXPORT_SYMBOL_GPL(regmap_reinit_cache);
diff --git a/drivers/block/aoe/aoecmd.c b/drivers/block/aoe/aoecmd.c
index de0435e..887f68f 100644
--- a/drivers/block/aoe/aoecmd.c
+++ b/drivers/block/aoe/aoecmd.c
@@ -35,6 +35,7 @@
skb_reset_mac_header(skb);
skb_reset_network_header(skb);
skb->protocol = __constant_htons(ETH_P_AOE);
+ skb_checksum_none_assert(skb);
}
return skb;
}
diff --git a/drivers/block/cciss_scsi.c b/drivers/block/cciss_scsi.c
index 38aa6dd..da33111 100644
--- a/drivers/block/cciss_scsi.c
+++ b/drivers/block/cciss_scsi.c
@@ -795,6 +795,7 @@
}
break;
case CMD_PROTOCOL_ERR:
+ cmd->result = DID_ERROR << 16;
dev_warn(&h->pdev->dev,
"%p has protocol error\n", c);
break;
diff --git a/drivers/block/drbd/drbd_bitmap.c b/drivers/block/drbd/drbd_bitmap.c
index ba91b40..d845664 100644
--- a/drivers/block/drbd/drbd_bitmap.c
+++ b/drivers/block/drbd/drbd_bitmap.c
@@ -889,6 +889,7 @@
unsigned int done;
unsigned flags;
#define BM_AIO_COPY_PAGES 1
+#define BM_WRITE_ALL_PAGES 2
int error;
struct kref kref;
};
@@ -1059,7 +1060,8 @@
if (lazy_writeout_upper_idx && i == lazy_writeout_upper_idx)
break;
if (rw & WRITE) {
- if (bm_test_page_unchanged(b->bm_pages[i])) {
+ if (!(flags & BM_WRITE_ALL_PAGES) &&
+ bm_test_page_unchanged(b->bm_pages[i])) {
dynamic_dev_dbg(DEV, "skipped bm write for idx %u\n", i);
continue;
}
@@ -1141,6 +1143,17 @@
}
/**
+ * drbd_bm_write_all() - Write the whole bitmap to its on disk location.
+ * @mdev: DRBD device.
+ *
+ * Will write all pages.
+ */
+int drbd_bm_write_all(struct drbd_conf *mdev) __must_hold(local)
+{
+ return bm_rw(mdev, WRITE, BM_WRITE_ALL_PAGES, 0);
+}
+
+/**
* drbd_bm_lazy_write_out() - Write bitmap pages 0 to @upper_idx-1, if they have changed.
* @mdev: DRBD device.
* @upper_idx: 0: write all changed pages; +ve: page index to stop scanning for changed pages
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index b2ca143..b953cc7 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1469,6 +1469,7 @@
extern int drbd_bm_write_page(struct drbd_conf *mdev, unsigned int idx) __must_hold(local);
extern int drbd_bm_read(struct drbd_conf *mdev) __must_hold(local);
extern int drbd_bm_write(struct drbd_conf *mdev) __must_hold(local);
+extern int drbd_bm_write_all(struct drbd_conf *mdev) __must_hold(local);
extern int drbd_bm_write_copy_pages(struct drbd_conf *mdev) __must_hold(local);
extern unsigned long drbd_bm_ALe_set_all(struct drbd_conf *mdev,
unsigned long al_enr);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index dbe6135..f93a032 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -79,6 +79,7 @@
static void md_sync_timer_fn(unsigned long data);
static int w_bitmap_io(struct drbd_conf *mdev, struct drbd_work *w, int unused);
static int w_go_diskless(struct drbd_conf *mdev, struct drbd_work *w, int unused);
+static void _tl_clear(struct drbd_conf *mdev);
MODULE_AUTHOR("Philipp Reisner <phil@linbit.com>, "
"Lars Ellenberg <lars@linbit.com>");
@@ -432,19 +433,10 @@
/* Actions operating on the disk state, also want to work on
requests that got barrier acked. */
- switch (what) {
- case fail_frozen_disk_io:
- case restart_frozen_disk_io:
- list_for_each_safe(le, tle, &mdev->barrier_acked_requests) {
- req = list_entry(le, struct drbd_request, tl_requests);
- _req_mod(req, what);
- }
- case connection_lost_while_pending:
- case resend:
- break;
- default:
- dev_err(DEV, "what = %d in _tl_restart()\n", what);
+ list_for_each_safe(le, tle, &mdev->barrier_acked_requests) {
+ req = list_entry(le, struct drbd_request, tl_requests);
+ _req_mod(req, what);
}
}
@@ -459,11 +451,16 @@
*/
void tl_clear(struct drbd_conf *mdev)
{
+ spin_lock_irq(&mdev->req_lock);
+ _tl_clear(mdev);
+ spin_unlock_irq(&mdev->req_lock);
+}
+
+static void _tl_clear(struct drbd_conf *mdev)
+{
struct list_head *le, *tle;
struct drbd_request *r;
- spin_lock_irq(&mdev->req_lock);
-
_tl_restart(mdev, connection_lost_while_pending);
/* we expect this list to be empty. */
@@ -482,7 +479,6 @@
memset(mdev->app_reads_hash, 0, APP_R_HSIZE*sizeof(void *));
- spin_unlock_irq(&mdev->req_lock);
}
void tl_restart(struct drbd_conf *mdev, enum drbd_req_event what)
@@ -1476,12 +1472,12 @@
if (ns.susp_fen) {
/* case1: The outdate peer handler is successful: */
if (os.pdsk > D_OUTDATED && ns.pdsk <= D_OUTDATED) {
- tl_clear(mdev);
if (test_bit(NEW_CUR_UUID, &mdev->flags)) {
drbd_uuid_new_current(mdev);
clear_bit(NEW_CUR_UUID, &mdev->flags);
}
spin_lock_irq(&mdev->req_lock);
+ _tl_clear(mdev);
_drbd_set_state(_NS(mdev, susp_fen, 0), CS_VERBOSE, NULL);
spin_unlock_irq(&mdev->req_lock);
}
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index fb9dce8..edb490a 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -674,8 +674,8 @@
la_size_changed && md_moved ? "size changed and md moved" :
la_size_changed ? "size changed" : "md moved");
/* next line implicitly does drbd_suspend_io()+drbd_resume_io() */
- err = drbd_bitmap_io(mdev, &drbd_bm_write,
- "size changed", BM_LOCKED_MASK);
+ err = drbd_bitmap_io(mdev, md_moved ? &drbd_bm_write_all : &drbd_bm_write,
+ "size changed", BM_LOCKED_MASK);
if (err) {
rv = dev_size_error;
goto out;
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index 910335c..01b2ac6 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -695,6 +695,12 @@
break;
case resend:
+ /* Simply complete (local only) READs. */
+ if (!(req->rq_state & RQ_WRITE) && !req->w.cb) {
+ _req_may_be_done(req, m);
+ break;
+ }
+
/* If RQ_NET_OK is already set, we got a P_WRITE_ACK or P_RECV_ACK
before the connection loss (B&C only); only P_BARRIER_ACK was missing.
Trowing them out of the TL here by pretending we got a BARRIER_ACK
@@ -834,7 +840,15 @@
req->private_bio = NULL;
}
if (rw == WRITE) {
- remote = 1;
+ /* Need to replicate writes. Unless it is an empty flush,
+ * which is better mapped to a DRBD P_BARRIER packet,
+ * also for drbd wire protocol compatibility reasons. */
+ if (unlikely(size == 0)) {
+ /* The only size==0 bios we expect are empty flushes. */
+ D_ASSERT(bio->bi_rw & REQ_FLUSH);
+ remote = 0;
+ } else
+ remote = 1;
} else {
/* READ || READA */
if (local) {
@@ -870,8 +884,11 @@
* extent. This waits for any resync activity in the corresponding
* resync extent to finish, and, if necessary, pulls in the target
* extent into the activity log, which involves further disk io because
- * of transactional on-disk meta data updates. */
- if (rw == WRITE && local && !test_bit(AL_SUSPENDED, &mdev->flags)) {
+ * of transactional on-disk meta data updates.
+ * Empty flushes don't need to go into the activity log, they can only
+ * flush data for pending writes which are already in there. */
+ if (rw == WRITE && local && size
+ && !test_bit(AL_SUSPENDED, &mdev->flags)) {
req->rq_state |= RQ_IN_ACT_LOG;
drbd_al_begin_io(mdev, sector);
}
@@ -994,7 +1011,10 @@
if (rw == WRITE && _req_conflicts(req))
goto fail_conflicting;
- list_add_tail(&req->tl_requests, &mdev->newest_tle->requests);
+ /* no point in adding empty flushes to the transfer log,
+ * they are mapped to drbd barriers already. */
+ if (likely(size!=0))
+ list_add_tail(&req->tl_requests, &mdev->newest_tle->requests);
/* NOTE remote first: to get the concurrent write detection right,
* we must register the request before start of local IO. */
@@ -1014,6 +1034,14 @@
mdev->net_conf->on_congestion != OC_BLOCK && mdev->agreed_pro_version >= 96)
maybe_pull_ahead(mdev);
+ /* If this was a flush, queue a drbd barrier/start a new epoch.
+ * Unless the current epoch was empty anyways, or we are not currently
+ * replicating, in which case there is no point. */
+ if (unlikely(bio->bi_rw & REQ_FLUSH)
+ && mdev->newest_tle->n_writes
+ && drbd_should_do_remote(mdev->state))
+ queue_barrier(mdev);
+
spin_unlock_irq(&mdev->req_lock);
kfree(b); /* if someone else has beaten us to it... */
diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c
index a8fddeb..f946d31 100644
--- a/drivers/block/mtip32xx/mtip32xx.c
+++ b/drivers/block/mtip32xx/mtip32xx.c
@@ -1148,11 +1148,15 @@
reply = port->rxfis + RX_FIS_D2H_REG;
task_file_data = readl(port->mmio+PORT_TFDATA);
- if ((task_file_data & 1) || (fis->command == ATA_CMD_SEC_ERASE_UNIT))
+ if (fis->command == ATA_CMD_SEC_ERASE_UNIT)
+ clear_bit(MTIP_DDF_SEC_LOCK_BIT, &port->dd->dd_flag);
+
+ if ((task_file_data & 1))
return false;
if (fis->command == ATA_CMD_SEC_ERASE_PREP) {
set_bit(MTIP_PF_SE_ACTIVE_BIT, &port->flags);
+ set_bit(MTIP_DDF_SEC_LOCK_BIT, &port->dd->dd_flag);
port->ic_pause_timer = jiffies;
return true;
} else if ((fis->command == ATA_CMD_DOWNLOAD_MICRO) &&
@@ -1900,7 +1904,7 @@
int rv = 0, xfer_sz = command[3];
if (xfer_sz) {
- if (user_buffer)
+ if (!user_buffer)
return -EFAULT;
buf = dmam_alloc_coherent(&port->dd->pdev->dev,
@@ -2043,7 +2047,7 @@
*timeout = 240000; /* 4 minutes */
break;
case ATA_CMD_STANDBYNOW1:
- *timeout = 10000; /* 10 seconds */
+ *timeout = 120000; /* 2 minutes */
break;
case 0xF7:
case 0xFA:
@@ -2588,9 +2592,6 @@
if (!len || size)
return 0;
- if (size < 0)
- return -EINVAL;
-
size += sprintf(&buf[size], "H/ S ACTive : [ 0x");
for (n = dd->slot_groups-1; n >= 0; n--)
@@ -2660,9 +2661,6 @@
if (!len || size)
return 0;
- if (size < 0)
- return -EINVAL;
-
size += sprintf(&buf[size], "Flag-port : [ %08lX ]\n",
dd->port->flags);
size += sprintf(&buf[size], "Flag-dd : [ %08lX ]\n",
@@ -3214,8 +3212,8 @@
"Unable to check write protect progress\n");
else
dev_info(&dd->pdev->dev,
- "Write protect progress: %d%% (%d blocks)\n",
- attr242.cur, attr242.data);
+ "Write protect progress: %u%% (%u blocks)\n",
+ attr242.cur, le32_to_cpu(attr242.data));
return rv;
out3:
@@ -3619,6 +3617,10 @@
bio_endio(bio, -ENODATA);
return;
}
+ if (unlikely(test_bit(MTIP_DDF_SEC_LOCK_BIT, &dd->dd_flag))) {
+ bio_endio(bio, -ENODATA);
+ return;
+ }
}
if (unlikely(!bio_has_data(bio))) {
@@ -4168,7 +4170,13 @@
/* Table of device ids supported by this driver. */
static DEFINE_PCI_DEVICE_TABLE(mtip_pci_tbl) = {
- { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P320_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P320H_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P320M_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P320S_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P325M_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P420H_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P420M_DEVICE_ID) },
+ { PCI_DEVICE(PCI_VENDOR_ID_MICRON, P425M_DEVICE_ID) },
{ 0 }
};
@@ -4199,12 +4207,12 @@
{
int error;
- printk(KERN_INFO MTIP_DRV_NAME " Version " MTIP_DRV_VERSION "\n");
+ pr_info(MTIP_DRV_NAME " Version " MTIP_DRV_VERSION "\n");
/* Allocate a major block device number to use with this driver. */
error = register_blkdev(0, MTIP_DRV_NAME);
if (error <= 0) {
- printk(KERN_ERR "Unable to register block device (%d)\n",
+ pr_err("Unable to register block device (%d)\n",
error);
return -EBUSY;
}
@@ -4213,7 +4221,7 @@
if (!dfs_parent) {
dfs_parent = debugfs_create_dir("rssd", NULL);
if (IS_ERR_OR_NULL(dfs_parent)) {
- printk(KERN_WARNING "Error creating debugfs parent\n");
+ pr_warn("Error creating debugfs parent\n");
dfs_parent = NULL;
}
}
diff --git a/drivers/block/mtip32xx/mtip32xx.h b/drivers/block/mtip32xx/mtip32xx.h
index f51fc23..18627a1 100644
--- a/drivers/block/mtip32xx/mtip32xx.h
+++ b/drivers/block/mtip32xx/mtip32xx.h
@@ -76,7 +76,13 @@
/* Micron Vendor ID & P320x SSD Device ID */
#define PCI_VENDOR_ID_MICRON 0x1344
-#define P320_DEVICE_ID 0x5150
+#define P320H_DEVICE_ID 0x5150
+#define P320M_DEVICE_ID 0x5151
+#define P320S_DEVICE_ID 0x5152
+#define P325M_DEVICE_ID 0x5153
+#define P420H_DEVICE_ID 0x5160
+#define P420M_DEVICE_ID 0x5161
+#define P425M_DEVICE_ID 0x5163
/* Driver name and version strings */
#define MTIP_DRV_NAME "mtip32xx"
@@ -131,10 +137,12 @@
MTIP_PF_SVC_THD_STOP_BIT = 8,
/* below are bit numbers in 'dd_flag' defined in driver_data */
+ MTIP_DDF_SEC_LOCK_BIT = 0,
MTIP_DDF_REMOVE_PENDING_BIT = 1,
MTIP_DDF_OVER_TEMP_BIT = 2,
MTIP_DDF_WRITE_PROTECT_BIT = 3,
MTIP_DDF_STOP_IO = ((1 << MTIP_DDF_REMOVE_PENDING_BIT) | \
+ (1 << MTIP_DDF_SEC_LOCK_BIT) | \
(1 << MTIP_DDF_OVER_TEMP_BIT) | \
(1 << MTIP_DDF_WRITE_PROTECT_BIT)),
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index d07c9f7..0c03411 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -449,6 +449,14 @@
req->errors++;
nbd_end_request(req);
}
+
+ while (!list_empty(&nbd->waiting_queue)) {
+ req = list_entry(nbd->waiting_queue.next, struct request,
+ queuelist);
+ list_del_init(&req->queuelist);
+ req->errors++;
+ nbd_end_request(req);
+ }
}
@@ -598,6 +606,7 @@
nbd->file = NULL;
nbd_clear_que(nbd);
BUG_ON(!list_empty(&nbd->queue_head));
+ BUG_ON(!list_empty(&nbd->waiting_queue));
if (file)
fput(file);
return 0;
diff --git a/drivers/block/nvme.c b/drivers/block/nvme.c
index 38a2d06..931769e 100644
--- a/drivers/block/nvme.c
+++ b/drivers/block/nvme.c
@@ -79,6 +79,7 @@
char serial[20];
char model[40];
char firmware_rev[8];
+ u32 max_hw_sectors;
};
/*
@@ -835,15 +836,15 @@
}
static int nvme_get_features(struct nvme_dev *dev, unsigned fid,
- unsigned dword11, dma_addr_t dma_addr)
+ unsigned nsid, dma_addr_t dma_addr)
{
struct nvme_command c;
memset(&c, 0, sizeof(c));
c.features.opcode = nvme_admin_get_features;
+ c.features.nsid = cpu_to_le32(nsid);
c.features.prp1 = cpu_to_le64(dma_addr);
c.features.fid = cpu_to_le32(fid);
- c.features.dword11 = cpu_to_le32(dword11);
return nvme_submit_admin_cmd(dev, &c, NULL);
}
@@ -862,11 +863,51 @@
return nvme_submit_admin_cmd(dev, &c, result);
}
+/**
+ * nvme_cancel_ios - Cancel outstanding I/Os
+ * @queue: The queue to cancel I/Os on
+ * @timeout: True to only cancel I/Os which have timed out
+ */
+static void nvme_cancel_ios(struct nvme_queue *nvmeq, bool timeout)
+{
+ int depth = nvmeq->q_depth - 1;
+ struct nvme_cmd_info *info = nvme_cmd_info(nvmeq);
+ unsigned long now = jiffies;
+ int cmdid;
+
+ for_each_set_bit(cmdid, nvmeq->cmdid_data, depth) {
+ void *ctx;
+ nvme_completion_fn fn;
+ static struct nvme_completion cqe = {
+ .status = cpu_to_le16(NVME_SC_ABORT_REQ) << 1,
+ };
+
+ if (timeout && !time_after(now, info[cmdid].timeout))
+ continue;
+ dev_warn(nvmeq->q_dmadev, "Cancelling I/O %d\n", cmdid);
+ ctx = cancel_cmdid(nvmeq, cmdid, &fn);
+ fn(nvmeq->dev, ctx, &cqe);
+ }
+}
+
+static void nvme_free_queue_mem(struct nvme_queue *nvmeq)
+{
+ dma_free_coherent(nvmeq->q_dmadev, CQ_SIZE(nvmeq->q_depth),
+ (void *)nvmeq->cqes, nvmeq->cq_dma_addr);
+ dma_free_coherent(nvmeq->q_dmadev, SQ_SIZE(nvmeq->q_depth),
+ nvmeq->sq_cmds, nvmeq->sq_dma_addr);
+ kfree(nvmeq);
+}
+
static void nvme_free_queue(struct nvme_dev *dev, int qid)
{
struct nvme_queue *nvmeq = dev->queues[qid];
int vector = dev->entry[nvmeq->cq_vector].vector;
+ spin_lock_irq(&nvmeq->q_lock);
+ nvme_cancel_ios(nvmeq, false);
+ spin_unlock_irq(&nvmeq->q_lock);
+
irq_set_affinity_hint(vector, NULL);
free_irq(vector, nvmeq);
@@ -876,18 +917,15 @@
adapter_delete_cq(dev, qid);
}
- dma_free_coherent(nvmeq->q_dmadev, CQ_SIZE(nvmeq->q_depth),
- (void *)nvmeq->cqes, nvmeq->cq_dma_addr);
- dma_free_coherent(nvmeq->q_dmadev, SQ_SIZE(nvmeq->q_depth),
- nvmeq->sq_cmds, nvmeq->sq_dma_addr);
- kfree(nvmeq);
+ nvme_free_queue_mem(nvmeq);
}
static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev, int qid,
int depth, int vector)
{
struct device *dmadev = &dev->pci_dev->dev;
- unsigned extra = (depth / 8) + (depth * sizeof(struct nvme_cmd_info));
+ unsigned extra = DIV_ROUND_UP(depth, 8) + (depth *
+ sizeof(struct nvme_cmd_info));
struct nvme_queue *nvmeq = kzalloc(sizeof(*nvmeq) + extra, GFP_KERNEL);
if (!nvmeq)
return NULL;
@@ -975,7 +1013,7 @@
static int __devinit nvme_configure_admin_queue(struct nvme_dev *dev)
{
- int result;
+ int result = 0;
u32 aqa;
u64 cap;
unsigned long timeout;
@@ -1005,17 +1043,22 @@
timeout = ((NVME_CAP_TIMEOUT(cap) + 1) * HZ / 2) + jiffies;
dev->db_stride = NVME_CAP_STRIDE(cap);
- while (!(readl(&dev->bar->csts) & NVME_CSTS_RDY)) {
+ while (!result && !(readl(&dev->bar->csts) & NVME_CSTS_RDY)) {
msleep(100);
if (fatal_signal_pending(current))
- return -EINTR;
+ result = -EINTR;
if (time_after(jiffies, timeout)) {
dev_err(&dev->pci_dev->dev,
"Device not ready; aborting initialisation\n");
- return -ENODEV;
+ result = -ENODEV;
}
}
+ if (result) {
+ nvme_free_queue_mem(nvmeq);
+ return result;
+ }
+
result = queue_request_irq(dev, nvmeq, "nvme admin");
dev->queues[0] = nvmeq;
return result;
@@ -1037,6 +1080,8 @@
offset = offset_in_page(addr);
count = DIV_ROUND_UP(offset + length, PAGE_SIZE);
pages = kcalloc(count, sizeof(*pages), GFP_KERNEL);
+ if (!pages)
+ return ERR_PTR(-ENOMEM);
err = get_user_pages_fast(addr, count, 1, pages);
if (err < count) {
@@ -1146,14 +1191,13 @@
return status;
}
-static int nvme_user_admin_cmd(struct nvme_ns *ns,
+static int nvme_user_admin_cmd(struct nvme_dev *dev,
struct nvme_admin_cmd __user *ucmd)
{
- struct nvme_dev *dev = ns->dev;
struct nvme_admin_cmd cmd;
struct nvme_command c;
int status, length;
- struct nvme_iod *iod;
+ struct nvme_iod *uninitialized_var(iod);
if (!capable(CAP_SYS_ADMIN))
return -EACCES;
@@ -1204,7 +1248,7 @@
case NVME_IOCTL_ID:
return ns->ns_id;
case NVME_IOCTL_ADMIN_CMD:
- return nvme_user_admin_cmd(ns, (void __user *)arg);
+ return nvme_user_admin_cmd(ns->dev, (void __user *)arg);
case NVME_IOCTL_SUBMIT_IO:
return nvme_submit_io(ns, (void __user *)arg);
default:
@@ -1218,26 +1262,6 @@
.compat_ioctl = nvme_ioctl,
};
-static void nvme_timeout_ios(struct nvme_queue *nvmeq)
-{
- int depth = nvmeq->q_depth - 1;
- struct nvme_cmd_info *info = nvme_cmd_info(nvmeq);
- unsigned long now = jiffies;
- int cmdid;
-
- for_each_set_bit(cmdid, nvmeq->cmdid_data, depth) {
- void *ctx;
- nvme_completion_fn fn;
- static struct nvme_completion cqe = { .status = cpu_to_le16(NVME_SC_ABORT_REQ) << 1, };
-
- if (!time_after(now, info[cmdid].timeout))
- continue;
- dev_warn(nvmeq->q_dmadev, "Timing out I/O %d\n", cmdid);
- ctx = cancel_cmdid(nvmeq, cmdid, &fn);
- fn(nvmeq->dev, ctx, &cqe);
- }
-}
-
static void nvme_resubmit_bios(struct nvme_queue *nvmeq)
{
while (bio_list_peek(&nvmeq->sq_cong)) {
@@ -1269,7 +1293,7 @@
spin_lock_irq(&nvmeq->q_lock);
if (nvme_process_cq(nvmeq))
printk("process_cq did something\n");
- nvme_timeout_ios(nvmeq);
+ nvme_cancel_ios(nvmeq, true);
nvme_resubmit_bios(nvmeq);
spin_unlock_irq(&nvmeq->q_lock);
}
@@ -1339,6 +1363,9 @@
ns->disk = disk;
lbaf = id->flbas & 0xf;
ns->lba_shift = id->lbaf[lbaf].ds;
+ blk_queue_logical_block_size(ns->queue, 1 << ns->lba_shift);
+ if (dev->max_hw_sectors)
+ blk_queue_max_hw_sectors(ns->queue, dev->max_hw_sectors);
disk->major = nvme_major;
disk->minors = NVME_MINORS;
@@ -1383,7 +1410,7 @@
static int __devinit nvme_setup_io_queues(struct nvme_dev *dev)
{
- int result, cpu, i, nr_io_queues, db_bar_size;
+ int result, cpu, i, nr_io_queues, db_bar_size, q_depth;
nr_io_queues = num_online_cpus();
result = set_queue_count(dev, nr_io_queues);
@@ -1429,9 +1456,10 @@
cpu = cpumask_next(cpu, cpu_online_mask);
}
+ q_depth = min_t(int, NVME_CAP_MQES(readq(&dev->bar->cap)) + 1,
+ NVME_Q_DEPTH);
for (i = 0; i < nr_io_queues; i++) {
- dev->queues[i + 1] = nvme_create_queue(dev, i + 1,
- NVME_Q_DEPTH, i);
+ dev->queues[i + 1] = nvme_create_queue(dev, i + 1, q_depth, i);
if (IS_ERR(dev->queues[i + 1]))
return PTR_ERR(dev->queues[i + 1]);
dev->queue_count++;
@@ -1480,6 +1508,10 @@
memcpy(dev->serial, ctrl->sn, sizeof(ctrl->sn));
memcpy(dev->model, ctrl->mn, sizeof(ctrl->mn));
memcpy(dev->firmware_rev, ctrl->fr, sizeof(ctrl->fr));
+ if (ctrl->mdts) {
+ int shift = NVME_CAP_MPSMIN(readq(&dev->bar->cap)) + 12;
+ dev->max_hw_sectors = 1 << (ctrl->mdts + shift - 9);
+ }
id_ns = mem;
for (i = 1; i <= nn; i++) {
@@ -1523,8 +1555,6 @@
list_del(&dev->node);
spin_unlock(&dev_list_lock);
- /* TODO: wait all I/O finished or cancel them */
-
list_for_each_entry_safe(ns, next, &dev->namespaces, list) {
list_del(&ns->list);
del_gendisk(ns->disk);
@@ -1560,15 +1590,33 @@
dma_pool_destroy(dev->prp_small_pool);
}
-/* XXX: Use an ida or something to let remove / add work correctly */
-static void nvme_set_instance(struct nvme_dev *dev)
+static DEFINE_IDA(nvme_instance_ida);
+
+static int nvme_set_instance(struct nvme_dev *dev)
{
- static int instance;
- dev->instance = instance++;
+ int instance, error;
+
+ do {
+ if (!ida_pre_get(&nvme_instance_ida, GFP_KERNEL))
+ return -ENODEV;
+
+ spin_lock(&dev_list_lock);
+ error = ida_get_new(&nvme_instance_ida, &instance);
+ spin_unlock(&dev_list_lock);
+ } while (error == -EAGAIN);
+
+ if (error)
+ return -ENODEV;
+
+ dev->instance = instance;
+ return 0;
}
static void nvme_release_instance(struct nvme_dev *dev)
{
+ spin_lock(&dev_list_lock);
+ ida_remove(&nvme_instance_ida, dev->instance);
+ spin_unlock(&dev_list_lock);
}
static int __devinit nvme_probe(struct pci_dev *pdev,
@@ -1601,7 +1649,10 @@
pci_set_drvdata(pdev, dev);
dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
- nvme_set_instance(dev);
+ result = nvme_set_instance(dev);
+ if (result)
+ goto disable;
+
dev->entry[0].vector = pdev->irq;
result = nvme_setup_prp_pools(dev);
@@ -1675,7 +1726,7 @@
#define nvme_suspend NULL
#define nvme_resume NULL
-static struct pci_error_handlers nvme_err_handler = {
+static const struct pci_error_handlers nvme_err_handler = {
.error_detected = nvme_error_detected,
.mmio_enabled = nvme_dump_registers,
.link_reset = nvme_link_reset,
@@ -1704,15 +1755,17 @@
static int __init nvme_init(void)
{
- int result = -EBUSY;
+ int result;
nvme_thread = kthread_run(nvme_kthread, NULL, "nvme");
if (IS_ERR(nvme_thread))
return PTR_ERR(nvme_thread);
- nvme_major = register_blkdev(nvme_major, "nvme");
- if (nvme_major <= 0)
+ result = register_blkdev(nvme_major, "nvme");
+ if (result < 0)
goto kill_kthread;
+ else if (result > 0)
+ nvme_major = result;
result = pci_register_driver(&nvme_driver);
if (result)
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index 9917943..54a55f0 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -246,13 +246,12 @@
{
struct rbd_device *rbd_dev = bdev->bd_disk->private_data;
- rbd_get_dev(rbd_dev);
-
- set_device_ro(bdev, rbd_dev->read_only);
-
if ((mode & FMODE_WRITE) && rbd_dev->read_only)
return -EROFS;
+ rbd_get_dev(rbd_dev);
+ set_device_ro(bdev, rbd_dev->read_only);
+
return 0;
}
diff --git a/drivers/block/xen-blkback/blkback.c b/drivers/block/xen-blkback/blkback.c
index 73f196c..c6decb9 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -337,7 +337,7 @@
invcount++;
}
- ret = gnttab_unmap_refs(unmap, pages, invcount, false);
+ ret = gnttab_unmap_refs(unmap, NULL, pages, invcount);
BUG_ON(ret);
}
diff --git a/drivers/bluetooth/ath3k.c b/drivers/bluetooth/ath3k.c
index 11f36e5..fc2de55 100644
--- a/drivers/bluetooth/ath3k.c
+++ b/drivers/bluetooth/ath3k.c
@@ -86,6 +86,7 @@
/* Atheros AR5BBU22 with sflash firmware */
{ USB_DEVICE(0x0489, 0xE03C) },
+ { USB_DEVICE(0x0489, 0xE036) },
{ } /* Terminating entry */
};
@@ -109,6 +110,7 @@
/* Atheros AR5BBU22 with sflash firmware */
{ USB_DEVICE(0x0489, 0xE03C), .driver_info = BTUSB_ATH3012 },
+ { USB_DEVICE(0x0489, 0xE036), .driver_info = BTUSB_ATH3012 },
{ } /* Terminating entry */
};
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index cef3bac..654e248 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -52,6 +52,9 @@
/* Generic Bluetooth USB device */
{ USB_DEVICE_INFO(0xe0, 0x01, 0x01) },
+ /* Apple-specific (Broadcom) devices */
+ { USB_VENDOR_AND_INTERFACE_INFO(0x05ac, 0xff, 0x01, 0x01) },
+
/* Broadcom SoftSailing reporting vendor specific */
{ USB_DEVICE(0x0a5c, 0x21e1) },
@@ -94,16 +97,14 @@
/* Broadcom BCM20702A0 */
{ USB_DEVICE(0x0489, 0xe042) },
- { USB_DEVICE(0x0a5c, 0x21e3) },
- { USB_DEVICE(0x0a5c, 0x21e6) },
- { USB_DEVICE(0x0a5c, 0x21e8) },
- { USB_DEVICE(0x0a5c, 0x21f3) },
- { USB_DEVICE(0x0a5c, 0x21f4) },
{ USB_DEVICE(0x413c, 0x8197) },
/* Foxconn - Hon Hai */
{ USB_DEVICE(0x0489, 0xe033) },
+ /*Broadcom devices with vendor specific id */
+ { USB_VENDOR_AND_INTERFACE_INFO(0x0a5c, 0xff, 0x01, 0x01) },
+
{ } /* Terminating entry */
};
@@ -141,6 +142,7 @@
/* Atheros AR5BBU12 with sflash firmware */
{ USB_DEVICE(0x0489, 0xe03c), .driver_info = BTUSB_ATH3012 },
+ { USB_DEVICE(0x0489, 0xe036), .driver_info = BTUSB_ATH3012 },
/* Broadcom BCM2035 */
{ USB_DEVICE(0x0a5c, 0x2035), .driver_info = BTUSB_WRONG_SCO_MTU },
diff --git a/drivers/char/agp/sgi-agp.c b/drivers/char/agp/sgi-agp.c
index 1920003..3a5af2f 100644
--- a/drivers/char/agp/sgi-agp.c
+++ b/drivers/char/agp/sgi-agp.c
@@ -289,12 +289,11 @@
j = 0;
list_for_each_entry(info, &tioca_list, ca_list) {
- struct list_head *tmp;
if (list_empty(info->ca_devices))
continue;
- list_for_each(tmp, info->ca_devices) {
+ list_for_each_entry(pdev, info->ca_devices, bus_list) {
u8 cap_ptr;
- pdev = pci_dev_b(tmp);
+
if (pdev->class != (PCI_CLASS_DISPLAY_VGA << 8))
continue;
cap_ptr = pci_find_capability(pdev, PCI_CAP_ID_AGP);
diff --git a/drivers/clk/Kconfig b/drivers/clk/Kconfig
index 7f0b5ca..bace9e9 100644
--- a/drivers/clk/Kconfig
+++ b/drivers/clk/Kconfig
@@ -40,4 +40,17 @@
Supports the clocking subsystem of the WM831x/2x series of
PMICs from Wolfson Microlectronics.
+config COMMON_CLK_VERSATILE
+ bool "Clock driver for ARM Reference designs"
+ depends on ARCH_INTEGRATOR || ARCH_REALVIEW
+ ---help---
+ Supports clocking on ARM Reference designs Integrator/AP,
+ Integrator/CP, RealView PB1176, EB, PB11MP and PBX.
+
+config COMMON_CLK_MAX77686
+ tristate "Clock driver for Maxim 77686 MFD"
+ depends on MFD_MAX77686
+ ---help---
+ This driver supports Maxim 77686 crystal oscillator clock.
+
endmenu
diff --git a/drivers/clk/Makefile b/drivers/clk/Makefile
index 5869ea3..b7b8620 100644
--- a/drivers/clk/Makefile
+++ b/drivers/clk/Makefile
@@ -1,4 +1,5 @@
# common clock types
+obj-$(CONFIG_HAVE_CLK) += clk-devres.o
obj-$(CONFIG_CLKDEV_LOOKUP) += clkdev.o
obj-$(CONFIG_COMMON_CLK) += clk.o clk-fixed-rate.o clk-gate.o \
clk-mux.o clk-divider.o clk-fixed-factor.o
@@ -9,7 +10,14 @@
obj-$(CONFIG_ARCH_SOCFPGA) += socfpga/
obj-$(CONFIG_PLAT_SPEAR) += spear/
obj-$(CONFIG_ARCH_U300) += clk-u300.o
-obj-$(CONFIG_ARCH_INTEGRATOR) += versatile/
+obj-$(CONFIG_COMMON_CLK_VERSATILE) += versatile/
+obj-$(CONFIG_ARCH_PRIMA2) += clk-prima2.o
+ifeq ($(CONFIG_COMMON_CLK), y)
+obj-$(CONFIG_ARCH_MMP) += mmp/
+endif
+obj-$(CONFIG_MACH_LOONGSON1) += clk-ls1x.o
+obj-$(CONFIG_ARCH_U8500) += ux500/
# Chip specific
obj-$(CONFIG_COMMON_CLK_WM831X) += clk-wm831x.o
+obj-$(CONFIG_COMMON_CLK_MAX77686) += clk-max77686.o
diff --git a/drivers/clk/clk-devres.c b/drivers/clk/clk-devres.c
new file mode 100644
index 0000000..8f57154
--- /dev/null
+++ b/drivers/clk/clk-devres.c
@@ -0,0 +1,55 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/export.h>
+#include <linux/gfp.h>
+
+static void devm_clk_release(struct device *dev, void *res)
+{
+ clk_put(*(struct clk **)res);
+}
+
+struct clk *devm_clk_get(struct device *dev, const char *id)
+{
+ struct clk **ptr, *clk;
+
+ ptr = devres_alloc(devm_clk_release, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+
+ clk = clk_get(dev, id);
+ if (!IS_ERR(clk)) {
+ *ptr = clk;
+ devres_add(dev, ptr);
+ } else {
+ devres_free(ptr);
+ }
+
+ return clk;
+}
+EXPORT_SYMBOL(devm_clk_get);
+
+static int devm_clk_match(struct device *dev, void *res, void *data)
+{
+ struct clk **c = res;
+ if (!c || !*c) {
+ WARN_ON(!c || !*c);
+ return 0;
+ }
+ return *c == data;
+}
+
+void devm_clk_put(struct device *dev, struct clk *clk)
+{
+ int ret;
+
+ ret = devres_release(dev, devm_clk_release, devm_clk_match, clk);
+
+ WARN_ON(ret);
+}
+EXPORT_SYMBOL(devm_clk_put);
diff --git a/drivers/clk/clk-ls1x.c b/drivers/clk/clk-ls1x.c
new file mode 100644
index 0000000..f20b750
--- /dev/null
+++ b/drivers/clk/clk-ls1x.c
@@ -0,0 +1,111 @@
+/*
+ * Copyright (c) 2012 Zhang, Keguang <keguang.zhang@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/io.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+
+#include <loongson1.h>
+
+#define OSC 33
+
+static DEFINE_SPINLOCK(_lock);
+
+static int ls1x_pll_clk_enable(struct clk_hw *hw)
+{
+ return 0;
+}
+
+static void ls1x_pll_clk_disable(struct clk_hw *hw)
+{
+}
+
+static unsigned long ls1x_pll_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ u32 pll, rate;
+
+ pll = __raw_readl(LS1X_CLK_PLL_FREQ);
+ rate = ((12 + (pll & 0x3f)) * 1000000) +
+ ((((pll >> 8) & 0x3ff) * 1000000) >> 10);
+ rate *= OSC;
+ rate >>= 1;
+
+ return rate;
+}
+
+static const struct clk_ops ls1x_pll_clk_ops = {
+ .enable = ls1x_pll_clk_enable,
+ .disable = ls1x_pll_clk_disable,
+ .recalc_rate = ls1x_pll_recalc_rate,
+};
+
+static struct clk * __init clk_register_pll(struct device *dev,
+ const char *name, const char *parent_name, unsigned long flags)
+{
+ struct clk_hw *hw;
+ struct clk *clk;
+ struct clk_init_data init;
+
+ /* allocate the divider */
+ hw = kzalloc(sizeof(struct clk_hw), GFP_KERNEL);
+ if (!hw) {
+ pr_err("%s: could not allocate clk_hw\n", __func__);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ init.name = name;
+ init.ops = &ls1x_pll_clk_ops;
+ init.flags = flags | CLK_IS_BASIC;
+ init.parent_names = (parent_name ? &parent_name : NULL);
+ init.num_parents = (parent_name ? 1 : 0);
+ hw->init = &init;
+
+ /* register the clock */
+ clk = clk_register(dev, hw);
+
+ if (IS_ERR(clk))
+ kfree(hw);
+
+ return clk;
+}
+
+void __init ls1x_clk_init(void)
+{
+ struct clk *clk;
+
+ clk = clk_register_pll(NULL, "pll_clk", NULL, CLK_IS_ROOT);
+ clk_prepare_enable(clk);
+
+ clk = clk_register_divider(NULL, "cpu_clk", "pll_clk",
+ CLK_SET_RATE_PARENT, LS1X_CLK_PLL_DIV, DIV_CPU_SHIFT,
+ DIV_CPU_WIDTH, CLK_DIVIDER_ONE_BASED, &_lock);
+ clk_prepare_enable(clk);
+ clk_register_clkdev(clk, "cpu", NULL);
+
+ clk = clk_register_divider(NULL, "dc_clk", "pll_clk",
+ CLK_SET_RATE_PARENT, LS1X_CLK_PLL_DIV, DIV_DC_SHIFT,
+ DIV_DC_WIDTH, CLK_DIVIDER_ONE_BASED, &_lock);
+ clk_prepare_enable(clk);
+ clk_register_clkdev(clk, "dc", NULL);
+
+ clk = clk_register_divider(NULL, "ahb_clk", "pll_clk",
+ CLK_SET_RATE_PARENT, LS1X_CLK_PLL_DIV, DIV_DDR_SHIFT,
+ DIV_DDR_WIDTH, CLK_DIVIDER_ONE_BASED, &_lock);
+ clk_prepare_enable(clk);
+ clk_register_clkdev(clk, "ahb", NULL);
+ clk_register_clkdev(clk, "stmmaceth", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "apb_clk", "ahb_clk", 0, 1, 2);
+ clk_prepare_enable(clk);
+ clk_register_clkdev(clk, "apb", NULL);
+ clk_register_clkdev(clk, "serial8250", NULL);
+}
diff --git a/drivers/clk/clk-max77686.c b/drivers/clk/clk-max77686.c
new file mode 100644
index 0000000..ac5f543
--- /dev/null
+++ b/drivers/clk/clk-max77686.c
@@ -0,0 +1,244 @@
+/*
+ * clk-max77686.c - Clock driver for Maxim 77686
+ *
+ * Copyright (C) 2012 Samsung Electornics
+ * Jonghwa Lee <jonghwa3.lee@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+#include <linux/platform_device.h>
+#include <linux/mfd/max77686.h>
+#include <linux/mfd/max77686-private.h>
+#include <linux/clk-provider.h>
+#include <linux/mutex.h>
+#include <linux/clkdev.h>
+
+enum {
+ MAX77686_CLK_AP = 0,
+ MAX77686_CLK_CP,
+ MAX77686_CLK_PMIC,
+ MAX77686_CLKS_NUM,
+};
+
+struct max77686_clk {
+ struct max77686_dev *iodev;
+ u32 mask;
+ struct clk_hw hw;
+ struct clk_lookup *lookup;
+};
+
+static struct max77686_clk *get_max77686_clk(struct clk_hw *hw)
+{
+ return container_of(hw, struct max77686_clk, hw);
+}
+
+static int max77686_clk_prepare(struct clk_hw *hw)
+{
+ struct max77686_clk *max77686;
+ int ret;
+
+ max77686 = get_max77686_clk(hw);
+ if (!max77686)
+ return -ENOMEM;
+
+ ret = regmap_update_bits(max77686->iodev->regmap,
+ MAX77686_REG_32KHZ, max77686->mask, max77686->mask);
+
+ return ret;
+}
+
+static void max77686_clk_unprepare(struct clk_hw *hw)
+{
+ struct max77686_clk *max77686;
+
+ max77686 = get_max77686_clk(hw);
+ if (!max77686)
+ return;
+
+ regmap_update_bits(max77686->iodev->regmap,
+ MAX77686_REG_32KHZ, max77686->mask, ~max77686->mask);
+}
+
+static int max77686_clk_is_enabled(struct clk_hw *hw)
+{
+ struct max77686_clk *max77686;
+ int ret;
+ u32 val;
+
+ max77686 = get_max77686_clk(hw);
+ if (!max77686)
+ return -ENOMEM;
+
+ ret = regmap_read(max77686->iodev->regmap,
+ MAX77686_REG_32KHZ, &val);
+
+ if (ret < 0)
+ return -EINVAL;
+
+ return val & max77686->mask;
+}
+
+static struct clk_ops max77686_clk_ops = {
+ .prepare = max77686_clk_prepare,
+ .unprepare = max77686_clk_unprepare,
+ .is_enabled = max77686_clk_is_enabled,
+};
+
+static struct clk_init_data max77686_clks_init[MAX77686_CLKS_NUM] = {
+ [MAX77686_CLK_AP] = {
+ .name = "32khz_ap",
+ .ops = &max77686_clk_ops,
+ .flags = CLK_IS_ROOT,
+ },
+ [MAX77686_CLK_CP] = {
+ .name = "32khz_cp",
+ .ops = &max77686_clk_ops,
+ .flags = CLK_IS_ROOT,
+ },
+ [MAX77686_CLK_PMIC] = {
+ .name = "32khz_pmic",
+ .ops = &max77686_clk_ops,
+ .flags = CLK_IS_ROOT,
+ },
+};
+
+static int max77686_clk_register(struct device *dev,
+ struct max77686_clk *max77686)
+{
+ struct clk *clk;
+ struct clk_hw *hw = &max77686->hw;
+
+ clk = clk_register(dev, hw);
+
+ if (IS_ERR(clk))
+ return -ENOMEM;
+
+ max77686->lookup = devm_kzalloc(dev, sizeof(struct clk_lookup),
+ GFP_KERNEL);
+ if (IS_ERR(max77686->lookup))
+ return -ENOMEM;
+
+ max77686->lookup->con_id = hw->init->name;
+ max77686->lookup->clk = clk;
+
+ clkdev_add(max77686->lookup);
+
+ return 0;
+}
+
+static __devinit int max77686_clk_probe(struct platform_device *pdev)
+{
+ struct max77686_dev *iodev = dev_get_drvdata(pdev->dev.parent);
+ struct max77686_clk **max77686_clks;
+ int i, ret;
+
+ max77686_clks = devm_kzalloc(&pdev->dev, sizeof(struct max77686_clk *)
+ * MAX77686_CLKS_NUM, GFP_KERNEL);
+ if (IS_ERR(max77686_clks))
+ return -ENOMEM;
+
+ for (i = 0; i < MAX77686_CLKS_NUM; i++) {
+ max77686_clks[i] = devm_kzalloc(&pdev->dev,
+ sizeof(struct max77686_clk), GFP_KERNEL);
+ if (IS_ERR(max77686_clks[i]))
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < MAX77686_CLKS_NUM; i++) {
+ max77686_clks[i]->iodev = iodev;
+ max77686_clks[i]->mask = 1 << i;
+ max77686_clks[i]->hw.init = &max77686_clks_init[i];
+
+ ret = max77686_clk_register(&pdev->dev, max77686_clks[i]);
+ if (ret) {
+ switch (i) {
+ case MAX77686_CLK_AP:
+ dev_err(&pdev->dev, "Fail to register CLK_AP\n");
+ goto err_clk_ap;
+ break;
+ case MAX77686_CLK_CP:
+ dev_err(&pdev->dev, "Fail to register CLK_CP\n");
+ goto err_clk_cp;
+ break;
+ case MAX77686_CLK_PMIC:
+ dev_err(&pdev->dev, "Fail to register CLK_PMIC\n");
+ goto err_clk_pmic;
+ }
+ }
+ }
+
+ platform_set_drvdata(pdev, max77686_clks);
+
+ goto out;
+
+err_clk_pmic:
+ clkdev_drop(max77686_clks[MAX77686_CLK_CP]->lookup);
+ kfree(max77686_clks[MAX77686_CLK_CP]->hw.clk);
+err_clk_cp:
+ clkdev_drop(max77686_clks[MAX77686_CLK_AP]->lookup);
+ kfree(max77686_clks[MAX77686_CLK_AP]->hw.clk);
+err_clk_ap:
+out:
+ return ret;
+}
+
+static int __devexit max77686_clk_remove(struct platform_device *pdev)
+{
+ struct max77686_clk **max77686_clks = platform_get_drvdata(pdev);
+ int i;
+
+ for (i = 0; i < MAX77686_CLKS_NUM; i++) {
+ clkdev_drop(max77686_clks[i]->lookup);
+ kfree(max77686_clks[i]->hw.clk);
+ }
+ return 0;
+}
+
+static const struct platform_device_id max77686_clk_id[] = {
+ { "max77686-clk", 0},
+ { },
+};
+MODULE_DEVICE_TABLE(platform, max77686_clk_id);
+
+static struct platform_driver max77686_clk_driver = {
+ .driver = {
+ .name = "max77686-clk",
+ .owner = THIS_MODULE,
+ },
+ .probe = max77686_clk_probe,
+ .remove = __devexit_p(max77686_clk_remove),
+ .id_table = max77686_clk_id,
+};
+
+static int __init max77686_clk_init(void)
+{
+ return platform_driver_register(&max77686_clk_driver);
+}
+subsys_initcall(max77686_clk_init);
+
+static void __init max77686_clk_cleanup(void)
+{
+ platform_driver_unregister(&max77686_clk_driver);
+}
+module_exit(max77686_clk_cleanup);
+
+MODULE_DESCRIPTION("MAXIM 77686 Clock Driver");
+MODULE_AUTHOR("Jonghwa Lee <jonghwa3.lee@samsung.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/clk/clk-prima2.c b/drivers/clk/clk-prima2.c
new file mode 100644
index 0000000..517874f
--- /dev/null
+++ b/drivers/clk/clk-prima2.c
@@ -0,0 +1,1171 @@
+/*
+ * Clock tree for CSR SiRFprimaII
+ *
+ * Copyright (c) 2011 Cambridge Silicon Radio Limited, a CSR plc group company.
+ *
+ * Licensed under GPLv2 or later.
+ */
+
+#include <linux/module.h>
+#include <linux/bitops.h>
+#include <linux/io.h>
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/of_address.h>
+#include <linux/syscore_ops.h>
+
+#define SIRFSOC_CLKC_CLK_EN0 0x0000
+#define SIRFSOC_CLKC_CLK_EN1 0x0004
+#define SIRFSOC_CLKC_REF_CFG 0x0014
+#define SIRFSOC_CLKC_CPU_CFG 0x0018
+#define SIRFSOC_CLKC_MEM_CFG 0x001c
+#define SIRFSOC_CLKC_SYS_CFG 0x0020
+#define SIRFSOC_CLKC_IO_CFG 0x0024
+#define SIRFSOC_CLKC_DSP_CFG 0x0028
+#define SIRFSOC_CLKC_GFX_CFG 0x002c
+#define SIRFSOC_CLKC_MM_CFG 0x0030
+#define SIRFSOC_CLKC_LCD_CFG 0x0034
+#define SIRFSOC_CLKC_MMC_CFG 0x0038
+#define SIRFSOC_CLKC_PLL1_CFG0 0x0040
+#define SIRFSOC_CLKC_PLL2_CFG0 0x0044
+#define SIRFSOC_CLKC_PLL3_CFG0 0x0048
+#define SIRFSOC_CLKC_PLL1_CFG1 0x004c
+#define SIRFSOC_CLKC_PLL2_CFG1 0x0050
+#define SIRFSOC_CLKC_PLL3_CFG1 0x0054
+#define SIRFSOC_CLKC_PLL1_CFG2 0x0058
+#define SIRFSOC_CLKC_PLL2_CFG2 0x005c
+#define SIRFSOC_CLKC_PLL3_CFG2 0x0060
+#define SIRFSOC_USBPHY_PLL_CTRL 0x0008
+#define SIRFSOC_USBPHY_PLL_POWERDOWN BIT(1)
+#define SIRFSOC_USBPHY_PLL_BYPASS BIT(2)
+#define SIRFSOC_USBPHY_PLL_LOCK BIT(3)
+
+static void *sirfsoc_clk_vbase, *sirfsoc_rsc_vbase;
+
+#define KHZ 1000
+#define MHZ (KHZ * KHZ)
+
+/*
+ * SiRFprimaII clock controller
+ * - 2 oscillators: osc-26MHz, rtc-32.768KHz
+ * - 3 standard configurable plls: pll1, pll2 & pll3
+ * - 2 exclusive plls: usb phy pll and sata phy pll
+ * - 8 clock domains: cpu/cpudiv, mem/memdiv, sys/io, dsp, graphic, multimedia,
+ * display and sdphy.
+ * Each clock domain can select its own clock source from five clock sources,
+ * X_XIN, X_XINW, PLL1, PLL2 and PLL3. The domain clock is used as the source
+ * clock of the group clock.
+ * - dsp domain: gps, mf
+ * - io domain: dmac, nand, audio, uart, i2c, spi, usp, pwm, pulse
+ * - sys domain: security
+ */
+
+struct clk_pll {
+ struct clk_hw hw;
+ unsigned short regofs; /* register offset */
+};
+
+#define to_pllclk(_hw) container_of(_hw, struct clk_pll, hw)
+
+struct clk_dmn {
+ struct clk_hw hw;
+ signed char enable_bit; /* enable bit: 0 ~ 63 */
+ unsigned short regofs; /* register offset */
+};
+
+#define to_dmnclk(_hw) container_of(_hw, struct clk_dmn, hw)
+
+struct clk_std {
+ struct clk_hw hw;
+ signed char enable_bit; /* enable bit: 0 ~ 63 */
+};
+
+#define to_stdclk(_hw) container_of(_hw, struct clk_std, hw)
+
+static int std_clk_is_enabled(struct clk_hw *hw);
+static int std_clk_enable(struct clk_hw *hw);
+static void std_clk_disable(struct clk_hw *hw);
+
+static inline unsigned long clkc_readl(unsigned reg)
+{
+ return readl(sirfsoc_clk_vbase + reg);
+}
+
+static inline void clkc_writel(u32 val, unsigned reg)
+{
+ writel(val, sirfsoc_clk_vbase + reg);
+}
+
+/*
+ * std pll
+ */
+
+static unsigned long pll_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ unsigned long fin = parent_rate;
+ struct clk_pll *clk = to_pllclk(hw);
+ u32 regcfg2 = clk->regofs + SIRFSOC_CLKC_PLL1_CFG2 -
+ SIRFSOC_CLKC_PLL1_CFG0;
+
+ if (clkc_readl(regcfg2) & BIT(2)) {
+ /* pll bypass mode */
+ return fin;
+ } else {
+ /* fout = fin * nf / nr / od */
+ u32 cfg0 = clkc_readl(clk->regofs);
+ u32 nf = (cfg0 & (BIT(13) - 1)) + 1;
+ u32 nr = ((cfg0 >> 13) & (BIT(6) - 1)) + 1;
+ u32 od = ((cfg0 >> 19) & (BIT(4) - 1)) + 1;
+ WARN_ON(fin % MHZ);
+ return fin / MHZ * nf / nr / od * MHZ;
+ }
+}
+
+static long pll_clk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ unsigned long fin, nf, nr, od;
+
+ /*
+ * fout = fin * nf / (nr * od);
+ * set od = 1, nr = fin/MHz, so fout = nf * MHz
+ */
+ rate = rate - rate % MHZ;
+
+ nf = rate / MHZ;
+ if (nf > BIT(13))
+ nf = BIT(13);
+ if (nf < 1)
+ nf = 1;
+
+ fin = *parent_rate;
+
+ nr = fin / MHZ;
+ if (nr > BIT(6))
+ nr = BIT(6);
+ od = 1;
+
+ return fin * nf / (nr * od);
+}
+
+static int pll_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_pll *clk = to_pllclk(hw);
+ unsigned long fin, nf, nr, od, reg;
+
+ /*
+ * fout = fin * nf / (nr * od);
+ * set od = 1, nr = fin/MHz, so fout = nf * MHz
+ */
+
+ nf = rate / MHZ;
+ if (unlikely((rate % MHZ) || nf > BIT(13) || nf < 1))
+ return -EINVAL;
+
+ fin = parent_rate;
+ BUG_ON(fin < MHZ);
+
+ nr = fin / MHZ;
+ BUG_ON((fin % MHZ) || nr > BIT(6));
+
+ od = 1;
+
+ reg = (nf - 1) | ((nr - 1) << 13) | ((od - 1) << 19);
+ clkc_writel(reg, clk->regofs);
+
+ reg = clk->regofs + SIRFSOC_CLKC_PLL1_CFG1 - SIRFSOC_CLKC_PLL1_CFG0;
+ clkc_writel((nf >> 1) - 1, reg);
+
+ reg = clk->regofs + SIRFSOC_CLKC_PLL1_CFG2 - SIRFSOC_CLKC_PLL1_CFG0;
+ while (!(clkc_readl(reg) & BIT(6)))
+ cpu_relax();
+
+ return 0;
+}
+
+static struct clk_ops std_pll_ops = {
+ .recalc_rate = pll_clk_recalc_rate,
+ .round_rate = pll_clk_round_rate,
+ .set_rate = pll_clk_set_rate,
+};
+
+static const char *pll_clk_parents[] = {
+ "osc",
+};
+
+static struct clk_init_data clk_pll1_init = {
+ .name = "pll1",
+ .ops = &std_pll_ops,
+ .parent_names = pll_clk_parents,
+ .num_parents = ARRAY_SIZE(pll_clk_parents),
+};
+
+static struct clk_init_data clk_pll2_init = {
+ .name = "pll2",
+ .ops = &std_pll_ops,
+ .parent_names = pll_clk_parents,
+ .num_parents = ARRAY_SIZE(pll_clk_parents),
+};
+
+static struct clk_init_data clk_pll3_init = {
+ .name = "pll3",
+ .ops = &std_pll_ops,
+ .parent_names = pll_clk_parents,
+ .num_parents = ARRAY_SIZE(pll_clk_parents),
+};
+
+static struct clk_pll clk_pll1 = {
+ .regofs = SIRFSOC_CLKC_PLL1_CFG0,
+ .hw = {
+ .init = &clk_pll1_init,
+ },
+};
+
+static struct clk_pll clk_pll2 = {
+ .regofs = SIRFSOC_CLKC_PLL2_CFG0,
+ .hw = {
+ .init = &clk_pll2_init,
+ },
+};
+
+static struct clk_pll clk_pll3 = {
+ .regofs = SIRFSOC_CLKC_PLL3_CFG0,
+ .hw = {
+ .init = &clk_pll3_init,
+ },
+};
+
+/*
+ * usb uses specified pll
+ */
+
+static int usb_pll_clk_enable(struct clk_hw *hw)
+{
+ u32 reg = readl(sirfsoc_rsc_vbase + SIRFSOC_USBPHY_PLL_CTRL);
+ reg &= ~(SIRFSOC_USBPHY_PLL_POWERDOWN | SIRFSOC_USBPHY_PLL_BYPASS);
+ writel(reg, sirfsoc_rsc_vbase + SIRFSOC_USBPHY_PLL_CTRL);
+ while (!(readl(sirfsoc_rsc_vbase + SIRFSOC_USBPHY_PLL_CTRL) &
+ SIRFSOC_USBPHY_PLL_LOCK))
+ cpu_relax();
+
+ return 0;
+}
+
+static void usb_pll_clk_disable(struct clk_hw *clk)
+{
+ u32 reg = readl(sirfsoc_rsc_vbase + SIRFSOC_USBPHY_PLL_CTRL);
+ reg |= (SIRFSOC_USBPHY_PLL_POWERDOWN | SIRFSOC_USBPHY_PLL_BYPASS);
+ writel(reg, sirfsoc_rsc_vbase + SIRFSOC_USBPHY_PLL_CTRL);
+}
+
+static unsigned long usb_pll_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+ u32 reg = readl(sirfsoc_rsc_vbase + SIRFSOC_USBPHY_PLL_CTRL);
+ return (reg & SIRFSOC_USBPHY_PLL_BYPASS) ? parent_rate : 48*MHZ;
+}
+
+static struct clk_ops usb_pll_ops = {
+ .enable = usb_pll_clk_enable,
+ .disable = usb_pll_clk_disable,
+ .recalc_rate = usb_pll_clk_recalc_rate,
+};
+
+static struct clk_init_data clk_usb_pll_init = {
+ .name = "usb_pll",
+ .ops = &usb_pll_ops,
+ .parent_names = pll_clk_parents,
+ .num_parents = ARRAY_SIZE(pll_clk_parents),
+};
+
+static struct clk_hw usb_pll_clk_hw = {
+ .init = &clk_usb_pll_init,
+};
+
+/*
+ * clock domains - cpu, mem, sys/io, dsp, gfx
+ */
+
+static const char *dmn_clk_parents[] = {
+ "rtc",
+ "osc",
+ "pll1",
+ "pll2",
+ "pll3",
+};
+
+static u8 dmn_clk_get_parent(struct clk_hw *hw)
+{
+ struct clk_dmn *clk = to_dmnclk(hw);
+ u32 cfg = clkc_readl(clk->regofs);
+
+ /* parent of io domain can only be pll3 */
+ if (strcmp(hw->init->name, "io") == 0)
+ return 4;
+
+ WARN_ON((cfg & (BIT(3) - 1)) > 4);
+
+ return cfg & (BIT(3) - 1);
+}
+
+static int dmn_clk_set_parent(struct clk_hw *hw, u8 parent)
+{
+ struct clk_dmn *clk = to_dmnclk(hw);
+ u32 cfg = clkc_readl(clk->regofs);
+
+ /* parent of io domain can only be pll3 */
+ if (strcmp(hw->init->name, "io") == 0)
+ return -EINVAL;
+
+ cfg &= ~(BIT(3) - 1);
+ clkc_writel(cfg | parent, clk->regofs);
+ /* BIT(3) - switching status: 1 - busy, 0 - done */
+ while (clkc_readl(clk->regofs) & BIT(3))
+ cpu_relax();
+
+ return 0;
+}
+
+static unsigned long dmn_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+
+{
+ unsigned long fin = parent_rate;
+ struct clk_dmn *clk = to_dmnclk(hw);
+
+ u32 cfg = clkc_readl(clk->regofs);
+
+ if (cfg & BIT(24)) {
+ /* fcd bypass mode */
+ return fin;
+ } else {
+ /*
+ * wait count: bit[19:16], hold count: bit[23:20]
+ */
+ u32 wait = (cfg >> 16) & (BIT(4) - 1);
+ u32 hold = (cfg >> 20) & (BIT(4) - 1);
+
+ return fin / (wait + hold + 2);
+ }
+}
+
+static long dmn_clk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ unsigned long fin;
+ unsigned ratio, wait, hold;
+ unsigned bits = (strcmp(hw->init->name, "mem") == 0) ? 3 : 4;
+
+ fin = *parent_rate;
+ ratio = fin / rate;
+
+ if (ratio < 2)
+ ratio = 2;
+ if (ratio > BIT(bits + 1))
+ ratio = BIT(bits + 1);
+
+ wait = (ratio >> 1) - 1;
+ hold = ratio - wait - 2;
+
+ return fin / (wait + hold + 2);
+}
+
+static int dmn_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_dmn *clk = to_dmnclk(hw);
+ unsigned long fin;
+ unsigned ratio, wait, hold, reg;
+ unsigned bits = (strcmp(hw->init->name, "mem") == 0) ? 3 : 4;
+
+ fin = parent_rate;
+ ratio = fin / rate;
+
+ if (unlikely(ratio < 2 || ratio > BIT(bits + 1)))
+ return -EINVAL;
+
+ WARN_ON(fin % rate);
+
+ wait = (ratio >> 1) - 1;
+ hold = ratio - wait - 2;
+
+ reg = clkc_readl(clk->regofs);
+ reg &= ~(((BIT(bits) - 1) << 16) | ((BIT(bits) - 1) << 20));
+ reg |= (wait << 16) | (hold << 20) | BIT(25);
+ clkc_writel(reg, clk->regofs);
+
+ /* waiting FCD been effective */
+ while (clkc_readl(clk->regofs) & BIT(25))
+ cpu_relax();
+
+ return 0;
+}
+
+static struct clk_ops msi_ops = {
+ .set_rate = dmn_clk_set_rate,
+ .round_rate = dmn_clk_round_rate,
+ .recalc_rate = dmn_clk_recalc_rate,
+ .set_parent = dmn_clk_set_parent,
+ .get_parent = dmn_clk_get_parent,
+};
+
+static struct clk_init_data clk_mem_init = {
+ .name = "mem",
+ .ops = &msi_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_mem = {
+ .regofs = SIRFSOC_CLKC_MEM_CFG,
+ .hw = {
+ .init = &clk_mem_init,
+ },
+};
+
+static struct clk_init_data clk_sys_init = {
+ .name = "sys",
+ .ops = &msi_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+ .flags = CLK_SET_RATE_GATE,
+};
+
+static struct clk_dmn clk_sys = {
+ .regofs = SIRFSOC_CLKC_SYS_CFG,
+ .hw = {
+ .init = &clk_sys_init,
+ },
+};
+
+static struct clk_init_data clk_io_init = {
+ .name = "io",
+ .ops = &msi_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_io = {
+ .regofs = SIRFSOC_CLKC_IO_CFG,
+ .hw = {
+ .init = &clk_io_init,
+ },
+};
+
+static struct clk_ops cpu_ops = {
+ .set_parent = dmn_clk_set_parent,
+ .get_parent = dmn_clk_get_parent,
+};
+
+static struct clk_init_data clk_cpu_init = {
+ .name = "cpu",
+ .ops = &cpu_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+ .flags = CLK_SET_RATE_PARENT,
+};
+
+static struct clk_dmn clk_cpu = {
+ .regofs = SIRFSOC_CLKC_CPU_CFG,
+ .hw = {
+ .init = &clk_cpu_init,
+ },
+};
+
+static struct clk_ops dmn_ops = {
+ .is_enabled = std_clk_is_enabled,
+ .enable = std_clk_enable,
+ .disable = std_clk_disable,
+ .set_rate = dmn_clk_set_rate,
+ .round_rate = dmn_clk_round_rate,
+ .recalc_rate = dmn_clk_recalc_rate,
+ .set_parent = dmn_clk_set_parent,
+ .get_parent = dmn_clk_get_parent,
+};
+
+/* dsp, gfx, mm, lcd and vpp domain */
+
+static struct clk_init_data clk_dsp_init = {
+ .name = "dsp",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_dsp = {
+ .regofs = SIRFSOC_CLKC_DSP_CFG,
+ .enable_bit = 0,
+ .hw = {
+ .init = &clk_dsp_init,
+ },
+};
+
+static struct clk_init_data clk_gfx_init = {
+ .name = "gfx",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_gfx = {
+ .regofs = SIRFSOC_CLKC_GFX_CFG,
+ .enable_bit = 8,
+ .hw = {
+ .init = &clk_gfx_init,
+ },
+};
+
+static struct clk_init_data clk_mm_init = {
+ .name = "mm",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_mm = {
+ .regofs = SIRFSOC_CLKC_MM_CFG,
+ .enable_bit = 9,
+ .hw = {
+ .init = &clk_mm_init,
+ },
+};
+
+static struct clk_init_data clk_lcd_init = {
+ .name = "lcd",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_lcd = {
+ .regofs = SIRFSOC_CLKC_LCD_CFG,
+ .enable_bit = 10,
+ .hw = {
+ .init = &clk_lcd_init,
+ },
+};
+
+static struct clk_init_data clk_vpp_init = {
+ .name = "vpp",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_vpp = {
+ .regofs = SIRFSOC_CLKC_LCD_CFG,
+ .enable_bit = 11,
+ .hw = {
+ .init = &clk_vpp_init,
+ },
+};
+
+static struct clk_init_data clk_mmc01_init = {
+ .name = "mmc01",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_mmc01 = {
+ .regofs = SIRFSOC_CLKC_MMC_CFG,
+ .enable_bit = 59,
+ .hw = {
+ .init = &clk_mmc01_init,
+ },
+};
+
+static struct clk_init_data clk_mmc23_init = {
+ .name = "mmc23",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_mmc23 = {
+ .regofs = SIRFSOC_CLKC_MMC_CFG,
+ .enable_bit = 60,
+ .hw = {
+ .init = &clk_mmc23_init,
+ },
+};
+
+static struct clk_init_data clk_mmc45_init = {
+ .name = "mmc45",
+ .ops = &dmn_ops,
+ .parent_names = dmn_clk_parents,
+ .num_parents = ARRAY_SIZE(dmn_clk_parents),
+};
+
+static struct clk_dmn clk_mmc45 = {
+ .regofs = SIRFSOC_CLKC_MMC_CFG,
+ .enable_bit = 61,
+ .hw = {
+ .init = &clk_mmc45_init,
+ },
+};
+
+/*
+ * peripheral controllers in io domain
+ */
+
+static int std_clk_is_enabled(struct clk_hw *hw)
+{
+ u32 reg;
+ int bit;
+ struct clk_std *clk = to_stdclk(hw);
+
+ bit = clk->enable_bit % 32;
+ reg = clk->enable_bit / 32;
+ reg = SIRFSOC_CLKC_CLK_EN0 + reg * sizeof(reg);
+
+ return !!(clkc_readl(reg) & BIT(bit));
+}
+
+static int std_clk_enable(struct clk_hw *hw)
+{
+ u32 val, reg;
+ int bit;
+ struct clk_std *clk = to_stdclk(hw);
+
+ BUG_ON(clk->enable_bit < 0 || clk->enable_bit > 63);
+
+ bit = clk->enable_bit % 32;
+ reg = clk->enable_bit / 32;
+ reg = SIRFSOC_CLKC_CLK_EN0 + reg * sizeof(reg);
+
+ val = clkc_readl(reg) | BIT(bit);
+ clkc_writel(val, reg);
+ return 0;
+}
+
+static void std_clk_disable(struct clk_hw *hw)
+{
+ u32 val, reg;
+ int bit;
+ struct clk_std *clk = to_stdclk(hw);
+
+ BUG_ON(clk->enable_bit < 0 || clk->enable_bit > 63);
+
+ bit = clk->enable_bit % 32;
+ reg = clk->enable_bit / 32;
+ reg = SIRFSOC_CLKC_CLK_EN0 + reg * sizeof(reg);
+
+ val = clkc_readl(reg) & ~BIT(bit);
+ clkc_writel(val, reg);
+}
+
+static const char *std_clk_io_parents[] = {
+ "io",
+};
+
+static struct clk_ops ios_ops = {
+ .is_enabled = std_clk_is_enabled,
+ .enable = std_clk_enable,
+ .disable = std_clk_disable,
+};
+
+static struct clk_init_data clk_dmac0_init = {
+ .name = "dmac0",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_dmac0 = {
+ .enable_bit = 32,
+ .hw = {
+ .init = &clk_dmac0_init,
+ },
+};
+
+static struct clk_init_data clk_dmac1_init = {
+ .name = "dmac1",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_dmac1 = {
+ .enable_bit = 33,
+ .hw = {
+ .init = &clk_dmac1_init,
+ },
+};
+
+static struct clk_init_data clk_nand_init = {
+ .name = "nand",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_nand = {
+ .enable_bit = 34,
+ .hw = {
+ .init = &clk_nand_init,
+ },
+};
+
+static struct clk_init_data clk_audio_init = {
+ .name = "audio",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_audio = {
+ .enable_bit = 35,
+ .hw = {
+ .init = &clk_audio_init,
+ },
+};
+
+static struct clk_init_data clk_uart0_init = {
+ .name = "uart0",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_uart0 = {
+ .enable_bit = 36,
+ .hw = {
+ .init = &clk_uart0_init,
+ },
+};
+
+static struct clk_init_data clk_uart1_init = {
+ .name = "uart1",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_uart1 = {
+ .enable_bit = 37,
+ .hw = {
+ .init = &clk_uart1_init,
+ },
+};
+
+static struct clk_init_data clk_uart2_init = {
+ .name = "uart2",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_uart2 = {
+ .enable_bit = 38,
+ .hw = {
+ .init = &clk_uart2_init,
+ },
+};
+
+static struct clk_init_data clk_usp0_init = {
+ .name = "usp0",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_usp0 = {
+ .enable_bit = 39,
+ .hw = {
+ .init = &clk_usp0_init,
+ },
+};
+
+static struct clk_init_data clk_usp1_init = {
+ .name = "usp1",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_usp1 = {
+ .enable_bit = 40,
+ .hw = {
+ .init = &clk_usp1_init,
+ },
+};
+
+static struct clk_init_data clk_usp2_init = {
+ .name = "usp2",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_usp2 = {
+ .enable_bit = 41,
+ .hw = {
+ .init = &clk_usp2_init,
+ },
+};
+
+static struct clk_init_data clk_vip_init = {
+ .name = "vip",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_vip = {
+ .enable_bit = 42,
+ .hw = {
+ .init = &clk_vip_init,
+ },
+};
+
+static struct clk_init_data clk_spi0_init = {
+ .name = "spi0",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_spi0 = {
+ .enable_bit = 43,
+ .hw = {
+ .init = &clk_spi0_init,
+ },
+};
+
+static struct clk_init_data clk_spi1_init = {
+ .name = "spi1",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_spi1 = {
+ .enable_bit = 44,
+ .hw = {
+ .init = &clk_spi1_init,
+ },
+};
+
+static struct clk_init_data clk_tsc_init = {
+ .name = "tsc",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_tsc = {
+ .enable_bit = 45,
+ .hw = {
+ .init = &clk_tsc_init,
+ },
+};
+
+static struct clk_init_data clk_i2c0_init = {
+ .name = "i2c0",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_i2c0 = {
+ .enable_bit = 46,
+ .hw = {
+ .init = &clk_i2c0_init,
+ },
+};
+
+static struct clk_init_data clk_i2c1_init = {
+ .name = "i2c1",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_i2c1 = {
+ .enable_bit = 47,
+ .hw = {
+ .init = &clk_i2c1_init,
+ },
+};
+
+static struct clk_init_data clk_pwmc_init = {
+ .name = "pwmc",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_pwmc = {
+ .enable_bit = 48,
+ .hw = {
+ .init = &clk_pwmc_init,
+ },
+};
+
+static struct clk_init_data clk_efuse_init = {
+ .name = "efuse",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_efuse = {
+ .enable_bit = 49,
+ .hw = {
+ .init = &clk_efuse_init,
+ },
+};
+
+static struct clk_init_data clk_pulse_init = {
+ .name = "pulse",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_pulse = {
+ .enable_bit = 50,
+ .hw = {
+ .init = &clk_pulse_init,
+ },
+};
+
+static const char *std_clk_dsp_parents[] = {
+ "dsp",
+};
+
+static struct clk_init_data clk_gps_init = {
+ .name = "gps",
+ .ops = &ios_ops,
+ .parent_names = std_clk_dsp_parents,
+ .num_parents = ARRAY_SIZE(std_clk_dsp_parents),
+};
+
+static struct clk_std clk_gps = {
+ .enable_bit = 1,
+ .hw = {
+ .init = &clk_gps_init,
+ },
+};
+
+static struct clk_init_data clk_mf_init = {
+ .name = "mf",
+ .ops = &ios_ops,
+ .parent_names = std_clk_io_parents,
+ .num_parents = ARRAY_SIZE(std_clk_io_parents),
+};
+
+static struct clk_std clk_mf = {
+ .enable_bit = 2,
+ .hw = {
+ .init = &clk_mf_init,
+ },
+};
+
+static const char *std_clk_sys_parents[] = {
+ "sys",
+};
+
+static struct clk_init_data clk_security_init = {
+ .name = "mf",
+ .ops = &ios_ops,
+ .parent_names = std_clk_sys_parents,
+ .num_parents = ARRAY_SIZE(std_clk_sys_parents),
+};
+
+static struct clk_std clk_security = {
+ .enable_bit = 19,
+ .hw = {
+ .init = &clk_security_init,
+ },
+};
+
+static const char *std_clk_usb_parents[] = {
+ "usb_pll",
+};
+
+static struct clk_init_data clk_usb0_init = {
+ .name = "usb0",
+ .ops = &ios_ops,
+ .parent_names = std_clk_usb_parents,
+ .num_parents = ARRAY_SIZE(std_clk_usb_parents),
+};
+
+static struct clk_std clk_usb0 = {
+ .enable_bit = 16,
+ .hw = {
+ .init = &clk_usb0_init,
+ },
+};
+
+static struct clk_init_data clk_usb1_init = {
+ .name = "usb1",
+ .ops = &ios_ops,
+ .parent_names = std_clk_usb_parents,
+ .num_parents = ARRAY_SIZE(std_clk_usb_parents),
+};
+
+static struct clk_std clk_usb1 = {
+ .enable_bit = 17,
+ .hw = {
+ .init = &clk_usb1_init,
+ },
+};
+
+static struct of_device_id clkc_ids[] = {
+ { .compatible = "sirf,prima2-clkc" },
+ {},
+};
+
+static struct of_device_id rsc_ids[] = {
+ { .compatible = "sirf,prima2-rsc" },
+ {},
+};
+
+void __init sirfsoc_of_clk_init(void)
+{
+ struct clk *clk;
+ struct device_node *np;
+
+ np = of_find_matching_node(NULL, clkc_ids);
+ if (!np)
+ panic("unable to find compatible clkc node in dtb\n");
+
+ sirfsoc_clk_vbase = of_iomap(np, 0);
+ if (!sirfsoc_clk_vbase)
+ panic("unable to map clkc registers\n");
+
+ of_node_put(np);
+
+ np = of_find_matching_node(NULL, rsc_ids);
+ if (!np)
+ panic("unable to find compatible rsc node in dtb\n");
+
+ sirfsoc_rsc_vbase = of_iomap(np, 0);
+ if (!sirfsoc_rsc_vbase)
+ panic("unable to map rsc registers\n");
+
+ of_node_put(np);
+
+
+ /* These are always available (RTC and 26MHz OSC)*/
+ clk = clk_register_fixed_rate(NULL, "rtc", NULL,
+ CLK_IS_ROOT, 32768);
+ BUG_ON(!clk);
+ clk = clk_register_fixed_rate(NULL, "osc", NULL,
+ CLK_IS_ROOT, 26000000);
+ BUG_ON(!clk);
+
+ clk = clk_register(NULL, &clk_pll1.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_pll2.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_pll3.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_mem.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_sys.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_security.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b8030000.security");
+ clk = clk_register(NULL, &clk_dsp.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_gps.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "a8010000.gps");
+ clk = clk_register(NULL, &clk_mf.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_io.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "io");
+ clk = clk_register(NULL, &clk_cpu.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "cpu");
+ clk = clk_register(NULL, &clk_uart0.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0050000.uart");
+ clk = clk_register(NULL, &clk_uart1.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0060000.uart");
+ clk = clk_register(NULL, &clk_uart2.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0070000.uart");
+ clk = clk_register(NULL, &clk_tsc.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0110000.tsc");
+ clk = clk_register(NULL, &clk_i2c0.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00e0000.i2c");
+ clk = clk_register(NULL, &clk_i2c1.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00f0000.i2c");
+ clk = clk_register(NULL, &clk_spi0.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00d0000.spi");
+ clk = clk_register(NULL, &clk_spi1.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0170000.spi");
+ clk = clk_register(NULL, &clk_pwmc.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0130000.pwm");
+ clk = clk_register(NULL, &clk_efuse.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0140000.efusesys");
+ clk = clk_register(NULL, &clk_pulse.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0150000.pulsec");
+ clk = clk_register(NULL, &clk_dmac0.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00b0000.dma-controller");
+ clk = clk_register(NULL, &clk_dmac1.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0160000.dma-controller");
+ clk = clk_register(NULL, &clk_nand.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0030000.nand");
+ clk = clk_register(NULL, &clk_audio.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0040000.audio");
+ clk = clk_register(NULL, &clk_usp0.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0080000.usp");
+ clk = clk_register(NULL, &clk_usp1.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b0090000.usp");
+ clk = clk_register(NULL, &clk_usp2.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00a0000.usp");
+ clk = clk_register(NULL, &clk_vip.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00c0000.vip");
+ clk = clk_register(NULL, &clk_gfx.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "98000000.graphics");
+ clk = clk_register(NULL, &clk_mm.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "a0000000.multimedia");
+ clk = clk_register(NULL, &clk_lcd.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "90010000.display");
+ clk = clk_register(NULL, &clk_vpp.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "90020000.vpp");
+ clk = clk_register(NULL, &clk_mmc01.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_mmc23.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_mmc45.hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &usb_pll_clk_hw);
+ BUG_ON(!clk);
+ clk = clk_register(NULL, &clk_usb0.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00e0000.usb");
+ clk = clk_register(NULL, &clk_usb1.hw);
+ BUG_ON(!clk);
+ clk_register_clkdev(clk, NULL, "b00f0000.usb");
+}
diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index efdfd00..56e4495e 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -558,25 +558,6 @@
EXPORT_SYMBOL_GPL(clk_enable);
/**
- * clk_get_rate - return the rate of clk
- * @clk: the clk whose rate is being returned
- *
- * Simply returns the cached rate of the clk. Does not query the hardware. If
- * clk is NULL then returns 0.
- */
-unsigned long clk_get_rate(struct clk *clk)
-{
- unsigned long rate;
-
- mutex_lock(&prepare_lock);
- rate = __clk_get_rate(clk);
- mutex_unlock(&prepare_lock);
-
- return rate;
-}
-EXPORT_SYMBOL_GPL(clk_get_rate);
-
-/**
* __clk_round_rate - round the given rate for a clk
* @clk: round the rate of this clock
*
@@ -702,6 +683,30 @@
}
/**
+ * clk_get_rate - return the rate of clk
+ * @clk: the clk whose rate is being returned
+ *
+ * Simply returns the cached rate of the clk, unless CLK_GET_RATE_NOCACHE flag
+ * is set, which means a recalc_rate will be issued.
+ * If clk is NULL then returns 0.
+ */
+unsigned long clk_get_rate(struct clk *clk)
+{
+ unsigned long rate;
+
+ mutex_lock(&prepare_lock);
+
+ if (clk && (clk->flags & CLK_GET_RATE_NOCACHE))
+ __clk_recalc_rates(clk, 0);
+
+ rate = __clk_get_rate(clk);
+ mutex_unlock(&prepare_lock);
+
+ return rate;
+}
+EXPORT_SYMBOL_GPL(clk_get_rate);
+
+/**
* __clk_speculate_rates
* @clk: first clk in the subtree
* @parent_rate: the "future" rate of clk's parent
@@ -1582,6 +1587,20 @@
}
EXPORT_SYMBOL_GPL(of_clk_src_simple_get);
+struct clk *of_clk_src_onecell_get(struct of_phandle_args *clkspec, void *data)
+{
+ struct clk_onecell_data *clk_data = data;
+ unsigned int idx = clkspec->args[0];
+
+ if (idx >= clk_data->clk_num) {
+ pr_err("%s: invalid clock index %d\n", __func__, idx);
+ return ERR_PTR(-EINVAL);
+ }
+
+ return clk_data->clks[idx];
+}
+EXPORT_SYMBOL_GPL(of_clk_src_onecell_get);
+
/**
* of_clk_add_provider() - Register a clock provider for a node
* @np: Device node pointer associated with clock provider
diff --git a/drivers/clk/clkdev.c b/drivers/clk/clkdev.c
index d423c9b..442a3136 100644
--- a/drivers/clk/clkdev.c
+++ b/drivers/clk/clkdev.c
@@ -171,51 +171,6 @@
}
EXPORT_SYMBOL(clk_put);
-static void devm_clk_release(struct device *dev, void *res)
-{
- clk_put(*(struct clk **)res);
-}
-
-struct clk *devm_clk_get(struct device *dev, const char *id)
-{
- struct clk **ptr, *clk;
-
- ptr = devres_alloc(devm_clk_release, sizeof(*ptr), GFP_KERNEL);
- if (!ptr)
- return ERR_PTR(-ENOMEM);
-
- clk = clk_get(dev, id);
- if (!IS_ERR(clk)) {
- *ptr = clk;
- devres_add(dev, ptr);
- } else {
- devres_free(ptr);
- }
-
- return clk;
-}
-EXPORT_SYMBOL(devm_clk_get);
-
-static int devm_clk_match(struct device *dev, void *res, void *data)
-{
- struct clk **c = res;
- if (!c || !*c) {
- WARN_ON(!c || !*c);
- return 0;
- }
- return *c == data;
-}
-
-void devm_clk_put(struct device *dev, struct clk *clk)
-{
- int ret;
-
- ret = devres_destroy(dev, devm_clk_release, devm_clk_match, clk);
-
- WARN_ON(ret);
-}
-EXPORT_SYMBOL(devm_clk_put);
-
void clkdev_add(struct clk_lookup *cl)
{
mutex_lock(&clocks_mutex);
diff --git a/drivers/clk/mmp/Makefile b/drivers/clk/mmp/Makefile
new file mode 100644
index 0000000..392d780
--- /dev/null
+++ b/drivers/clk/mmp/Makefile
@@ -0,0 +1,9 @@
+#
+# Makefile for mmp specific clk
+#
+
+obj-y += clk-apbc.o clk-apmu.o clk-frac.o
+
+obj-$(CONFIG_CPU_PXA168) += clk-pxa168.o
+obj-$(CONFIG_CPU_PXA910) += clk-pxa910.o
+obj-$(CONFIG_CPU_MMP2) += clk-mmp2.o
diff --git a/drivers/clk/mmp/clk-apbc.c b/drivers/clk/mmp/clk-apbc.c
new file mode 100644
index 0000000..d14120e
--- /dev/null
+++ b/drivers/clk/mmp/clk-apbc.c
@@ -0,0 +1,152 @@
+/*
+ * mmp APB clock operation source file
+ *
+ * Copyright (C) 2012 Marvell
+ * Chao Xie <xiechao.mail@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/kernel.h>
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/err.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+
+#include "clk.h"
+
+/* Common APB clock register bit definitions */
+#define APBC_APBCLK (1 << 0) /* APB Bus Clock Enable */
+#define APBC_FNCLK (1 << 1) /* Functional Clock Enable */
+#define APBC_RST (1 << 2) /* Reset Generation */
+#define APBC_POWER (1 << 7) /* Reset Generation */
+
+#define to_clk_apbc(hw) container_of(hw, struct clk_apbc, hw)
+struct clk_apbc {
+ struct clk_hw hw;
+ void __iomem *base;
+ unsigned int delay;
+ unsigned int flags;
+ spinlock_t *lock;
+};
+
+static int clk_apbc_prepare(struct clk_hw *hw)
+{
+ struct clk_apbc *apbc = to_clk_apbc(hw);
+ unsigned int data;
+ unsigned long flags = 0;
+
+ /*
+ * It may share same register as MUX clock,
+ * and it will impact FNCLK enable. Spinlock is needed
+ */
+ if (apbc->lock)
+ spin_lock_irqsave(apbc->lock, flags);
+
+ data = readl_relaxed(apbc->base);
+ if (apbc->flags & APBC_POWER_CTRL)
+ data |= APBC_POWER;
+ data |= APBC_FNCLK;
+ writel_relaxed(data, apbc->base);
+
+ if (apbc->lock)
+ spin_unlock_irqrestore(apbc->lock, flags);
+
+ udelay(apbc->delay);
+
+ if (apbc->lock)
+ spin_lock_irqsave(apbc->lock, flags);
+
+ data = readl_relaxed(apbc->base);
+ data |= APBC_APBCLK;
+ writel_relaxed(data, apbc->base);
+
+ if (apbc->lock)
+ spin_unlock_irqrestore(apbc->lock, flags);
+
+ udelay(apbc->delay);
+
+ if (!(apbc->flags & APBC_NO_BUS_CTRL)) {
+ if (apbc->lock)
+ spin_lock_irqsave(apbc->lock, flags);
+
+ data = readl_relaxed(apbc->base);
+ data &= ~APBC_RST;
+ writel_relaxed(data, apbc->base);
+
+ if (apbc->lock)
+ spin_unlock_irqrestore(apbc->lock, flags);
+ }
+
+ return 0;
+}
+
+static void clk_apbc_unprepare(struct clk_hw *hw)
+{
+ struct clk_apbc *apbc = to_clk_apbc(hw);
+ unsigned long data;
+ unsigned long flags = 0;
+
+ if (apbc->lock)
+ spin_lock_irqsave(apbc->lock, flags);
+
+ data = readl_relaxed(apbc->base);
+ if (apbc->flags & APBC_POWER_CTRL)
+ data &= ~APBC_POWER;
+ data &= ~APBC_FNCLK;
+ writel_relaxed(data, apbc->base);
+
+ if (apbc->lock)
+ spin_unlock_irqrestore(apbc->lock, flags);
+
+ udelay(10);
+
+ if (apbc->lock)
+ spin_lock_irqsave(apbc->lock, flags);
+
+ data = readl_relaxed(apbc->base);
+ data &= ~APBC_APBCLK;
+ writel_relaxed(data, apbc->base);
+
+ if (apbc->lock)
+ spin_unlock_irqrestore(apbc->lock, flags);
+}
+
+struct clk_ops clk_apbc_ops = {
+ .prepare = clk_apbc_prepare,
+ .unprepare = clk_apbc_unprepare,
+};
+
+struct clk *mmp_clk_register_apbc(const char *name, const char *parent_name,
+ void __iomem *base, unsigned int delay,
+ unsigned int apbc_flags, spinlock_t *lock)
+{
+ struct clk_apbc *apbc;
+ struct clk *clk;
+ struct clk_init_data init;
+
+ apbc = kzalloc(sizeof(*apbc), GFP_KERNEL);
+ if (!apbc)
+ return NULL;
+
+ init.name = name;
+ init.ops = &clk_apbc_ops;
+ init.flags = CLK_SET_RATE_PARENT;
+ init.parent_names = (parent_name ? &parent_name : NULL);
+ init.num_parents = (parent_name ? 1 : 0);
+
+ apbc->base = base;
+ apbc->delay = delay;
+ apbc->flags = apbc_flags;
+ apbc->lock = lock;
+ apbc->hw.init = &init;
+
+ clk = clk_register(NULL, &apbc->hw);
+ if (IS_ERR(clk))
+ kfree(apbc);
+
+ return clk;
+}
diff --git a/drivers/clk/mmp/clk-apmu.c b/drivers/clk/mmp/clk-apmu.c
new file mode 100644
index 0000000..abe182b
--- /dev/null
+++ b/drivers/clk/mmp/clk-apmu.c
@@ -0,0 +1,97 @@
+/*
+ * mmp AXI peripharal clock operation source file
+ *
+ * Copyright (C) 2012 Marvell
+ * Chao Xie <xiechao.mail@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/kernel.h>
+#include <linux/clk.h>
+#include <linux/io.h>
+#include <linux/err.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+
+#include "clk.h"
+
+#define to_clk_apmu(clk) (container_of(clk, struct clk_apmu, clk))
+struct clk_apmu {
+ struct clk_hw hw;
+ void __iomem *base;
+ u32 rst_mask;
+ u32 enable_mask;
+ spinlock_t *lock;
+};
+
+static int clk_apmu_enable(struct clk_hw *hw)
+{
+ struct clk_apmu *apmu = to_clk_apmu(hw);
+ unsigned long data;
+ unsigned long flags = 0;
+
+ if (apmu->lock)
+ spin_lock_irqsave(apmu->lock, flags);
+
+ data = readl_relaxed(apmu->base) | apmu->enable_mask;
+ writel_relaxed(data, apmu->base);
+
+ if (apmu->lock)
+ spin_unlock_irqrestore(apmu->lock, flags);
+
+ return 0;
+}
+
+static void clk_apmu_disable(struct clk_hw *hw)
+{
+ struct clk_apmu *apmu = to_clk_apmu(hw);
+ unsigned long data;
+ unsigned long flags = 0;
+
+ if (apmu->lock)
+ spin_lock_irqsave(apmu->lock, flags);
+
+ data = readl_relaxed(apmu->base) & ~apmu->enable_mask;
+ writel_relaxed(data, apmu->base);
+
+ if (apmu->lock)
+ spin_unlock_irqrestore(apmu->lock, flags);
+}
+
+struct clk_ops clk_apmu_ops = {
+ .enable = clk_apmu_enable,
+ .disable = clk_apmu_disable,
+};
+
+struct clk *mmp_clk_register_apmu(const char *name, const char *parent_name,
+ void __iomem *base, u32 enable_mask, spinlock_t *lock)
+{
+ struct clk_apmu *apmu;
+ struct clk *clk;
+ struct clk_init_data init;
+
+ apmu = kzalloc(sizeof(*apmu), GFP_KERNEL);
+ if (!apmu)
+ return NULL;
+
+ init.name = name;
+ init.ops = &clk_apmu_ops;
+ init.flags = CLK_SET_RATE_PARENT;
+ init.parent_names = (parent_name ? &parent_name : NULL);
+ init.num_parents = (parent_name ? 1 : 0);
+
+ apmu->base = base;
+ apmu->enable_mask = enable_mask;
+ apmu->lock = lock;
+ apmu->hw.init = &init;
+
+ clk = clk_register(NULL, &apmu->hw);
+
+ if (IS_ERR(clk))
+ kfree(apmu);
+
+ return clk;
+}
diff --git a/drivers/clk/mmp/clk-frac.c b/drivers/clk/mmp/clk-frac.c
new file mode 100644
index 0000000..80c1dd1
--- /dev/null
+++ b/drivers/clk/mmp/clk-frac.c
@@ -0,0 +1,153 @@
+/*
+ * mmp factor clock operation source file
+ *
+ * Copyright (C) 2012 Marvell
+ * Chao Xie <xiechao.mail@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/slab.h>
+#include <linux/io.h>
+#include <linux/err.h>
+
+#include "clk.h"
+/*
+ * It is M/N clock
+ *
+ * Fout from synthesizer can be given from two equations:
+ * numerator/denominator = Fin / (Fout * factor)
+ */
+
+#define to_clk_factor(hw) container_of(hw, struct clk_factor, hw)
+struct clk_factor {
+ struct clk_hw hw;
+ void __iomem *base;
+ struct clk_factor_masks *masks;
+ struct clk_factor_tbl *ftbl;
+ unsigned int ftbl_cnt;
+};
+
+static long clk_factor_round_rate(struct clk_hw *hw, unsigned long drate,
+ unsigned long *prate)
+{
+ struct clk_factor *factor = to_clk_factor(hw);
+ unsigned long rate = 0, prev_rate;
+ int i;
+
+ for (i = 0; i < factor->ftbl_cnt; i++) {
+ prev_rate = rate;
+ rate = (((*prate / 10000) * factor->ftbl[i].num) /
+ (factor->ftbl[i].den * factor->masks->factor)) * 10000;
+ if (rate > drate)
+ break;
+ }
+ if (i == 0)
+ return rate;
+ else
+ return prev_rate;
+}
+
+static unsigned long clk_factor_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct clk_factor *factor = to_clk_factor(hw);
+ struct clk_factor_masks *masks = factor->masks;
+ unsigned int val, num, den;
+
+ val = readl_relaxed(factor->base);
+
+ /* calculate numerator */
+ num = (val >> masks->num_shift) & masks->num_mask;
+
+ /* calculate denominator */
+ den = (val >> masks->den_shift) & masks->num_mask;
+
+ if (!den)
+ return 0;
+
+ return (((parent_rate / 10000) * den) /
+ (num * factor->masks->factor)) * 10000;
+}
+
+/* Configures new clock rate*/
+static int clk_factor_set_rate(struct clk_hw *hw, unsigned long drate,
+ unsigned long prate)
+{
+ struct clk_factor *factor = to_clk_factor(hw);
+ struct clk_factor_masks *masks = factor->masks;
+ int i;
+ unsigned long val;
+ unsigned long prev_rate, rate = 0;
+
+ for (i = 0; i < factor->ftbl_cnt; i++) {
+ prev_rate = rate;
+ rate = (((prate / 10000) * factor->ftbl[i].num) /
+ (factor->ftbl[i].den * factor->masks->factor)) * 10000;
+ if (rate > drate)
+ break;
+ }
+ if (i > 0)
+ i--;
+
+ val = readl_relaxed(factor->base);
+
+ val &= ~(masks->num_mask << masks->num_shift);
+ val |= (factor->ftbl[i].num & masks->num_mask) << masks->num_shift;
+
+ val &= ~(masks->den_mask << masks->den_shift);
+ val |= (factor->ftbl[i].den & masks->den_mask) << masks->den_shift;
+
+ writel_relaxed(val, factor->base);
+
+ return 0;
+}
+
+static struct clk_ops clk_factor_ops = {
+ .recalc_rate = clk_factor_recalc_rate,
+ .round_rate = clk_factor_round_rate,
+ .set_rate = clk_factor_set_rate,
+};
+
+struct clk *mmp_clk_register_factor(const char *name, const char *parent_name,
+ unsigned long flags, void __iomem *base,
+ struct clk_factor_masks *masks, struct clk_factor_tbl *ftbl,
+ unsigned int ftbl_cnt)
+{
+ struct clk_factor *factor;
+ struct clk_init_data init;
+ struct clk *clk;
+
+ if (!masks) {
+ pr_err("%s: must pass a clk_factor_mask\n", __func__);
+ return ERR_PTR(-EINVAL);
+ }
+
+ factor = kzalloc(sizeof(*factor), GFP_KERNEL);
+ if (!factor) {
+ pr_err("%s: could not allocate factor clk\n", __func__);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* struct clk_aux assignments */
+ factor->base = base;
+ factor->masks = masks;
+ factor->ftbl = ftbl;
+ factor->ftbl_cnt = ftbl_cnt;
+ factor->hw.init = &init;
+
+ init.name = name;
+ init.ops = &clk_factor_ops;
+ init.flags = flags;
+ init.parent_names = &parent_name;
+ init.num_parents = 1;
+
+ clk = clk_register(NULL, &factor->hw);
+ if (IS_ERR_OR_NULL(clk))
+ kfree(factor);
+
+ return clk;
+}
diff --git a/drivers/clk/mmp/clk-mmp2.c b/drivers/clk/mmp/clk-mmp2.c
new file mode 100644
index 0000000..ade4358
--- /dev/null
+++ b/drivers/clk/mmp/clk-mmp2.c
@@ -0,0 +1,449 @@
+/*
+ * mmp2 clock framework source file
+ *
+ * Copyright (C) 2012 Marvell
+ * Chao Xie <xiechao.mail@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/io.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+
+#include <mach/addr-map.h>
+
+#include "clk.h"
+
+#define APBC_RTC 0x0
+#define APBC_TWSI0 0x4
+#define APBC_TWSI1 0x8
+#define APBC_TWSI2 0xc
+#define APBC_TWSI3 0x10
+#define APBC_TWSI4 0x7c
+#define APBC_TWSI5 0x80
+#define APBC_KPC 0x18
+#define APBC_UART0 0x2c
+#define APBC_UART1 0x30
+#define APBC_UART2 0x34
+#define APBC_UART3 0x88
+#define APBC_GPIO 0x38
+#define APBC_PWM0 0x3c
+#define APBC_PWM1 0x40
+#define APBC_PWM2 0x44
+#define APBC_PWM3 0x48
+#define APBC_SSP0 0x50
+#define APBC_SSP1 0x54
+#define APBC_SSP2 0x58
+#define APBC_SSP3 0x5c
+#define APMU_SDH0 0x54
+#define APMU_SDH1 0x58
+#define APMU_SDH2 0xe8
+#define APMU_SDH3 0xec
+#define APMU_USB 0x5c
+#define APMU_DISP0 0x4c
+#define APMU_DISP1 0x110
+#define APMU_CCIC0 0x50
+#define APMU_CCIC1 0xf4
+#define MPMU_UART_PLL 0x14
+
+static DEFINE_SPINLOCK(clk_lock);
+
+static struct clk_factor_masks uart_factor_masks = {
+ .factor = 2,
+ .num_mask = 0x1fff,
+ .den_mask = 0x1fff,
+ .num_shift = 16,
+ .den_shift = 0,
+};
+
+static struct clk_factor_tbl uart_factor_tbl[] = {
+ {.num = 14634, .den = 2165}, /*14.745MHZ */
+ {.num = 3521, .den = 689}, /*19.23MHZ */
+ {.num = 9679, .den = 5728}, /*58.9824MHZ */
+ {.num = 15850, .den = 9451}, /*59.429MHZ */
+};
+
+static const char *uart_parent[] = {"uart_pll", "vctcxo"};
+static const char *ssp_parent[] = {"vctcxo_4", "vctcxo_2", "vctcxo", "pll1_16"};
+static const char *sdh_parent[] = {"pll1_4", "pll2", "usb_pll", "pll1"};
+static const char *disp_parent[] = {"pll1", "pll1_16", "pll2", "vctcxo"};
+static const char *ccic_parent[] = {"pll1_2", "pll1_16", "vctcxo"};
+
+void __init mmp2_clk_init(void)
+{
+ struct clk *clk;
+ struct clk *vctcxo;
+ void __iomem *mpmu_base;
+ void __iomem *apmu_base;
+ void __iomem *apbc_base;
+
+ mpmu_base = ioremap(APB_PHYS_BASE + 0x50000, SZ_4K);
+ if (mpmu_base == NULL) {
+ pr_err("error to ioremap MPMU base\n");
+ return;
+ }
+
+ apmu_base = ioremap(AXI_PHYS_BASE + 0x82800, SZ_4K);
+ if (apmu_base == NULL) {
+ pr_err("error to ioremap APMU base\n");
+ return;
+ }
+
+ apbc_base = ioremap(APB_PHYS_BASE + 0x15000, SZ_4K);
+ if (apbc_base == NULL) {
+ pr_err("error to ioremap APBC base\n");
+ return;
+ }
+
+ clk = clk_register_fixed_rate(NULL, "clk32", NULL, CLK_IS_ROOT, 3200);
+ clk_register_clkdev(clk, "clk32", NULL);
+
+ vctcxo = clk_register_fixed_rate(NULL, "vctcxo", NULL, CLK_IS_ROOT,
+ 26000000);
+ clk_register_clkdev(vctcxo, "vctcxo", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "pll1", NULL, CLK_IS_ROOT,
+ 800000000);
+ clk_register_clkdev(clk, "pll1", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "usb_pll", NULL, CLK_IS_ROOT,
+ 480000000);
+ clk_register_clkdev(clk, "usb_pll", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "pll2", NULL, CLK_IS_ROOT,
+ 960000000);
+ clk_register_clkdev(clk, "pll2", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_2", "pll1",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_2", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_4", "pll1_2",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_4", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_8", "pll1_4",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_8", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_16", "pll1_8",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_16", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_20", "pll1_4",
+ CLK_SET_RATE_PARENT, 1, 5);
+ clk_register_clkdev(clk, "pll1_20", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_3", "pll1",
+ CLK_SET_RATE_PARENT, 1, 3);
+ clk_register_clkdev(clk, "pll1_3", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_6", "pll1_3",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_6", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_12", "pll1_6",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_12", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_2", "pll2",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll2_2", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_4", "pll2_2",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll2_4", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_8", "pll2_4",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll2_8", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_16", "pll2_8",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll2_16", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_3", "pll2",
+ CLK_SET_RATE_PARENT, 1, 3);
+ clk_register_clkdev(clk, "pll2_3", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_6", "pll2_3",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll2_6", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll2_12", "pll2_6",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll2_12", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "vctcxo_2", "vctcxo",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "vctcxo_2", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "vctcxo_4", "vctcxo_2",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "vctcxo_4", NULL);
+
+ clk = mmp_clk_register_factor("uart_pll", "pll1_4", 0,
+ mpmu_base + MPMU_UART_PLL,
+ &uart_factor_masks, uart_factor_tbl,
+ ARRAY_SIZE(uart_factor_tbl));
+ clk_set_rate(clk, 14745600);
+ clk_register_clkdev(clk, "uart_pll", NULL);
+
+ clk = mmp_clk_register_apbc("twsi0", "vctcxo",
+ apbc_base + APBC_TWSI0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.0");
+
+ clk = mmp_clk_register_apbc("twsi1", "vctcxo",
+ apbc_base + APBC_TWSI1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.1");
+
+ clk = mmp_clk_register_apbc("twsi2", "vctcxo",
+ apbc_base + APBC_TWSI2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.2");
+
+ clk = mmp_clk_register_apbc("twsi3", "vctcxo",
+ apbc_base + APBC_TWSI3, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.3");
+
+ clk = mmp_clk_register_apbc("twsi4", "vctcxo",
+ apbc_base + APBC_TWSI4, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.4");
+
+ clk = mmp_clk_register_apbc("twsi5", "vctcxo",
+ apbc_base + APBC_TWSI5, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.5");
+
+ clk = mmp_clk_register_apbc("gpio", "vctcxo",
+ apbc_base + APBC_GPIO, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa-gpio");
+
+ clk = mmp_clk_register_apbc("kpc", "clk32",
+ apbc_base + APBC_KPC, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa27x-keypad");
+
+ clk = mmp_clk_register_apbc("rtc", "clk32",
+ apbc_base + APBC_RTC, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-rtc");
+
+ clk = mmp_clk_register_apbc("pwm0", "vctcxo",
+ apbc_base + APBC_PWM0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp2-pwm.0");
+
+ clk = mmp_clk_register_apbc("pwm1", "vctcxo",
+ apbc_base + APBC_PWM1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp2-pwm.1");
+
+ clk = mmp_clk_register_apbc("pwm2", "vctcxo",
+ apbc_base + APBC_PWM2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp2-pwm.2");
+
+ clk = mmp_clk_register_apbc("pwm3", "vctcxo",
+ apbc_base + APBC_PWM3, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp2-pwm.3");
+
+ clk = clk_register_mux(NULL, "uart0_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART0, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, vctcxo);
+ clk_register_clkdev(clk, "uart_mux.0", NULL);
+
+ clk = mmp_clk_register_apbc("uart0", "uart0_mux",
+ apbc_base + APBC_UART0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.0");
+
+ clk = clk_register_mux(NULL, "uart1_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART1, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, vctcxo);
+ clk_register_clkdev(clk, "uart_mux.1", NULL);
+
+ clk = mmp_clk_register_apbc("uart1", "uart1_mux",
+ apbc_base + APBC_UART1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.1");
+
+ clk = clk_register_mux(NULL, "uart2_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART2, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, vctcxo);
+ clk_register_clkdev(clk, "uart_mux.2", NULL);
+
+ clk = mmp_clk_register_apbc("uart2", "uart2_mux",
+ apbc_base + APBC_UART2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.2");
+
+ clk = clk_register_mux(NULL, "uart3_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART3, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, vctcxo);
+ clk_register_clkdev(clk, "uart_mux.3", NULL);
+
+ clk = mmp_clk_register_apbc("uart3", "uart3_mux",
+ apbc_base + APBC_UART3, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.3");
+
+ clk = clk_register_mux(NULL, "ssp0_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP0, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "uart_mux.0", NULL);
+
+ clk = mmp_clk_register_apbc("ssp0", "ssp0_mux",
+ apbc_base + APBC_SSP0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.0");
+
+ clk = clk_register_mux(NULL, "ssp1_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP1, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.1", NULL);
+
+ clk = mmp_clk_register_apbc("ssp1", "ssp1_mux",
+ apbc_base + APBC_SSP1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.1");
+
+ clk = clk_register_mux(NULL, "ssp2_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP2, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.2", NULL);
+
+ clk = mmp_clk_register_apbc("ssp2", "ssp2_mux",
+ apbc_base + APBC_SSP2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.2");
+
+ clk = clk_register_mux(NULL, "ssp3_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP3, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.3", NULL);
+
+ clk = mmp_clk_register_apbc("ssp3", "ssp3_mux",
+ apbc_base + APBC_SSP3, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.3");
+
+ clk = clk_register_mux(NULL, "sdh_mux", sdh_parent,
+ ARRAY_SIZE(sdh_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_SDH0, 8, 2, 0, &clk_lock);
+ clk_register_clkdev(clk, "sdh_mux", NULL);
+
+ clk = clk_register_divider(NULL, "sdh_div", "sdh_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_SDH0,
+ 10, 4, CLK_DIVIDER_ONE_BASED, &clk_lock);
+ clk_register_clkdev(clk, "sdh_div", NULL);
+
+ clk = mmp_clk_register_apmu("sdh0", "sdh_div", apmu_base + APMU_SDH0,
+ 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxav3.0");
+
+ clk = mmp_clk_register_apmu("sdh1", "sdh_div", apmu_base + APMU_SDH1,
+ 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxav3.1");
+
+ clk = mmp_clk_register_apmu("sdh2", "sdh_div", apmu_base + APMU_SDH2,
+ 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxav3.2");
+
+ clk = mmp_clk_register_apmu("sdh3", "sdh_div", apmu_base + APMU_SDH3,
+ 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxav3.3");
+
+ clk = mmp_clk_register_apmu("usb", "usb_pll", apmu_base + APMU_USB,
+ 0x9, &clk_lock);
+ clk_register_clkdev(clk, "usb_clk", NULL);
+
+ clk = clk_register_mux(NULL, "disp0_mux", disp_parent,
+ ARRAY_SIZE(disp_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_DISP0, 6, 2, 0, &clk_lock);
+ clk_register_clkdev(clk, "disp_mux.0", NULL);
+
+ clk = clk_register_divider(NULL, "disp0_div", "disp0_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_DISP0,
+ 8, 4, CLK_DIVIDER_ONE_BASED, &clk_lock);
+ clk_register_clkdev(clk, "disp_div.0", NULL);
+
+ clk = mmp_clk_register_apmu("disp0", "disp0_div",
+ apmu_base + APMU_DISP0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-disp.0");
+
+ clk = clk_register_divider(NULL, "disp0_sphy_div", "disp0_mux", 0,
+ apmu_base + APMU_DISP0, 15, 5, 0, &clk_lock);
+ clk_register_clkdev(clk, "disp_sphy_div.0", NULL);
+
+ clk = mmp_clk_register_apmu("disp0_sphy", "disp0_sphy_div",
+ apmu_base + APMU_DISP0, 0x1024, &clk_lock);
+ clk_register_clkdev(clk, "disp_sphy.0", NULL);
+
+ clk = clk_register_mux(NULL, "disp1_mux", disp_parent,
+ ARRAY_SIZE(disp_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_DISP1, 6, 2, 0, &clk_lock);
+ clk_register_clkdev(clk, "disp_mux.1", NULL);
+
+ clk = clk_register_divider(NULL, "disp1_div", "disp1_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_DISP1,
+ 8, 4, CLK_DIVIDER_ONE_BASED, &clk_lock);
+ clk_register_clkdev(clk, "disp_div.1", NULL);
+
+ clk = mmp_clk_register_apmu("disp1", "disp1_div",
+ apmu_base + APMU_DISP1, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-disp.1");
+
+ clk = mmp_clk_register_apmu("ccic_arbiter", "vctcxo",
+ apmu_base + APMU_CCIC0, 0x1800, &clk_lock);
+ clk_register_clkdev(clk, "ccic_arbiter", NULL);
+
+ clk = clk_register_mux(NULL, "ccic0_mux", ccic_parent,
+ ARRAY_SIZE(ccic_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_CCIC0, 6, 2, 0, &clk_lock);
+ clk_register_clkdev(clk, "ccic_mux.0", NULL);
+
+ clk = clk_register_divider(NULL, "ccic0_div", "ccic0_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC0,
+ 17, 4, CLK_DIVIDER_ONE_BASED, &clk_lock);
+ clk_register_clkdev(clk, "ccic_div.0", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0", "ccic0_div",
+ apmu_base + APMU_CCIC0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, "fnclk", "mmp-ccic.0");
+
+ clk = mmp_clk_register_apmu("ccic0_phy", "ccic0_div",
+ apmu_base + APMU_CCIC0, 0x24, &clk_lock);
+ clk_register_clkdev(clk, "phyclk", "mmp-ccic.0");
+
+ clk = clk_register_divider(NULL, "ccic0_sphy_div", "ccic0_div",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC0,
+ 10, 5, 0, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk_div", "mmp-ccic.0");
+
+ clk = mmp_clk_register_apmu("ccic0_sphy", "ccic0_sphy_div",
+ apmu_base + APMU_CCIC0, 0x300, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk", "mmp-ccic.0");
+
+ clk = clk_register_mux(NULL, "ccic1_mux", ccic_parent,
+ ARRAY_SIZE(ccic_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_CCIC1, 6, 2, 0, &clk_lock);
+ clk_register_clkdev(clk, "ccic_mux.1", NULL);
+
+ clk = clk_register_divider(NULL, "ccic1_div", "ccic1_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC1,
+ 16, 4, CLK_DIVIDER_ONE_BASED, &clk_lock);
+ clk_register_clkdev(clk, "ccic_div.1", NULL);
+
+ clk = mmp_clk_register_apmu("ccic1", "ccic1_div",
+ apmu_base + APMU_CCIC1, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, "fnclk", "mmp-ccic.1");
+
+ clk = mmp_clk_register_apmu("ccic1_phy", "ccic1_div",
+ apmu_base + APMU_CCIC1, 0x24, &clk_lock);
+ clk_register_clkdev(clk, "phyclk", "mmp-ccic.1");
+
+ clk = clk_register_divider(NULL, "ccic1_sphy_div", "ccic1_div",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC1,
+ 10, 5, 0, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk_div", "mmp-ccic.1");
+
+ clk = mmp_clk_register_apmu("ccic1_sphy", "ccic1_sphy_div",
+ apmu_base + APMU_CCIC1, 0x300, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk", "mmp-ccic.1");
+}
diff --git a/drivers/clk/mmp/clk-pxa168.c b/drivers/clk/mmp/clk-pxa168.c
new file mode 100644
index 0000000..e8d036c
--- /dev/null
+++ b/drivers/clk/mmp/clk-pxa168.c
@@ -0,0 +1,346 @@
+/*
+ * pxa168 clock framework source file
+ *
+ * Copyright (C) 2012 Marvell
+ * Chao Xie <xiechao.mail@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/io.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+
+#include <mach/addr-map.h>
+
+#include "clk.h"
+
+#define APBC_RTC 0x28
+#define APBC_TWSI0 0x2c
+#define APBC_KPC 0x30
+#define APBC_UART0 0x0
+#define APBC_UART1 0x4
+#define APBC_GPIO 0x8
+#define APBC_PWM0 0xc
+#define APBC_PWM1 0x10
+#define APBC_PWM2 0x14
+#define APBC_PWM3 0x18
+#define APBC_SSP0 0x81c
+#define APBC_SSP1 0x820
+#define APBC_SSP2 0x84c
+#define APBC_SSP3 0x858
+#define APBC_SSP4 0x85c
+#define APBC_TWSI1 0x6c
+#define APBC_UART2 0x70
+#define APMU_SDH0 0x54
+#define APMU_SDH1 0x58
+#define APMU_USB 0x5c
+#define APMU_DISP0 0x4c
+#define APMU_CCIC0 0x50
+#define APMU_DFC 0x60
+#define MPMU_UART_PLL 0x14
+
+static DEFINE_SPINLOCK(clk_lock);
+
+static struct clk_factor_masks uart_factor_masks = {
+ .factor = 2,
+ .num_mask = 0x1fff,
+ .den_mask = 0x1fff,
+ .num_shift = 16,
+ .den_shift = 0,
+};
+
+static struct clk_factor_tbl uart_factor_tbl[] = {
+ {.num = 8125, .den = 1536}, /*14.745MHZ */
+};
+
+static const char *uart_parent[] = {"pll1_3_16", "uart_pll"};
+static const char *ssp_parent[] = {"pll1_96", "pll1_48", "pll1_24", "pll1_12"};
+static const char *sdh_parent[] = {"pll1_12", "pll1_13"};
+static const char *disp_parent[] = {"pll1_2", "pll1_12"};
+static const char *ccic_parent[] = {"pll1_2", "pll1_12"};
+static const char *ccic_phy_parent[] = {"pll1_6", "pll1_12"};
+
+void __init pxa168_clk_init(void)
+{
+ struct clk *clk;
+ struct clk *uart_pll;
+ void __iomem *mpmu_base;
+ void __iomem *apmu_base;
+ void __iomem *apbc_base;
+
+ mpmu_base = ioremap(APB_PHYS_BASE + 0x50000, SZ_4K);
+ if (mpmu_base == NULL) {
+ pr_err("error to ioremap MPMU base\n");
+ return;
+ }
+
+ apmu_base = ioremap(AXI_PHYS_BASE + 0x82800, SZ_4K);
+ if (apmu_base == NULL) {
+ pr_err("error to ioremap APMU base\n");
+ return;
+ }
+
+ apbc_base = ioremap(APB_PHYS_BASE + 0x15000, SZ_4K);
+ if (apbc_base == NULL) {
+ pr_err("error to ioremap APBC base\n");
+ return;
+ }
+
+ clk = clk_register_fixed_rate(NULL, "clk32", NULL, CLK_IS_ROOT, 3200);
+ clk_register_clkdev(clk, "clk32", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "vctcxo", NULL, CLK_IS_ROOT,
+ 26000000);
+ clk_register_clkdev(clk, "vctcxo", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "pll1", NULL, CLK_IS_ROOT,
+ 624000000);
+ clk_register_clkdev(clk, "pll1", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_2", "pll1",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_2", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_4", "pll1_2",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_4", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_8", "pll1_4",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_8", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_16", "pll1_8",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_16", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_6", "pll1_2",
+ CLK_SET_RATE_PARENT, 1, 3);
+ clk_register_clkdev(clk, "pll1_6", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_12", "pll1_6",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_12", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_24", "pll1_12",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_24", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_48", "pll1_24",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_48", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_96", "pll1_48",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_96", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_13", "pll1",
+ CLK_SET_RATE_PARENT, 1, 13);
+ clk_register_clkdev(clk, "pll1_13", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_13_1_5", "pll1",
+ CLK_SET_RATE_PARENT, 2, 3);
+ clk_register_clkdev(clk, "pll1_13_1_5", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_2_1_5", "pll1",
+ CLK_SET_RATE_PARENT, 2, 3);
+ clk_register_clkdev(clk, "pll1_2_1_5", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_3_16", "pll1",
+ CLK_SET_RATE_PARENT, 3, 16);
+ clk_register_clkdev(clk, "pll1_3_16", NULL);
+
+ uart_pll = mmp_clk_register_factor("uart_pll", "pll1_4", 0,
+ mpmu_base + MPMU_UART_PLL,
+ &uart_factor_masks, uart_factor_tbl,
+ ARRAY_SIZE(uart_factor_tbl));
+ clk_set_rate(uart_pll, 14745600);
+ clk_register_clkdev(uart_pll, "uart_pll", NULL);
+
+ clk = mmp_clk_register_apbc("twsi0", "pll1_13_1_5",
+ apbc_base + APBC_TWSI0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.0");
+
+ clk = mmp_clk_register_apbc("twsi1", "pll1_13_1_5",
+ apbc_base + APBC_TWSI1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.1");
+
+ clk = mmp_clk_register_apbc("gpio", "vctcxo",
+ apbc_base + APBC_GPIO, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa-gpio");
+
+ clk = mmp_clk_register_apbc("kpc", "clk32",
+ apbc_base + APBC_KPC, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa27x-keypad");
+
+ clk = mmp_clk_register_apbc("rtc", "clk32",
+ apbc_base + APBC_RTC, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sa1100-rtc");
+
+ clk = mmp_clk_register_apbc("pwm0", "pll1_48",
+ apbc_base + APBC_PWM0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa168-pwm.0");
+
+ clk = mmp_clk_register_apbc("pwm1", "pll1_48",
+ apbc_base + APBC_PWM1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa168-pwm.1");
+
+ clk = mmp_clk_register_apbc("pwm2", "pll1_48",
+ apbc_base + APBC_PWM2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa168-pwm.2");
+
+ clk = mmp_clk_register_apbc("pwm3", "pll1_48",
+ apbc_base + APBC_PWM3, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa168-pwm.3");
+
+ clk = clk_register_mux(NULL, "uart0_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART0, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, uart_pll);
+ clk_register_clkdev(clk, "uart_mux.0", NULL);
+
+ clk = mmp_clk_register_apbc("uart0", "uart0_mux",
+ apbc_base + APBC_UART0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.0");
+
+ clk = clk_register_mux(NULL, "uart1_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART1, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, uart_pll);
+ clk_register_clkdev(clk, "uart_mux.1", NULL);
+
+ clk = mmp_clk_register_apbc("uart1", "uart1_mux",
+ apbc_base + APBC_UART1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.1");
+
+ clk = clk_register_mux(NULL, "uart2_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART2, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, uart_pll);
+ clk_register_clkdev(clk, "uart_mux.2", NULL);
+
+ clk = mmp_clk_register_apbc("uart2", "uart2_mux",
+ apbc_base + APBC_UART2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.2");
+
+ clk = clk_register_mux(NULL, "ssp0_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP0, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "uart_mux.0", NULL);
+
+ clk = mmp_clk_register_apbc("ssp0", "ssp0_mux", apbc_base + APBC_SSP0,
+ 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.0");
+
+ clk = clk_register_mux(NULL, "ssp1_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP1, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.1", NULL);
+
+ clk = mmp_clk_register_apbc("ssp1", "ssp1_mux", apbc_base + APBC_SSP1,
+ 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.1");
+
+ clk = clk_register_mux(NULL, "ssp2_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP2, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.2", NULL);
+
+ clk = mmp_clk_register_apbc("ssp2", "ssp1_mux", apbc_base + APBC_SSP2,
+ 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.2");
+
+ clk = clk_register_mux(NULL, "ssp3_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP3, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.3", NULL);
+
+ clk = mmp_clk_register_apbc("ssp3", "ssp1_mux", apbc_base + APBC_SSP3,
+ 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.3");
+
+ clk = clk_register_mux(NULL, "ssp4_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP4, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.4", NULL);
+
+ clk = mmp_clk_register_apbc("ssp4", "ssp1_mux", apbc_base + APBC_SSP4,
+ 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.4");
+
+ clk = mmp_clk_register_apmu("dfc", "pll1_4", apmu_base + APMU_DFC,
+ 0x19b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa3xx-nand.0");
+
+ clk = clk_register_mux(NULL, "sdh0_mux", sdh_parent,
+ ARRAY_SIZE(sdh_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_SDH0, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "sdh0_mux", NULL);
+
+ clk = mmp_clk_register_apmu("sdh0", "sdh_mux", apmu_base + APMU_SDH0,
+ 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxa.0");
+
+ clk = clk_register_mux(NULL, "sdh1_mux", sdh_parent,
+ ARRAY_SIZE(sdh_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_SDH1, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "sdh1_mux", NULL);
+
+ clk = mmp_clk_register_apmu("sdh1", "sdh1_mux", apmu_base + APMU_SDH1,
+ 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxa.1");
+
+ clk = mmp_clk_register_apmu("usb", "usb_pll", apmu_base + APMU_USB,
+ 0x9, &clk_lock);
+ clk_register_clkdev(clk, "usb_clk", NULL);
+
+ clk = mmp_clk_register_apmu("sph", "usb_pll", apmu_base + APMU_USB,
+ 0x12, &clk_lock);
+ clk_register_clkdev(clk, "sph_clk", NULL);
+
+ clk = clk_register_mux(NULL, "disp0_mux", disp_parent,
+ ARRAY_SIZE(disp_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_DISP0, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "disp_mux.0", NULL);
+
+ clk = mmp_clk_register_apmu("disp0", "disp0_mux",
+ apmu_base + APMU_DISP0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, "fnclk", "mmp-disp.0");
+
+ clk = mmp_clk_register_apmu("disp0_hclk", "disp0_mux",
+ apmu_base + APMU_DISP0, 0x24, &clk_lock);
+ clk_register_clkdev(clk, "hclk", "mmp-disp.0");
+
+ clk = clk_register_mux(NULL, "ccic0_mux", ccic_parent,
+ ARRAY_SIZE(ccic_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_CCIC0, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "ccic_mux.0", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0", "ccic0_mux",
+ apmu_base + APMU_CCIC0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, "fnclk", "mmp-ccic.0");
+
+ clk = clk_register_mux(NULL, "ccic0_phy_mux", ccic_phy_parent,
+ ARRAY_SIZE(ccic_phy_parent),
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC0,
+ 7, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "ccic_phy_mux.0", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0_phy", "ccic0_phy_mux",
+ apmu_base + APMU_CCIC0, 0x24, &clk_lock);
+ clk_register_clkdev(clk, "phyclk", "mmp-ccic.0");
+
+ clk = clk_register_divider(NULL, "ccic0_sphy_div", "ccic0_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC0,
+ 10, 5, 0, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk_div", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0_sphy", "ccic0_sphy_div",
+ apmu_base + APMU_CCIC0, 0x300, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk", "mmp-ccic.0");
+}
diff --git a/drivers/clk/mmp/clk-pxa910.c b/drivers/clk/mmp/clk-pxa910.c
new file mode 100644
index 0000000..7048c31
--- /dev/null
+++ b/drivers/clk/mmp/clk-pxa910.c
@@ -0,0 +1,320 @@
+/*
+ * pxa910 clock framework source file
+ *
+ * Copyright (C) 2012 Marvell
+ * Chao Xie <xiechao.mail@gmail.com>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/spinlock.h>
+#include <linux/io.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+
+#include <mach/addr-map.h>
+
+#include "clk.h"
+
+#define APBC_RTC 0x28
+#define APBC_TWSI0 0x2c
+#define APBC_KPC 0x18
+#define APBC_UART0 0x0
+#define APBC_UART1 0x4
+#define APBC_GPIO 0x8
+#define APBC_PWM0 0xc
+#define APBC_PWM1 0x10
+#define APBC_PWM2 0x14
+#define APBC_PWM3 0x18
+#define APBC_SSP0 0x1c
+#define APBC_SSP1 0x20
+#define APBC_SSP2 0x4c
+#define APBCP_TWSI1 0x28
+#define APBCP_UART2 0x1c
+#define APMU_SDH0 0x54
+#define APMU_SDH1 0x58
+#define APMU_USB 0x5c
+#define APMU_DISP0 0x4c
+#define APMU_CCIC0 0x50
+#define APMU_DFC 0x60
+#define MPMU_UART_PLL 0x14
+
+static DEFINE_SPINLOCK(clk_lock);
+
+static struct clk_factor_masks uart_factor_masks = {
+ .factor = 2,
+ .num_mask = 0x1fff,
+ .den_mask = 0x1fff,
+ .num_shift = 16,
+ .den_shift = 0,
+};
+
+static struct clk_factor_tbl uart_factor_tbl[] = {
+ {.num = 8125, .den = 1536}, /*14.745MHZ */
+};
+
+static const char *uart_parent[] = {"pll1_3_16", "uart_pll"};
+static const char *ssp_parent[] = {"pll1_96", "pll1_48", "pll1_24", "pll1_12"};
+static const char *sdh_parent[] = {"pll1_12", "pll1_13"};
+static const char *disp_parent[] = {"pll1_2", "pll1_12"};
+static const char *ccic_parent[] = {"pll1_2", "pll1_12"};
+static const char *ccic_phy_parent[] = {"pll1_6", "pll1_12"};
+
+void __init pxa910_clk_init(void)
+{
+ struct clk *clk;
+ struct clk *uart_pll;
+ void __iomem *mpmu_base;
+ void __iomem *apmu_base;
+ void __iomem *apbcp_base;
+ void __iomem *apbc_base;
+
+ mpmu_base = ioremap(APB_PHYS_BASE + 0x50000, SZ_4K);
+ if (mpmu_base == NULL) {
+ pr_err("error to ioremap MPMU base\n");
+ return;
+ }
+
+ apmu_base = ioremap(AXI_PHYS_BASE + 0x82800, SZ_4K);
+ if (apmu_base == NULL) {
+ pr_err("error to ioremap APMU base\n");
+ return;
+ }
+
+ apbcp_base = ioremap(APB_PHYS_BASE + 0x3b000, SZ_4K);
+ if (apbcp_base == NULL) {
+ pr_err("error to ioremap APBC extension base\n");
+ return;
+ }
+
+ apbc_base = ioremap(APB_PHYS_BASE + 0x15000, SZ_4K);
+ if (apbc_base == NULL) {
+ pr_err("error to ioremap APBC base\n");
+ return;
+ }
+
+ clk = clk_register_fixed_rate(NULL, "clk32", NULL, CLK_IS_ROOT, 3200);
+ clk_register_clkdev(clk, "clk32", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "vctcxo", NULL, CLK_IS_ROOT,
+ 26000000);
+ clk_register_clkdev(clk, "vctcxo", NULL);
+
+ clk = clk_register_fixed_rate(NULL, "pll1", NULL, CLK_IS_ROOT,
+ 624000000);
+ clk_register_clkdev(clk, "pll1", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_2", "pll1",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_2", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_4", "pll1_2",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_4", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_8", "pll1_4",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_8", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_16", "pll1_8",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_16", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_6", "pll1_2",
+ CLK_SET_RATE_PARENT, 1, 3);
+ clk_register_clkdev(clk, "pll1_6", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_12", "pll1_6",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_12", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_24", "pll1_12",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_24", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_48", "pll1_24",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_48", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_96", "pll1_48",
+ CLK_SET_RATE_PARENT, 1, 2);
+ clk_register_clkdev(clk, "pll1_96", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_13", "pll1",
+ CLK_SET_RATE_PARENT, 1, 13);
+ clk_register_clkdev(clk, "pll1_13", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_13_1_5", "pll1",
+ CLK_SET_RATE_PARENT, 2, 3);
+ clk_register_clkdev(clk, "pll1_13_1_5", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_2_1_5", "pll1",
+ CLK_SET_RATE_PARENT, 2, 3);
+ clk_register_clkdev(clk, "pll1_2_1_5", NULL);
+
+ clk = clk_register_fixed_factor(NULL, "pll1_3_16", "pll1",
+ CLK_SET_RATE_PARENT, 3, 16);
+ clk_register_clkdev(clk, "pll1_3_16", NULL);
+
+ uart_pll = mmp_clk_register_factor("uart_pll", "pll1_4", 0,
+ mpmu_base + MPMU_UART_PLL,
+ &uart_factor_masks, uart_factor_tbl,
+ ARRAY_SIZE(uart_factor_tbl));
+ clk_set_rate(uart_pll, 14745600);
+ clk_register_clkdev(uart_pll, "uart_pll", NULL);
+
+ clk = mmp_clk_register_apbc("twsi0", "pll1_13_1_5",
+ apbc_base + APBC_TWSI0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.0");
+
+ clk = mmp_clk_register_apbc("twsi1", "pll1_13_1_5",
+ apbcp_base + APBCP_TWSI1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-i2c.1");
+
+ clk = mmp_clk_register_apbc("gpio", "vctcxo",
+ apbc_base + APBC_GPIO, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa-gpio");
+
+ clk = mmp_clk_register_apbc("kpc", "clk32",
+ apbc_base + APBC_KPC, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa27x-keypad");
+
+ clk = mmp_clk_register_apbc("rtc", "clk32",
+ apbc_base + APBC_RTC, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sa1100-rtc");
+
+ clk = mmp_clk_register_apbc("pwm0", "pll1_48",
+ apbc_base + APBC_PWM0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa910-pwm.0");
+
+ clk = mmp_clk_register_apbc("pwm1", "pll1_48",
+ apbc_base + APBC_PWM1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa910-pwm.1");
+
+ clk = mmp_clk_register_apbc("pwm2", "pll1_48",
+ apbc_base + APBC_PWM2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa910-pwm.2");
+
+ clk = mmp_clk_register_apbc("pwm3", "pll1_48",
+ apbc_base + APBC_PWM3, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa910-pwm.3");
+
+ clk = clk_register_mux(NULL, "uart0_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART0, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, uart_pll);
+ clk_register_clkdev(clk, "uart_mux.0", NULL);
+
+ clk = mmp_clk_register_apbc("uart0", "uart0_mux",
+ apbc_base + APBC_UART0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.0");
+
+ clk = clk_register_mux(NULL, "uart1_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_UART1, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, uart_pll);
+ clk_register_clkdev(clk, "uart_mux.1", NULL);
+
+ clk = mmp_clk_register_apbc("uart1", "uart1_mux",
+ apbc_base + APBC_UART1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.1");
+
+ clk = clk_register_mux(NULL, "uart2_mux", uart_parent,
+ ARRAY_SIZE(uart_parent), CLK_SET_RATE_PARENT,
+ apbcp_base + APBCP_UART2, 4, 3, 0, &clk_lock);
+ clk_set_parent(clk, uart_pll);
+ clk_register_clkdev(clk, "uart_mux.2", NULL);
+
+ clk = mmp_clk_register_apbc("uart2", "uart2_mux",
+ apbcp_base + APBCP_UART2, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa2xx-uart.2");
+
+ clk = clk_register_mux(NULL, "ssp0_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP0, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "uart_mux.0", NULL);
+
+ clk = mmp_clk_register_apbc("ssp0", "ssp0_mux",
+ apbc_base + APBC_SSP0, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.0");
+
+ clk = clk_register_mux(NULL, "ssp1_mux", ssp_parent,
+ ARRAY_SIZE(ssp_parent), CLK_SET_RATE_PARENT,
+ apbc_base + APBC_SSP1, 4, 3, 0, &clk_lock);
+ clk_register_clkdev(clk, "ssp_mux.1", NULL);
+
+ clk = mmp_clk_register_apbc("ssp1", "ssp1_mux",
+ apbc_base + APBC_SSP1, 10, 0, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-ssp.1");
+
+ clk = mmp_clk_register_apmu("dfc", "pll1_4",
+ apmu_base + APMU_DFC, 0x19b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "pxa3xx-nand.0");
+
+ clk = clk_register_mux(NULL, "sdh0_mux", sdh_parent,
+ ARRAY_SIZE(sdh_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_SDH0, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "sdh0_mux", NULL);
+
+ clk = mmp_clk_register_apmu("sdh0", "sdh_mux",
+ apmu_base + APMU_SDH0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxa.0");
+
+ clk = clk_register_mux(NULL, "sdh1_mux", sdh_parent,
+ ARRAY_SIZE(sdh_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_SDH1, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "sdh1_mux", NULL);
+
+ clk = mmp_clk_register_apmu("sdh1", "sdh1_mux",
+ apmu_base + APMU_SDH1, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "sdhci-pxa.1");
+
+ clk = mmp_clk_register_apmu("usb", "usb_pll",
+ apmu_base + APMU_USB, 0x9, &clk_lock);
+ clk_register_clkdev(clk, "usb_clk", NULL);
+
+ clk = mmp_clk_register_apmu("sph", "usb_pll",
+ apmu_base + APMU_USB, 0x12, &clk_lock);
+ clk_register_clkdev(clk, "sph_clk", NULL);
+
+ clk = clk_register_mux(NULL, "disp0_mux", disp_parent,
+ ARRAY_SIZE(disp_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_DISP0, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "disp_mux.0", NULL);
+
+ clk = mmp_clk_register_apmu("disp0", "disp0_mux",
+ apmu_base + APMU_DISP0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, NULL, "mmp-disp.0");
+
+ clk = clk_register_mux(NULL, "ccic0_mux", ccic_parent,
+ ARRAY_SIZE(ccic_parent), CLK_SET_RATE_PARENT,
+ apmu_base + APMU_CCIC0, 6, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "ccic_mux.0", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0", "ccic0_mux",
+ apmu_base + APMU_CCIC0, 0x1b, &clk_lock);
+ clk_register_clkdev(clk, "fnclk", "mmp-ccic.0");
+
+ clk = clk_register_mux(NULL, "ccic0_phy_mux", ccic_phy_parent,
+ ARRAY_SIZE(ccic_phy_parent),
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC0,
+ 7, 1, 0, &clk_lock);
+ clk_register_clkdev(clk, "ccic_phy_mux.0", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0_phy", "ccic0_phy_mux",
+ apmu_base + APMU_CCIC0, 0x24, &clk_lock);
+ clk_register_clkdev(clk, "phyclk", "mmp-ccic.0");
+
+ clk = clk_register_divider(NULL, "ccic0_sphy_div", "ccic0_mux",
+ CLK_SET_RATE_PARENT, apmu_base + APMU_CCIC0,
+ 10, 5, 0, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk_div", NULL);
+
+ clk = mmp_clk_register_apmu("ccic0_sphy", "ccic0_sphy_div",
+ apmu_base + APMU_CCIC0, 0x300, &clk_lock);
+ clk_register_clkdev(clk, "sphyclk", "mmp-ccic.0");
+}
diff --git a/drivers/clk/mmp/clk.h b/drivers/clk/mmp/clk.h
new file mode 100644
index 0000000..ab86dd4
--- /dev/null
+++ b/drivers/clk/mmp/clk.h
@@ -0,0 +1,35 @@
+#ifndef __MACH_MMP_CLK_H
+#define __MACH_MMP_CLK_H
+
+#include <linux/clk-provider.h>
+#include <linux/clkdev.h>
+
+#define APBC_NO_BUS_CTRL BIT(0)
+#define APBC_POWER_CTRL BIT(1)
+
+struct clk_factor_masks {
+ unsigned int factor;
+ unsigned int num_mask;
+ unsigned int den_mask;
+ unsigned int num_shift;
+ unsigned int den_shift;
+};
+
+struct clk_factor_tbl {
+ unsigned int num;
+ unsigned int den;
+};
+
+extern struct clk *mmp_clk_register_pll2(const char *name,
+ const char *parent_name, unsigned long flags);
+extern struct clk *mmp_clk_register_apbc(const char *name,
+ const char *parent_name, void __iomem *base,
+ unsigned int delay, unsigned int apbc_flags, spinlock_t *lock);
+extern struct clk *mmp_clk_register_apmu(const char *name,
+ const char *parent_name, void __iomem *base, u32 enable_mask,
+ spinlock_t *lock);
+extern struct clk *mmp_clk_register_factor(const char *name,
+ const char *parent_name, unsigned long flags,
+ void __iomem *base, struct clk_factor_masks *masks,
+ struct clk_factor_tbl *ftbl, unsigned int ftbl_cnt);
+#endif
diff --git a/drivers/clk/ux500/Makefile b/drivers/clk/ux500/Makefile
new file mode 100644
index 0000000..858fbfe
--- /dev/null
+++ b/drivers/clk/ux500/Makefile
@@ -0,0 +1,12 @@
+#
+# Makefile for ux500 clocks
+#
+
+# Clock types
+obj-y += clk-prcc.o
+obj-y += clk-prcmu.o
+
+# Clock definitions
+obj-y += u8500_clk.o
+obj-y += u9540_clk.o
+obj-y += u8540_clk.o
diff --git a/drivers/clk/ux500/clk-prcc.c b/drivers/clk/ux500/clk-prcc.c
new file mode 100644
index 0000000..7eee7f7
--- /dev/null
+++ b/drivers/clk/ux500/clk-prcc.c
@@ -0,0 +1,164 @@
+/*
+ * PRCC clock implementation for ux500 platform.
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/clk-private.h>
+#include <linux/slab.h>
+#include <linux/io.h>
+#include <linux/err.h>
+#include <linux/types.h>
+#include <mach/hardware.h>
+
+#include "clk.h"
+
+#define PRCC_PCKEN 0x000
+#define PRCC_PCKDIS 0x004
+#define PRCC_KCKEN 0x008
+#define PRCC_KCKDIS 0x00C
+#define PRCC_PCKSR 0x010
+#define PRCC_KCKSR 0x014
+
+#define to_clk_prcc(_hw) container_of(_hw, struct clk_prcc, hw)
+
+struct clk_prcc {
+ struct clk_hw hw;
+ void __iomem *base;
+ u32 cg_sel;
+ int is_enabled;
+};
+
+/* PRCC clock operations. */
+
+static int clk_prcc_pclk_enable(struct clk_hw *hw)
+{
+ struct clk_prcc *clk = to_clk_prcc(hw);
+
+ writel(clk->cg_sel, (clk->base + PRCC_PCKEN));
+ while (!(readl(clk->base + PRCC_PCKSR) & clk->cg_sel))
+ cpu_relax();
+
+ clk->is_enabled = 1;
+ return 0;
+}
+
+static void clk_prcc_pclk_disable(struct clk_hw *hw)
+{
+ struct clk_prcc *clk = to_clk_prcc(hw);
+
+ writel(clk->cg_sel, (clk->base + PRCC_PCKDIS));
+ clk->is_enabled = 0;
+}
+
+static int clk_prcc_kclk_enable(struct clk_hw *hw)
+{
+ struct clk_prcc *clk = to_clk_prcc(hw);
+
+ writel(clk->cg_sel, (clk->base + PRCC_KCKEN));
+ while (!(readl(clk->base + PRCC_KCKSR) & clk->cg_sel))
+ cpu_relax();
+
+ clk->is_enabled = 1;
+ return 0;
+}
+
+static void clk_prcc_kclk_disable(struct clk_hw *hw)
+{
+ struct clk_prcc *clk = to_clk_prcc(hw);
+
+ writel(clk->cg_sel, (clk->base + PRCC_KCKDIS));
+ clk->is_enabled = 0;
+}
+
+static int clk_prcc_is_enabled(struct clk_hw *hw)
+{
+ struct clk_prcc *clk = to_clk_prcc(hw);
+ return clk->is_enabled;
+}
+
+static struct clk_ops clk_prcc_pclk_ops = {
+ .enable = clk_prcc_pclk_enable,
+ .disable = clk_prcc_pclk_disable,
+ .is_enabled = clk_prcc_is_enabled,
+};
+
+static struct clk_ops clk_prcc_kclk_ops = {
+ .enable = clk_prcc_kclk_enable,
+ .disable = clk_prcc_kclk_disable,
+ .is_enabled = clk_prcc_is_enabled,
+};
+
+static struct clk *clk_reg_prcc(const char *name,
+ const char *parent_name,
+ resource_size_t phy_base,
+ u32 cg_sel,
+ unsigned long flags,
+ struct clk_ops *clk_prcc_ops)
+{
+ struct clk_prcc *clk;
+ struct clk_init_data clk_prcc_init;
+ struct clk *clk_reg;
+
+ if (!name) {
+ pr_err("clk_prcc: %s invalid arguments passed\n", __func__);
+ return ERR_PTR(-EINVAL);
+ }
+
+ clk = kzalloc(sizeof(struct clk_prcc), GFP_KERNEL);
+ if (!clk) {
+ pr_err("clk_prcc: %s could not allocate clk\n", __func__);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ clk->base = ioremap(phy_base, SZ_4K);
+ if (!clk->base)
+ goto free_clk;
+
+ clk->cg_sel = cg_sel;
+ clk->is_enabled = 1;
+
+ clk_prcc_init.name = name;
+ clk_prcc_init.ops = clk_prcc_ops;
+ clk_prcc_init.flags = flags;
+ clk_prcc_init.parent_names = (parent_name ? &parent_name : NULL);
+ clk_prcc_init.num_parents = (parent_name ? 1 : 0);
+ clk->hw.init = &clk_prcc_init;
+
+ clk_reg = clk_register(NULL, &clk->hw);
+ if (IS_ERR_OR_NULL(clk_reg))
+ goto unmap_clk;
+
+ return clk_reg;
+
+unmap_clk:
+ iounmap(clk->base);
+free_clk:
+ kfree(clk);
+ pr_err("clk_prcc: %s failed to register clk\n", __func__);
+ return ERR_PTR(-ENOMEM);
+}
+
+struct clk *clk_reg_prcc_pclk(const char *name,
+ const char *parent_name,
+ resource_size_t phy_base,
+ u32 cg_sel,
+ unsigned long flags)
+{
+ return clk_reg_prcc(name, parent_name, phy_base, cg_sel, flags,
+ &clk_prcc_pclk_ops);
+}
+
+struct clk *clk_reg_prcc_kclk(const char *name,
+ const char *parent_name,
+ resource_size_t phy_base,
+ u32 cg_sel,
+ unsigned long flags)
+{
+ return clk_reg_prcc(name, parent_name, phy_base, cg_sel, flags,
+ &clk_prcc_kclk_ops);
+}
diff --git a/drivers/clk/ux500/clk-prcmu.c b/drivers/clk/ux500/clk-prcmu.c
new file mode 100644
index 0000000..930cdfe
--- /dev/null
+++ b/drivers/clk/ux500/clk-prcmu.c
@@ -0,0 +1,252 @@
+/*
+ * PRCMU clock implementation for ux500 platform.
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/clk-private.h>
+#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/slab.h>
+#include <linux/io.h>
+#include <linux/err.h>
+#include "clk.h"
+
+#define to_clk_prcmu(_hw) container_of(_hw, struct clk_prcmu, hw)
+
+struct clk_prcmu {
+ struct clk_hw hw;
+ u8 cg_sel;
+ int is_enabled;
+};
+
+/* PRCMU clock operations. */
+
+static int clk_prcmu_prepare(struct clk_hw *hw)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ return prcmu_request_clock(clk->cg_sel, true);
+}
+
+static void clk_prcmu_unprepare(struct clk_hw *hw)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ if (prcmu_request_clock(clk->cg_sel, false))
+ pr_err("clk_prcmu: %s failed to disable %s.\n", __func__,
+ hw->init->name);
+}
+
+static int clk_prcmu_enable(struct clk_hw *hw)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ clk->is_enabled = 1;
+ return 0;
+}
+
+static void clk_prcmu_disable(struct clk_hw *hw)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ clk->is_enabled = 0;
+}
+
+static int clk_prcmu_is_enabled(struct clk_hw *hw)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ return clk->is_enabled;
+}
+
+static unsigned long clk_prcmu_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ return prcmu_clock_rate(clk->cg_sel);
+}
+
+static long clk_prcmu_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ return prcmu_round_clock_rate(clk->cg_sel, rate);
+}
+
+static int clk_prcmu_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+ return prcmu_set_clock_rate(clk->cg_sel, rate);
+}
+
+static int request_ape_opp100(bool enable)
+{
+ static int reqs;
+ int err = 0;
+
+ if (enable) {
+ if (!reqs)
+ err = prcmu_qos_add_requirement(PRCMU_QOS_APE_OPP,
+ "clock", 100);
+ if (!err)
+ reqs++;
+ } else {
+ reqs--;
+ if (!reqs)
+ prcmu_qos_remove_requirement(PRCMU_QOS_APE_OPP,
+ "clock");
+ }
+ return err;
+}
+
+static int clk_prcmu_opp_prepare(struct clk_hw *hw)
+{
+ int err;
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+
+ err = request_ape_opp100(true);
+ if (err) {
+ pr_err("clk_prcmu: %s failed to request APE OPP100 for %s.\n",
+ __func__, hw->init->name);
+ return err;
+ }
+
+ err = prcmu_request_clock(clk->cg_sel, true);
+ if (err)
+ request_ape_opp100(false);
+
+ return err;
+}
+
+static void clk_prcmu_opp_unprepare(struct clk_hw *hw)
+{
+ struct clk_prcmu *clk = to_clk_prcmu(hw);
+
+ if (prcmu_request_clock(clk->cg_sel, false))
+ goto out_error;
+ if (request_ape_opp100(false))
+ goto out_error;
+ return;
+
+out_error:
+ pr_err("clk_prcmu: %s failed to disable %s.\n", __func__,
+ hw->init->name);
+}
+
+static struct clk_ops clk_prcmu_scalable_ops = {
+ .prepare = clk_prcmu_prepare,
+ .unprepare = clk_prcmu_unprepare,
+ .enable = clk_prcmu_enable,
+ .disable = clk_prcmu_disable,
+ .is_enabled = clk_prcmu_is_enabled,
+ .recalc_rate = clk_prcmu_recalc_rate,
+ .round_rate = clk_prcmu_round_rate,
+ .set_rate = clk_prcmu_set_rate,
+};
+
+static struct clk_ops clk_prcmu_gate_ops = {
+ .prepare = clk_prcmu_prepare,
+ .unprepare = clk_prcmu_unprepare,
+ .enable = clk_prcmu_enable,
+ .disable = clk_prcmu_disable,
+ .is_enabled = clk_prcmu_is_enabled,
+ .recalc_rate = clk_prcmu_recalc_rate,
+};
+
+static struct clk_ops clk_prcmu_rate_ops = {
+ .is_enabled = clk_prcmu_is_enabled,
+ .recalc_rate = clk_prcmu_recalc_rate,
+};
+
+static struct clk_ops clk_prcmu_opp_gate_ops = {
+ .prepare = clk_prcmu_opp_prepare,
+ .unprepare = clk_prcmu_opp_unprepare,
+ .enable = clk_prcmu_enable,
+ .disable = clk_prcmu_disable,
+ .is_enabled = clk_prcmu_is_enabled,
+ .recalc_rate = clk_prcmu_recalc_rate,
+};
+
+static struct clk *clk_reg_prcmu(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long rate,
+ unsigned long flags,
+ struct clk_ops *clk_prcmu_ops)
+{
+ struct clk_prcmu *clk;
+ struct clk_init_data clk_prcmu_init;
+ struct clk *clk_reg;
+
+ if (!name) {
+ pr_err("clk_prcmu: %s invalid arguments passed\n", __func__);
+ return ERR_PTR(-EINVAL);
+ }
+
+ clk = kzalloc(sizeof(struct clk_prcmu), GFP_KERNEL);
+ if (!clk) {
+ pr_err("clk_prcmu: %s could not allocate clk\n", __func__);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ clk->cg_sel = cg_sel;
+ clk->is_enabled = 1;
+ /* "rate" can be used for changing the initial frequency */
+ if (rate)
+ prcmu_set_clock_rate(cg_sel, rate);
+
+ clk_prcmu_init.name = name;
+ clk_prcmu_init.ops = clk_prcmu_ops;
+ clk_prcmu_init.flags = flags;
+ clk_prcmu_init.parent_names = (parent_name ? &parent_name : NULL);
+ clk_prcmu_init.num_parents = (parent_name ? 1 : 0);
+ clk->hw.init = &clk_prcmu_init;
+
+ clk_reg = clk_register(NULL, &clk->hw);
+ if (IS_ERR_OR_NULL(clk_reg))
+ goto free_clk;
+
+ return clk_reg;
+
+free_clk:
+ kfree(clk);
+ pr_err("clk_prcmu: %s failed to register clk\n", __func__);
+ return ERR_PTR(-ENOMEM);
+}
+
+struct clk *clk_reg_prcmu_scalable(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long rate,
+ unsigned long flags)
+{
+ return clk_reg_prcmu(name, parent_name, cg_sel, rate, flags,
+ &clk_prcmu_scalable_ops);
+}
+
+struct clk *clk_reg_prcmu_gate(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long flags)
+{
+ return clk_reg_prcmu(name, parent_name, cg_sel, 0, flags,
+ &clk_prcmu_gate_ops);
+}
+
+struct clk *clk_reg_prcmu_rate(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long flags)
+{
+ return clk_reg_prcmu(name, parent_name, cg_sel, 0, flags,
+ &clk_prcmu_rate_ops);
+}
+
+struct clk *clk_reg_prcmu_opp_gate(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long flags)
+{
+ return clk_reg_prcmu(name, parent_name, cg_sel, 0, flags,
+ &clk_prcmu_opp_gate_ops);
+}
diff --git a/drivers/clk/ux500/clk.h b/drivers/clk/ux500/clk.h
new file mode 100644
index 0000000..836d7d1
--- /dev/null
+++ b/drivers/clk/ux500/clk.h
@@ -0,0 +1,48 @@
+/*
+ * Clocks for ux500 platforms
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#ifndef __UX500_CLK_H
+#define __UX500_CLK_H
+
+#include <linux/clk.h>
+
+struct clk *clk_reg_prcc_pclk(const char *name,
+ const char *parent_name,
+ unsigned int phy_base,
+ u32 cg_sel,
+ unsigned long flags);
+
+struct clk *clk_reg_prcc_kclk(const char *name,
+ const char *parent_name,
+ unsigned int phy_base,
+ u32 cg_sel,
+ unsigned long flags);
+
+struct clk *clk_reg_prcmu_scalable(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long rate,
+ unsigned long flags);
+
+struct clk *clk_reg_prcmu_gate(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long flags);
+
+struct clk *clk_reg_prcmu_rate(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long flags);
+
+struct clk *clk_reg_prcmu_opp_gate(const char *name,
+ const char *parent_name,
+ u8 cg_sel,
+ unsigned long flags);
+
+#endif /* __UX500_CLK_H */
diff --git a/drivers/clk/ux500/u8500_clk.c b/drivers/clk/ux500/u8500_clk.c
new file mode 100644
index 0000000..ca4a25e
--- /dev/null
+++ b/drivers/clk/ux500/u8500_clk.c
@@ -0,0 +1,477 @@
+/*
+ * Clock definitions for u8500 platform.
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/platform_data/clk-ux500.h>
+
+#include "clk.h"
+
+void u8500_clk_init(void)
+{
+ struct prcmu_fw_version *fw_version;
+ const char *sgaclk_parent = NULL;
+ struct clk *clk;
+
+ /* Clock sources */
+ clk = clk_reg_prcmu_gate("soc0_pll", NULL, PRCMU_PLLSOC0,
+ CLK_IS_ROOT|CLK_IGNORE_UNUSED);
+ clk_register_clkdev(clk, "soc0_pll", NULL);
+
+ clk = clk_reg_prcmu_gate("soc1_pll", NULL, PRCMU_PLLSOC1,
+ CLK_IS_ROOT|CLK_IGNORE_UNUSED);
+ clk_register_clkdev(clk, "soc1_pll", NULL);
+
+ clk = clk_reg_prcmu_gate("ddr_pll", NULL, PRCMU_PLLDDR,
+ CLK_IS_ROOT|CLK_IGNORE_UNUSED);
+ clk_register_clkdev(clk, "ddr_pll", NULL);
+
+ /* FIXME: Add sys, ulp and int clocks here. */
+
+ clk = clk_register_fixed_rate(NULL, "rtc32k", "NULL",
+ CLK_IS_ROOT|CLK_IGNORE_UNUSED,
+ 32768);
+ clk_register_clkdev(clk, "clk32k", NULL);
+ clk_register_clkdev(clk, NULL, "rtc-pl031");
+
+ /* PRCMU clocks */
+ fw_version = prcmu_get_fw_version();
+ if (fw_version != NULL) {
+ switch (fw_version->project) {
+ case PRCMU_FW_PROJECT_U8500_C2:
+ case PRCMU_FW_PROJECT_U8520:
+ case PRCMU_FW_PROJECT_U8420:
+ sgaclk_parent = "soc0_pll";
+ break;
+ default:
+ break;
+ }
+ }
+
+ if (sgaclk_parent)
+ clk = clk_reg_prcmu_gate("sgclk", sgaclk_parent,
+ PRCMU_SGACLK, 0);
+ else
+ clk = clk_reg_prcmu_gate("sgclk", NULL,
+ PRCMU_SGACLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "mali");
+
+ clk = clk_reg_prcmu_gate("uartclk", NULL, PRCMU_UARTCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "UART");
+
+ clk = clk_reg_prcmu_gate("msp02clk", NULL, PRCMU_MSP02CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "MSP02");
+
+ clk = clk_reg_prcmu_gate("msp1clk", NULL, PRCMU_MSP1CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "MSP1");
+
+ clk = clk_reg_prcmu_gate("i2cclk", NULL, PRCMU_I2CCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "I2C");
+
+ clk = clk_reg_prcmu_gate("slimclk", NULL, PRCMU_SLIMCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "slim");
+
+ clk = clk_reg_prcmu_gate("per1clk", NULL, PRCMU_PER1CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "PERIPH1");
+
+ clk = clk_reg_prcmu_gate("per2clk", NULL, PRCMU_PER2CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "PERIPH2");
+
+ clk = clk_reg_prcmu_gate("per3clk", NULL, PRCMU_PER3CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "PERIPH3");
+
+ clk = clk_reg_prcmu_gate("per5clk", NULL, PRCMU_PER5CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "PERIPH5");
+
+ clk = clk_reg_prcmu_gate("per6clk", NULL, PRCMU_PER6CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "PERIPH6");
+
+ clk = clk_reg_prcmu_gate("per7clk", NULL, PRCMU_PER7CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "PERIPH7");
+
+ clk = clk_reg_prcmu_scalable("lcdclk", NULL, PRCMU_LCDCLK, 0,
+ CLK_IS_ROOT|CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "lcd");
+ clk_register_clkdev(clk, "lcd", "mcde");
+
+ clk = clk_reg_prcmu_opp_gate("bmlclk", NULL, PRCMU_BMLCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "bml");
+
+ clk = clk_reg_prcmu_scalable("hsitxclk", NULL, PRCMU_HSITXCLK, 0,
+ CLK_IS_ROOT|CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcmu_scalable("hsirxclk", NULL, PRCMU_HSIRXCLK, 0,
+ CLK_IS_ROOT|CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcmu_scalable("hdmiclk", NULL, PRCMU_HDMICLK, 0,
+ CLK_IS_ROOT|CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "hdmi");
+ clk_register_clkdev(clk, "hdmi", "mcde");
+
+ clk = clk_reg_prcmu_gate("apeatclk", NULL, PRCMU_APEATCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "apeat");
+
+ clk = clk_reg_prcmu_gate("apetraceclk", NULL, PRCMU_APETRACECLK,
+ CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "apetrace");
+
+ clk = clk_reg_prcmu_gate("mcdeclk", NULL, PRCMU_MCDECLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "mcde");
+ clk_register_clkdev(clk, "mcde", "mcde");
+ clk_register_clkdev(clk, "dsisys", "dsilink.0");
+ clk_register_clkdev(clk, "dsisys", "dsilink.1");
+ clk_register_clkdev(clk, "dsisys", "dsilink.2");
+
+ clk = clk_reg_prcmu_opp_gate("ipi2cclk", NULL, PRCMU_IPI2CCLK,
+ CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "ipi2");
+
+ clk = clk_reg_prcmu_gate("dsialtclk", NULL, PRCMU_DSIALTCLK,
+ CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "dsialt");
+
+ clk = clk_reg_prcmu_gate("dmaclk", NULL, PRCMU_DMACLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "dma40.0");
+
+ clk = clk_reg_prcmu_gate("b2r2clk", NULL, PRCMU_B2R2CLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "b2r2");
+ clk_register_clkdev(clk, NULL, "b2r2_core");
+ clk_register_clkdev(clk, NULL, "U8500-B2R2.0");
+
+ clk = clk_reg_prcmu_scalable("tvclk", NULL, PRCMU_TVCLK, 0,
+ CLK_IS_ROOT|CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "tv");
+ clk_register_clkdev(clk, "tv", "mcde");
+
+ clk = clk_reg_prcmu_gate("sspclk", NULL, PRCMU_SSPCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "SSP");
+
+ clk = clk_reg_prcmu_gate("rngclk", NULL, PRCMU_RNGCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "rngclk");
+
+ clk = clk_reg_prcmu_gate("uiccclk", NULL, PRCMU_UICCCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "uicc");
+
+ /*
+ * FIXME: The MTU clocks might need some kind of "parent muxed join"
+ * and these have no K-clocks. For now, we ignore the missing
+ * connection to the corresponding P-clocks, p6_mtu0_clk and
+ * p6_mtu1_clk. Instead timclk is used which is the valid parent.
+ */
+ clk = clk_reg_prcmu_gate("timclk", NULL, PRCMU_TIMCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "mtu0");
+ clk_register_clkdev(clk, NULL, "mtu1");
+
+ clk = clk_reg_prcmu_gate("sdmmcclk", NULL, PRCMU_SDMMCCLK, CLK_IS_ROOT);
+ clk_register_clkdev(clk, NULL, "sdmmc");
+
+
+ clk = clk_reg_prcmu_scalable("dsi_pll", "hdmiclk",
+ PRCMU_PLLDSI, 0, CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, "dsihs2", "mcde");
+ clk_register_clkdev(clk, "dsihs2", "dsilink.2");
+
+
+ clk = clk_reg_prcmu_scalable("dsi0clk", "dsi_pll",
+ PRCMU_DSI0CLK, 0, CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, "dsihs0", "mcde");
+ clk_register_clkdev(clk, "dsihs0", "dsilink.0");
+
+ clk = clk_reg_prcmu_scalable("dsi1clk", "dsi_pll",
+ PRCMU_DSI1CLK, 0, CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, "dsihs1", "mcde");
+ clk_register_clkdev(clk, "dsihs1", "dsilink.1");
+
+ clk = clk_reg_prcmu_scalable("dsi0escclk", "tvclk",
+ PRCMU_DSI0ESCCLK, 0, CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, "dsilp0", "dsilink.0");
+ clk_register_clkdev(clk, "dsilp0", "mcde");
+
+ clk = clk_reg_prcmu_scalable("dsi1escclk", "tvclk",
+ PRCMU_DSI1ESCCLK, 0, CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, "dsilp1", "dsilink.1");
+ clk_register_clkdev(clk, "dsilp1", "mcde");
+
+ clk = clk_reg_prcmu_scalable("dsi2escclk", "tvclk",
+ PRCMU_DSI2ESCCLK, 0, CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, "dsilp2", "dsilink.2");
+ clk_register_clkdev(clk, "dsilp2", "mcde");
+
+ clk = clk_reg_prcmu_rate("smp_twd", NULL, PRCMU_ARMSS,
+ CLK_IS_ROOT|CLK_GET_RATE_NOCACHE|
+ CLK_IGNORE_UNUSED);
+ clk_register_clkdev(clk, NULL, "smp_twd");
+
+ /*
+ * FIXME: Add special handled PRCMU clocks here:
+ * 1. clk_arm, use PRCMU_ARMCLK.
+ * 2. clkout0yuv, use PRCMU as parent + need regulator + pinctrl.
+ * 3. ab9540_clkout1yuv, see clkout0yuv
+ */
+
+ /* PRCC P-clocks */
+ clk = clk_reg_prcc_pclk("p1_pclk0", "per1clk", U8500_CLKRST1_BASE,
+ BIT(0), 0);
+ clk_register_clkdev(clk, "apb_pclk", "uart0");
+
+ clk = clk_reg_prcc_pclk("p1_pclk1", "per1clk", U8500_CLKRST1_BASE,
+ BIT(1), 0);
+ clk_register_clkdev(clk, "apb_pclk", "uart1");
+
+ clk = clk_reg_prcc_pclk("p1_pclk2", "per1clk", U8500_CLKRST1_BASE,
+ BIT(2), 0);
+ clk = clk_reg_prcc_pclk("p1_pclk3", "per1clk", U8500_CLKRST1_BASE,
+ BIT(3), 0);
+ clk = clk_reg_prcc_pclk("p1_pclk4", "per1clk", U8500_CLKRST1_BASE,
+ BIT(4), 0);
+
+ clk = clk_reg_prcc_pclk("p1_pclk5", "per1clk", U8500_CLKRST1_BASE,
+ BIT(5), 0);
+ clk_register_clkdev(clk, "apb_pclk", "sdi0");
+
+ clk = clk_reg_prcc_pclk("p1_pclk6", "per1clk", U8500_CLKRST1_BASE,
+ BIT(6), 0);
+
+ clk = clk_reg_prcc_pclk("p1_pclk7", "per1clk", U8500_CLKRST1_BASE,
+ BIT(7), 0);
+ clk_register_clkdev(clk, NULL, "spi3");
+
+ clk = clk_reg_prcc_pclk("p1_pclk8", "per1clk", U8500_CLKRST1_BASE,
+ BIT(8), 0);
+
+ clk = clk_reg_prcc_pclk("p1_pclk9", "per1clk", U8500_CLKRST1_BASE,
+ BIT(9), 0);
+ clk_register_clkdev(clk, NULL, "gpio.0");
+ clk_register_clkdev(clk, NULL, "gpio.1");
+ clk_register_clkdev(clk, NULL, "gpioblock0");
+
+ clk = clk_reg_prcc_pclk("p1_pclk10", "per1clk", U8500_CLKRST1_BASE,
+ BIT(10), 0);
+ clk = clk_reg_prcc_pclk("p1_pclk11", "per1clk", U8500_CLKRST1_BASE,
+ BIT(11), 0);
+
+ clk = clk_reg_prcc_pclk("p2_pclk0", "per2clk", U8500_CLKRST2_BASE,
+ BIT(0), 0);
+
+ clk = clk_reg_prcc_pclk("p2_pclk1", "per2clk", U8500_CLKRST2_BASE,
+ BIT(1), 0);
+ clk_register_clkdev(clk, NULL, "spi2");
+
+ clk = clk_reg_prcc_pclk("p2_pclk2", "per2clk", U8500_CLKRST2_BASE,
+ BIT(2), 0);
+ clk_register_clkdev(clk, NULL, "spi1");
+
+ clk = clk_reg_prcc_pclk("p2_pclk3", "per2clk", U8500_CLKRST2_BASE,
+ BIT(3), 0);
+ clk_register_clkdev(clk, NULL, "pwl");
+
+ clk = clk_reg_prcc_pclk("p2_pclk4", "per2clk", U8500_CLKRST2_BASE,
+ BIT(4), 0);
+ clk_register_clkdev(clk, "apb_pclk", "sdi4");
+
+ clk = clk_reg_prcc_pclk("p2_pclk5", "per2clk", U8500_CLKRST2_BASE,
+ BIT(5), 0);
+
+ clk = clk_reg_prcc_pclk("p2_pclk6", "per2clk", U8500_CLKRST2_BASE,
+ BIT(6), 0);
+ clk_register_clkdev(clk, "apb_pclk", "sdi1");
+
+
+ clk = clk_reg_prcc_pclk("p2_pclk7", "per2clk", U8500_CLKRST2_BASE,
+ BIT(7), 0);
+ clk_register_clkdev(clk, "apb_pclk", "sdi3");
+
+ clk = clk_reg_prcc_pclk("p2_pclk8", "per2clk", U8500_CLKRST2_BASE,
+ BIT(8), 0);
+ clk_register_clkdev(clk, NULL, "spi0");
+
+ clk = clk_reg_prcc_pclk("p2_pclk9", "per2clk", U8500_CLKRST2_BASE,
+ BIT(9), 0);
+ clk_register_clkdev(clk, "hsir_hclk", "ste_hsi.0");
+
+ clk = clk_reg_prcc_pclk("p2_pclk10", "per2clk", U8500_CLKRST2_BASE,
+ BIT(10), 0);
+ clk_register_clkdev(clk, "hsit_hclk", "ste_hsi.0");
+
+ clk = clk_reg_prcc_pclk("p2_pclk11", "per2clk", U8500_CLKRST2_BASE,
+ BIT(11), 0);
+ clk_register_clkdev(clk, NULL, "gpio.6");
+ clk_register_clkdev(clk, NULL, "gpio.7");
+ clk_register_clkdev(clk, NULL, "gpioblock1");
+
+ clk = clk_reg_prcc_pclk("p2_pclk12", "per2clk", U8500_CLKRST2_BASE,
+ BIT(11), 0);
+
+ clk = clk_reg_prcc_pclk("p3_pclk0", "per3clk", U8500_CLKRST3_BASE,
+ BIT(0), 0);
+ clk_register_clkdev(clk, NULL, "fsmc");
+
+ clk = clk_reg_prcc_pclk("p3_pclk1", "per3clk", U8500_CLKRST3_BASE,
+ BIT(1), 0);
+ clk = clk_reg_prcc_pclk("p3_pclk2", "per3clk", U8500_CLKRST3_BASE,
+ BIT(2), 0);
+ clk = clk_reg_prcc_pclk("p3_pclk3", "per3clk", U8500_CLKRST3_BASE,
+ BIT(3), 0);
+
+ clk = clk_reg_prcc_pclk("p3_pclk4", "per3clk", U8500_CLKRST3_BASE,
+ BIT(4), 0);
+ clk_register_clkdev(clk, "apb_pclk", "sdi2");
+
+ clk = clk_reg_prcc_pclk("p3_pclk5", "per3clk", U8500_CLKRST3_BASE,
+ BIT(5), 0);
+
+ clk = clk_reg_prcc_pclk("p3_pclk6", "per3clk", U8500_CLKRST3_BASE,
+ BIT(6), 0);
+ clk_register_clkdev(clk, "apb_pclk", "uart2");
+
+ clk = clk_reg_prcc_pclk("p3_pclk7", "per3clk", U8500_CLKRST3_BASE,
+ BIT(7), 0);
+ clk_register_clkdev(clk, "apb_pclk", "sdi5");
+
+ clk = clk_reg_prcc_pclk("p3_pclk8", "per3clk", U8500_CLKRST3_BASE,
+ BIT(8), 0);
+ clk_register_clkdev(clk, NULL, "gpio.2");
+ clk_register_clkdev(clk, NULL, "gpio.3");
+ clk_register_clkdev(clk, NULL, "gpio.4");
+ clk_register_clkdev(clk, NULL, "gpio.5");
+ clk_register_clkdev(clk, NULL, "gpioblock2");
+
+ clk = clk_reg_prcc_pclk("p5_pclk0", "per5clk", U8500_CLKRST5_BASE,
+ BIT(0), 0);
+ clk_register_clkdev(clk, "usb", "musb-ux500.0");
+
+ clk = clk_reg_prcc_pclk("p5_pclk1", "per5clk", U8500_CLKRST5_BASE,
+ BIT(1), 0);
+ clk_register_clkdev(clk, NULL, "gpio.8");
+ clk_register_clkdev(clk, NULL, "gpioblock3");
+
+ clk = clk_reg_prcc_pclk("p6_pclk0", "per6clk", U8500_CLKRST6_BASE,
+ BIT(0), 0);
+
+ clk = clk_reg_prcc_pclk("p6_pclk1", "per6clk", U8500_CLKRST6_BASE,
+ BIT(1), 0);
+ clk_register_clkdev(clk, NULL, "cryp0");
+ clk_register_clkdev(clk, NULL, "cryp1");
+
+ clk = clk_reg_prcc_pclk("p6_pclk2", "per6clk", U8500_CLKRST6_BASE,
+ BIT(2), 0);
+ clk_register_clkdev(clk, NULL, "hash0");
+
+ clk = clk_reg_prcc_pclk("p6_pclk3", "per6clk", U8500_CLKRST6_BASE,
+ BIT(3), 0);
+ clk_register_clkdev(clk, NULL, "pka");
+
+ clk = clk_reg_prcc_pclk("p6_pclk4", "per6clk", U8500_CLKRST6_BASE,
+ BIT(4), 0);
+ clk_register_clkdev(clk, NULL, "hash1");
+
+ clk = clk_reg_prcc_pclk("p6_pclk5", "per6clk", U8500_CLKRST6_BASE,
+ BIT(5), 0);
+ clk_register_clkdev(clk, NULL, "cfgreg");
+
+ clk = clk_reg_prcc_pclk("p6_pclk6", "per6clk", U8500_CLKRST6_BASE,
+ BIT(6), 0);
+ clk = clk_reg_prcc_pclk("p6_pclk7", "per6clk", U8500_CLKRST6_BASE,
+ BIT(7), 0);
+
+ /* PRCC K-clocks
+ *
+ * FIXME: Some drivers requires PERPIH[n| to be automatically enabled
+ * by enabling just the K-clock, even if it is not a valid parent to
+ * the K-clock. Until drivers get fixed we might need some kind of
+ * "parent muxed join".
+ */
+
+ /* Periph1 */
+ clk = clk_reg_prcc_kclk("p1_uart0_kclk", "uartclk",
+ U8500_CLKRST1_BASE, BIT(0), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "uart0");
+
+ clk = clk_reg_prcc_kclk("p1_uart1_kclk", "uartclk",
+ U8500_CLKRST1_BASE, BIT(1), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "uart1");
+
+ clk = clk_reg_prcc_kclk("p1_i2c1_kclk", "i2cclk",
+ U8500_CLKRST1_BASE, BIT(2), CLK_SET_RATE_GATE);
+ clk = clk_reg_prcc_kclk("p1_msp0_kclk", "msp02clk",
+ U8500_CLKRST1_BASE, BIT(3), CLK_SET_RATE_GATE);
+ clk = clk_reg_prcc_kclk("p1_msp1_kclk", "msp1clk",
+ U8500_CLKRST1_BASE, BIT(4), CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcc_kclk("p1_sdi0_kclk", "sdmmcclk",
+ U8500_CLKRST1_BASE, BIT(5), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "sdi0");
+
+ clk = clk_reg_prcc_kclk("p1_i2c2_kclk", "i2cclk",
+ U8500_CLKRST1_BASE, BIT(6), CLK_SET_RATE_GATE);
+ clk = clk_reg_prcc_kclk("p1_slimbus0_kclk", "slimclk",
+ U8500_CLKRST1_BASE, BIT(3), CLK_SET_RATE_GATE);
+ /* FIXME: Redefinition of BIT(3). */
+ clk = clk_reg_prcc_kclk("p1_i2c4_kclk", "i2cclk",
+ U8500_CLKRST1_BASE, BIT(9), CLK_SET_RATE_GATE);
+ clk = clk_reg_prcc_kclk("p1_msp3_kclk", "msp1clk",
+ U8500_CLKRST1_BASE, BIT(10), CLK_SET_RATE_GATE);
+
+ /* Periph2 */
+ clk = clk_reg_prcc_kclk("p2_i2c3_kclk", "i2cclk",
+ U8500_CLKRST2_BASE, BIT(0), CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcc_kclk("p2_sdi4_kclk", "sdmmcclk",
+ U8500_CLKRST2_BASE, BIT(2), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "sdi4");
+
+ clk = clk_reg_prcc_kclk("p2_msp2_kclk", "msp02clk",
+ U8500_CLKRST2_BASE, BIT(3), CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcc_kclk("p2_sdi1_kclk", "sdmmcclk",
+ U8500_CLKRST2_BASE, BIT(4), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "sdi1");
+
+ clk = clk_reg_prcc_kclk("p2_sdi3_kclk", "sdmmcclk",
+ U8500_CLKRST2_BASE, BIT(5), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "sdi3");
+
+ /* Note that rate is received from parent. */
+ clk = clk_reg_prcc_kclk("p2_ssirx_kclk", "hsirxclk",
+ U8500_CLKRST2_BASE, BIT(6),
+ CLK_SET_RATE_GATE|CLK_SET_RATE_PARENT);
+ clk = clk_reg_prcc_kclk("p2_ssitx_kclk", "hsitxclk",
+ U8500_CLKRST2_BASE, BIT(7),
+ CLK_SET_RATE_GATE|CLK_SET_RATE_PARENT);
+
+ /* Periph3 */
+ clk = clk_reg_prcc_kclk("p3_ssp0_kclk", "sspclk",
+ U8500_CLKRST3_BASE, BIT(1), CLK_SET_RATE_GATE);
+ clk = clk_reg_prcc_kclk("p3_ssp1_kclk", "sspclk",
+ U8500_CLKRST3_BASE, BIT(2), CLK_SET_RATE_GATE);
+ clk = clk_reg_prcc_kclk("p3_i2c0_kclk", "i2cclk",
+ U8500_CLKRST3_BASE, BIT(3), CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcc_kclk("p3_sdi2_kclk", "sdmmcclk",
+ U8500_CLKRST3_BASE, BIT(4), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "sdi2");
+
+ clk = clk_reg_prcc_kclk("p3_ske_kclk", "rtc32k",
+ U8500_CLKRST3_BASE, BIT(5), CLK_SET_RATE_GATE);
+
+ clk = clk_reg_prcc_kclk("p3_uart2_kclk", "uartclk",
+ U8500_CLKRST3_BASE, BIT(6), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "uart2");
+
+ clk = clk_reg_prcc_kclk("p3_sdi5_kclk", "sdmmcclk",
+ U8500_CLKRST3_BASE, BIT(7), CLK_SET_RATE_GATE);
+ clk_register_clkdev(clk, NULL, "sdi5");
+
+ /* Periph6 */
+ clk = clk_reg_prcc_kclk("p3_rng_kclk", "rngclk",
+ U8500_CLKRST6_BASE, BIT(0), CLK_SET_RATE_GATE);
+
+}
diff --git a/drivers/clk/ux500/u8540_clk.c b/drivers/clk/ux500/u8540_clk.c
new file mode 100644
index 0000000..10adfd2
--- /dev/null
+++ b/drivers/clk/ux500/u8540_clk.c
@@ -0,0 +1,21 @@
+/*
+ * Clock definitions for u8540 platform.
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/platform_data/clk-ux500.h>
+
+#include "clk.h"
+
+void u8540_clk_init(void)
+{
+ /* register clocks here */
+}
diff --git a/drivers/clk/ux500/u9540_clk.c b/drivers/clk/ux500/u9540_clk.c
new file mode 100644
index 0000000..dbc0191
--- /dev/null
+++ b/drivers/clk/ux500/u9540_clk.c
@@ -0,0 +1,21 @@
+/*
+ * Clock definitions for u9540 platform.
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/mfd/dbx500-prcmu.h>
+#include <linux/platform_data/clk-ux500.h>
+
+#include "clk.h"
+
+void u9540_clk_init(void)
+{
+ /* register clocks here */
+}
diff --git a/drivers/clk/versatile/Makefile b/drivers/clk/versatile/Makefile
index 50cf6a2..c0a0f64 100644
--- a/drivers/clk/versatile/Makefile
+++ b/drivers/clk/versatile/Makefile
@@ -1,3 +1,4 @@
# Makefile for Versatile-specific clocks
obj-$(CONFIG_ICST) += clk-icst.o
obj-$(CONFIG_ARCH_INTEGRATOR) += clk-integrator.o
+obj-$(CONFIG_ARCH_REALVIEW) += clk-realview.o
diff --git a/drivers/clk/versatile/clk-realview.c b/drivers/clk/versatile/clk-realview.c
new file mode 100644
index 0000000..e21a99c
--- /dev/null
+++ b/drivers/clk/versatile/clk-realview.c
@@ -0,0 +1,114 @@
+#include <linux/clk.h>
+#include <linux/clkdev.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/clk-provider.h>
+
+#include <mach/hardware.h>
+#include <mach/platform.h>
+
+#include "clk-icst.h"
+
+/*
+ * Implementation of the ARM RealView clock trees.
+ */
+
+static void __iomem *sys_lock;
+static void __iomem *sys_vcoreg;
+
+/**
+ * realview_oscvco_get() - get ICST OSC settings for the RealView
+ */
+static struct icst_vco realview_oscvco_get(void)
+{
+ u32 val;
+ struct icst_vco vco;
+
+ val = readl(sys_vcoreg);
+ vco.v = val & 0x1ff;
+ vco.r = (val >> 9) & 0x7f;
+ vco.s = (val >> 16) & 03;
+ return vco;
+}
+
+static void realview_oscvco_set(struct icst_vco vco)
+{
+ u32 val;
+
+ val = readl(sys_vcoreg) & ~0x7ffff;
+ val |= vco.v | (vco.r << 9) | (vco.s << 16);
+
+ /* This magic unlocks the CM VCO so it can be controlled */
+ writel(0xa05f, sys_lock);
+ writel(val, sys_vcoreg);
+ /* This locks the CM again */
+ writel(0, sys_lock);
+}
+
+static const struct icst_params realview_oscvco_params = {
+ .ref = 24000000,
+ .vco_max = ICST307_VCO_MAX,
+ .vco_min = ICST307_VCO_MIN,
+ .vd_min = 4 + 8,
+ .vd_max = 511 + 8,
+ .rd_min = 1 + 2,
+ .rd_max = 127 + 2,
+ .s2div = icst307_s2div,
+ .idx2s = icst307_idx2s,
+};
+
+static const struct clk_icst_desc __initdata realview_icst_desc = {
+ .params = &realview_oscvco_params,
+ .getvco = realview_oscvco_get,
+ .setvco = realview_oscvco_set,
+};
+
+/*
+ * realview_clk_init() - set up the RealView clock tree
+ */
+void __init realview_clk_init(void __iomem *sysbase, bool is_pb1176)
+{
+ struct clk *clk;
+
+ sys_lock = sysbase + REALVIEW_SYS_LOCK_OFFSET;
+ if (is_pb1176)
+ sys_vcoreg = sysbase + REALVIEW_SYS_OSC0_OFFSET;
+ else
+ sys_vcoreg = sysbase + REALVIEW_SYS_OSC4_OFFSET;
+
+
+ /* APB clock dummy */
+ clk = clk_register_fixed_rate(NULL, "apb_pclk", NULL, CLK_IS_ROOT, 0);
+ clk_register_clkdev(clk, "apb_pclk", NULL);
+
+ /* 24 MHz clock */
+ clk = clk_register_fixed_rate(NULL, "clk24mhz", NULL, CLK_IS_ROOT,
+ 24000000);
+ clk_register_clkdev(clk, NULL, "dev:uart0");
+ clk_register_clkdev(clk, NULL, "dev:uart1");
+ clk_register_clkdev(clk, NULL, "dev:uart2");
+ clk_register_clkdev(clk, NULL, "fpga:kmi0");
+ clk_register_clkdev(clk, NULL, "fpga:kmi1");
+ clk_register_clkdev(clk, NULL, "fpga:mmc0");
+ clk_register_clkdev(clk, NULL, "dev:ssp0");
+ if (is_pb1176) {
+ /*
+ * UART3 is on the dev chip in PB1176
+ * UART4 only exists in PB1176
+ */
+ clk_register_clkdev(clk, NULL, "dev:uart3");
+ clk_register_clkdev(clk, NULL, "dev:uart4");
+ } else
+ clk_register_clkdev(clk, NULL, "fpga:uart3");
+
+
+ /* 1 MHz clock */
+ clk = clk_register_fixed_rate(NULL, "clk1mhz", NULL, CLK_IS_ROOT,
+ 1000000);
+ clk_register_clkdev(clk, NULL, "sp804");
+
+ /* ICST VCO clock */
+ clk = icst_clk_register(NULL, &realview_icst_desc);
+ clk_register_clkdev(clk, NULL, "dev:clcd");
+ clk_register_clkdev(clk, NULL, "issp:clcd");
+}
diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig
index d53cd0a..6a78073 100644
--- a/drivers/clocksource/Kconfig
+++ b/drivers/clocksource/Kconfig
@@ -35,3 +35,8 @@
default y
help
Use the always on PRCMU Timer as sched_clock
+
+config CLKSRC_ARM_GENERIC
+ def_bool y if ARM64
+ help
+ This option enables support for the ARM generic timer.
diff --git a/drivers/clocksource/Makefile b/drivers/clocksource/Makefile
index b65d0c5..6591990 100644
--- a/drivers/clocksource/Makefile
+++ b/drivers/clocksource/Makefile
@@ -13,3 +13,4 @@
obj-$(CONFIG_DW_APB_TIMER_OF) += dw_apb_timer_of.o
obj-$(CONFIG_CLKSRC_DBX500_PRCMU) += clksrc-dbx500-prcmu.o
obj-$(CONFIG_ARMADA_370_XP_TIMER) += time-armada-370-xp.o
+obj-$(CONFIG_CLKSRC_ARM_GENERIC) += arm_generic.o
diff --git a/drivers/clocksource/arm_generic.c b/drivers/clocksource/arm_generic.c
new file mode 100644
index 0000000..c4d9f95
--- /dev/null
+++ b/drivers/clocksource/arm_generic.c
@@ -0,0 +1,232 @@
+/*
+ * Generic timers support
+ *
+ * Copyright (C) 2012 ARM Ltd.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/smp.h>
+#include <linux/cpu.h>
+#include <linux/jiffies.h>
+#include <linux/interrupt.h>
+#include <linux/clockchips.h>
+#include <linux/of_irq.h>
+#include <linux/io.h>
+
+#include <clocksource/arm_generic.h>
+
+#include <asm/arm_generic.h>
+
+static u32 arch_timer_rate;
+static u64 sched_clock_mult __read_mostly;
+static DEFINE_PER_CPU(struct clock_event_device, arch_timer_evt);
+static int arch_timer_ppi;
+
+static irqreturn_t arch_timer_handle_irq(int irq, void *dev_id)
+{
+ struct clock_event_device *evt = dev_id;
+ unsigned long ctrl;
+
+ ctrl = arch_timer_reg_read(ARCH_TIMER_REG_CTRL);
+ if (ctrl & ARCH_TIMER_CTRL_ISTATUS) {
+ ctrl |= ARCH_TIMER_CTRL_IMASK;
+ arch_timer_reg_write(ARCH_TIMER_REG_CTRL, ctrl);
+ evt->event_handler(evt);
+ return IRQ_HANDLED;
+ }
+
+ return IRQ_NONE;
+}
+
+static void arch_timer_stop(void)
+{
+ unsigned long ctrl;
+
+ ctrl = arch_timer_reg_read(ARCH_TIMER_REG_CTRL);
+ ctrl &= ~ARCH_TIMER_CTRL_ENABLE;
+ arch_timer_reg_write(ARCH_TIMER_REG_CTRL, ctrl);
+}
+
+static void arch_timer_set_mode(enum clock_event_mode mode,
+ struct clock_event_device *clk)
+{
+ switch (mode) {
+ case CLOCK_EVT_MODE_UNUSED:
+ case CLOCK_EVT_MODE_SHUTDOWN:
+ arch_timer_stop();
+ break;
+ default:
+ break;
+ }
+}
+
+static int arch_timer_set_next_event(unsigned long evt,
+ struct clock_event_device *unused)
+{
+ unsigned long ctrl;
+
+ ctrl = arch_timer_reg_read(ARCH_TIMER_REG_CTRL);
+ ctrl |= ARCH_TIMER_CTRL_ENABLE;
+ ctrl &= ~ARCH_TIMER_CTRL_IMASK;
+
+ arch_timer_reg_write(ARCH_TIMER_REG_TVAL, evt);
+ arch_timer_reg_write(ARCH_TIMER_REG_CTRL, ctrl);
+
+ return 0;
+}
+
+static void __cpuinit arch_timer_setup(struct clock_event_device *clk)
+{
+ /* Let's make sure the timer is off before doing anything else */
+ arch_timer_stop();
+
+ clk->features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP;
+ clk->name = "arch_sys_timer";
+ clk->rating = 400;
+ clk->set_mode = arch_timer_set_mode;
+ clk->set_next_event = arch_timer_set_next_event;
+ clk->irq = arch_timer_ppi;
+ clk->cpumask = cpumask_of(smp_processor_id());
+
+ clockevents_config_and_register(clk, arch_timer_rate,
+ 0xf, 0x7fffffff);
+
+ enable_percpu_irq(clk->irq, 0);
+
+ /* Ensure the physical counter is visible to userspace for the vDSO. */
+ arch_counter_enable_user_access();
+}
+
+static void __init arch_timer_calibrate(void)
+{
+ if (arch_timer_rate == 0) {
+ arch_timer_reg_write(ARCH_TIMER_REG_CTRL, 0);
+ arch_timer_rate = arch_timer_reg_read(ARCH_TIMER_REG_FREQ);
+
+ /* Check the timer frequency. */
+ if (arch_timer_rate == 0)
+ panic("Architected timer frequency is set to zero.\n"
+ "You must set this in your .dts file\n");
+ }
+
+ /* Cache the sched_clock multiplier to save a divide in the hot path. */
+
+ sched_clock_mult = NSEC_PER_SEC / arch_timer_rate;
+
+ pr_info("Architected local timer running at %u.%02uMHz.\n",
+ arch_timer_rate / 1000000, (arch_timer_rate / 10000) % 100);
+}
+
+static cycle_t arch_counter_read(struct clocksource *cs)
+{
+ return arch_counter_get_cntpct();
+}
+
+static struct clocksource clocksource_counter = {
+ .name = "arch_sys_counter",
+ .rating = 400,
+ .read = arch_counter_read,
+ .mask = CLOCKSOURCE_MASK(56),
+ .flags = (CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_VALID_FOR_HRES),
+};
+
+int read_current_timer(unsigned long *timer_value)
+{
+ *timer_value = arch_counter_get_cntpct();
+ return 0;
+}
+
+unsigned long long notrace sched_clock(void)
+{
+ return arch_counter_get_cntvct() * sched_clock_mult;
+}
+
+static int __cpuinit arch_timer_cpu_notify(struct notifier_block *self,
+ unsigned long action, void *hcpu)
+{
+ int cpu = (long)hcpu;
+ struct clock_event_device *clk = per_cpu_ptr(&arch_timer_evt, cpu);
+
+ switch(action) {
+ case CPU_STARTING:
+ case CPU_STARTING_FROZEN:
+ arch_timer_setup(clk);
+ break;
+
+ case CPU_DYING:
+ case CPU_DYING_FROZEN:
+ pr_debug("arch_timer_teardown disable IRQ%d cpu #%d\n",
+ clk->irq, cpu);
+ disable_percpu_irq(clk->irq);
+ arch_timer_set_mode(CLOCK_EVT_MODE_UNUSED, clk);
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block __cpuinitdata arch_timer_cpu_nb = {
+ .notifier_call = arch_timer_cpu_notify,
+};
+
+static const struct of_device_id arch_timer_of_match[] __initconst = {
+ { .compatible = "arm,armv8-timer" },
+ {},
+};
+
+int __init arm_generic_timer_init(void)
+{
+ struct device_node *np;
+ int err;
+ u32 freq;
+
+ np = of_find_matching_node(NULL, arch_timer_of_match);
+ if (!np) {
+ pr_err("arch_timer: can't find DT node\n");
+ return -ENODEV;
+ }
+
+ /* Try to determine the frequency from the device tree or CNTFRQ */
+ if (!of_property_read_u32(np, "clock-frequency", &freq))
+ arch_timer_rate = freq;
+ arch_timer_calibrate();
+
+ arch_timer_ppi = irq_of_parse_and_map(np, 0);
+ pr_info("arch_timer: found %s irq %d\n", np->name, arch_timer_ppi);
+
+ err = request_percpu_irq(arch_timer_ppi, arch_timer_handle_irq,
+ np->name, &arch_timer_evt);
+ if (err) {
+ pr_err("arch_timer: can't register interrupt %d (%d)\n",
+ arch_timer_ppi, err);
+ return err;
+ }
+
+ clocksource_register_hz(&clocksource_counter, arch_timer_rate);
+
+ /* Calibrate the delay loop directly */
+ lpj_fine = arch_timer_rate / HZ;
+
+ /* Immediately configure the timer on the boot CPU */
+ arch_timer_setup(per_cpu_ptr(&arch_timer_evt, smp_processor_id()));
+
+ register_cpu_notifier(&arch_timer_cpu_nb);
+
+ return 0;
+}
diff --git a/drivers/cpufreq/omap-cpufreq.c b/drivers/cpufreq/omap-cpufreq.c
index 17fa04d0..b47034e 100644
--- a/drivers/cpufreq/omap-cpufreq.c
+++ b/drivers/cpufreq/omap-cpufreq.c
@@ -218,7 +218,7 @@
policy->cur = policy->min = policy->max = omap_getspeed(policy->cpu);
- if (atomic_inc_return(&freq_table_users) == 1)
+ if (!freq_table)
result = opp_init_cpufreq_table(mpu_dev, &freq_table);
if (result) {
@@ -227,6 +227,8 @@
goto fail_ck;
}
+ atomic_inc_return(&freq_table_users);
+
result = cpufreq_frequency_table_cpuinfo(policy, freq_table);
if (result)
goto fail_table;
diff --git a/drivers/cpufreq/powernow-k8.c b/drivers/cpufreq/powernow-k8.c
index c0e8164..1a40935 100644
--- a/drivers/cpufreq/powernow-k8.c
+++ b/drivers/cpufreq/powernow-k8.c
@@ -35,7 +35,6 @@
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/cpumask.h>
-#include <linux/sched.h> /* for current / set_cpus_allowed() */
#include <linux/io.h>
#include <linux/delay.h>
@@ -1139,16 +1138,23 @@
return res;
}
-/* Driver entry point to switch to the target frequency */
-static int powernowk8_target(struct cpufreq_policy *pol,
- unsigned targfreq, unsigned relation)
+struct powernowk8_target_arg {
+ struct cpufreq_policy *pol;
+ unsigned targfreq;
+ unsigned relation;
+};
+
+static long powernowk8_target_fn(void *arg)
{
- cpumask_var_t oldmask;
+ struct powernowk8_target_arg *pta = arg;
+ struct cpufreq_policy *pol = pta->pol;
+ unsigned targfreq = pta->targfreq;
+ unsigned relation = pta->relation;
struct powernow_k8_data *data = per_cpu(powernow_data, pol->cpu);
u32 checkfid;
u32 checkvid;
unsigned int newstate;
- int ret = -EIO;
+ int ret;
if (!data)
return -EINVAL;
@@ -1156,29 +1162,16 @@
checkfid = data->currfid;
checkvid = data->currvid;
- /* only run on specific CPU from here on. */
- /* This is poor form: use a workqueue or smp_call_function_single */
- if (!alloc_cpumask_var(&oldmask, GFP_KERNEL))
- return -ENOMEM;
-
- cpumask_copy(oldmask, tsk_cpus_allowed(current));
- set_cpus_allowed_ptr(current, cpumask_of(pol->cpu));
-
- if (smp_processor_id() != pol->cpu) {
- printk(KERN_ERR PFX "limiting to cpu %u failed\n", pol->cpu);
- goto err_out;
- }
-
if (pending_bit_stuck()) {
printk(KERN_ERR PFX "failing targ, change pending bit set\n");
- goto err_out;
+ return -EIO;
}
pr_debug("targ: cpu %d, %d kHz, min %d, max %d, relation %d\n",
pol->cpu, targfreq, pol->min, pol->max, relation);
if (query_current_values_with_pending_wait(data))
- goto err_out;
+ return -EIO;
if (cpu_family != CPU_HW_PSTATE) {
pr_debug("targ: curr fid 0x%x, vid 0x%x\n",
@@ -1196,7 +1189,7 @@
if (cpufreq_frequency_table_target(pol, data->powernow_table,
targfreq, relation, &newstate))
- goto err_out;
+ return -EIO;
mutex_lock(&fidvid_mutex);
@@ -1209,9 +1202,8 @@
ret = transition_frequency_fidvid(data, newstate);
if (ret) {
printk(KERN_ERR PFX "transition frequency failed\n");
- ret = 1;
mutex_unlock(&fidvid_mutex);
- goto err_out;
+ return 1;
}
mutex_unlock(&fidvid_mutex);
@@ -1220,12 +1212,25 @@
data->powernow_table[newstate].index);
else
pol->cur = find_khz_freq_from_fid(data->currfid);
- ret = 0;
-err_out:
- set_cpus_allowed_ptr(current, oldmask);
- free_cpumask_var(oldmask);
- return ret;
+ return 0;
+}
+
+/* Driver entry point to switch to the target frequency */
+static int powernowk8_target(struct cpufreq_policy *pol,
+ unsigned targfreq, unsigned relation)
+{
+ struct powernowk8_target_arg pta = { .pol = pol, .targfreq = targfreq,
+ .relation = relation };
+
+ /*
+ * Must run on @pol->cpu. cpufreq core is responsible for ensuring
+ * that we're bound to the current CPU and pol->cpu stays online.
+ */
+ if (smp_processor_id() == pol->cpu)
+ return powernowk8_target_fn(&pta);
+ else
+ return work_on_cpu(pol->cpu, powernowk8_target_fn, &pta);
}
/* Driver entry point to verify the policy and range of frequencies */
diff --git a/drivers/crypto/caam/jr.c b/drivers/crypto/caam/jr.c
index 53c8c51..93d1407 100644
--- a/drivers/crypto/caam/jr.c
+++ b/drivers/crypto/caam/jr.c
@@ -63,7 +63,7 @@
head = ACCESS_ONCE(jrp->head);
- spin_lock_bh(&jrp->outlock);
+ spin_lock(&jrp->outlock);
sw_idx = tail = jrp->tail;
hw_idx = jrp->out_ring_read_index;
@@ -115,7 +115,7 @@
jrp->tail = tail;
}
- spin_unlock_bh(&jrp->outlock);
+ spin_unlock(&jrp->outlock);
/* Finally, execute user's callback */
usercall(dev, userdesc, userstatus, userarg);
@@ -236,14 +236,14 @@
return -EIO;
}
- spin_lock(&jrp->inplock);
+ spin_lock_bh(&jrp->inplock);
head = jrp->head;
tail = ACCESS_ONCE(jrp->tail);
if (!rd_reg32(&jrp->rregs->inpring_avail) ||
CIRC_SPACE(head, tail, JOBR_DEPTH) <= 0) {
- spin_unlock(&jrp->inplock);
+ spin_unlock_bh(&jrp->inplock);
dma_unmap_single(dev, desc_dma, desc_size, DMA_TO_DEVICE);
return -EBUSY;
}
@@ -265,7 +265,7 @@
wr_reg32(&jrp->rregs->inpring_jobadd, 1);
- spin_unlock(&jrp->inplock);
+ spin_unlock_bh(&jrp->inplock);
return 0;
}
diff --git a/drivers/crypto/caam/key_gen.c b/drivers/crypto/caam/key_gen.c
index 0028881..d216cd3 100644
--- a/drivers/crypto/caam/key_gen.c
+++ b/drivers/crypto/caam/key_gen.c
@@ -120,3 +120,4 @@
return ret;
}
+EXPORT_SYMBOL(gen_split_key);
diff --git a/drivers/crypto/hifn_795x.c b/drivers/crypto/hifn_795x.c
index c9c4bef..df14358 100644
--- a/drivers/crypto/hifn_795x.c
+++ b/drivers/crypto/hifn_795x.c
@@ -821,8 +821,8 @@
/*
* We must wait at least 256 Pk_clk cycles between two reads of the rng.
*/
- dev->rng_wait_time = DIV_ROUND_UP(NSEC_PER_SEC, dev->pk_clk_freq) *
- 256;
+ dev->rng_wait_time = DIV_ROUND_UP_ULL(NSEC_PER_SEC,
+ dev->pk_clk_freq) * 256;
dev->rng.name = dev->name;
dev->rng.data_present = hifn_rng_data_present,
diff --git a/drivers/dma/at_hdmac.c b/drivers/dma/at_hdmac.c
index 3934fcc..17d6958 100644
--- a/drivers/dma/at_hdmac.c
+++ b/drivers/dma/at_hdmac.c
@@ -168,9 +168,9 @@
}
/**
- * atc_desc_chain - build chain adding a descripor
- * @first: address of first descripor of the chain
- * @prev: address of previous descripor of the chain
+ * atc_desc_chain - build chain adding a descriptor
+ * @first: address of first descriptor of the chain
+ * @prev: address of previous descriptor of the chain
* @desc: descriptor to queue
*
* Called from prep_* functions
@@ -661,7 +661,7 @@
flags);
if (unlikely(!atslave || !sg_len)) {
- dev_dbg(chan2dev(chan), "prep_dma_memcpy: length is zero!\n");
+ dev_dbg(chan2dev(chan), "prep_slave_sg: sg length is zero!\n");
return NULL;
}
@@ -689,6 +689,11 @@
mem = sg_dma_address(sg);
len = sg_dma_len(sg);
+ if (unlikely(!len)) {
+ dev_dbg(chan2dev(chan),
+ "prep_slave_sg: sg(%d) data length is zero\n", i);
+ goto err;
+ }
mem_width = 2;
if (unlikely(mem & 3 || len & 3))
mem_width = 0;
@@ -724,6 +729,11 @@
mem = sg_dma_address(sg);
len = sg_dma_len(sg);
+ if (unlikely(!len)) {
+ dev_dbg(chan2dev(chan),
+ "prep_slave_sg: sg(%d) data length is zero\n", i);
+ goto err;
+ }
mem_width = 2;
if (unlikely(mem & 3 || len & 3))
mem_width = 0;
@@ -757,6 +767,7 @@
err_desc_get:
dev_err(chan2dev(chan), "not enough descriptors available\n");
+err:
atc_desc_put(atchan, first);
return NULL;
}
@@ -785,7 +796,7 @@
}
/**
- * atc_dma_cyclic_fill_desc - Fill one period decriptor
+ * atc_dma_cyclic_fill_desc - Fill one period descriptor
*/
static int
atc_dma_cyclic_fill_desc(struct dma_chan *chan, struct at_desc *desc,
diff --git a/drivers/dma/ep93xx_dma.c b/drivers/dma/ep93xx_dma.c
index c64917e..bb02fd9 100644
--- a/drivers/dma/ep93xx_dma.c
+++ b/drivers/dma/ep93xx_dma.c
@@ -1118,7 +1118,7 @@
* @chan: channel
* @dma_addr: DMA mapped address of the buffer
* @buf_len: length of the buffer (in bytes)
- * @period_len: lenght of a single period
+ * @period_len: length of a single period
* @dir: direction of the operation
* @context: operation context (ignored)
*
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 8f84761..094437b 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -1015,7 +1015,7 @@
/*
* Programming Error
* The DMA_INTERRUPT async_tx is a NULL transfer, which will
- * triger a PE interrupt.
+ * trigger a PE interrupt.
*/
if (stat & FSL_DMA_SR_PE) {
chan_dbg(chan, "irq: Programming Error INT\n");
diff --git a/drivers/dma/imx-dma.c b/drivers/dma/imx-dma.c
index 5084975..54f580b 100644
--- a/drivers/dma/imx-dma.c
+++ b/drivers/dma/imx-dma.c
@@ -572,8 +572,8 @@
if (desc->desc.callback)
desc->desc.callback(desc->desc.callback_param);
- /* If we are dealing with a cyclic descriptor keep it on ld_active
- * and dont mark the descripor as complete.
+ /* If we are dealing with a cyclic descriptor, keep it on ld_active
+ * and dont mark the descriptor as complete.
* Only in non-cyclic cases it would be marked as complete
*/
if (imxdma_chan_is_doing_cyclic(imxdmac))
diff --git a/drivers/dma/intel_mid_dma.c b/drivers/dma/intel_mid_dma.c
index 222e907..02b21d7 100644
--- a/drivers/dma/intel_mid_dma.c
+++ b/drivers/dma/intel_mid_dma.c
@@ -427,7 +427,7 @@
* intel_mid_dma_tx_submit - callback to submit DMA transaction
* @tx: dma engine descriptor
*
- * Submit the DMA trasaction for this descriptor, start if ch idle
+ * Submit the DMA transaction for this descriptor, start if ch idle
*/
static dma_cookie_t intel_mid_dma_tx_submit(struct dma_async_tx_descriptor *tx)
{
diff --git a/drivers/dma/intel_mid_dma_regs.h b/drivers/dma/intel_mid_dma_regs.h
index 1bfa926..17b4219 100644
--- a/drivers/dma/intel_mid_dma_regs.h
+++ b/drivers/dma/intel_mid_dma_regs.h
@@ -168,9 +168,9 @@
* @active_list: current active descriptors
* @queue: current queued up descriptors
* @free_list: current free descriptors
- * @slave: dma slave struture
- * @descs_allocated: total number of decsiptors allocated
- * @dma: dma device struture pointer
+ * @slave: dma slave structure
+ * @descs_allocated: total number of descriptors allocated
+ * @dma: dma device structure pointer
* @busy: bool representing if ch is busy (active txn) or not
* @in_use: bool representing if ch is in use or not
* @raw_tfr: raw trf interrupt received
diff --git a/drivers/dma/ioat/hw.h b/drivers/dma/ioat/hw.h
index 60e6754..d2ff3fd 100644
--- a/drivers/dma/ioat/hw.h
+++ b/drivers/dma/ioat/hw.h
@@ -22,7 +22,6 @@
#define _IOAT_HW_H_
/* PCI Configuration Space Values */
-#define IOAT_PCI_VID 0x8086
#define IOAT_MMIO_BAR 0
/* CB device ID's */
@@ -31,9 +30,6 @@
#define IOAT_PCI_DID_SCNB 0x65FF
#define IOAT_PCI_DID_SNB 0x402F
-#define IOAT_PCI_RID 0x00
-#define IOAT_PCI_SVID 0x8086
-#define IOAT_PCI_SID 0x8086
#define IOAT_VER_1_2 0x12 /* Version 1.2 */
#define IOAT_VER_2_0 0x20 /* Version 2.0 */
#define IOAT_VER_3_0 0x30 /* Version 3.0 */
diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c
index e4feba6..5d3bbcd 100644
--- a/drivers/dma/pl330.c
+++ b/drivers/dma/pl330.c
@@ -522,7 +522,7 @@
/* In the DMAC pool */
FREE,
/*
- * Allocted to some channel during prep_xxx
+ * Allocated to some channel during prep_xxx
* Also may be sitting on the work_list.
*/
PREP,
@@ -1567,17 +1567,19 @@
goto xfer_exit;
}
- /* Prefer Secure Channel */
- if (!_manager_ns(thrd))
- r->cfg->nonsecure = 0;
- else
- r->cfg->nonsecure = 1;
/* Use last settings, if not provided */
- if (r->cfg)
+ if (r->cfg) {
+ /* Prefer Secure Channel */
+ if (!_manager_ns(thrd))
+ r->cfg->nonsecure = 0;
+ else
+ r->cfg->nonsecure = 1;
+
ccr = _prepare_ccr(r->cfg);
- else
+ } else {
ccr = readl(regs + CC(thrd->id));
+ }
/* If this req doesn't have valid xfer settings */
if (!_is_valid(ccr)) {
@@ -2928,6 +2930,11 @@
num_chan = max_t(int, pi->pcfg.num_peri, pi->pcfg.num_chan);
pdmac->peripherals = kzalloc(num_chan * sizeof(*pch), GFP_KERNEL);
+ if (!pdmac->peripherals) {
+ ret = -ENOMEM;
+ dev_err(&adev->dev, "unable to allocate pdmac->peripherals\n");
+ goto probe_err5;
+ }
for (i = 0; i < num_chan; i++) {
pch = &pdmac->peripherals[i];
diff --git a/drivers/dma/ppc4xx/adma.c b/drivers/dma/ppc4xx/adma.c
index ced9882..f72348d 100644
--- a/drivers/dma/ppc4xx/adma.c
+++ b/drivers/dma/ppc4xx/adma.c
@@ -4446,7 +4446,7 @@
ret = -ENOMEM;
goto err_dma_alloc;
}
- dev_dbg(&ofdev->dev, "allocted descriptor pool virt 0x%p phys 0x%llx\n",
+ dev_dbg(&ofdev->dev, "allocated descriptor pool virt 0x%p phys 0x%llx\n",
adev->dma_desc_pool_virt, (u64)adev->dma_desc_pool);
regs = ioremap(res.start, resource_size(&res));
diff --git a/drivers/dma/ste_dma40_ll.h b/drivers/dma/ste_dma40_ll.h
index 51e8e53..6d47373 100644
--- a/drivers/dma/ste_dma40_ll.h
+++ b/drivers/dma/ste_dma40_ll.h
@@ -202,7 +202,7 @@
/* LLI related structures */
/**
- * struct d40_phy_lli - The basic configration register for each physical
+ * struct d40_phy_lli - The basic configuration register for each physical
* channel.
*
* @reg_cfg: The configuration register.
diff --git a/drivers/edac/edac_mc.c b/drivers/edac/edac_mc.c
index 616d90b..d5dc9da 100644
--- a/drivers/edac/edac_mc.c
+++ b/drivers/edac/edac_mc.c
@@ -199,6 +199,36 @@
return (void *)(((unsigned long)ptr) + align - r);
}
+static void _edac_mc_free(struct mem_ctl_info *mci)
+{
+ int i, chn, row;
+ struct csrow_info *csr;
+ const unsigned int tot_dimms = mci->tot_dimms;
+ const unsigned int tot_channels = mci->num_cschannel;
+ const unsigned int tot_csrows = mci->nr_csrows;
+
+ if (mci->dimms) {
+ for (i = 0; i < tot_dimms; i++)
+ kfree(mci->dimms[i]);
+ kfree(mci->dimms);
+ }
+ if (mci->csrows) {
+ for (row = 0; row < tot_csrows; row++) {
+ csr = mci->csrows[row];
+ if (csr) {
+ if (csr->channels) {
+ for (chn = 0; chn < tot_channels; chn++)
+ kfree(csr->channels[chn]);
+ kfree(csr->channels);
+ }
+ kfree(csr);
+ }
+ }
+ kfree(mci->csrows);
+ }
+ kfree(mci);
+}
+
/**
* edac_mc_alloc: Allocate and partially fill a struct mem_ctl_info structure
* @mc_num: Memory controller number
@@ -413,24 +443,7 @@
return mci;
error:
- if (mci->dimms) {
- for (i = 0; i < tot_dimms; i++)
- kfree(mci->dimms[i]);
- kfree(mci->dimms);
- }
- if (mci->csrows) {
- for (chn = 0; chn < tot_channels; chn++) {
- csr = mci->csrows[chn];
- if (csr) {
- for (chn = 0; chn < tot_channels; chn++)
- kfree(csr->channels[chn]);
- kfree(csr);
- }
- kfree(mci->csrows[i]);
- }
- kfree(mci->csrows);
- }
- kfree(mci);
+ _edac_mc_free(mci);
return NULL;
}
@@ -445,6 +458,14 @@
{
edac_dbg(1, "\n");
+ /* If we're not yet registered with sysfs free only what was allocated
+ * in edac_mc_alloc().
+ */
+ if (!device_is_registered(&mci->dev)) {
+ _edac_mc_free(mci);
+ return;
+ }
+
/* the mci instance is freed here, when the sysfs object is dropped */
edac_unregister_sysfs(mci);
}
diff --git a/drivers/edac/i3200_edac.c b/drivers/edac/i3200_edac.c
index 47180a0..b6653a6 100644
--- a/drivers/edac/i3200_edac.c
+++ b/drivers/edac/i3200_edac.c
@@ -391,7 +391,7 @@
for (j = 0; j < nr_channels; j++) {
struct dimm_info *dimm = csrow->channels[j]->dimm;
- dimm->nr_pages = nr_pages / nr_channels;
+ dimm->nr_pages = nr_pages;
dimm->grain = nr_pages << PAGE_SHIFT;
dimm->mtype = MEM_DDR2;
dimm->dtype = DEV_UNKNOWN;
diff --git a/drivers/edac/i5000_edac.c b/drivers/edac/i5000_edac.c
index 39c6375..6a49dd0 100644
--- a/drivers/edac/i5000_edac.c
+++ b/drivers/edac/i5000_edac.c
@@ -1012,6 +1012,10 @@
/* add the number of COLUMN bits */
addrBits += MTR_DIMM_COLS_ADDR_BITS(mtr);
+ /* Dual-rank memories have twice the size */
+ if (dinfo->dual_rank)
+ addrBits++;
+
addrBits += 6; /* add 64 bits per DIMM */
addrBits -= 20; /* divide by 2^^20 */
addrBits -= 3; /* 8 bits per bytes */
diff --git a/drivers/edac/sb_edac.c b/drivers/edac/sb_edac.c
index f3b1f9f..5715b7c 100644
--- a/drivers/edac/sb_edac.c
+++ b/drivers/edac/sb_edac.c
@@ -513,7 +513,8 @@
{
struct sbridge_pvt *pvt = mci->pvt_info;
struct dimm_info *dimm;
- int i, j, banks, ranks, rows, cols, size, npages;
+ unsigned i, j, banks, ranks, rows, cols, npages;
+ u64 size;
u32 reg;
enum edac_type mode;
enum mem_type mtype;
@@ -585,10 +586,10 @@
cols = numcol(mtr);
/* DDR3 has 8 I/O banks */
- size = (rows * cols * banks * ranks) >> (20 - 3);
+ size = ((u64)rows * cols * banks * ranks) >> (20 - 3);
npages = MiB_TO_PAGES(size);
- edac_dbg(0, "mc#%d: channel %d, dimm %d, %d Mb (%d pages) bank: %d, rank: %d, row: %#x, col: %#x\n",
+ edac_dbg(0, "mc#%d: channel %d, dimm %d, %Ld Mb (%d pages) bank: %d, rank: %d, row: %#x, col: %#x\n",
pvt->sbridge_dev->mc, i, j,
size, npages,
banks, ranks, rows, cols);
diff --git a/drivers/extcon/extcon-arizona.c b/drivers/extcon/extcon-arizona.c
index 427a289..6c19833 100644
--- a/drivers/extcon/extcon-arizona.c
+++ b/drivers/extcon/extcon-arizona.c
@@ -434,6 +434,11 @@
regmap_update_bits(arizona->regmap, ARIZONA_JACK_DETECT_ANALOGUE,
ARIZONA_JD1_ENA, ARIZONA_JD1_ENA);
+ ret = regulator_allow_bypass(info->micvdd, true);
+ if (ret != 0)
+ dev_warn(arizona->dev, "Failed to set MICVDD to bypass: %d\n",
+ ret);
+
pm_runtime_put(&pdev->dev);
return 0;
diff --git a/drivers/extcon/extcon-max77693.c b/drivers/extcon/extcon-max77693.c
index 920a609..38f9e52 100644
--- a/drivers/extcon/extcon-max77693.c
+++ b/drivers/extcon/extcon-max77693.c
@@ -669,13 +669,18 @@
}
info->dev = &pdev->dev;
info->max77693 = max77693;
- info->max77693->regmap_muic = regmap_init_i2c(info->max77693->muic,
- &max77693_muic_regmap_config);
- if (IS_ERR(info->max77693->regmap_muic)) {
- ret = PTR_ERR(info->max77693->regmap_muic);
- dev_err(max77693->dev,
- "failed to allocate register map: %d\n", ret);
- goto err_regmap;
+ if (info->max77693->regmap_muic)
+ dev_dbg(&pdev->dev, "allocate register map\n");
+ else {
+ info->max77693->regmap_muic = devm_regmap_init_i2c(
+ info->max77693->muic,
+ &max77693_muic_regmap_config);
+ if (IS_ERR(info->max77693->regmap_muic)) {
+ ret = PTR_ERR(info->max77693->regmap_muic);
+ dev_err(max77693->dev,
+ "failed to allocate register map: %d\n", ret);
+ goto err_regmap;
+ }
}
platform_set_drvdata(pdev, info);
mutex_init(&info->mutex);
diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig
index b16c8a7..ba7926f5 100644
--- a/drivers/gpio/Kconfig
+++ b/drivers/gpio/Kconfig
@@ -294,7 +294,7 @@
config GPIO_MC9S08DZ60
bool "MX35 3DS BOARD MC9S08DZ60 GPIO functions"
- depends on I2C && MACH_MX35_3DS
+ depends on I2C=y && MACH_MX35_3DS
help
Select this to enable the MC9S08DZ60 GPIO driver
diff --git a/drivers/gpio/gpio-em.c b/drivers/gpio/gpio-em.c
index ae37181..ec48ed5 100644
--- a/drivers/gpio/gpio-em.c
+++ b/drivers/gpio/gpio-em.c
@@ -247,9 +247,9 @@
p->irq_base = irq_alloc_descs(pdata->irq_base, 0,
pdata->number_of_pins, numa_node_id());
- if (IS_ERR_VALUE(p->irq_base)) {
+ if (p->irq_base < 0) {
dev_err(&pdev->dev, "cannot get irq_desc\n");
- return -ENXIO;
+ return p->irq_base;
}
pr_debug("gio: hw base = %d, nr = %d, sw base = %d\n",
pdata->gpio_base, pdata->number_of_pins, p->irq_base);
diff --git a/drivers/gpio/gpio-lpc32xx.c b/drivers/gpio/gpio-lpc32xx.c
index 8a420f1..ed94b4e 100644
--- a/drivers/gpio/gpio-lpc32xx.c
+++ b/drivers/gpio/gpio-lpc32xx.c
@@ -308,6 +308,7 @@
{
struct lpc32xx_gpio_chip *group = to_lpc32xx_gpio(chip);
+ __set_gpio_level_p012(group, pin, value);
__set_gpio_dir_p012(group, pin, 0);
return 0;
@@ -318,6 +319,7 @@
{
struct lpc32xx_gpio_chip *group = to_lpc32xx_gpio(chip);
+ __set_gpio_level_p3(group, pin, value);
__set_gpio_dir_p3(group, pin, 0);
return 0;
@@ -326,6 +328,9 @@
static int lpc32xx_gpio_dir_out_always(struct gpio_chip *chip, unsigned pin,
int value)
{
+ struct lpc32xx_gpio_chip *group = to_lpc32xx_gpio(chip);
+
+ __set_gpo_level_p3(group, pin, value);
return 0;
}
diff --git a/drivers/gpio/gpio-rdc321x.c b/drivers/gpio/gpio-rdc321x.c
index e97016a..b62d443 100644
--- a/drivers/gpio/gpio-rdc321x.c
+++ b/drivers/gpio/gpio-rdc321x.c
@@ -170,6 +170,7 @@
rdc321x_gpio_dev->reg2_data_base = r->start + 0x4;
rdc321x_gpio_dev->chip.label = "rdc321x-gpio";
+ rdc321x_gpio_dev->chip.owner = THIS_MODULE;
rdc321x_gpio_dev->chip.direction_input = rdc_gpio_direction_input;
rdc321x_gpio_dev->chip.direction_output = rdc_gpio_config;
rdc321x_gpio_dev->chip.get = rdc_gpio_get_value;
diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
index a18c4aa..f1a4599 100644
--- a/drivers/gpio/gpiolib-of.c
+++ b/drivers/gpio/gpiolib-of.c
@@ -82,7 +82,7 @@
gpiochip_find(&gg_data, of_gpiochip_find_and_xlate);
of_node_put(gg_data.gpiospec.np);
- pr_debug("%s exited with status %d\n", __func__, ret);
+ pr_debug("%s exited with status %d\n", __func__, gg_data.out_gpio);
return gg_data.out_gpio;
}
EXPORT_SYMBOL(of_get_named_gpio_flags);
diff --git a/drivers/gpu/drm/ast/ast_drv.c b/drivers/gpu/drm/ast/ast_drv.c
index d0c4574..3616480 100644
--- a/drivers/gpu/drm/ast/ast_drv.c
+++ b/drivers/gpu/drm/ast/ast_drv.c
@@ -193,6 +193,9 @@
.mmap = ast_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.read = drm_read,
};
diff --git a/drivers/gpu/drm/ast/ast_mode.c b/drivers/gpu/drm/ast/ast_mode.c
index 7282c08..a712caf 100644
--- a/drivers/gpu/drm/ast/ast_mode.c
+++ b/drivers/gpu/drm/ast/ast_mode.c
@@ -841,7 +841,7 @@
ast->cursor_cache = obj;
ast->cursor_cache_gpu_addr = gpu_addr;
- DRM_ERROR("pinned cursor cache at %llx\n", ast->cursor_cache_gpu_addr);
+ DRM_DEBUG_KMS("pinned cursor cache at %llx\n", ast->cursor_cache_gpu_addr);
return 0;
fail:
return ret;
diff --git a/drivers/gpu/drm/cirrus/cirrus_drv.c b/drivers/gpu/drm/cirrus/cirrus_drv.c
index 7053140..b83a2d7 100644
--- a/drivers/gpu/drm/cirrus/cirrus_drv.c
+++ b/drivers/gpu/drm/cirrus/cirrus_drv.c
@@ -74,6 +74,9 @@
.unlocked_ioctl = drm_ioctl,
.mmap = cirrus_mmap,
.poll = drm_poll,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.fasync = drm_fasync,
};
static struct drm_driver driver = {
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 08a7aa7..6fbfc24 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -1981,7 +1981,7 @@
if (!drm_core_check_feature(dev, DRIVER_MODESET))
return -EINVAL;
- if (!req->flags)
+ if (!req->flags || (~DRM_MODE_CURSOR_FLAGS & req->flags))
return -EINVAL;
mutex_lock(&dev->mode_config.mutex);
diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c
index a8743c3..b7ee230 100644
--- a/drivers/gpu/drm/drm_edid.c
+++ b/drivers/gpu/drm/drm_edid.c
@@ -87,6 +87,9 @@
int product_id;
u32 quirks;
} edid_quirk_list[] = {
+ /* ASUS VW222S */
+ { "ACI", 0x22a2, EDID_QUIRK_FORCE_REDUCED_BLANKING },
+
/* Acer AL1706 */
{ "ACR", 44358, EDID_QUIRK_PREFER_LARGE_60 },
/* Acer F51 */
diff --git a/drivers/gpu/drm/exynos/Kconfig b/drivers/gpu/drm/exynos/Kconfig
index 7f50967..59a26e5 100644
--- a/drivers/gpu/drm/exynos/Kconfig
+++ b/drivers/gpu/drm/exynos/Kconfig
@@ -36,6 +36,6 @@
config DRM_EXYNOS_G2D
bool "Exynos DRM G2D"
- depends on DRM_EXYNOS
+ depends on DRM_EXYNOS && !VIDEO_SAMSUNG_S5P_G2D
help
Choose this option if you want to use Exynos G2D for DRM.
diff --git a/drivers/gpu/drm/exynos/exynos_drm_dmabuf.c b/drivers/gpu/drm/exynos/exynos_drm_dmabuf.c
index 613bf8a..ae13feb 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_dmabuf.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_dmabuf.c
@@ -163,6 +163,12 @@
/* TODO */
}
+static int exynos_gem_dmabuf_mmap(struct dma_buf *dma_buf,
+ struct vm_area_struct *vma)
+{
+ return -ENOTTY;
+}
+
static struct dma_buf_ops exynos_dmabuf_ops = {
.map_dma_buf = exynos_gem_map_dma_buf,
.unmap_dma_buf = exynos_gem_unmap_dma_buf,
@@ -170,6 +176,7 @@
.kmap_atomic = exynos_gem_dmabuf_kmap_atomic,
.kunmap = exynos_gem_dmabuf_kunmap,
.kunmap_atomic = exynos_gem_dmabuf_kunmap_atomic,
+ .mmap = exynos_gem_dmabuf_mmap,
.release = exynos_dmabuf_release,
};
diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.c b/drivers/gpu/drm/exynos/exynos_drm_drv.c
index ebacec6..d070719 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_drv.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_drv.c
@@ -160,7 +160,6 @@
if (!file_priv)
return -ENOMEM;
- drm_prime_init_file_private(&file->prime);
file->driver_priv = file_priv;
return exynos_drm_subdrv_open(dev, file);
@@ -184,7 +183,6 @@
e->base.destroy(&e->base);
}
}
- drm_prime_destroy_file_private(&file->prime);
spin_unlock_irqrestore(&dev->event_lock, flags);
exynos_drm_subdrv_close(dev, file);
@@ -241,6 +239,9 @@
.poll = drm_poll,
.read = drm_read,
.unlocked_ioctl = drm_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.release = drm_release,
};
diff --git a/drivers/gpu/drm/exynos/exynos_drm_fimd.c b/drivers/gpu/drm/exynos/exynos_drm_fimd.c
index a68d2b3..b19cd93 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_fimd.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_fimd.c
@@ -831,11 +831,6 @@
}
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res) {
- dev_err(dev, "failed to find registers\n");
- ret = -ENOENT;
- goto err_clk;
- }
ctx->regs = devm_request_and_ioremap(&pdev->dev, res);
if (!ctx->regs) {
diff --git a/drivers/gpu/drm/exynos/exynos_drm_g2d.c b/drivers/gpu/drm/exynos/exynos_drm_g2d.c
index d2d88f2..1065e90 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_g2d.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_g2d.c
@@ -129,7 +129,6 @@
struct g2d_data {
struct device *dev;
struct clk *gate_clk;
- struct resource *regs_res;
void __iomem *regs;
int irq;
struct workqueue_struct *g2d_workq;
@@ -751,7 +750,7 @@
struct exynos_drm_subdrv *subdrv;
int ret;
- g2d = kzalloc(sizeof(*g2d), GFP_KERNEL);
+ g2d = devm_kzalloc(&pdev->dev, sizeof(*g2d), GFP_KERNEL);
if (!g2d) {
dev_err(dev, "failed to allocate driver data\n");
return -ENOMEM;
@@ -759,10 +758,8 @@
g2d->runqueue_slab = kmem_cache_create("g2d_runqueue_slab",
sizeof(struct g2d_runqueue_node), 0, 0, NULL);
- if (!g2d->runqueue_slab) {
- ret = -ENOMEM;
- goto err_free_mem;
- }
+ if (!g2d->runqueue_slab)
+ return -ENOMEM;
g2d->dev = dev;
@@ -794,38 +791,26 @@
pm_runtime_enable(dev);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res) {
- dev_err(dev, "failed to get I/O memory\n");
- ret = -ENOENT;
- goto err_put_clk;
- }
- g2d->regs_res = request_mem_region(res->start, resource_size(res),
- dev_name(dev));
- if (!g2d->regs_res) {
- dev_err(dev, "failed to request I/O memory\n");
- ret = -ENOENT;
- goto err_put_clk;
- }
-
- g2d->regs = ioremap(res->start, resource_size(res));
+ g2d->regs = devm_request_and_ioremap(&pdev->dev, res);
if (!g2d->regs) {
dev_err(dev, "failed to remap I/O memory\n");
ret = -ENXIO;
- goto err_release_res;
+ goto err_put_clk;
}
g2d->irq = platform_get_irq(pdev, 0);
if (g2d->irq < 0) {
dev_err(dev, "failed to get irq\n");
ret = g2d->irq;
- goto err_unmap_base;
+ goto err_put_clk;
}
- ret = request_irq(g2d->irq, g2d_irq_handler, 0, "drm_g2d", g2d);
+ ret = devm_request_irq(&pdev->dev, g2d->irq, g2d_irq_handler, 0,
+ "drm_g2d", g2d);
if (ret < 0) {
dev_err(dev, "irq request failed\n");
- goto err_unmap_base;
+ goto err_put_clk;
}
platform_set_drvdata(pdev, g2d);
@@ -838,7 +823,7 @@
ret = exynos_drm_subdrv_register(subdrv);
if (ret < 0) {
dev_err(dev, "failed to register drm g2d device\n");
- goto err_free_irq;
+ goto err_put_clk;
}
dev_info(dev, "The exynos g2d(ver %d.%d) successfully probed\n",
@@ -846,13 +831,6 @@
return 0;
-err_free_irq:
- free_irq(g2d->irq, g2d);
-err_unmap_base:
- iounmap(g2d->regs);
-err_release_res:
- release_resource(g2d->regs_res);
- kfree(g2d->regs_res);
err_put_clk:
pm_runtime_disable(dev);
clk_put(g2d->gate_clk);
@@ -862,8 +840,6 @@
destroy_workqueue(g2d->g2d_workq);
err_destroy_slab:
kmem_cache_destroy(g2d->runqueue_slab);
-err_free_mem:
- kfree(g2d);
return ret;
}
@@ -873,24 +849,18 @@
cancel_work_sync(&g2d->runqueue_work);
exynos_drm_subdrv_unregister(&g2d->subdrv);
- free_irq(g2d->irq, g2d);
while (g2d->runqueue_node) {
g2d_free_runqueue_node(g2d, g2d->runqueue_node);
g2d->runqueue_node = g2d_get_runqueue_node(g2d);
}
- iounmap(g2d->regs);
- release_resource(g2d->regs_res);
- kfree(g2d->regs_res);
-
pm_runtime_disable(&pdev->dev);
clk_put(g2d->gate_clk);
g2d_fini_cmdlist(g2d);
destroy_workqueue(g2d->g2d_workq);
kmem_cache_destroy(g2d->runqueue_slab);
- kfree(g2d);
return 0;
}
@@ -924,7 +894,7 @@
}
#endif
-SIMPLE_DEV_PM_OPS(g2d_pm_ops, g2d_suspend, g2d_resume);
+static SIMPLE_DEV_PM_OPS(g2d_pm_ops, g2d_suspend, g2d_resume);
struct platform_driver g2d_driver = {
.probe = g2d_probe,
diff --git a/drivers/gpu/drm/exynos/exynos_drm_gem.c b/drivers/gpu/drm/exynos/exynos_drm_gem.c
index f9efde4..a38051c 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_gem.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_gem.c
@@ -122,7 +122,7 @@
__free_page(pages[i]);
drm_free_large(pages);
- return ERR_PTR(PTR_ERR(p));
+ return ERR_CAST(p);
}
static void exynos_gem_put_pages(struct drm_gem_object *obj,
@@ -662,7 +662,7 @@
*/
args->pitch = args->width * ((args->bpp + 7) / 8);
- args->size = PAGE_ALIGN(args->pitch * args->height);
+ args->size = args->pitch * args->height;
exynos_gem_obj = exynos_drm_gem_create(dev, args->flags, args->size);
if (IS_ERR(exynos_gem_obj))
diff --git a/drivers/gpu/drm/exynos/exynos_drm_hdmi.c b/drivers/gpu/drm/exynos/exynos_drm_hdmi.c
index 8ffcdf8..3fdf0b6 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_hdmi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_hdmi.c
@@ -345,7 +345,7 @@
DRM_DEBUG_KMS("%s\n", __FILE__);
- ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ ctx = devm_kzalloc(&pdev->dev, sizeof(*ctx), GFP_KERNEL);
if (!ctx) {
DRM_LOG_KMS("failed to alloc common hdmi context.\n");
return -ENOMEM;
@@ -371,7 +371,6 @@
DRM_DEBUG_KMS("%s\n", __FILE__);
exynos_drm_subdrv_unregister(&ctx->subdrv);
- kfree(ctx);
return 0;
}
diff --git a/drivers/gpu/drm/exynos/exynos_drm_plane.c b/drivers/gpu/drm/exynos/exynos_drm_plane.c
index b89829e..e1f94b7 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_plane.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_plane.c
@@ -29,7 +29,6 @@
DRM_FORMAT_XRGB8888,
DRM_FORMAT_ARGB8888,
DRM_FORMAT_NV12,
- DRM_FORMAT_NV12M,
DRM_FORMAT_NV12MT,
};
diff --git a/drivers/gpu/drm/exynos/exynos_drm_vidi.c b/drivers/gpu/drm/exynos/exynos_drm_vidi.c
index bb1550c..537027a 100644
--- a/drivers/gpu/drm/exynos/exynos_drm_vidi.c
+++ b/drivers/gpu/drm/exynos/exynos_drm_vidi.c
@@ -633,7 +633,7 @@
DRM_DEBUG_KMS("%s\n", __FILE__);
- ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ ctx = devm_kzalloc(&pdev->dev, sizeof(*ctx), GFP_KERNEL);
if (!ctx)
return -ENOMEM;
@@ -673,8 +673,6 @@
ctx->raw_edid = NULL;
}
- kfree(ctx);
-
return 0;
}
diff --git a/drivers/gpu/drm/exynos/exynos_hdmi.c b/drivers/gpu/drm/exynos/exynos_hdmi.c
index 409e2ec..a6aea6f 100644
--- a/drivers/gpu/drm/exynos/exynos_hdmi.c
+++ b/drivers/gpu/drm/exynos/exynos_hdmi.c
@@ -2172,7 +2172,7 @@
DRM_DEBUG_KMS("HDMI resource init\n");
- memset(res, 0, sizeof *res);
+ memset(res, 0, sizeof(*res));
/* get clocks, power */
res->hdmi = clk_get(dev, "hdmi");
@@ -2204,7 +2204,7 @@
clk_set_parent(res->sclk_hdmi, res->sclk_pixel);
res->regul_bulk = kzalloc(ARRAY_SIZE(supply) *
- sizeof res->regul_bulk[0], GFP_KERNEL);
+ sizeof(res->regul_bulk[0]), GFP_KERNEL);
if (!res->regul_bulk) {
DRM_ERROR("failed to get memory for regulators\n");
goto fail;
@@ -2243,7 +2243,7 @@
clk_put(res->sclk_hdmi);
if (!IS_ERR_OR_NULL(res->hdmi))
clk_put(res->hdmi);
- memset(res, 0, sizeof *res);
+ memset(res, 0, sizeof(*res));
return 0;
}
@@ -2312,11 +2312,6 @@
}
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res) {
- DRM_ERROR("failed to find registers\n");
- ret = -ENOENT;
- goto err_resource;
- }
hdata->regs = devm_request_and_ioremap(&pdev->dev, res);
if (!hdata->regs) {
diff --git a/drivers/gpu/drm/exynos/exynos_mixer.c b/drivers/gpu/drm/exynos/exynos_mixer.c
index 30fcc12..25b97d5 100644
--- a/drivers/gpu/drm/exynos/exynos_mixer.c
+++ b/drivers/gpu/drm/exynos/exynos_mixer.c
@@ -236,11 +236,11 @@
static void vp_default_filter(struct mixer_resources *res)
{
vp_filter_set(res, VP_POLY8_Y0_LL,
- filter_y_horiz_tap8, sizeof filter_y_horiz_tap8);
+ filter_y_horiz_tap8, sizeof(filter_y_horiz_tap8));
vp_filter_set(res, VP_POLY4_Y0_LL,
- filter_y_vert_tap4, sizeof filter_y_vert_tap4);
+ filter_y_vert_tap4, sizeof(filter_y_vert_tap4));
vp_filter_set(res, VP_POLY4_C0_LL,
- filter_cr_horiz_tap4, sizeof filter_cr_horiz_tap4);
+ filter_cr_horiz_tap4, sizeof(filter_cr_horiz_tap4));
}
static void mixer_vsync_set_update(struct mixer_context *ctx, bool enable)
diff --git a/drivers/gpu/drm/gma500/oaktrail_device.c b/drivers/gpu/drm/gma500/oaktrail_device.c
index 0f9b7db..cf49ba5 100644
--- a/drivers/gpu/drm/gma500/oaktrail_device.c
+++ b/drivers/gpu/drm/gma500/oaktrail_device.c
@@ -476,6 +476,7 @@
.pos = DSPAPOS,
.surf = DSPASURF,
.addr = MRST_DSPABASE,
+ .base = MRST_DSPABASE,
.status = PIPEASTAT,
.linoff = DSPALINOFF,
.tileoff = DSPATILEOFF,
@@ -499,6 +500,7 @@
.pos = DSPBPOS,
.surf = DSPBSURF,
.addr = DSPBBASE,
+ .base = DSPBBASE,
.status = PIPEBSTAT,
.linoff = DSPBLINOFF,
.tileoff = DSPBTILEOFF,
diff --git a/drivers/gpu/drm/gma500/psb_intel_display.c b/drivers/gpu/drm/gma500/psb_intel_display.c
index 30dc22a..8033526 100644
--- a/drivers/gpu/drm/gma500/psb_intel_display.c
+++ b/drivers/gpu/drm/gma500/psb_intel_display.c
@@ -1362,6 +1362,9 @@
(struct drm_connector **) (psb_intel_crtc + 1);
psb_intel_crtc->mode_set.num_connectors = 0;
psb_intel_cursor_init(dev, psb_intel_crtc);
+
+ /* Set to true so that the pipe is forced off on initial config. */
+ psb_intel_crtc->active = true;
}
int psb_intel_get_pipe_from_crtc_id(struct drm_device *dev, void *data,
diff --git a/drivers/gpu/drm/i810/i810_dma.c b/drivers/gpu/drm/i810/i810_dma.c
index 57d892e..463ec68 100644
--- a/drivers/gpu/drm/i810/i810_dma.c
+++ b/drivers/gpu/drm/i810/i810_dma.c
@@ -115,6 +115,9 @@
.unlocked_ioctl = drm_ioctl,
.mmap = i810_mmap_buffers,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/i810/i810_drv.c b/drivers/gpu/drm/i810/i810_drv.c
index f9924ad..48cfcca 100644
--- a/drivers/gpu/drm/i810/i810_drv.c
+++ b/drivers/gpu/drm/i810/i810_drv.c
@@ -51,6 +51,9 @@
.mmap = drm_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index 9cf7dfe..914c0df 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -1587,6 +1587,7 @@
spin_lock_init(&dev_priv->irq_lock);
spin_lock_init(&dev_priv->error_lock);
spin_lock_init(&dev_priv->rps_lock);
+ spin_lock_init(&dev_priv->dpio_lock);
if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev))
dev_priv->num_pipe = 3;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 489e2b1..274d25d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3242,7 +3242,8 @@
{
int ret;
- BUG_ON(obj->pin_count == DRM_I915_GEM_OBJECT_MAX_PIN_COUNT);
+ if (WARN_ON(obj->pin_count == DRM_I915_GEM_OBJECT_MAX_PIN_COUNT))
+ return -EBUSY;
if (obj->gtt_space != NULL) {
if ((alignment && obj->gtt_offset & (alignment - 1)) ||
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index d9a5372..60815b8 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -72,7 +72,7 @@
/* ppgtt PDEs reside in the global gtt pagetable, which has 512*1024
* entries. For aliasing ppgtt support we just steal them at the end for
* now. */
- first_pd_entry_in_global_pt = 512*1024 - I915_PPGTT_PD_ENTRIES;
+ first_pd_entry_in_global_pt = dev_priv->mm.gtt->gtt_total_entries - I915_PPGTT_PD_ENTRIES;
ppgtt = kzalloc(sizeof(*ppgtt), GFP_KERNEL);
if (!ppgtt)
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index 8a38285..5249640 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -2700,9 +2700,6 @@
dev->driver->irq_handler = i8xx_irq_handler;
dev->driver->irq_uninstall = i8xx_irq_uninstall;
} else if (INTEL_INFO(dev)->gen == 3) {
- /* IIR "flip pending" means done if this bit is set */
- I915_WRITE(ECOSKPD, _MASKED_BIT_DISABLE(ECO_FLIP_DONE));
-
dev->driver->irq_preinstall = i915_irq_preinstall;
dev->driver->irq_postinstall = i915_irq_postinstall;
dev->driver->irq_uninstall = i915_irq_uninstall;
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index a69a3d0..c040aee 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -1376,7 +1376,8 @@
"PCH DP (0x%08x) enabled on transcoder %c, should be disabled\n",
reg, pipe_name(pipe));
- WARN(HAS_PCH_IBX(dev_priv->dev) && (val & SDVO_PIPE_B_SELECT),
+ WARN(HAS_PCH_IBX(dev_priv->dev) && (val & DP_PORT_EN) == 0
+ && (val & DP_PIPEB_SELECT),
"IBX PCH dp port still using transcoder B\n");
}
@@ -1384,11 +1385,12 @@
enum pipe pipe, int reg)
{
u32 val = I915_READ(reg);
- WARN(hdmi_pipe_enabled(dev_priv, val, pipe),
+ WARN(hdmi_pipe_enabled(dev_priv, pipe, val),
"PCH HDMI (0x%08x) enabled on transcoder %c, should be disabled\n",
reg, pipe_name(pipe));
- WARN(HAS_PCH_IBX(dev_priv->dev) && (val & SDVO_PIPE_B_SELECT),
+ WARN(HAS_PCH_IBX(dev_priv->dev) && (val & PORT_ENABLE) == 0
+ && (val & SDVO_PIPE_B_SELECT),
"IBX PCH hdmi port still using transcoder B\n");
}
@@ -1404,13 +1406,13 @@
reg = PCH_ADPA;
val = I915_READ(reg);
- WARN(adpa_pipe_enabled(dev_priv, val, pipe),
+ WARN(adpa_pipe_enabled(dev_priv, pipe, val),
"PCH VGA enabled on transcoder %c, should be disabled\n",
pipe_name(pipe));
reg = PCH_LVDS;
val = I915_READ(reg);
- WARN(lvds_pipe_enabled(dev_priv, val, pipe),
+ WARN(lvds_pipe_enabled(dev_priv, pipe, val),
"PCH LVDS enabled on transcoder %c, should be disabled\n",
pipe_name(pipe));
@@ -1872,7 +1874,7 @@
enum pipe pipe, int reg)
{
u32 val = I915_READ(reg);
- if (hdmi_pipe_enabled(dev_priv, val, pipe)) {
+ if (hdmi_pipe_enabled(dev_priv, pipe, val)) {
DRM_DEBUG_KMS("Disabling pch HDMI %x on pipe %d\n",
reg, pipe);
I915_WRITE(reg, val & ~PORT_ENABLE);
@@ -1894,12 +1896,12 @@
reg = PCH_ADPA;
val = I915_READ(reg);
- if (adpa_pipe_enabled(dev_priv, val, pipe))
+ if (adpa_pipe_enabled(dev_priv, pipe, val))
I915_WRITE(reg, val & ~ADPA_DAC_ENABLE);
reg = PCH_LVDS;
val = I915_READ(reg);
- if (lvds_pipe_enabled(dev_priv, val, pipe)) {
+ if (lvds_pipe_enabled(dev_priv, pipe, val)) {
DRM_DEBUG_KMS("disable lvds on pipe %d val 0x%08x\n", pipe, val);
I915_WRITE(reg, val & ~LVDS_PORT_EN);
POSTING_READ(reg);
@@ -4189,12 +4191,6 @@
POSTING_READ(DPLL(pipe));
udelay(150);
- I915_WRITE(DPLL(pipe), dpll);
-
- /* Wait for the clocks to stabilize. */
- POSTING_READ(DPLL(pipe));
- udelay(150);
-
/* The LVDS pin pair needs to be on before the DPLLs are enabled.
* This is an exception to the general rule that mode_set doesn't turn
* things on.
@@ -4202,6 +4198,12 @@
if (intel_pipe_has_type(crtc, INTEL_OUTPUT_LVDS))
intel_update_lvds(crtc, clock, adjusted_mode);
+ I915_WRITE(DPLL(pipe), dpll);
+
+ /* Wait for the clocks to stabilize. */
+ POSTING_READ(DPLL(pipe));
+ udelay(150);
+
/* The pixel multiplier can only be updated once the
* DPLL is enabled and the clocks are stable.
*
diff --git a/drivers/gpu/drm/i915/intel_dp.c b/drivers/gpu/drm/i915/intel_dp.c
index a6c426a..ace757a 100644
--- a/drivers/gpu/drm/i915/intel_dp.c
+++ b/drivers/gpu/drm/i915/intel_dp.c
@@ -2533,14 +2533,10 @@
break;
}
- intel_dp_i2c_init(intel_dp, intel_connector, name);
-
/* Cache some DPCD data in the eDP case */
if (is_edp(intel_dp)) {
- bool ret;
struct edp_power_seq cur, vbt;
u32 pp_on, pp_off, pp_div;
- struct edid *edid;
pp_on = I915_READ(PCH_PP_ON_DELAYS);
pp_off = I915_READ(PCH_PP_OFF_DELAYS);
@@ -2591,6 +2587,13 @@
DRM_DEBUG_KMS("backlight on delay %d, off delay %d\n",
intel_dp->backlight_on_delay, intel_dp->backlight_off_delay);
+ }
+
+ intel_dp_i2c_init(intel_dp, intel_connector, name);
+
+ if (is_edp(intel_dp)) {
+ bool ret;
+ struct edid *edid;
ironlake_edp_panel_vdd_on(intel_dp);
ret = intel_dp_get_dpcd(intel_dp);
diff --git a/drivers/gpu/drm/i915/intel_hdmi.c b/drivers/gpu/drm/i915/intel_hdmi.c
index 98f6024..12dc330 100644
--- a/drivers/gpu/drm/i915/intel_hdmi.c
+++ b/drivers/gpu/drm/i915/intel_hdmi.c
@@ -609,7 +609,7 @@
u32 temp;
u32 enable_bits = SDVO_ENABLE;
- if (intel_hdmi->has_audio)
+ if (intel_hdmi->has_audio || mode != DRM_MODE_DPMS_ON)
enable_bits |= SDVO_AUDIO_ENABLE;
temp = I915_READ(intel_hdmi->sdvox_reg);
diff --git a/drivers/gpu/drm/i915/intel_lvds.c b/drivers/gpu/drm/i915/intel_lvds.c
index e05c0d3..e9a6f6a 100644
--- a/drivers/gpu/drm/i915/intel_lvds.c
+++ b/drivers/gpu/drm/i915/intel_lvds.c
@@ -780,6 +780,14 @@
DMI_MATCH(DMI_BOARD_NAME, "ZBOXSD-ID12/ID13"),
},
},
+ {
+ .callback = intel_no_lvds_dmi_callback,
+ .ident = "Gigabyte GA-D525TUD",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "Gigabyte Technology Co., Ltd."),
+ DMI_MATCH(DMI_BOARD_NAME, "D525TUD"),
+ },
+ },
{ } /* terminating entry */
};
diff --git a/drivers/gpu/drm/i915/intel_panel.c b/drivers/gpu/drm/i915/intel_panel.c
index 3df4f5f..e019b23 100644
--- a/drivers/gpu/drm/i915/intel_panel.c
+++ b/drivers/gpu/drm/i915/intel_panel.c
@@ -162,19 +162,12 @@
return val;
}
-u32 intel_panel_get_max_backlight(struct drm_device *dev)
+static u32 _intel_panel_get_max_backlight(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
u32 max;
max = i915_read_blc_pwm_ctl(dev_priv);
- if (max == 0) {
- /* XXX add code here to query mode clock or hardware clock
- * and program max PWM appropriately.
- */
- pr_warn_once("fixme: max PWM is zero\n");
- return 1;
- }
if (HAS_PCH_SPLIT(dev)) {
max >>= 16;
@@ -188,6 +181,22 @@
max *= 0xff;
}
+ return max;
+}
+
+u32 intel_panel_get_max_backlight(struct drm_device *dev)
+{
+ u32 max;
+
+ max = _intel_panel_get_max_backlight(dev);
+ if (max == 0) {
+ /* XXX add code here to query mode clock or hardware clock
+ * and program max PWM appropriately.
+ */
+ pr_warn_once("fixme: max PWM is zero\n");
+ return 1;
+ }
+
DRM_DEBUG_DRIVER("max backlight PWM = %d\n", max);
return max;
}
@@ -424,7 +433,11 @@
memset(&props, 0, sizeof(props));
props.type = BACKLIGHT_RAW;
- props.max_brightness = intel_panel_get_max_backlight(dev);
+ props.max_brightness = _intel_panel_get_max_backlight(dev);
+ if (props.max_brightness == 0) {
+ DRM_ERROR("Failed to get maximum backlight value\n");
+ return -ENODEV;
+ }
dev_priv->backlight =
backlight_device_register("intel_backlight",
&connector->kdev, dev,
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 1881c8c..ba8a27b 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -3672,6 +3672,9 @@
if (IS_PINEVIEW(dev))
I915_WRITE(ECOSKPD, _MASKED_BIT_ENABLE(ECO_GATING_CX_ONLY));
+
+ /* IIR "flip pending" means done if this bit is set */
+ I915_WRITE(ECOSKPD, _MASKED_BIT_DISABLE(ECO_FLIP_DONE));
}
static void i85x_init_clock_gating(struct drm_device *dev)
diff --git a/drivers/gpu/drm/i915/intel_sdvo.c b/drivers/gpu/drm/i915/intel_sdvo.c
index d81bb0b..123afd3 100644
--- a/drivers/gpu/drm/i915/intel_sdvo.c
+++ b/drivers/gpu/drm/i915/intel_sdvo.c
@@ -2573,7 +2573,6 @@
hotplug_mask = intel_sdvo->is_sdvob ?
SDVOB_HOTPLUG_INT_STATUS_I915 : SDVOC_HOTPLUG_INT_STATUS_I915;
}
- dev_priv->hotplug_supported_mask |= hotplug_mask;
drm_encoder_helper_add(&intel_encoder->base, &intel_sdvo_helper_funcs);
@@ -2581,14 +2580,6 @@
if (!intel_sdvo_get_capabilities(intel_sdvo, &intel_sdvo->caps))
goto err;
- /* Set up hotplug command - note paranoia about contents of reply.
- * We assume that the hardware is in a sane state, and only touch
- * the bits we think we understand.
- */
- intel_sdvo_get_value(intel_sdvo, SDVO_CMD_GET_ACTIVE_HOT_PLUG,
- &intel_sdvo->hotplug_active, 2);
- intel_sdvo->hotplug_active[0] &= ~0x3;
-
if (intel_sdvo_output_setup(intel_sdvo,
intel_sdvo->caps.output_flags) != true) {
DRM_DEBUG_KMS("SDVO output failed to setup on %s\n",
@@ -2596,6 +2587,12 @@
goto err;
}
+ /* Only enable the hotplug irq if we need it, to work around noisy
+ * hotplug lines.
+ */
+ if (intel_sdvo->hotplug_active[0])
+ dev_priv->hotplug_supported_mask |= hotplug_mask;
+
intel_sdvo_select_ddc_bus(dev_priv, intel_sdvo, sdvo_reg);
/* Set the input timing to the screen. Assume always input 0. */
diff --git a/drivers/gpu/drm/i915/intel_sprite.c b/drivers/gpu/drm/i915/intel_sprite.c
index cc8df4d..7644f31 100644
--- a/drivers/gpu/drm/i915/intel_sprite.c
+++ b/drivers/gpu/drm/i915/intel_sprite.c
@@ -60,11 +60,11 @@
switch (fb->pixel_format) {
case DRM_FORMAT_XBGR8888:
- sprctl |= SPRITE_FORMAT_RGBX888;
+ sprctl |= SPRITE_FORMAT_RGBX888 | SPRITE_RGB_ORDER_RGBX;
pixel_size = 4;
break;
case DRM_FORMAT_XRGB8888:
- sprctl |= SPRITE_FORMAT_RGBX888 | SPRITE_RGB_ORDER_RGBX;
+ sprctl |= SPRITE_FORMAT_RGBX888;
pixel_size = 4;
break;
case DRM_FORMAT_YUYV:
diff --git a/drivers/gpu/drm/mgag200/mgag200_drv.c b/drivers/gpu/drm/mgag200/mgag200_drv.c
index ea1024d..e5f145d 100644
--- a/drivers/gpu/drm/mgag200/mgag200_drv.c
+++ b/drivers/gpu/drm/mgag200/mgag200_drv.c
@@ -84,6 +84,9 @@
.mmap = mgag200_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.read = drm_read,
};
diff --git a/drivers/gpu/drm/nouveau/nouveau_abi16.c b/drivers/gpu/drm/nouveau/nouveau_abi16.c
index ff23d88..3ca240b 100644
--- a/drivers/gpu/drm/nouveau/nouveau_abi16.c
+++ b/drivers/gpu/drm/nouveau/nouveau_abi16.c
@@ -179,7 +179,7 @@
return 0;
} else
if (init->class == 0x906e) {
- NV_ERROR(dev, "906e not supported yet\n");
+ NV_DEBUG(dev, "906e not supported yet\n");
return -EINVAL;
}
diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index 69688ef..7e16dc5 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -598,7 +598,7 @@
args->size = args->pitch * args->height;
args->size = roundup(args->size, PAGE_SIZE);
- ret = nouveau_gem_new(dev, args->size, 0, TTM_PL_FLAG_VRAM, 0, 0, &bo);
+ ret = nouveau_gem_new(dev, args->size, 0, NOUVEAU_GEM_DOMAIN_VRAM, 0, 0, &bo);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/nouveau/nouveau_state.c b/drivers/gpu/drm/nouveau/nouveau_state.c
index 1866dbb..c610144 100644
--- a/drivers/gpu/drm/nouveau/nouveau_state.c
+++ b/drivers/gpu/drm/nouveau/nouveau_state.c
@@ -736,9 +736,11 @@
}
break;
case NV_C0:
- nvc0_copy_create(dev, 1);
+ if (!(nv_rd32(dev, 0x022500) & 0x00000200))
+ nvc0_copy_create(dev, 1);
case NV_D0:
- nvc0_copy_create(dev, 0);
+ if (!(nv_rd32(dev, 0x022500) & 0x00000100))
+ nvc0_copy_create(dev, 0);
break;
default:
break;
diff --git a/drivers/gpu/drm/nouveau/nv50_gpio.c b/drivers/gpu/drm/nouveau/nv50_gpio.c
index f429e6a..c399d51 100644
--- a/drivers/gpu/drm/nouveau/nv50_gpio.c
+++ b/drivers/gpu/drm/nouveau/nv50_gpio.c
@@ -22,6 +22,7 @@
* Authors: Ben Skeggs
*/
+#include <linux/dmi.h>
#include "drmP.h"
#include "nouveau_drv.h"
#include "nouveau_hw.h"
@@ -110,11 +111,26 @@
nv_wr32(dev, 0xe074, intr1);
}
+static struct dmi_system_id gpio_reset_ids[] = {
+ {
+ .ident = "Apple Macbook 10,1",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Apple Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "MacBookPro10,1"),
+ }
+ },
+ { }
+};
+
int
nv50_gpio_init(struct drm_device *dev)
{
struct drm_nouveau_private *dev_priv = dev->dev_private;
+ /* initialise gpios and routing to vbios defaults */
+ if (dmi_check_system(gpio_reset_ids))
+ nouveau_gpio_reset(dev);
+
/* disable, and ack any pending gpio interrupts */
nv_wr32(dev, 0xe050, 0x00000000);
nv_wr32(dev, 0xe054, 0xffffffff);
diff --git a/drivers/gpu/drm/nouveau/nvc0_fb.c b/drivers/gpu/drm/nouveau/nvc0_fb.c
index f704e94..f376c39 100644
--- a/drivers/gpu/drm/nouveau/nvc0_fb.c
+++ b/drivers/gpu/drm/nouveau/nvc0_fb.c
@@ -124,6 +124,7 @@
priv = dev_priv->engine.fb.priv;
nv_wr32(dev, 0x100c10, priv->r100c10 >> 8);
+ nv_mask(dev, 0x17e820, 0x00100000, 0x00000000); /* NV_PLTCG_INTR_EN */
return 0;
}
diff --git a/drivers/gpu/drm/nouveau/nvc0_fifo.c b/drivers/gpu/drm/nouveau/nvc0_fifo.c
index 7d85553..cd39eb9 100644
--- a/drivers/gpu/drm/nouveau/nvc0_fifo.c
+++ b/drivers/gpu/drm/nouveau/nvc0_fifo.c
@@ -373,7 +373,8 @@
static void
nvc0_fifo_isr(struct drm_device *dev)
{
- u32 stat = nv_rd32(dev, 0x002100);
+ u32 mask = nv_rd32(dev, 0x002140);
+ u32 stat = nv_rd32(dev, 0x002100) & mask;
if (stat & 0x00000100) {
NV_INFO(dev, "PFIFO: unknown status 0x00000100\n");
diff --git a/drivers/gpu/drm/nouveau/nvd0_display.c b/drivers/gpu/drm/nouveau/nvd0_display.c
index dac525b..8a2fc89 100644
--- a/drivers/gpu/drm/nouveau/nvd0_display.c
+++ b/drivers/gpu/drm/nouveau/nvd0_display.c
@@ -1510,10 +1510,10 @@
case OUTPUT_DP:
if (nv_connector->base.display_info.bpc == 6) {
nv_encoder->dp.datarate = mode->clock * 18 / 8;
- syncs |= 0x00000140;
+ syncs |= 0x00000002 << 6;
} else {
nv_encoder->dp.datarate = mode->clock * 24 / 8;
- syncs |= 0x00000180;
+ syncs |= 0x00000005 << 6;
}
if (nv_encoder->dcb->sorconf.link & 1)
diff --git a/drivers/gpu/drm/nouveau/nve0_fifo.c b/drivers/gpu/drm/nouveau/nve0_fifo.c
index e98d144..281bece 100644
--- a/drivers/gpu/drm/nouveau/nve0_fifo.c
+++ b/drivers/gpu/drm/nouveau/nve0_fifo.c
@@ -345,7 +345,8 @@
static void
nve0_fifo_isr(struct drm_device *dev)
{
- u32 stat = nv_rd32(dev, 0x002100);
+ u32 mask = nv_rd32(dev, 0x002140);
+ u32 stat = nv_rd32(dev, 0x002100) & mask;
if (stat & 0x00000100) {
NV_INFO(dev, "PFIFO: unknown status 0x00000100\n");
diff --git a/drivers/gpu/drm/radeon/atombios_crtc.c b/drivers/gpu/drm/radeon/atombios_crtc.c
index f4d4505..2817101 100644
--- a/drivers/gpu/drm/radeon/atombios_crtc.c
+++ b/drivers/gpu/drm/radeon/atombios_crtc.c
@@ -258,7 +258,6 @@
radeon_crtc->enabled = true;
/* adjust pm to dpms changes BEFORE enabling crtcs */
radeon_pm_compute_clocks(rdev);
- /* disable crtc pair power gating before programming */
if (ASIC_IS_DCE6(rdev) && !radeon_crtc->in_mode_set)
atombios_powergate_crtc(crtc, ATOM_DISABLE);
atombios_enable_crtc(crtc, ATOM_ENABLE);
@@ -278,25 +277,8 @@
atombios_enable_crtc_memreq(crtc, ATOM_DISABLE);
atombios_enable_crtc(crtc, ATOM_DISABLE);
radeon_crtc->enabled = false;
- /* power gating is per-pair */
- if (ASIC_IS_DCE6(rdev) && !radeon_crtc->in_mode_set) {
- struct drm_crtc *other_crtc;
- struct radeon_crtc *other_radeon_crtc;
- list_for_each_entry(other_crtc, &rdev->ddev->mode_config.crtc_list, head) {
- other_radeon_crtc = to_radeon_crtc(other_crtc);
- if (((radeon_crtc->crtc_id == 0) && (other_radeon_crtc->crtc_id == 1)) ||
- ((radeon_crtc->crtc_id == 1) && (other_radeon_crtc->crtc_id == 0)) ||
- ((radeon_crtc->crtc_id == 2) && (other_radeon_crtc->crtc_id == 3)) ||
- ((radeon_crtc->crtc_id == 3) && (other_radeon_crtc->crtc_id == 2)) ||
- ((radeon_crtc->crtc_id == 4) && (other_radeon_crtc->crtc_id == 5)) ||
- ((radeon_crtc->crtc_id == 5) && (other_radeon_crtc->crtc_id == 4))) {
- /* if both crtcs in the pair are off, enable power gating */
- if (other_radeon_crtc->enabled == false)
- atombios_powergate_crtc(crtc, ATOM_ENABLE);
- break;
- }
- }
- }
+ if (ASIC_IS_DCE6(rdev) && !radeon_crtc->in_mode_set)
+ atombios_powergate_crtc(crtc, ATOM_ENABLE);
/* adjust pm to dpms changes AFTER disabling crtcs */
radeon_pm_compute_clocks(rdev);
break;
@@ -1682,9 +1664,22 @@
struct drm_device *dev = crtc->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_atom_ss ss;
+ int i;
atombios_crtc_dpms(crtc, DRM_MODE_DPMS_OFF);
+ for (i = 0; i < rdev->num_crtc; i++) {
+ if (rdev->mode_info.crtcs[i] &&
+ rdev->mode_info.crtcs[i]->enabled &&
+ i != radeon_crtc->crtc_id &&
+ radeon_crtc->pll_id == rdev->mode_info.crtcs[i]->pll_id) {
+ /* one other crtc is using this pll don't turn
+ * off the pll
+ */
+ goto done;
+ }
+ }
+
switch (radeon_crtc->pll_id) {
case ATOM_PPLL1:
case ATOM_PPLL2:
@@ -1701,6 +1696,7 @@
default:
break;
}
+done:
radeon_crtc->pll_id = -1;
}
diff --git a/drivers/gpu/drm/radeon/atombios_dp.c b/drivers/gpu/drm/radeon/atombios_dp.c
index 7712cf5..3623b98 100644
--- a/drivers/gpu/drm/radeon/atombios_dp.c
+++ b/drivers/gpu/drm/radeon/atombios_dp.c
@@ -577,30 +577,25 @@
struct radeon_device *rdev = dev->dev_private;
struct radeon_connector *radeon_connector = to_radeon_connector(connector);
int panel_mode = DP_PANEL_MODE_EXTERNAL_DP_MODE;
+ u16 dp_bridge = radeon_connector_encoder_get_dp_bridge_encoder_id(connector);
+ u8 tmp;
if (!ASIC_IS_DCE4(rdev))
return panel_mode;
- if (radeon_connector_encoder_get_dp_bridge_encoder_id(connector) ==
- ENCODER_OBJECT_ID_NUTMEG)
- panel_mode = DP_PANEL_MODE_INTERNAL_DP1_MODE;
- else if (radeon_connector_encoder_get_dp_bridge_encoder_id(connector) ==
- ENCODER_OBJECT_ID_TRAVIS) {
- u8 id[6];
- int i;
- for (i = 0; i < 6; i++)
- id[i] = radeon_read_dpcd_reg(radeon_connector, 0x503 + i);
- if (id[0] == 0x73 &&
- id[1] == 0x69 &&
- id[2] == 0x76 &&
- id[3] == 0x61 &&
- id[4] == 0x72 &&
- id[5] == 0x54)
+ if (dp_bridge != ENCODER_OBJECT_ID_NONE) {
+ /* DP bridge chips */
+ tmp = radeon_read_dpcd_reg(radeon_connector, DP_EDP_CONFIGURATION_CAP);
+ if (tmp & 1)
+ panel_mode = DP_PANEL_MODE_INTERNAL_DP2_MODE;
+ else if ((dp_bridge == ENCODER_OBJECT_ID_NUTMEG) ||
+ (dp_bridge == ENCODER_OBJECT_ID_TRAVIS))
panel_mode = DP_PANEL_MODE_INTERNAL_DP1_MODE;
else
- panel_mode = DP_PANEL_MODE_INTERNAL_DP2_MODE;
+ panel_mode = DP_PANEL_MODE_EXTERNAL_DP_MODE;
} else if (connector->connector_type == DRM_MODE_CONNECTOR_eDP) {
- u8 tmp = radeon_read_dpcd_reg(radeon_connector, DP_EDP_CONFIGURATION_CAP);
+ /* eDP */
+ tmp = radeon_read_dpcd_reg(radeon_connector, DP_EDP_CONFIGURATION_CAP);
if (tmp & 1)
panel_mode = DP_PANEL_MODE_INTERNAL_DP2_MODE;
}
diff --git a/drivers/gpu/drm/radeon/atombios_encoders.c b/drivers/gpu/drm/radeon/atombios_encoders.c
index f9bc27f..6e8803a 100644
--- a/drivers/gpu/drm/radeon/atombios_encoders.c
+++ b/drivers/gpu/drm/radeon/atombios_encoders.c
@@ -1379,6 +1379,8 @@
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
+ struct drm_encoder *ext_encoder = radeon_get_external_encoder(encoder);
+ struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
struct radeon_connector *radeon_connector = NULL;
struct radeon_connector_atom_dig *radeon_dig_connector = NULL;
@@ -1390,19 +1392,37 @@
switch (mode) {
case DRM_MODE_DPMS_ON:
- /* some early dce3.2 boards have a bug in their transmitter control table */
- if ((rdev->family == CHIP_RV710) || (rdev->family == CHIP_RV730) ||
- ASIC_IS_DCE41(rdev) || ASIC_IS_DCE5(rdev)) {
- if (ASIC_IS_DCE6(rdev)) {
- /* It seems we need to call ATOM_ENCODER_CMD_SETUP again
- * before reenabling encoder on DPMS ON, otherwise we never
- * get picture
- */
- atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_SETUP, 0);
+ if (ASIC_IS_DCE41(rdev) || ASIC_IS_DCE5(rdev)) {
+ if (!connector)
+ dig->panel_mode = DP_PANEL_MODE_EXTERNAL_DP_MODE;
+ else
+ dig->panel_mode = radeon_dp_get_panel_mode(encoder, connector);
+
+ /* setup and enable the encoder */
+ atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_SETUP, 0);
+ atombios_dig_encoder_setup(encoder,
+ ATOM_ENCODER_CMD_SETUP_PANEL_MODE,
+ dig->panel_mode);
+ if (ext_encoder) {
+ if (ASIC_IS_DCE41(rdev) || ASIC_IS_DCE61(rdev))
+ atombios_external_encoder_setup(encoder, ext_encoder,
+ EXTERNAL_ENCODER_ACTION_V3_ENCODER_SETUP);
}
atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE, 0, 0);
- } else {
+ } else if (ASIC_IS_DCE4(rdev)) {
+ /* setup and enable the encoder */
+ atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_SETUP, 0);
+ /* enable the transmitter */
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE, 0, 0);
atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE_OUTPUT, 0, 0);
+ } else {
+ /* setup and enable the encoder and transmitter */
+ atombios_dig_encoder_setup(encoder, ATOM_ENABLE, 0);
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_SETUP, 0, 0);
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE, 0, 0);
+ /* some early dce3.2 boards have a bug in their transmitter control table */
+ if ((rdev->family != CHIP_RV710) || (rdev->family != CHIP_RV730))
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE_OUTPUT, 0, 0);
}
if (ENCODER_MODE_IS_DP(atombios_get_encoder_mode(encoder)) && connector) {
if (connector->connector_type == DRM_MODE_CONNECTOR_eDP) {
@@ -1420,10 +1440,19 @@
case DRM_MODE_DPMS_STANDBY:
case DRM_MODE_DPMS_SUSPEND:
case DRM_MODE_DPMS_OFF:
- if (ASIC_IS_DCE41(rdev) || ASIC_IS_DCE5(rdev))
+ if (ASIC_IS_DCE41(rdev) || ASIC_IS_DCE5(rdev)) {
+ /* disable the transmitter */
atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
- else
+ } else if (ASIC_IS_DCE4(rdev)) {
+ /* disable the transmitter */
atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE_OUTPUT, 0, 0);
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
+ } else {
+ /* disable the encoder and transmitter */
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE_OUTPUT, 0, 0);
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
+ atombios_dig_encoder_setup(encoder, ATOM_DISABLE, 0);
+ }
if (ENCODER_MODE_IS_DP(atombios_get_encoder_mode(encoder)) && connector) {
if (ASIC_IS_DCE4(rdev))
atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_DP_VIDEO_OFF, 0);
@@ -1740,13 +1769,34 @@
struct radeon_crtc *radeon_crtc = to_radeon_crtc(encoder->crtc);
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
struct drm_encoder *test_encoder;
- struct radeon_encoder_atom_dig *dig;
+ struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
uint32_t dig_enc_in_use = 0;
- /* DCE4/5 */
- if (ASIC_IS_DCE4(rdev)) {
- dig = radeon_encoder->enc_priv;
- if (ASIC_IS_DCE41(rdev)) {
+ if (ASIC_IS_DCE6(rdev)) {
+ /* DCE6 */
+ switch (radeon_encoder->encoder_id) {
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
+ if (dig->linkb)
+ return 1;
+ else
+ return 0;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
+ if (dig->linkb)
+ return 3;
+ else
+ return 2;
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
+ if (dig->linkb)
+ return 5;
+ else
+ return 4;
+ break;
+ }
+ } else if (ASIC_IS_DCE4(rdev)) {
+ /* DCE4/5 */
+ if (ASIC_IS_DCE41(rdev) && !ASIC_IS_DCE61(rdev)) {
/* ontario follows DCE4 */
if (rdev->family == CHIP_PALM) {
if (dig->linkb)
@@ -1848,10 +1898,12 @@
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
- struct drm_encoder *ext_encoder = radeon_get_external_encoder(encoder);
radeon_encoder->pixel_clock = adjusted_mode->clock;
+ /* need to call this here rather than in prepare() since we need some crtc info */
+ radeon_atom_encoder_dpms(encoder, DRM_MODE_DPMS_OFF);
+
if (ASIC_IS_AVIVO(rdev) && !ASIC_IS_DCE4(rdev)) {
if (radeon_encoder->active_device & (ATOM_DEVICE_CV_SUPPORT | ATOM_DEVICE_TV_SUPPORT))
atombios_yuv_setup(encoder, true);
@@ -1870,38 +1922,7 @@
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
- if (ASIC_IS_DCE41(rdev) || ASIC_IS_DCE5(rdev)) {
- struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
- struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
-
- if (!connector)
- dig->panel_mode = DP_PANEL_MODE_EXTERNAL_DP_MODE;
- else
- dig->panel_mode = radeon_dp_get_panel_mode(encoder, connector);
-
- /* setup and enable the encoder */
- atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_SETUP, 0);
- atombios_dig_encoder_setup(encoder,
- ATOM_ENCODER_CMD_SETUP_PANEL_MODE,
- dig->panel_mode);
- } else if (ASIC_IS_DCE4(rdev)) {
- /* disable the transmitter */
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
- /* setup and enable the encoder */
- atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_SETUP, 0);
-
- /* enable the transmitter */
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE, 0, 0);
- } else {
- /* disable the encoder and transmitter */
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
- atombios_dig_encoder_setup(encoder, ATOM_DISABLE, 0);
-
- /* setup and enable the encoder and transmitter */
- atombios_dig_encoder_setup(encoder, ATOM_ENABLE, 0);
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_SETUP, 0, 0);
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE, 0, 0);
- }
+ /* handled in dpms */
break;
case ENCODER_OBJECT_ID_INTERNAL_DDI:
case ENCODER_OBJECT_ID_INTERNAL_DVO1:
@@ -1922,14 +1943,6 @@
break;
}
- if (ext_encoder) {
- if (ASIC_IS_DCE41(rdev) || ASIC_IS_DCE61(rdev))
- atombios_external_encoder_setup(encoder, ext_encoder,
- EXTERNAL_ENCODER_ACTION_V3_ENCODER_SETUP);
- else
- atombios_external_encoder_setup(encoder, ext_encoder, ATOM_ENABLE);
- }
-
atombios_apply_encoder_quirks(encoder, adjusted_mode);
if (atombios_get_encoder_mode(encoder) == ATOM_ENCODER_MODE_HDMI) {
@@ -2116,7 +2129,6 @@
}
radeon_atom_output_lock(encoder, true);
- radeon_atom_encoder_dpms(encoder, DRM_MODE_DPMS_OFF);
if (connector) {
struct radeon_connector *radeon_connector = to_radeon_connector(connector);
@@ -2137,6 +2149,7 @@
static void radeon_atom_encoder_commit(struct drm_encoder *encoder)
{
+ /* need to call this here as we need the crtc set up */
radeon_atom_encoder_dpms(encoder, DRM_MODE_DPMS_ON);
radeon_atom_output_lock(encoder, false);
}
@@ -2177,14 +2190,7 @@
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
- if (ASIC_IS_DCE4(rdev))
- /* disable the transmitter */
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
- else {
- /* disable the encoder and transmitter */
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE, 0, 0);
- atombios_dig_encoder_setup(encoder, ATOM_DISABLE, 0);
- }
+ /* handled in dpms */
break;
case ENCODER_OBJECT_ID_INTERNAL_DDI:
case ENCODER_OBJECT_ID_INTERNAL_DVO1:
diff --git a/drivers/gpu/drm/radeon/evergreen.c b/drivers/gpu/drm/radeon/evergreen.c
index e93b80a..ed3340a 100644
--- a/drivers/gpu/drm/radeon/evergreen.c
+++ b/drivers/gpu/drm/radeon/evergreen.c
@@ -77,13 +77,9 @@
void evergreen_fix_pci_max_read_req_size(struct radeon_device *rdev)
{
u16 ctl, v;
- int cap, err;
+ int err;
- cap = pci_pcie_cap(rdev->pdev);
- if (!cap)
- return;
-
- err = pci_read_config_word(rdev->pdev, cap + PCI_EXP_DEVCTL, &ctl);
+ err = pcie_capability_read_word(rdev->pdev, PCI_EXP_DEVCTL, &ctl);
if (err)
return;
@@ -95,7 +91,7 @@
if ((v == 0) || (v == 6) || (v == 7)) {
ctl &= ~PCI_EXP_DEVCTL_READRQ;
ctl |= (2 << 12);
- pci_write_config_word(rdev->pdev, cap + PCI_EXP_DEVCTL, ctl);
+ pcie_capability_write_word(rdev->pdev, PCI_EXP_DEVCTL, ctl);
}
}
diff --git a/drivers/gpu/drm/radeon/r100.c b/drivers/gpu/drm/radeon/r100.c
index 8acb34f..8d7e33a 100644
--- a/drivers/gpu/drm/radeon/r100.c
+++ b/drivers/gpu/drm/radeon/r100.c
@@ -1182,7 +1182,8 @@
ring->ready = true;
radeon_ttm_set_active_vram_size(rdev, rdev->mc.real_vram_size);
- if (radeon_ring_supports_scratch_reg(rdev, ring)) {
+ if (!ring->rptr_save_reg /* not resuming from suspend */
+ && radeon_ring_supports_scratch_reg(rdev, ring)) {
r = radeon_scratch_get(rdev, &ring->rptr_save_reg);
if (r) {
DRM_ERROR("failed to get scratch reg for rptr save (%d).\n", r);
diff --git a/drivers/gpu/drm/radeon/r600_cs.c b/drivers/gpu/drm/radeon/r600_cs.c
index ab74e6b..f37676d 100644
--- a/drivers/gpu/drm/radeon/r600_cs.c
+++ b/drivers/gpu/drm/radeon/r600_cs.c
@@ -63,6 +63,7 @@
u32 cb_color_size_idx[8]; /* unused */
u32 cb_target_mask;
u32 cb_shader_mask; /* unused */
+ bool is_resolve;
u32 cb_color_size[8];
u32 vgt_strmout_en;
u32 vgt_strmout_buffer_en;
@@ -315,7 +316,15 @@
track->cb_color_bo[i] = NULL;
track->cb_color_bo_offset[i] = 0xFFFFFFFF;
track->cb_color_bo_mc[i] = 0xFFFFFFFF;
+ track->cb_color_frag_bo[i] = NULL;
+ track->cb_color_frag_offset[i] = 0xFFFFFFFF;
+ track->cb_color_tile_bo[i] = NULL;
+ track->cb_color_tile_offset[i] = 0xFFFFFFFF;
+ track->cb_color_mask[i] = 0xFFFFFFFF;
}
+ track->is_resolve = false;
+ track->nsamples = 16;
+ track->log_nsamples = 4;
track->cb_target_mask = 0xFFFFFFFF;
track->cb_shader_mask = 0xFFFFFFFF;
track->cb_dirty = true;
@@ -352,6 +361,8 @@
volatile u32 *ib = p->ib.ptr;
unsigned array_mode;
u32 format;
+ /* When resolve is used, the second colorbuffer has always 1 sample. */
+ unsigned nsamples = track->is_resolve && i == 1 ? 1 : track->nsamples;
size = radeon_bo_size(track->cb_color_bo[i]) - track->cb_color_bo_offset[i];
format = G_0280A0_FORMAT(track->cb_color_info[i]);
@@ -375,7 +386,7 @@
array_check.group_size = track->group_size;
array_check.nbanks = track->nbanks;
array_check.npipes = track->npipes;
- array_check.nsamples = track->nsamples;
+ array_check.nsamples = nsamples;
array_check.blocksize = r600_fmt_get_blocksize(format);
if (r600_get_array_mode_alignment(&array_check,
&pitch_align, &height_align, &depth_align, &base_align)) {
@@ -421,7 +432,7 @@
/* check offset */
tmp = r600_fmt_get_nblocksy(format, height) * r600_fmt_get_nblocksx(format, pitch) *
- r600_fmt_get_blocksize(format) * track->nsamples;
+ r600_fmt_get_blocksize(format) * nsamples;
switch (array_mode) {
default:
case V_0280A0_ARRAY_LINEAR_GENERAL:
@@ -792,6 +803,12 @@
*/
if (track->cb_dirty) {
tmp = track->cb_target_mask;
+
+ /* We must check both colorbuffers for RESOLVE. */
+ if (track->is_resolve) {
+ tmp |= 0xff;
+ }
+
for (i = 0; i < 8; i++) {
if ((tmp >> (i * 4)) & 0xF) {
/* at least one component is enabled */
@@ -1281,6 +1298,11 @@
track->nsamples = 1 << tmp;
track->cb_dirty = true;
break;
+ case R_028808_CB_COLOR_CONTROL:
+ tmp = G_028808_SPECIAL_OP(radeon_get_ib_value(p, idx));
+ track->is_resolve = tmp == V_028808_SPECIAL_RESOLVE_BOX;
+ track->cb_dirty = true;
+ break;
case R_0280A0_CB_COLOR0_INFO:
case R_0280A4_CB_COLOR1_INFO:
case R_0280A8_CB_COLOR2_INFO:
@@ -1416,7 +1438,7 @@
case R_028118_CB_COLOR6_MASK:
case R_02811C_CB_COLOR7_MASK:
tmp = (reg - R_028100_CB_COLOR0_MASK) / 4;
- track->cb_color_mask[tmp] = ib[idx];
+ track->cb_color_mask[tmp] = radeon_get_ib_value(p, idx);
if (G_0280A0_TILE_MODE(track->cb_color_info[tmp])) {
track->cb_dirty = true;
}
diff --git a/drivers/gpu/drm/radeon/r600d.h b/drivers/gpu/drm/radeon/r600d.h
index bdb69a6..fa6f370 100644
--- a/drivers/gpu/drm/radeon/r600d.h
+++ b/drivers/gpu/drm/radeon/r600d.h
@@ -66,6 +66,14 @@
#define CC_RB_BACKEND_DISABLE 0x98F4
#define BACKEND_DISABLE(x) ((x) << 16)
+#define R_028808_CB_COLOR_CONTROL 0x28808
+#define S_028808_SPECIAL_OP(x) (((x) & 0x7) << 4)
+#define G_028808_SPECIAL_OP(x) (((x) >> 4) & 0x7)
+#define C_028808_SPECIAL_OP 0xFFFFFF8F
+#define V_028808_SPECIAL_NORMAL 0x00
+#define V_028808_SPECIAL_DISABLE 0x01
+#define V_028808_SPECIAL_RESOLVE_BOX 0x07
+
#define CB_COLOR0_BASE 0x28040
#define CB_COLOR1_BASE 0x28044
#define CB_COLOR2_BASE 0x28048
diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c
index d2e2438..7a3daeb 100644
--- a/drivers/gpu/drm/radeon/radeon_device.c
+++ b/drivers/gpu/drm/radeon/radeon_device.c
@@ -1051,7 +1051,7 @@
if (rdev->flags & RADEON_IS_AGP)
rdev->need_dma32 = true;
if ((rdev->flags & RADEON_IS_PCI) &&
- (rdev->family < CHIP_RS400))
+ (rdev->family <= CHIP_RS740))
rdev->need_dma32 = true;
dma_bits = rdev->need_dma32 ? 32 : 40;
@@ -1346,12 +1346,15 @@
for (i = 0; i < RADEON_NUM_RINGS; ++i) {
radeon_ring_restore(rdev, &rdev->ring[i],
ring_sizes[i], ring_data[i]);
+ ring_sizes[i] = 0;
+ ring_data[i] = NULL;
}
r = radeon_ib_ring_tests(rdev);
if (r) {
dev_err(rdev->dev, "ib ring test failed (%d).\n", r);
if (saved) {
+ saved = false;
radeon_suspend(rdev);
goto retry;
}
diff --git a/drivers/gpu/drm/radeon/radeon_drv.c b/drivers/gpu/drm/radeon/radeon_drv.c
index 27d22d7..8c593ea 100644
--- a/drivers/gpu/drm/radeon/radeon_drv.c
+++ b/drivers/gpu/drm/radeon/radeon_drv.c
@@ -63,9 +63,10 @@
* 2.19.0 - r600-eg: MSAA textures
* 2.20.0 - r600-si: RADEON_INFO_TIMESTAMP query
* 2.21.0 - r600-r700: FMASK and CMASK
+ * 2.22.0 - r600 only: RESOLVE_BOX allowed
*/
#define KMS_DRIVER_MAJOR 2
-#define KMS_DRIVER_MINOR 21
+#define KMS_DRIVER_MINOR 22
#define KMS_DRIVER_PATCHLEVEL 0
int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
int radeon_driver_unload_kms(struct drm_device *dev);
diff --git a/drivers/gpu/drm/radeon/radeon_fence.c b/drivers/gpu/drm/radeon/radeon_fence.c
index 7b737b9..2a59375 100644
--- a/drivers/gpu/drm/radeon/radeon_fence.c
+++ b/drivers/gpu/drm/radeon/radeon_fence.c
@@ -131,7 +131,7 @@
*/
void radeon_fence_process(struct radeon_device *rdev, int ring)
{
- uint64_t seq, last_seq;
+ uint64_t seq, last_seq, last_emitted;
unsigned count_loop = 0;
bool wake = false;
@@ -158,13 +158,15 @@
*/
last_seq = atomic64_read(&rdev->fence_drv[ring].last_seq);
do {
+ last_emitted = rdev->fence_drv[ring].sync_seq[ring];
seq = radeon_fence_read(rdev, ring);
seq |= last_seq & 0xffffffff00000000LL;
if (seq < last_seq) {
- seq += 0x100000000LL;
+ seq &= 0xffffffff;
+ seq |= last_emitted & 0xffffffff00000000LL;
}
- if (seq == last_seq) {
+ if (seq <= last_seq || seq > last_emitted) {
break;
}
/* If we loop over we don't want to return without
diff --git a/drivers/gpu/drm/radeon/reg_srcs/r600 b/drivers/gpu/drm/radeon/reg_srcs/r600
index f93e45d..20bfbda 100644
--- a/drivers/gpu/drm/radeon/reg_srcs/r600
+++ b/drivers/gpu/drm/radeon/reg_srcs/r600
@@ -744,7 +744,6 @@
0x00028C38 CB_CLRCMP_DST
0x00028C3C CB_CLRCMP_MSK
0x00028C34 CB_CLRCMP_SRC
-0x00028808 CB_COLOR_CONTROL
0x0002842C CB_FOG_BLUE
0x00028428 CB_FOG_GREEN
0x00028424 CB_FOG_RED
diff --git a/drivers/gpu/drm/savage/savage_drv.c b/drivers/gpu/drm/savage/savage_drv.c
index d31d4cc..c5a1643 100644
--- a/drivers/gpu/drm/savage/savage_drv.c
+++ b/drivers/gpu/drm/savage/savage_drv.c
@@ -43,6 +43,9 @@
.mmap = drm_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/sis/sis_drv.c b/drivers/gpu/drm/sis/sis_drv.c
index 7f11987..867dc03 100644
--- a/drivers/gpu/drm/sis/sis_drv.c
+++ b/drivers/gpu/drm/sis/sis_drv.c
@@ -74,6 +74,9 @@
.mmap = drm_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/tdfx/tdfx_drv.c b/drivers/gpu/drm/tdfx/tdfx_drv.c
index 90f6b13..a7f4d6b 100644
--- a/drivers/gpu/drm/tdfx/tdfx_drv.c
+++ b/drivers/gpu/drm/tdfx/tdfx_drv.c
@@ -49,6 +49,9 @@
.mmap = drm_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/udl/udl_connector.c b/drivers/gpu/drm/udl/udl_connector.c
index ba055e9..8d9dc44 100644
--- a/drivers/gpu/drm/udl/udl_connector.c
+++ b/drivers/gpu/drm/udl/udl_connector.c
@@ -69,6 +69,13 @@
static int udl_mode_valid(struct drm_connector *connector,
struct drm_display_mode *mode)
{
+ struct udl_device *udl = connector->dev->dev_private;
+ if (!udl->sku_pixel_limit)
+ return 0;
+
+ if (mode->vdisplay * mode->hdisplay > udl->sku_pixel_limit)
+ return MODE_VIRTUAL_Y;
+
return 0;
}
diff --git a/drivers/gpu/drm/udl/udl_drv.c b/drivers/gpu/drm/udl/udl_drv.c
index 6e52069..9f84128 100644
--- a/drivers/gpu/drm/udl/udl_drv.c
+++ b/drivers/gpu/drm/udl/udl_drv.c
@@ -66,6 +66,9 @@
.unlocked_ioctl = drm_ioctl,
.release = drm_release,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/via/via_drv.c b/drivers/gpu/drm/via/via_drv.c
index e927b4c..af1b914 100644
--- a/drivers/gpu/drm/via/via_drv.c
+++ b/drivers/gpu/drm/via/via_drv.c
@@ -65,6 +65,9 @@
.mmap = drm_mmap,
.poll = drm_poll,
.fasync = drm_fasync,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = drm_compat_ioctl,
+#endif
.llseek = noop_llseek,
};
diff --git a/drivers/gpu/drm/vmwgfx/Kconfig b/drivers/gpu/drm/vmwgfx/Kconfig
index 794ff67..b71bcd0 100644
--- a/drivers/gpu/drm/vmwgfx/Kconfig
+++ b/drivers/gpu/drm/vmwgfx/Kconfig
@@ -12,3 +12,11 @@
This is a KMS enabled DRM driver for the VMware SVGA2
virtual hardware.
The compiled module will be called "vmwgfx.ko".
+
+config DRM_VMWGFX_FBCON
+ depends on DRM_VMWGFX
+ bool "Enable framebuffer console under vmwgfx by default"
+ help
+ Choose this option if you are shipping a new vmwgfx
+ userspace driver that supports using the kernel driver.
+
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
index 4d9edea..ba2c35d 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c
@@ -182,8 +182,9 @@
{0x15ad, 0x0405, PCI_ANY_ID, PCI_ANY_ID, 0, 0, VMWGFX_CHIP_SVGAII},
{0, 0, 0}
};
+MODULE_DEVICE_TABLE(pci, vmw_pci_id_list);
-static int enable_fbdev;
+static int enable_fbdev = IS_ENABLED(CONFIG_DRM_VMWGFX_FBCON);
static int vmw_probe(struct pci_dev *, const struct pci_device_id *);
static void vmw_master_init(struct vmw_master *);
@@ -1154,6 +1155,11 @@
.open = vmw_driver_open,
.preclose = vmw_preclose,
.postclose = vmw_postclose,
+
+ .dumb_create = vmw_dumb_create,
+ .dumb_map_offset = vmw_dumb_map_offset,
+ .dumb_destroy = vmw_dumb_destroy,
+
.fops = &vmwgfx_driver_fops,
.name = VMWGFX_DRIVER_NAME,
.desc = VMWGFX_DRIVER_DESC,
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
index d0f2c07..29c984f 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h
@@ -645,6 +645,16 @@
int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
+int vmw_dumb_create(struct drm_file *file_priv,
+ struct drm_device *dev,
+ struct drm_mode_create_dumb *args);
+
+int vmw_dumb_map_offset(struct drm_file *file_priv,
+ struct drm_device *dev, uint32_t handle,
+ uint64_t *offset);
+int vmw_dumb_destroy(struct drm_file *file_priv,
+ struct drm_device *dev,
+ uint32_t handle);
/**
* Overlay control - vmwgfx_overlay.c
*/
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c
index f2fb8f1..7e07433 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c
@@ -1018,7 +1018,7 @@
}
- event = kzalloc(sizeof(event->event), GFP_KERNEL);
+ event = kzalloc(sizeof(*event), GFP_KERNEL);
if (unlikely(event == NULL)) {
DRM_ERROR("Failed to allocate an event.\n");
ret = -ENOMEM;
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
index 22bf9a2..2c6ffe0 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c
@@ -1917,3 +1917,76 @@
vmw_resource_unreference(&res);
return ret;
}
+
+
+int vmw_dumb_create(struct drm_file *file_priv,
+ struct drm_device *dev,
+ struct drm_mode_create_dumb *args)
+{
+ struct vmw_private *dev_priv = vmw_priv(dev);
+ struct vmw_master *vmaster = vmw_master(file_priv->master);
+ struct vmw_user_dma_buffer *vmw_user_bo;
+ struct ttm_buffer_object *tmp;
+ int ret;
+
+ args->pitch = args->width * ((args->bpp + 7) / 8);
+ args->size = args->pitch * args->height;
+
+ vmw_user_bo = kzalloc(sizeof(*vmw_user_bo), GFP_KERNEL);
+ if (vmw_user_bo == NULL)
+ return -ENOMEM;
+
+ ret = ttm_read_lock(&vmaster->lock, true);
+ if (ret != 0) {
+ kfree(vmw_user_bo);
+ return ret;
+ }
+
+ ret = vmw_dmabuf_init(dev_priv, &vmw_user_bo->dma, args->size,
+ &vmw_vram_sys_placement, true,
+ &vmw_user_dmabuf_destroy);
+ if (ret != 0)
+ goto out_no_dmabuf;
+
+ tmp = ttm_bo_reference(&vmw_user_bo->dma.base);
+ ret = ttm_base_object_init(vmw_fpriv(file_priv)->tfile,
+ &vmw_user_bo->base,
+ false,
+ ttm_buffer_type,
+ &vmw_user_dmabuf_release, NULL);
+ if (unlikely(ret != 0))
+ goto out_no_base_object;
+
+ args->handle = vmw_user_bo->base.hash.key;
+
+out_no_base_object:
+ ttm_bo_unref(&tmp);
+out_no_dmabuf:
+ ttm_read_unlock(&vmaster->lock);
+ return ret;
+}
+
+int vmw_dumb_map_offset(struct drm_file *file_priv,
+ struct drm_device *dev, uint32_t handle,
+ uint64_t *offset)
+{
+ struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile;
+ struct vmw_dma_buffer *out_buf;
+ int ret;
+
+ ret = vmw_user_dmabuf_lookup(tfile, handle, &out_buf);
+ if (ret != 0)
+ return -EINVAL;
+
+ *offset = out_buf->base.addr_space_offset;
+ vmw_dmabuf_unreference(&out_buf);
+ return 0;
+}
+
+int vmw_dumb_destroy(struct drm_file *file_priv,
+ struct drm_device *dev,
+ uint32_t handle)
+{
+ return ttm_ref_object_base_unref(vmw_fpriv(file_priv)->tfile,
+ handle, TTM_REF_USAGE);
+}
diff --git a/drivers/gpu/vga/vgaarb.c b/drivers/gpu/vga/vgaarb.c
index 3df8fc0..e893f6e 100644
--- a/drivers/gpu/vga/vgaarb.c
+++ b/drivers/gpu/vga/vgaarb.c
@@ -141,7 +141,11 @@
void vga_set_default_device(struct pci_dev *pdev)
{
- vga_default = pdev;
+ if (vga_default == pdev)
+ return;
+
+ pci_dev_put(vga_default);
+ vga_default = pci_dev_get(pdev);
}
#endif
@@ -577,7 +581,7 @@
#ifndef __ARCH_HAS_VGA_DEFAULT_DEVICE
if (vga_default == NULL &&
((vgadev->owns & VGA_RSRC_LEGACY_MASK) == VGA_RSRC_LEGACY_MASK))
- vga_default = pci_dev_get(pdev);
+ vga_set_default_device(pdev);
#endif
vga_arbiter_check_bridge_sharing(vgadev);
@@ -613,10 +617,8 @@
}
#ifndef __ARCH_HAS_VGA_DEFAULT_DEVICE
- if (vga_default == pdev) {
- pci_dev_put(vga_default);
- vga_default = NULL;
- }
+ if (vga_default == pdev)
+ vga_set_default_device(NULL);
#endif
if (vgadev->decodes & (VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM))
@@ -1066,7 +1068,6 @@
}
} else if (strncmp(curr_pos, "target ", 7) == 0) {
- struct pci_bus *pbus;
unsigned int domain, bus, devfn;
struct vga_device *vgadev;
@@ -1085,19 +1086,11 @@
pr_debug("vgaarb: %s ==> %x:%x:%x.%x\n", curr_pos,
domain, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
- pbus = pci_find_bus(domain, bus);
- pr_debug("vgaarb: pbus %p\n", pbus);
- if (pbus == NULL) {
- pr_err("vgaarb: invalid PCI domain and/or bus address %x:%x\n",
- domain, bus);
- ret_val = -ENODEV;
- goto done;
- }
- pdev = pci_get_slot(pbus, devfn);
+ pdev = pci_get_domain_bus_and_slot(domain, bus, devfn);
pr_debug("vgaarb: pdev %p\n", pdev);
if (!pdev) {
- pr_err("vgaarb: invalid PCI address %x:%x\n",
- bus, devfn);
+ pr_err("vgaarb: invalid PCI address %x:%x:%x\n",
+ domain, bus, devfn);
ret_val = -ENODEV;
goto done;
}
diff --git a/drivers/hid/Kconfig b/drivers/hid/Kconfig
index fbf4950..2af774a 100644
--- a/drivers/hid/Kconfig
+++ b/drivers/hid/Kconfig
@@ -307,7 +307,6 @@
config HID_LOGITECH_DJ
tristate "Logitech Unifying receivers full support"
depends on HID_LOGITECH
- default m
---help---
Say Y if you want support for Logitech Unifying receivers and devices.
Unifying receivers are capable of pairing up to 6 Logitech compliant
@@ -527,6 +526,14 @@
---help---
Provide access to PicoLCD's GPO pins via leds class.
+config HID_PICOLCD_CIR
+ bool "CIR via RC class" if EXPERT
+ default !EXPERT
+ depends on HID_PICOLCD
+ depends on HID_PICOLCD=RC_CORE || RC_CORE=y
+ ---help---
+ Provide access to PicoLCD's CIR interface via remote control (LIRC).
+
config HID_PRIMAX
tristate "Primax non-fully HID-compliant devices"
depends on USB_HID
@@ -534,6 +541,15 @@
Support for Primax devices that are not fully compliant with the
HID standard.
+config HID_PS3REMOTE
+ tristate "Sony PS3 BD Remote Control"
+ depends on BT_HIDP
+ ---help---
+ Support for the Sony PS3 Blue-ray Disk Remote Control and Logitech
+ Harmony Adapter for PS3, which connect over Bluetooth.
+
+ Support for the 6-axis controllers is provided by HID_SONY.
+
config HID_ROCCAT
tristate "Roccat device support"
depends on USB_HID
@@ -561,7 +577,9 @@
tristate "Sony PS3 controller"
depends on USB_HID
---help---
- Support for Sony PS3 controller.
+ Support for Sony PS3 6-axis controllers.
+
+ Support for the Sony PS3 BD Remote is provided by HID_PS3REMOTE.
config HID_SPEEDLINK
tristate "Speedlink VAD Cezanne mouse support"
diff --git a/drivers/hid/Makefile b/drivers/hid/Makefile
index f975485..5a3690f 100644
--- a/drivers/hid/Makefile
+++ b/drivers/hid/Makefile
@@ -69,7 +69,28 @@
obj-$(CONFIG_HID_PANTHERLORD) += hid-pl.o
obj-$(CONFIG_HID_PETALYNX) += hid-petalynx.o
obj-$(CONFIG_HID_PICOLCD) += hid-picolcd.o
+hid-picolcd-y += hid-picolcd_core.o
+ifdef CONFIG_HID_PICOLCD_FB
+hid-picolcd-y += hid-picolcd_fb.o
+endif
+ifdef CONFIG_HID_PICOLCD_BACKLIGHT
+hid-picolcd-y += hid-picolcd_backlight.o
+endif
+ifdef CONFIG_HID_PICOLCD_LCD
+hid-picolcd-y += hid-picolcd_lcd.o
+endif
+ifdef CONFIG_HID_PICOLCD_LEDS
+hid-picolcd-y += hid-picolcd_leds.o
+endif
+ifdef CONFIG_HID_PICOLCD_CIR
+hid-picolcd-y += hid-picolcd_cir.o
+endif
+ifdef CONFIG_DEBUG_FS
+hid-picolcd-y += hid-picolcd_debugfs.o
+endif
+
obj-$(CONFIG_HID_PRIMAX) += hid-primax.o
+obj-$(CONFIG_HID_PS3REMOTE) += hid-ps3remote.o
obj-$(CONFIG_HID_ROCCAT) += hid-roccat.o hid-roccat-common.o \
hid-roccat-arvo.o hid-roccat-isku.o hid-roccat-kone.o \
hid-roccat-koneplus.o hid-roccat-kovaplus.o hid-roccat-pyra.o \
diff --git a/drivers/hid/hid-a4tech.c b/drivers/hid/hid-a4tech.c
index 902d1df..0a23988 100644
--- a/drivers/hid/hid-a4tech.c
+++ b/drivers/hid/hid-a4tech.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-apple.c b/drivers/hid/hid-apple.c
index 585344b..06ebdbb 100644
--- a/drivers/hid/hid-apple.c
+++ b/drivers/hid/hid-apple.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby <jirislaby@gmail.com>
*/
diff --git a/drivers/hid/hid-aureal.c b/drivers/hid/hid-aureal.c
index ba64b04..7968187 100644
--- a/drivers/hid/hid-aureal.c
+++ b/drivers/hid/hid-aureal.c
@@ -9,7 +9,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
#include <linux/device.h>
diff --git a/drivers/hid/hid-belkin.c b/drivers/hid/hid-belkin.c
index a1a765a..a1a5a12 100644
--- a/drivers/hid/hid-belkin.c
+++ b/drivers/hid/hid-belkin.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-cherry.c b/drivers/hid/hid-cherry.c
index 888ece6..af034d3 100644
--- a/drivers/hid/hid-cherry.c
+++ b/drivers/hid/hid-cherry.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-core.c b/drivers/hid/hid-core.c
index 60ea284..2cd6880 100644
--- a/drivers/hid/hid-core.c
+++ b/drivers/hid/hid-core.c
@@ -126,7 +126,7 @@
if (parser->collection_stack_ptr == HID_COLLECTION_STACK_SIZE) {
hid_err(parser->device, "collection stack overflow\n");
- return -1;
+ return -EINVAL;
}
if (parser->device->maxcollection == parser->device->collection_size) {
@@ -134,7 +134,7 @@
parser->device->collection_size * 2, GFP_KERNEL);
if (collection == NULL) {
hid_err(parser->device, "failed to reallocate collection array\n");
- return -1;
+ return -ENOMEM;
}
memcpy(collection, parser->device->collection,
sizeof(struct hid_collection) *
@@ -170,7 +170,7 @@
{
if (!parser->collection_stack_ptr) {
hid_err(parser->device, "collection stack underflow\n");
- return -1;
+ return -EINVAL;
}
parser->collection_stack_ptr--;
return 0;
@@ -374,7 +374,7 @@
case HID_GLOBAL_ITEM_TAG_REPORT_SIZE:
parser->global.report_size = item_udata(item);
- if (parser->global.report_size > 96) {
+ if (parser->global.report_size > 128) {
hid_err(parser->device, "invalid report_size %d\n",
parser->global.report_size);
return -1;
@@ -757,6 +757,7 @@
struct hid_item item;
unsigned int size;
__u8 *start;
+ __u8 *buf;
__u8 *end;
int ret;
static int (*dispatch_type[])(struct hid_parser *parser,
@@ -775,12 +776,21 @@
return -ENODEV;
size = device->dev_rsize;
- if (device->driver->report_fixup)
- start = device->driver->report_fixup(device, start, &size);
-
- device->rdesc = kmemdup(start, size, GFP_KERNEL);
- if (device->rdesc == NULL)
+ buf = kmemdup(start, size, GFP_KERNEL);
+ if (buf == NULL)
return -ENOMEM;
+
+ if (device->driver->report_fixup)
+ start = device->driver->report_fixup(device, buf, &size);
+ else
+ start = buf;
+
+ start = kmemdup(start, size, GFP_KERNEL);
+ kfree(buf);
+ if (start == NULL)
+ return -ENOMEM;
+
+ device->rdesc = start;
device->rsize = size;
parser = vzalloc(sizeof(struct hid_parser));
@@ -996,7 +1006,8 @@
struct hid_driver *hdrv = hid->driver;
int ret;
- hid_dump_input(hid, usage, value);
+ if (!list_empty(&hid->debug_list))
+ hid_dump_input(hid, usage, value);
if (hdrv && hdrv->event && hid_match_usage(hid, usage)) {
ret = hdrv->event(hid, field, usage, value);
@@ -1447,7 +1458,14 @@
}
EXPORT_SYMBOL_GPL(hid_disconnect);
-/* a list of devices for which there is a specialized driver on HID bus */
+/*
+ * A list of devices for which there is a specialized driver on HID bus.
+ *
+ * Please note that for multitouch devices (driven by hid-multitouch driver),
+ * there is a proper autodetection and autoloading in place (based on presence
+ * of HID_DG_CONTACTID), so those devices don't need to be added to this list,
+ * as we are doing the right thing in hid_scan_usage().
+ */
static const struct hid_device_id hid_have_special_driver[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_A4TECH, USB_DEVICE_ID_A4TECH_WCP32PU) },
{ HID_USB_DEVICE(USB_VENDOR_ID_A4TECH, USB_DEVICE_ID_A4TECH_X5_005D) },
@@ -1558,11 +1576,14 @@
{ HID_USB_DEVICE(USB_VENDOR_ID_KYE, USB_DEVICE_ID_KYE_EASYPEN_M610X) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LABTEC, USB_DEVICE_ID_LABTEC_WIRELESS_KEYBOARD) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LCPOWER, USB_DEVICE_ID_LCPOWER_LC1000 ) },
- { HID_USB_DEVICE(USB_VENDOR_ID_LENOVO, USB_DEVICE_ID_LENOVO_TPKBD) },
+#if IS_ENABLED(CONFIG_HID_LENOVO_TPKBD)
+ { HID_USB_DEVICE(USB_VENDOR_ID_LENOVO, USB_DEVICE_ID_LENOVO_TPKBD) },
+#endif
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_MX3000_RECEIVER) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_S510_RECEIVER) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_S510_RECEIVER_2) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_RECEIVER) },
+ { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_HARMONY_PS3) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_DINOVO_DESKTOP) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_DINOVO_EDGE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_DINOVO_MINI) },
@@ -1624,7 +1645,7 @@
{ HID_USB_DEVICE(USB_VENDOR_ID_ORTEK, USB_DEVICE_ID_ORTEK_WKB2000) },
{ HID_USB_DEVICE(USB_VENDOR_ID_PETALYNX, USB_DEVICE_ID_PETALYNX_MAXTER_REMOTE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_PRIMAX, USB_DEVICE_ID_PRIMAX_KEYBOARD) },
- { HID_USB_DEVICE(USB_VENDOR_ID_QUANTA, USB_DEVICE_ID_PIXART_IMAGING_INC_OPTICAL_TOUCH_SCREEN) },
+#if IS_ENABLED(CONFIG_HID_ROCCAT)
{ HID_USB_DEVICE(USB_VENDOR_ID_ROCCAT, USB_DEVICE_ID_ROCCAT_KONE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_ROCCAT, USB_DEVICE_ID_ROCCAT_ARVO) },
{ HID_USB_DEVICE(USB_VENDOR_ID_ROCCAT, USB_DEVICE_ID_ROCCAT_ISKU) },
@@ -1633,10 +1654,12 @@
{ HID_USB_DEVICE(USB_VENDOR_ID_ROCCAT, USB_DEVICE_ID_ROCCAT_PYRA_WIRED) },
{ HID_USB_DEVICE(USB_VENDOR_ID_ROCCAT, USB_DEVICE_ID_ROCCAT_PYRA_WIRELESS) },
{ HID_USB_DEVICE(USB_VENDOR_ID_ROCCAT, USB_DEVICE_ID_ROCCAT_SAVU) },
+#endif
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_PS1000) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAMSUNG, USB_DEVICE_ID_SAMSUNG_IR_REMOTE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAMSUNG, USB_DEVICE_ID_SAMSUNG_WIRELESS_KBD_MOUSE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SKYCABLE, USB_DEVICE_ID_SKYCABLE_WIRELESS_PRESENTER) },
+ { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_SONY, USB_DEVICE_ID_SONY_PS3_BDREMOTE) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SONY, USB_DEVICE_ID_SONY_PS3_CONTROLLER) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SONY, USB_DEVICE_ID_SONY_NAVIGATION_CONTROLLER) },
{ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_SONY, USB_DEVICE_ID_SONY_PS3_CONTROLLER) },
@@ -1661,6 +1684,7 @@
{ HID_USB_DEVICE(USB_VENDOR_ID_UCLOGIC, USB_DEVICE_ID_UCLOGIC_TABLET_WP8060U) },
{ HID_USB_DEVICE(USB_VENDOR_ID_UCLOGIC, USB_DEVICE_ID_UCLOGIC_TABLET_WP1062) },
{ HID_USB_DEVICE(USB_VENDOR_ID_UCLOGIC, USB_DEVICE_ID_UCLOGIC_WIRELESS_TABLET_TWHL850) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_UCLOGIC, USB_DEVICE_ID_UCLOGIC_TABLET_TWHA60) },
{ HID_USB_DEVICE(USB_VENDOR_ID_WISEGROUP, USB_DEVICE_ID_SMARTJOY_PLUS) },
{ HID_USB_DEVICE(USB_VENDOR_ID_WISEGROUP, USB_DEVICE_ID_SUPER_JOY_BOX_3) },
{ HID_USB_DEVICE(USB_VENDOR_ID_WISEGROUP, USB_DEVICE_ID_DUAL_USB_JOYPAD) },
diff --git a/drivers/hid/hid-cypress.c b/drivers/hid/hid-cypress.c
index 9e43aac..3e159a5 100644
--- a/drivers/hid/hid-cypress.c
+++ b/drivers/hid/hid-cypress.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-debug.c b/drivers/hid/hid-debug.c
index 01dd9a7..933fff0 100644
--- a/drivers/hid/hid-debug.c
+++ b/drivers/hid/hid-debug.c
@@ -911,15 +911,21 @@
}
-
static int hid_debug_rdesc_show(struct seq_file *f, void *p)
{
struct hid_device *hdev = f->private;
+ const __u8 *rdesc = hdev->rdesc;
+ unsigned rsize = hdev->rsize;
int i;
+ if (!rdesc) {
+ rdesc = hdev->dev_rdesc;
+ rsize = hdev->dev_rsize;
+ }
+
/* dump HID report descriptor */
- for (i = 0; i < hdev->rsize; i++)
- seq_printf(f, "%02x ", hdev->rdesc[i]);
+ for (i = 0; i < rsize; i++)
+ seq_printf(f, "%02x ", rdesc[i]);
seq_printf(f, "\n\n");
/* dump parsed data and input mappings */
diff --git a/drivers/hid/hid-ezkey.c b/drivers/hid/hid-ezkey.c
index ca1163e..6540af2 100644
--- a/drivers/hid/hid-ezkey.c
+++ b/drivers/hid/hid-ezkey.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-gyration.c b/drivers/hid/hid-gyration.c
index e88b951..4442c30 100644
--- a/drivers/hid/hid-gyration.c
+++ b/drivers/hid/hid-gyration.c
@@ -4,7 +4,6 @@
* Copyright (c) 1999 Andreas Gal
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
* Copyright (c) 2006-2008 Jiri Kosina
*/
diff --git a/drivers/hid/hid-holtekff.c b/drivers/hid/hid-holtekff.c
index 4e75421..ff295e6 100644
--- a/drivers/hid/hid-holtekff.c
+++ b/drivers/hid/hid-holtekff.c
@@ -100,8 +100,7 @@
holtekff->field->value[i] = data[i];
}
- dbg_hid("sending %02x %02x %02x %02x %02x %02x %02x\n", data[0],
- data[1], data[2], data[3], data[4], data[5], data[6]);
+ dbg_hid("sending %*ph\n", 7, data);
usbhid_submit_report(hid, holtekff->field->report, USB_DIR_OUT);
}
diff --git a/drivers/hid/hid-ids.h b/drivers/hid/hid-ids.h
index 1dcb76f..ca4d83e 100644
--- a/drivers/hid/hid-ids.h
+++ b/drivers/hid/hid-ids.h
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
*/
/*
@@ -269,7 +268,11 @@
#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_72FA 0x72fa
#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_7302 0x7302
#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_7349 0x7349
+#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_73F7 0x73f7
#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_A001 0xa001
+#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_7224 0x7224
+#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_72D0 0x72d0
+#define USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_72C4 0x72c4
#define USB_VENDOR_ID_ELECOM 0x056e
#define USB_DEVICE_ID_ELECOM_BM084 0x0061
@@ -283,6 +286,9 @@
#define USB_VENDOR_ID_EMS 0x2006
#define USB_DEVICE_ID_EMS_TRIO_LINKER_PLUS_II 0x0118
+#define USB_VENDOR_ID_FLATFROG 0x25b5
+#define USB_DEVICE_ID_MULTITOUCH_3200 0x0002
+
#define USB_VENDOR_ID_ESSENTIAL_REALITY 0x0d7f
#define USB_DEVICE_ID_ESSENTIAL_REALITY_P5 0x0100
@@ -296,6 +302,9 @@
#define USB_VENDOR_ID_EZKEY 0x0518
#define USB_DEVICE_ID_BTC_8193 0x0002
+#define USB_VENDOR_ID_FREESCALE 0x15A2
+#define USB_DEVICE_ID_FREESCALE_MX28 0x004F
+
#define USB_VENDOR_ID_FRUCTEL 0x25B6
#define USB_DEVICE_ID_GAMETEL_MT_MODE 0x0002
@@ -305,6 +314,7 @@
#define USB_VENDOR_ID_GENERAL_TOUCH 0x0dfc
#define USB_DEVICE_ID_GENERAL_TOUCH_WIN7_TWOFINGERS 0x0003
+#define USB_DEVICE_ID_GENERAL_TOUCH_WIN8_PWT_TENFINGERS 0x0100
#define USB_VENDOR_ID_GLAB 0x06c2
#define USB_DEVICE_ID_4_PHIDGETSERVO_30 0x0038
@@ -496,6 +506,7 @@
#define USB_DEVICE_ID_LOGITECH_RECEIVER 0xc101
#define USB_DEVICE_ID_LOGITECH_HARMONY_FIRST 0xc110
#define USB_DEVICE_ID_LOGITECH_HARMONY_LAST 0xc14f
+#define USB_DEVICE_ID_LOGITECH_HARMONY_PS3 0x0306
#define USB_DEVICE_ID_LOGITECH_RUMBLEPAD_CORD 0xc20a
#define USB_DEVICE_ID_LOGITECH_RUMBLEPAD 0xc211
#define USB_DEVICE_ID_LOGITECH_EXTREME_3D 0xc215
@@ -652,7 +663,6 @@
#define USB_DEVICE_ID_QUANTA_OPTICAL_TOUCH 0x3000
#define USB_DEVICE_ID_QUANTA_OPTICAL_TOUCH_3001 0x3001
#define USB_DEVICE_ID_QUANTA_OPTICAL_TOUCH_3008 0x3008
-#define USB_DEVICE_ID_PIXART_IMAGING_INC_OPTICAL_TOUCH_SCREEN 0x3001
#define USB_VENDOR_ID_ROCCAT 0x1e7d
#define USB_DEVICE_ID_ROCCAT_ARVO 0x30d4
@@ -683,6 +693,7 @@
#define USB_VENDOR_ID_SONY 0x054c
#define USB_DEVICE_ID_SONY_VAIO_VGX_MOUSE 0x024b
+#define USB_DEVICE_ID_SONY_PS3_BDREMOTE 0x0306
#define USB_DEVICE_ID_SONY_PS3_CONTROLLER 0x0268
#define USB_DEVICE_ID_SONY_NAVIGATION_CONTROLLER 0x042f
@@ -758,6 +769,7 @@
#define USB_DEVICE_ID_UCLOGIC_TABLET_WP8060U 0x0005
#define USB_DEVICE_ID_UCLOGIC_TABLET_WP1062 0x0064
#define USB_DEVICE_ID_UCLOGIC_WIRELESS_TABLET_TWHL850 0x0522
+#define USB_DEVICE_ID_UCLOGIC_TABLET_TWHA60 0x0781
#define USB_VENDOR_ID_UNITEC 0x227d
#define USB_DEVICE_ID_UNITEC_USB_TOUCH_0709 0x0709
diff --git a/drivers/hid/hid-input.c b/drivers/hid/hid-input.c
index 811bfad..d917c0d 100644
--- a/drivers/hid/hid-input.c
+++ b/drivers/hid/hid-input.c
@@ -1154,6 +1154,7 @@
int hidinput_connect(struct hid_device *hid, unsigned int force)
{
+ struct hid_driver *drv = hid->driver;
struct hid_report *report;
struct hid_input *hidinput = NULL;
struct input_dev *input_dev;
@@ -1228,6 +1229,8 @@
* UGCI) cram a lot of unrelated inputs into the
* same interface. */
hidinput->report = report;
+ if (drv->input_configured)
+ drv->input_configured(hid, hidinput);
if (input_register_device(hidinput->input))
goto out_cleanup;
hidinput = NULL;
@@ -1235,8 +1238,12 @@
}
}
- if (hidinput && input_register_device(hidinput->input))
- goto out_cleanup;
+ if (hidinput) {
+ if (drv->input_configured)
+ drv->input_configured(hid, hidinput);
+ if (input_register_device(hidinput->input))
+ goto out_cleanup;
+ }
return 0;
diff --git a/drivers/hid/hid-lcpower.c b/drivers/hid/hid-lcpower.c
index c4fe9bd0..22bc14a 100644
--- a/drivers/hid/hid-lcpower.c
+++ b/drivers/hid/hid-lcpower.c
@@ -24,7 +24,7 @@
struct hid_field *field, struct hid_usage *usage,
unsigned long **bit, int *max)
{
- if ((usage->hid & HID_USAGE_PAGE) != 0x0ffbc0000)
+ if ((usage->hid & HID_USAGE_PAGE) != HID_UP_LOGIVENDOR)
return 0;
switch (usage->hid & HID_USAGE) {
diff --git a/drivers/hid/hid-lenovo-tpkbd.c b/drivers/hid/hid-lenovo-tpkbd.c
index 77d2df0..cea016e 100644
--- a/drivers/hid/hid-lenovo-tpkbd.c
+++ b/drivers/hid/hid-lenovo-tpkbd.c
@@ -56,9 +56,8 @@
static int tpkbd_features_set(struct hid_device *hdev)
{
struct hid_report *report;
- struct tpkbd_data_pointer *data_pointer;
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
report = hdev->report_enum[HID_FEATURE_REPORT].report_id_hash[4];
report->field[0]->value[0] = data_pointer->press_to_select ? 0x01 : 0x02;
@@ -77,14 +76,8 @@
struct device_attribute *attr,
char *buf)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
-
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
return snprintf(buf, PAGE_SIZE, "%u\n", data_pointer->press_to_select);
}
@@ -94,16 +87,10 @@
const char *buf,
size_t count)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int value;
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (kstrtoint(buf, 10, &value))
return -EINVAL;
if (value < 0 || value > 1)
@@ -119,14 +106,8 @@
struct device_attribute *attr,
char *buf)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
-
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
return snprintf(buf, PAGE_SIZE, "%u\n", data_pointer->dragging);
}
@@ -136,16 +117,10 @@
const char *buf,
size_t count)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int value;
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (kstrtoint(buf, 10, &value))
return -EINVAL;
if (value < 0 || value > 1)
@@ -161,14 +136,8 @@
struct device_attribute *attr,
char *buf)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
-
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
return snprintf(buf, PAGE_SIZE, "%u\n", data_pointer->release_to_select);
}
@@ -178,16 +147,10 @@
const char *buf,
size_t count)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int value;
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (kstrtoint(buf, 10, &value))
return -EINVAL;
if (value < 0 || value > 1)
@@ -203,14 +166,8 @@
struct device_attribute *attr,
char *buf)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
-
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
return snprintf(buf, PAGE_SIZE, "%u\n", data_pointer->select_right);
}
@@ -220,16 +177,10 @@
const char *buf,
size_t count)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int value;
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (kstrtoint(buf, 10, &value))
return -EINVAL;
if (value < 0 || value > 1)
@@ -245,14 +196,8 @@
struct device_attribute *attr,
char *buf)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
-
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
return snprintf(buf, PAGE_SIZE, "%u\n",
data_pointer->sensitivity);
@@ -263,16 +208,10 @@
const char *buf,
size_t count)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int value;
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (kstrtoint(buf, 10, &value) || value < 1 || value > 255)
return -EINVAL;
@@ -286,14 +225,10 @@
struct device_attribute *attr,
char *buf)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
+ data_pointer = hid_get_drvdata(hdev);
return snprintf(buf, PAGE_SIZE, "%u\n",
data_pointer->press_speed);
@@ -304,16 +239,10 @@
const char *buf,
size_t count)
{
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int value;
- hdev = container_of(dev, struct hid_device, dev);
- if (hdev == NULL)
- return -ENODEV;
-
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (kstrtoint(buf, 10, &value) || value < 1 || value > 255)
return -EINVAL;
@@ -370,15 +299,11 @@
static enum led_brightness tpkbd_led_brightness_get(
struct led_classdev *led_cdev)
{
- struct device *dev;
- struct hid_device *hdev;
- struct tpkbd_data_pointer *data_pointer;
+ struct device *dev = led_cdev->dev->parent;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
int led_nr = 0;
- dev = led_cdev->dev->parent;
- hdev = container_of(dev, struct hid_device, dev);
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (led_cdev == &data_pointer->led_micmute)
led_nr = 1;
@@ -390,16 +315,12 @@
static void tpkbd_led_brightness_set(struct led_classdev *led_cdev,
enum led_brightness value)
{
- struct device *dev;
- struct hid_device *hdev;
+ struct device *dev = led_cdev->dev->parent;
+ struct hid_device *hdev = container_of(dev, struct hid_device, dev);
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
struct hid_report *report;
- struct tpkbd_data_pointer *data_pointer;
int led_nr = 0;
- dev = led_cdev->dev->parent;
- hdev = container_of(dev, struct hid_device, dev);
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
if (led_cdev == &data_pointer->led_micmute)
led_nr = 1;
@@ -508,17 +429,17 @@
static void tpkbd_remove_tp(struct hid_device *hdev)
{
- struct tpkbd_data_pointer *data_pointer;
+ struct tpkbd_data_pointer *data_pointer = hid_get_drvdata(hdev);
sysfs_remove_group(&hdev->dev.kobj,
&tpkbd_attr_group_pointer);
- data_pointer = (struct tpkbd_data_pointer *) hid_get_drvdata(hdev);
-
led_classdev_unregister(&data_pointer->led_micmute);
led_classdev_unregister(&data_pointer->led_mute);
hid_set_drvdata(hdev, NULL);
+ kfree(data_pointer->led_micmute.name);
+ kfree(data_pointer->led_mute.name);
kfree(data_pointer);
}
diff --git a/drivers/hid/hid-lg.c b/drivers/hid/hid-lg.c
index fc37ed6..a2f8e88 100644
--- a/drivers/hid/hid-lg.c
+++ b/drivers/hid/hid-lg.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
* Copyright (c) 2010 Hendrik Iben
*/
@@ -109,7 +108,7 @@
static __u8 *lg_report_fixup(struct hid_device *hdev, __u8 *rdesc,
unsigned int *rsize)
{
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hdev);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hdev);
if ((drv_data->quirks & LG_RDESC) && *rsize >= 90 && rdesc[83] == 0x26 &&
rdesc[84] == 0x8c && rdesc[85] == 0x02) {
@@ -278,7 +277,7 @@
0, 0, 0, 0, 0,183,184,185,186,187,
188,189,190,191,192,193,194, 0, 0, 0
};
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hdev);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hdev);
unsigned int hid = usage->hid;
if (hdev->product == USB_DEVICE_ID_LOGITECH_RECEIVER &&
@@ -319,7 +318,7 @@
struct hid_field *field, struct hid_usage *usage,
unsigned long **bit, int *max)
{
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hdev);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hdev);
if ((drv_data->quirks & LG_BAD_RELATIVE_KEYS) && usage->type == EV_KEY &&
(field->flags & HID_MAIN_ITEM_RELATIVE))
@@ -335,13 +334,16 @@
static int lg_event(struct hid_device *hdev, struct hid_field *field,
struct hid_usage *usage, __s32 value)
{
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hdev);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hdev);
if ((drv_data->quirks & LG_INVERT_HWHEEL) && usage->code == REL_HWHEEL) {
input_event(field->hidinput->input, usage->type, usage->code,
-value);
return 1;
}
+ if (drv_data->quirks & LG_FF4) {
+ return lg4ff_adjust_input_event(hdev, field, usage, value, drv_data);
+ }
return 0;
}
@@ -358,7 +360,7 @@
return -ENOMEM;
}
drv_data->quirks = id->driver_data;
-
+
hid_set_drvdata(hdev, (void *)drv_data);
if (drv_data->quirks & LG_NOGET)
@@ -380,7 +382,7 @@
}
/* Setup wireless link with Logitech Wii wheel */
- if(hdev->product == USB_DEVICE_ID_LOGITECH_WII_WHEEL) {
+ if (hdev->product == USB_DEVICE_ID_LOGITECH_WII_WHEEL) {
unsigned char buf[] = { 0x00, 0xAF, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
ret = hdev->hid_output_raw_report(hdev, buf, sizeof(buf), HID_FEATURE_REPORT);
@@ -416,7 +418,7 @@
static void lg_remove(struct hid_device *hdev)
{
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hdev);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hdev);
if (drv_data->quirks & LG_FF4)
lg4ff_deinit(hdev);
@@ -476,7 +478,7 @@
.driver_data = LG_NOGET | LG_FF4 },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_WII_WHEEL),
.driver_data = LG_FF4 },
- { HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_WINGMAN_FFG ),
+ { HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_WINGMAN_FFG),
.driver_data = LG_FF },
{ HID_USB_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_RUMBLEPAD2),
.driver_data = LG_FF2 },
diff --git a/drivers/hid/hid-lg.h b/drivers/hid/hid-lg.h
index d64cf8d..142ce3f 100644
--- a/drivers/hid/hid-lg.h
+++ b/drivers/hid/hid-lg.h
@@ -25,9 +25,13 @@
#endif
#ifdef CONFIG_LOGIWHEELS_FF
+int lg4ff_adjust_input_event(struct hid_device *hid, struct hid_field *field,
+ struct hid_usage *usage, __s32 value, struct lg_drv_data *drv_data);
int lg4ff_init(struct hid_device *hdev);
int lg4ff_deinit(struct hid_device *hdev);
#else
+static inline int lg4ff_adjust_input_event(struct hid_device *hid, struct hid_field *field,
+ struct hid_usage *usage, __s32 value, struct lg_drv_data *drv_data) { return 0; }
static inline int lg4ff_init(struct hid_device *hdev) { return -1; }
static inline int lg4ff_deinit(struct hid_device *hdev) { return -1; }
#endif
diff --git a/drivers/hid/hid-lg4ff.c b/drivers/hid/hid-lg4ff.c
index f3390ee..d7947c7 100644
--- a/drivers/hid/hid-lg4ff.c
+++ b/drivers/hid/hid-lg4ff.c
@@ -43,6 +43,11 @@
#define G27_REV_MAJ 0x12
#define G27_REV_MIN 0x38
+#define DFP_X_MIN 0
+#define DFP_X_MAX 16383
+#define DFP_PEDAL_MIN 0
+#define DFP_PEDAL_MAX 255
+
#define to_hid_device(pdev) container_of(pdev, struct hid_device, dev)
static void hid_lg4ff_set_range_dfp(struct hid_device *hid, u16 range);
@@ -53,6 +58,7 @@
static DEVICE_ATTR(range, S_IRWXU | S_IRWXG | S_IRWXO, lg4ff_range_show, lg4ff_range_store);
struct lg4ff_device_entry {
+ __u32 product_id;
__u16 range;
__u16 min_range;
__u16 max_range;
@@ -129,26 +135,77 @@
{G27_REV_MAJ, G27_REV_MIN, &native_g27}, /* G27 */
};
+/* Recalculates X axis value accordingly to currently selected range */
+static __s32 lg4ff_adjust_dfp_x_axis(__s32 value, __u16 range)
+{
+ __u16 max_range;
+ __s32 new_value;
+
+ if (range == 900)
+ return value;
+ else if (range == 200)
+ return value;
+ else if (range < 200)
+ max_range = 200;
+ else
+ max_range = 900;
+
+ new_value = 8192 + mult_frac(value - 8192, max_range, range);
+ if (new_value < 0)
+ return 0;
+ else if (new_value > 16383)
+ return 16383;
+ else
+ return new_value;
+}
+
+int lg4ff_adjust_input_event(struct hid_device *hid, struct hid_field *field,
+ struct hid_usage *usage, __s32 value, struct lg_drv_data *drv_data)
+{
+ struct lg4ff_device_entry *entry = drv_data->device_props;
+ __s32 new_value = 0;
+
+ if (!entry) {
+ hid_err(hid, "Device properties not found");
+ return 0;
+ }
+
+ switch (entry->product_id) {
+ case USB_DEVICE_ID_LOGITECH_DFP_WHEEL:
+ switch (usage->code) {
+ case ABS_X:
+ new_value = lg4ff_adjust_dfp_x_axis(value, entry->range);
+ input_event(field->hidinput->input, usage->type, usage->code, new_value);
+ return 1;
+ default:
+ return 0;
+ }
+ default:
+ return 0;
+ }
+}
+
static int hid_lg4ff_play(struct input_dev *dev, void *data, struct ff_effect *effect)
{
struct hid_device *hid = input_get_drvdata(dev);
struct list_head *report_list = &hid->report_enum[HID_OUTPUT_REPORT].report_list;
struct hid_report *report = list_entry(report_list->next, struct hid_report, list);
+ __s32 *value = report->field[0]->value;
int x;
-#define CLAMP(x) if (x < 0) x = 0; if (x > 0xff) x = 0xff
+#define CLAMP(x) do { if (x < 0) x = 0; else if (x > 0xff) x = 0xff; } while (0)
switch (effect->type) {
case FF_CONSTANT:
x = effect->u.ramp.start_level + 0x80; /* 0x80 is no force */
CLAMP(x);
- report->field[0]->value[0] = 0x11; /* Slot 1 */
- report->field[0]->value[1] = 0x08;
- report->field[0]->value[2] = x;
- report->field[0]->value[3] = 0x80;
- report->field[0]->value[4] = 0x00;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0x11; /* Slot 1 */
+ value[1] = 0x08;
+ value[2] = x;
+ value[3] = 0x80;
+ value[4] = 0x00;
+ value[5] = 0x00;
+ value[6] = 0x00;
usbhid_submit_report(hid, report, USB_DIR_OUT);
break;
@@ -163,14 +220,15 @@
struct hid_device *hid = input_get_drvdata(dev);
struct list_head *report_list = &hid->report_enum[HID_OUTPUT_REPORT].report_list;
struct hid_report *report = list_entry(report_list->next, struct hid_report, list);
+ __s32 *value = report->field[0]->value;
- report->field[0]->value[0] = 0xfe;
- report->field[0]->value[1] = 0x0d;
- report->field[0]->value[2] = magnitude >> 13;
- report->field[0]->value[3] = magnitude >> 13;
- report->field[0]->value[4] = magnitude >> 8;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0xfe;
+ value[1] = 0x0d;
+ value[2] = magnitude >> 13;
+ value[3] = magnitude >> 13;
+ value[4] = magnitude >> 8;
+ value[5] = 0x00;
+ value[6] = 0x00;
usbhid_submit_report(hid, report, USB_DIR_OUT);
}
@@ -181,16 +239,16 @@
struct hid_device *hid = input_get_drvdata(dev);
struct list_head *report_list = &hid->report_enum[HID_OUTPUT_REPORT].report_list;
struct hid_report *report = list_entry(report_list->next, struct hid_report, list);
+ __s32 *value = report->field[0]->value;
magnitude = magnitude * 90 / 65535;
-
- report->field[0]->value[0] = 0xfe;
- report->field[0]->value[1] = 0x03;
- report->field[0]->value[2] = magnitude >> 14;
- report->field[0]->value[3] = magnitude >> 14;
- report->field[0]->value[4] = magnitude;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0xfe;
+ value[1] = 0x03;
+ value[2] = magnitude >> 14;
+ value[3] = magnitude >> 14;
+ value[4] = magnitude;
+ value[5] = 0x00;
+ value[6] = 0x00;
usbhid_submit_report(hid, report, USB_DIR_OUT);
}
@@ -200,15 +258,17 @@
{
struct list_head *report_list = &hid->report_enum[HID_OUTPUT_REPORT].report_list;
struct hid_report *report = list_entry(report_list->next, struct hid_report, list);
+ __s32 *value = report->field[0]->value;
+
dbg_hid("G25/G27/DFGT: setting range to %u\n", range);
- report->field[0]->value[0] = 0xf8;
- report->field[0]->value[1] = 0x81;
- report->field[0]->value[2] = range & 0x00ff;
- report->field[0]->value[3] = (range & 0xff00) >> 8;
- report->field[0]->value[4] = 0x00;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0xf8;
+ value[1] = 0x81;
+ value[2] = range & 0x00ff;
+ value[3] = (range & 0xff00) >> 8;
+ value[4] = 0x00;
+ value[5] = 0x00;
+ value[6] = 0x00;
usbhid_submit_report(hid, report, USB_DIR_OUT);
}
@@ -219,16 +279,18 @@
struct list_head *report_list = &hid->report_enum[HID_OUTPUT_REPORT].report_list;
struct hid_report *report = list_entry(report_list->next, struct hid_report, list);
int start_left, start_right, full_range;
+ __s32 *value = report->field[0]->value;
+
dbg_hid("Driving Force Pro: setting range to %u\n", range);
/* Prepare "coarse" limit command */
- report->field[0]->value[0] = 0xf8;
- report->field[0]->value[1] = 0x00; /* Set later */
- report->field[0]->value[2] = 0x00;
- report->field[0]->value[3] = 0x00;
- report->field[0]->value[4] = 0x00;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0xf8;
+ value[1] = 0x00; /* Set later */
+ value[2] = 0x00;
+ value[3] = 0x00;
+ value[4] = 0x00;
+ value[5] = 0x00;
+ value[6] = 0x00;
if (range > 200) {
report->field[0]->value[1] = 0x03;
@@ -240,13 +302,13 @@
usbhid_submit_report(hid, report, USB_DIR_OUT);
/* Prepare "fine" limit command */
- report->field[0]->value[0] = 0x81;
- report->field[0]->value[1] = 0x0b;
- report->field[0]->value[2] = 0x00;
- report->field[0]->value[3] = 0x00;
- report->field[0]->value[4] = 0x00;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0x81;
+ value[1] = 0x0b;
+ value[2] = 0x00;
+ value[3] = 0x00;
+ value[4] = 0x00;
+ value[5] = 0x00;
+ value[6] = 0x00;
if (range == 200 || range == 900) { /* Do not apply any fine limit */
usbhid_submit_report(hid, report, USB_DIR_OUT);
@@ -257,11 +319,11 @@
start_left = (((full_range - range + 1) * 2047) / full_range);
start_right = 0xfff - start_left;
- report->field[0]->value[2] = start_left >> 4;
- report->field[0]->value[3] = start_right >> 4;
- report->field[0]->value[4] = 0xff;
- report->field[0]->value[5] = (start_right & 0xe) << 4 | (start_left & 0xe);
- report->field[0]->value[6] = 0xff;
+ value[2] = start_left >> 4;
+ value[3] = start_right >> 4;
+ value[4] = 0xff;
+ value[5] = (start_right & 0xe) << 4 | (start_left & 0xe);
+ value[6] = 0xff;
usbhid_submit_report(hid, report, USB_DIR_OUT);
}
@@ -344,14 +406,15 @@
{
struct list_head *report_list = &hid->report_enum[HID_OUTPUT_REPORT].report_list;
struct hid_report *report = list_entry(report_list->next, struct hid_report, list);
+ __s32 *value = report->field[0]->value;
- report->field[0]->value[0] = 0xf8;
- report->field[0]->value[1] = 0x12;
- report->field[0]->value[2] = leds;
- report->field[0]->value[3] = 0x00;
- report->field[0]->value[4] = 0x00;
- report->field[0]->value[5] = 0x00;
- report->field[0]->value[6] = 0x00;
+ value[0] = 0xf8;
+ value[1] = 0x12;
+ value[2] = leds;
+ value[3] = 0x00;
+ value[4] = 0x00;
+ value[5] = 0x00;
+ value[6] = 0x00;
usbhid_submit_report(hid, report, USB_DIR_OUT);
}
@@ -360,7 +423,7 @@
{
struct device *dev = led_cdev->dev->parent;
struct hid_device *hid = container_of(dev, struct hid_device, dev);
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hid);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hid);
struct lg4ff_device_entry *entry;
int i, state = 0;
@@ -395,7 +458,7 @@
{
struct device *dev = led_cdev->dev->parent;
struct hid_device *hid = container_of(dev, struct hid_device, dev);
- struct lg_drv_data *drv_data = (struct lg_drv_data *)hid_get_drvdata(hid);
+ struct lg_drv_data *drv_data = hid_get_drvdata(hid);
struct lg4ff_device_entry *entry;
int i, value = 0;
@@ -501,7 +564,7 @@
/* Check if autocentering is available and
* set the centering force to zero by default */
if (test_bit(FF_AUTOCENTER, dev->ffbit)) {
- if(rev_maj == FFEX_REV_MAJ && rev_min == FFEX_REV_MIN) /* Formula Force EX expects different autocentering command */
+ if (rev_maj == FFEX_REV_MAJ && rev_min == FFEX_REV_MIN) /* Formula Force EX expects different autocentering command */
dev->ff->set_autocenter = hid_lg4ff_set_autocenter_ffex;
else
dev->ff->set_autocenter = hid_lg4ff_set_autocenter_default;
@@ -524,6 +587,7 @@
}
drv_data->device_props = entry;
+ entry->product_id = lg4ff_devices[i].product_id;
entry->min_range = lg4ff_devices[i].min_range;
entry->max_range = lg4ff_devices[i].max_range;
entry->set_range = lg4ff_devices[i].set_range;
@@ -534,6 +598,18 @@
return error;
dbg_hid("sysfs interface created\n");
+ /* Set default axes parameters */
+ switch (lg4ff_devices[i].product_id) {
+ case USB_DEVICE_ID_LOGITECH_DFP_WHEEL:
+ dbg_hid("Setting axes parameters for Driving Force Pro\n");
+ input_set_abs_params(dev, ABS_X, DFP_X_MIN, DFP_X_MAX, 0, 0);
+ input_set_abs_params(dev, ABS_Y, DFP_PEDAL_MIN, DFP_PEDAL_MAX, 0, 0);
+ input_set_abs_params(dev, ABS_RZ, DFP_PEDAL_MIN, DFP_PEDAL_MAX, 0, 0);
+ break;
+ default:
+ break;
+ }
+
/* Set the maximum range to start with */
entry->range = entry->max_range;
if (entry->set_range != NULL)
@@ -594,6 +670,8 @@
return 0;
}
+
+
int lg4ff_deinit(struct hid_device *hid)
{
struct lg4ff_device_entry *entry;
diff --git a/drivers/hid/hid-logitech-dj.c b/drivers/hid/hid-logitech-dj.c
index 0f9c146..9500f2f 100644
--- a/drivers/hid/hid-logitech-dj.c
+++ b/drivers/hid/hid-logitech-dj.c
@@ -193,6 +193,7 @@
static int logi_dj_output_hidraw_report(struct hid_device *hid, u8 * buf,
size_t count,
unsigned char report_type);
+static int logi_dj_recv_query_paired_devices(struct dj_receiver_dev *djrcv_dev);
static void logi_dj_recv_destroy_djhid_device(struct dj_receiver_dev *djrcv_dev,
struct dj_report *dj_report)
@@ -233,6 +234,7 @@
if (dj_report->report_params[DEVICE_PAIRED_PARAM_SPFUNCTION] &
SPFUNCTION_DEVICE_LIST_EMPTY) {
dbg_hid("%s: device list is empty\n", __func__);
+ djrcv_dev->querying_devices = false;
return;
}
@@ -243,6 +245,12 @@
return;
}
+ if (djrcv_dev->paired_dj_devices[dj_report->device_index]) {
+ /* The device is already known. No need to reallocate it. */
+ dbg_hid("%s: device is already known\n", __func__);
+ return;
+ }
+
dj_hiddev = hid_allocate_device();
if (IS_ERR(dj_hiddev)) {
dev_err(&djrcv_hdev->dev, "%s: hid_allocate_device failed\n",
@@ -306,6 +314,7 @@
struct dj_report dj_report;
unsigned long flags;
int count;
+ int retval;
dbg_hid("%s\n", __func__);
@@ -338,6 +347,25 @@
logi_dj_recv_destroy_djhid_device(djrcv_dev, &dj_report);
break;
default:
+ /* A normal report (i. e. not belonging to a pair/unpair notification)
+ * arriving here, means that the report arrived but we did not have a
+ * paired dj_device associated to the report's device_index, this
+ * means that the original "device paired" notification corresponding
+ * to this dj_device never arrived to this driver. The reason is that
+ * hid-core discards all packets coming from a device while probe() is
+ * executing. */
+ if (!djrcv_dev->paired_dj_devices[dj_report.device_index]) {
+ /* ok, we don't know the device, just re-ask the
+ * receiver for the list of connected devices. */
+ retval = logi_dj_recv_query_paired_devices(djrcv_dev);
+ if (!retval) {
+ /* everything went fine, so just leave */
+ break;
+ }
+ dev_err(&djrcv_dev->hdev->dev,
+ "%s:logi_dj_recv_query_paired_devices "
+ "error:%d\n", __func__, retval);
+ }
dbg_hid("%s: unexpected report type\n", __func__);
}
}
@@ -368,6 +396,12 @@
if (!djdev) {
dbg_hid("djrcv_dev->paired_dj_devices[dj_report->device_index]"
" is NULL, index %d\n", dj_report->device_index);
+ kfifo_in(&djrcv_dev->notif_fifo, dj_report, sizeof(struct dj_report));
+
+ if (schedule_work(&djrcv_dev->work) == 0) {
+ dbg_hid("%s: did not schedule the work item, was already "
+ "queued\n", __func__);
+ }
return;
}
@@ -398,6 +432,12 @@
if (dj_device == NULL) {
dbg_hid("djrcv_dev->paired_dj_devices[dj_report->device_index]"
" is NULL, index %d\n", dj_report->device_index);
+ kfifo_in(&djrcv_dev->notif_fifo, dj_report, sizeof(struct dj_report));
+
+ if (schedule_work(&djrcv_dev->work) == 0) {
+ dbg_hid("%s: did not schedule the work item, was already "
+ "queued\n", __func__);
+ }
return;
}
@@ -439,7 +479,11 @@
struct dj_report *dj_report;
int retval;
- dj_report = kzalloc(sizeof(dj_report), GFP_KERNEL);
+ /* no need to protect djrcv_dev->querying_devices */
+ if (djrcv_dev->querying_devices)
+ return 0;
+
+ dj_report = kzalloc(sizeof(struct dj_report), GFP_KERNEL);
if (!dj_report)
return -ENOMEM;
dj_report->report_id = REPORT_ID_DJ_SHORT;
@@ -450,13 +494,14 @@
return retval;
}
+
static int logi_dj_recv_switch_to_dj_mode(struct dj_receiver_dev *djrcv_dev,
unsigned timeout)
{
struct dj_report *dj_report;
int retval;
- dj_report = kzalloc(sizeof(dj_report), GFP_KERNEL);
+ dj_report = kzalloc(sizeof(struct dj_report), GFP_KERNEL);
if (!dj_report)
return -ENOMEM;
dj_report->report_id = REPORT_ID_DJ_SHORT;
diff --git a/drivers/hid/hid-logitech-dj.h b/drivers/hid/hid-logitech-dj.h
index fd28a5e..4a40003 100644
--- a/drivers/hid/hid-logitech-dj.h
+++ b/drivers/hid/hid-logitech-dj.h
@@ -101,6 +101,7 @@
struct work_struct work;
struct kfifo notif_fifo;
spinlock_t lock;
+ bool querying_devices;
};
struct dj_device {
diff --git a/drivers/hid/hid-magicmouse.c b/drivers/hid/hid-magicmouse.c
index 7364726..25ddf3e 100644
--- a/drivers/hid/hid-magicmouse.c
+++ b/drivers/hid/hid-magicmouse.c
@@ -392,7 +392,7 @@
__set_bit(EV_ABS, input->evbit);
- error = input_mt_init_slots(input, 16);
+ error = input_mt_init_slots(input, 16, 0);
if (error)
return error;
input_set_abs_params(input, ABS_MT_TOUCH_MAJOR, 0, 255 << 2,
diff --git a/drivers/hid/hid-microsoft.c b/drivers/hid/hid-microsoft.c
index e5c699b..3acdcfc 100644
--- a/drivers/hid/hid-microsoft.c
+++ b/drivers/hid/hid-microsoft.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-monterey.c b/drivers/hid/hid-monterey.c
index dedf757..cd3643e 100644
--- a/drivers/hid/hid-monterey.c
+++ b/drivers/hid/hid-monterey.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c
index 59c8b5c..3eb02b9 100644
--- a/drivers/hid/hid-multitouch.c
+++ b/drivers/hid/hid-multitouch.c
@@ -51,12 +51,12 @@
#define MT_QUIRK_VALID_IS_INRANGE (1 << 5)
#define MT_QUIRK_VALID_IS_CONFIDENCE (1 << 6)
#define MT_QUIRK_SLOT_IS_CONTACTID_MINUS_ONE (1 << 8)
+#define MT_QUIRK_NO_AREA (1 << 9)
struct mt_slot {
__s32 x, y, p, w, h;
__s32 contactid; /* the device ContactID assigned to this slot */
bool touch_state; /* is the touch valid? */
- bool seen_in_this_frame;/* has this slot been updated */
};
struct mt_class {
@@ -92,8 +92,9 @@
__u8 touches_by_report; /* how many touches are present in one report:
* 1 means we should use a serial protocol
* > 1 means hybrid (multitouch) protocol */
+ bool serial_maybe; /* need to check for serial protocol */
bool curvalid; /* is the current contact valid? */
- struct mt_slot *slots;
+ unsigned mt_flags; /* flags to pass to input-mt */
};
/* classes of device behavior */
@@ -115,6 +116,9 @@
#define MT_CLS_EGALAX_SERIAL 0x0104
#define MT_CLS_TOPSEED 0x0105
#define MT_CLS_PANASONIC 0x0106
+#define MT_CLS_FLATFROG 0x0107
+#define MT_CLS_GENERALTOUCH_TWOFINGERS 0x0108
+#define MT_CLS_GENERALTOUCH_PWT_TENFINGERS 0x0109
#define MT_DEFAULT_MAXCONTACT 10
@@ -134,25 +138,6 @@
return -1;
}
-static int find_slot_from_contactid(struct mt_device *td)
-{
- int i;
- for (i = 0; i < td->maxcontacts; ++i) {
- if (td->slots[i].contactid == td->curdata.contactid &&
- td->slots[i].touch_state)
- return i;
- }
- for (i = 0; i < td->maxcontacts; ++i) {
- if (!td->slots[i].seen_in_this_frame &&
- !td->slots[i].touch_state)
- return i;
- }
- /* should not occurs. If this happens that means
- * that the device sent more touches that it says
- * in the report descriptor. It is ignored then. */
- return -1;
-}
-
static struct mt_class mt_classes[] = {
{ .name = MT_CLS_DEFAULT,
.quirks = MT_QUIRK_NOT_SEEN_MEANS_UP },
@@ -190,7 +175,9 @@
MT_QUIRK_SLOT_IS_CONTACTID,
.sn_move = 2048,
.sn_width = 128,
- .sn_height = 128 },
+ .sn_height = 128,
+ .maxcontacts = 60,
+ },
{ .name = MT_CLS_CYPRESS,
.quirks = MT_QUIRK_NOT_SEEN_MEANS_UP |
MT_QUIRK_CYPRESS,
@@ -215,7 +202,24 @@
{ .name = MT_CLS_PANASONIC,
.quirks = MT_QUIRK_NOT_SEEN_MEANS_UP,
.maxcontacts = 4 },
+ { .name = MT_CLS_GENERALTOUCH_TWOFINGERS,
+ .quirks = MT_QUIRK_NOT_SEEN_MEANS_UP |
+ MT_QUIRK_VALID_IS_INRANGE |
+ MT_QUIRK_SLOT_IS_CONTACTNUMBER,
+ .maxcontacts = 2
+ },
+ { .name = MT_CLS_GENERALTOUCH_PWT_TENFINGERS,
+ .quirks = MT_QUIRK_NOT_SEEN_MEANS_UP |
+ MT_QUIRK_SLOT_IS_CONTACTNUMBER,
+ .maxcontacts = 10
+ },
+ { .name = MT_CLS_FLATFROG,
+ .quirks = MT_QUIRK_NOT_SEEN_MEANS_UP |
+ MT_QUIRK_NO_AREA,
+ .sn_move = 2048,
+ .maxcontacts = 40,
+ },
{ }
};
@@ -319,24 +323,16 @@
* We need to ignore fields that belong to other collections
* such as Mouse that might have the same GenericDesktop usages. */
if (field->application == HID_DG_TOUCHSCREEN)
- set_bit(INPUT_PROP_DIRECT, hi->input->propbit);
+ td->mt_flags |= INPUT_MT_DIRECT;
else if (field->application != HID_DG_TOUCHPAD)
return 0;
- /* In case of an indirect device (touchpad), we need to add
- * specific BTN_TOOL_* to be handled by the synaptics xorg
- * driver.
- * We also consider that touchscreens providing buttons are touchpads.
+ /*
+ * Model touchscreens providing buttons as touchpads.
*/
if (field->application == HID_DG_TOUCHPAD ||
- (usage->hid & HID_USAGE_PAGE) == HID_UP_BUTTON ||
- cls->is_indirect) {
- set_bit(INPUT_PROP_POINTER, hi->input->propbit);
- set_bit(BTN_TOOL_FINGER, hi->input->keybit);
- set_bit(BTN_TOOL_DOUBLETAP, hi->input->keybit);
- set_bit(BTN_TOOL_TRIPLETAP, hi->input->keybit);
- set_bit(BTN_TOOL_QUADTAP, hi->input->keybit);
- }
+ (usage->hid & HID_USAGE_PAGE) == HID_UP_BUTTON)
+ td->mt_flags |= INPUT_MT_POINTER;
/* eGalax devices provide a Digitizer.Stylus input which overrides
* the correct Digitizers.Finger X/Y ranges.
@@ -353,8 +349,6 @@
EV_ABS, ABS_MT_POSITION_X);
set_abs(hi->input, ABS_MT_POSITION_X, field,
cls->sn_move);
- /* touchscreen emulation */
- set_abs(hi->input, ABS_X, field, cls->sn_move);
mt_store_field(usage, td, hi);
td->last_field_index = field->index;
return 1;
@@ -363,8 +357,6 @@
EV_ABS, ABS_MT_POSITION_Y);
set_abs(hi->input, ABS_MT_POSITION_Y, field,
cls->sn_move);
- /* touchscreen emulation */
- set_abs(hi->input, ABS_Y, field, cls->sn_move);
mt_store_field(usage, td, hi);
td->last_field_index = field->index;
return 1;
@@ -388,9 +380,6 @@
td->last_field_index = field->index;
return 1;
case HID_DG_CONTACTID:
- if (!td->maxcontacts)
- td->maxcontacts = MT_DEFAULT_MAXCONTACT;
- input_mt_init_slots(hi->input, td->maxcontacts);
mt_store_field(usage, td, hi);
td->last_field_index = field->index;
td->touches_by_report++;
@@ -398,18 +387,21 @@
case HID_DG_WIDTH:
hid_map_usage(hi, usage, bit, max,
EV_ABS, ABS_MT_TOUCH_MAJOR);
- set_abs(hi->input, ABS_MT_TOUCH_MAJOR, field,
- cls->sn_width);
+ if (!(cls->quirks & MT_QUIRK_NO_AREA))
+ set_abs(hi->input, ABS_MT_TOUCH_MAJOR, field,
+ cls->sn_width);
mt_store_field(usage, td, hi);
td->last_field_index = field->index;
return 1;
case HID_DG_HEIGHT:
hid_map_usage(hi, usage, bit, max,
EV_ABS, ABS_MT_TOUCH_MINOR);
- set_abs(hi->input, ABS_MT_TOUCH_MINOR, field,
- cls->sn_height);
- input_set_abs_params(hi->input,
+ if (!(cls->quirks & MT_QUIRK_NO_AREA)) {
+ set_abs(hi->input, ABS_MT_TOUCH_MINOR, field,
+ cls->sn_height);
+ input_set_abs_params(hi->input,
ABS_MT_ORIENTATION, 0, 1, 0, 0);
+ }
mt_store_field(usage, td, hi);
td->last_field_index = field->index;
return 1;
@@ -418,9 +410,6 @@
EV_ABS, ABS_MT_PRESSURE);
set_abs(hi->input, ABS_MT_PRESSURE, field,
cls->sn_pressure);
- /* touchscreen emulation */
- set_abs(hi->input, ABS_PRESSURE, field,
- cls->sn_pressure);
mt_store_field(usage, td, hi);
td->last_field_index = field->index;
return 1;
@@ -464,7 +453,7 @@
return -1;
}
-static int mt_compute_slot(struct mt_device *td)
+static int mt_compute_slot(struct mt_device *td, struct input_dev *input)
{
__s32 quirks = td->mtclass.quirks;
@@ -480,42 +469,23 @@
if (quirks & MT_QUIRK_SLOT_IS_CONTACTID_MINUS_ONE)
return td->curdata.contactid - 1;
- return find_slot_from_contactid(td);
+ return input_mt_get_slot_by_key(input, td->curdata.contactid);
}
/*
* this function is called when a whole contact has been processed,
* so that it can assign it to a slot and store the data there
*/
-static void mt_complete_slot(struct mt_device *td)
+static void mt_complete_slot(struct mt_device *td, struct input_dev *input)
{
- td->curdata.seen_in_this_frame = true;
if (td->curvalid) {
- int slotnum = mt_compute_slot(td);
+ int slotnum = mt_compute_slot(td, input);
+ struct mt_slot *s = &td->curdata;
- if (slotnum >= 0 && slotnum < td->maxcontacts)
- td->slots[slotnum] = td->curdata;
- }
- td->num_received++;
-}
+ if (slotnum < 0 || slotnum >= td->maxcontacts)
+ return;
-
-/*
- * this function is called when a whole packet has been received and processed,
- * so that it can decide what to send to the input layer.
- */
-static void mt_emit_event(struct mt_device *td, struct input_dev *input)
-{
- int i;
-
- for (i = 0; i < td->maxcontacts; ++i) {
- struct mt_slot *s = &(td->slots[i]);
- if ((td->mtclass.quirks & MT_QUIRK_NOT_SEEN_MEANS_UP) &&
- !s->seen_in_this_frame) {
- s->touch_state = false;
- }
-
- input_mt_slot(input, i);
+ input_mt_slot(input, slotnum);
input_mt_report_slot_state(input, MT_TOOL_FINGER,
s->touch_state);
if (s->touch_state) {
@@ -532,24 +502,29 @@
input_event(input, EV_ABS, ABS_MT_TOUCH_MAJOR, major);
input_event(input, EV_ABS, ABS_MT_TOUCH_MINOR, minor);
}
- s->seen_in_this_frame = false;
-
}
- input_mt_report_pointer_emulation(input, true);
+ td->num_received++;
+}
+
+/*
+ * this function is called when a whole packet has been received and processed,
+ * so that it can decide what to send to the input layer.
+ */
+static void mt_sync_frame(struct mt_device *td, struct input_dev *input)
+{
+ input_mt_sync_frame(input);
input_sync(input);
td->num_received = 0;
}
-
-
static int mt_event(struct hid_device *hid, struct hid_field *field,
struct hid_usage *usage, __s32 value)
{
struct mt_device *td = hid_get_drvdata(hid);
__s32 quirks = td->mtclass.quirks;
- if (hid->claimed & HID_CLAIMED_INPUT && td->slots) {
+ if (hid->claimed & HID_CLAIMED_INPUT) {
switch (usage->hid) {
case HID_DG_INRANGE:
if (quirks & MT_QUIRK_ALWAYS_VALID)
@@ -602,11 +577,11 @@
}
if (usage->hid == td->last_slot_field)
- mt_complete_slot(td);
+ mt_complete_slot(td, field->hidinput->input);
if (field->index == td->last_field_index
&& td->num_received >= td->num_expected)
- mt_emit_event(td, field->hidinput->input);
+ mt_sync_frame(td, field->hidinput->input);
}
@@ -685,18 +660,45 @@
}
}
+static void mt_input_configured(struct hid_device *hdev, struct hid_input *hi)
+
+{
+ struct mt_device *td = hid_get_drvdata(hdev);
+ struct mt_class *cls = &td->mtclass;
+ struct input_dev *input = hi->input;
+
+ /* Only initialize slots for MT input devices */
+ if (!test_bit(ABS_MT_POSITION_X, input->absbit))
+ return;
+
+ if (!td->maxcontacts)
+ td->maxcontacts = MT_DEFAULT_MAXCONTACT;
+
+ mt_post_parse(td);
+ if (td->serial_maybe)
+ mt_post_parse_default_settings(td);
+
+ if (cls->is_indirect)
+ td->mt_flags |= INPUT_MT_POINTER;
+
+ if (cls->quirks & MT_QUIRK_NOT_SEEN_MEANS_UP)
+ td->mt_flags |= INPUT_MT_DROP_UNUSED;
+
+ input_mt_init_slots(input, td->maxcontacts, td->mt_flags);
+
+ td->mt_flags = 0;
+}
+
static int mt_probe(struct hid_device *hdev, const struct hid_device_id *id)
{
int ret, i;
struct mt_device *td;
struct mt_class *mtclass = mt_classes; /* MT_CLS_DEFAULT */
- if (id) {
- for (i = 0; mt_classes[i].name ; i++) {
- if (id->driver_data == mt_classes[i].name) {
- mtclass = &(mt_classes[i]);
- break;
- }
+ for (i = 0; mt_classes[i].name ; i++) {
+ if (id->driver_data == mt_classes[i].name) {
+ mtclass = &(mt_classes[i]);
+ break;
}
}
@@ -722,6 +724,9 @@
goto fail;
}
+ if (id->vendor == HID_ANY_ID && id->product == HID_ANY_ID)
+ td->serial_maybe = true;
+
ret = hid_parse(hdev);
if (ret != 0)
goto fail;
@@ -730,20 +735,6 @@
if (ret)
goto fail;
- mt_post_parse(td);
-
- if (id->vendor == HID_ANY_ID && id->product == HID_ANY_ID)
- mt_post_parse_default_settings(td);
-
- td->slots = kzalloc(td->maxcontacts * sizeof(struct mt_slot),
- GFP_KERNEL);
- if (!td->slots) {
- dev_err(&hdev->dev, "cannot allocate multitouch slots\n");
- hid_hw_stop(hdev);
- ret = -ENOMEM;
- goto fail;
- }
-
ret = sysfs_create_group(&hdev->dev.kobj, &mt_attribute_group);
mt_set_maxcontacts(hdev);
@@ -767,6 +758,32 @@
mt_set_input_mode(hdev);
return 0;
}
+
+static int mt_resume(struct hid_device *hdev)
+{
+ struct usb_interface *intf;
+ struct usb_host_interface *interface;
+ struct usb_device *dev;
+
+ if (hdev->bus != BUS_USB)
+ return 0;
+
+ intf = to_usb_interface(hdev->dev.parent);
+ interface = intf->cur_altsetting;
+ dev = hid_to_usb_dev(hdev);
+
+ /* Some Elan legacy devices require SET_IDLE to be set on resume.
+ * It should be safe to send it to other devices too.
+ * Tested on 3M, Stantum, Cypress, Zytronic, eGalax, and Elan panels. */
+
+ usb_control_msg(dev, usb_sndctrlpipe(dev, 0),
+ HID_REQ_SET_IDLE,
+ USB_TYPE_CLASS | USB_RECIP_INTERFACE,
+ 0, interface->desc.bInterfaceNumber,
+ NULL, 0, USB_CTRL_SET_TIMEOUT);
+
+ return 0;
+}
#endif
static void mt_remove(struct hid_device *hdev)
@@ -774,7 +791,6 @@
struct mt_device *td = hid_get_drvdata(hdev);
sysfs_remove_group(&hdev->dev.kobj, &mt_attribute_group);
hid_hw_stop(hdev);
- kfree(td->slots);
kfree(td);
hid_set_drvdata(hdev, NULL);
}
@@ -885,17 +901,37 @@
USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_7349) },
{ .driver_data = MT_CLS_EGALAX_SERIAL,
MT_USB_DEVICE(USB_VENDOR_ID_DWAV,
+ USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_73F7) },
+ { .driver_data = MT_CLS_EGALAX_SERIAL,
+ MT_USB_DEVICE(USB_VENDOR_ID_DWAV,
USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_A001) },
+ { .driver_data = MT_CLS_EGALAX,
+ HID_USB_DEVICE(USB_VENDOR_ID_DWAV,
+ USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_7224) },
+ { .driver_data = MT_CLS_EGALAX,
+ HID_USB_DEVICE(USB_VENDOR_ID_DWAV,
+ USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_72D0) },
+ { .driver_data = MT_CLS_EGALAX,
+ HID_USB_DEVICE(USB_VENDOR_ID_DWAV,
+ USB_DEVICE_ID_DWAV_EGALAX_MULTITOUCH_72C4) },
/* Elo TouchSystems IntelliTouch Plus panel */
{ .driver_data = MT_CLS_DUAL_NSMU_CONTACTID,
MT_USB_DEVICE(USB_VENDOR_ID_ELO,
USB_DEVICE_ID_ELO_TS2515) },
+ /* Flatfrog Panels */
+ { .driver_data = MT_CLS_FLATFROG,
+ MT_USB_DEVICE(USB_VENDOR_ID_FLATFROG,
+ USB_DEVICE_ID_MULTITOUCH_3200) },
+
/* GeneralTouch panel */
- { .driver_data = MT_CLS_DUAL_INRANGE_CONTACTNUMBER,
+ { .driver_data = MT_CLS_GENERALTOUCH_TWOFINGERS,
MT_USB_DEVICE(USB_VENDOR_ID_GENERAL_TOUCH,
USB_DEVICE_ID_GENERAL_TOUCH_WIN7_TWOFINGERS) },
+ { .driver_data = MT_CLS_GENERALTOUCH_PWT_TENFINGERS,
+ MT_USB_DEVICE(USB_VENDOR_ID_GENERAL_TOUCH,
+ USB_DEVICE_ID_GENERAL_TOUCH_WIN8_PWT_TENFINGERS) },
/* Gametel game controller */
{ .driver_data = MT_CLS_DEFAULT,
@@ -1087,11 +1123,13 @@
.remove = mt_remove,
.input_mapping = mt_input_mapping,
.input_mapped = mt_input_mapped,
+ .input_configured = mt_input_configured,
.feature_mapping = mt_feature_mapping,
.usage_table = mt_grabbed_usages,
.event = mt_event,
#ifdef CONFIG_PM
.reset_resume = mt_reset_resume,
+ .resume = mt_resume,
#endif
};
diff --git a/drivers/hid/hid-ntrig.c b/drivers/hid/hid-ntrig.c
index 9fae2eb..86a969f 100644
--- a/drivers/hid/hid-ntrig.c
+++ b/drivers/hid/hid-ntrig.c
@@ -882,10 +882,10 @@
nd->activate_slack = activate_slack;
nd->act_state = activate_slack;
nd->deactivate_slack = -deactivate_slack;
- nd->sensor_logical_width = 0;
- nd->sensor_logical_height = 0;
- nd->sensor_physical_width = 0;
- nd->sensor_physical_height = 0;
+ nd->sensor_logical_width = 1;
+ nd->sensor_logical_height = 1;
+ nd->sensor_physical_width = 1;
+ nd->sensor_physical_height = 1;
hid_set_drvdata(hdev, nd);
diff --git a/drivers/hid/hid-petalynx.c b/drivers/hid/hid-petalynx.c
index f1ea3ff..4c521de 100644
--- a/drivers/hid/hid-petalynx.c
+++ b/drivers/hid/hid-petalynx.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-picolcd.c b/drivers/hid/hid-picolcd.c
deleted file mode 100644
index 27c8ebd..0000000
--- a/drivers/hid/hid-picolcd.c
+++ /dev/null
@@ -1,2748 +0,0 @@
-/***************************************************************************
- * Copyright (C) 2010 by Bruno Prémont <bonbons@linux-vserver.org> *
- * *
- * Based on Logitech G13 driver (v0.4) *
- * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
- * *
- * This program is free software: you can redistribute it and/or modify *
- * it under the terms of the GNU General Public License as published by *
- * the Free Software Foundation, version 2 of the License. *
- * *
- * This driver is distributed in the hope that it will be useful, but *
- * WITHOUT ANY WARRANTY; without even the implied warranty of *
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
- * General Public License for more details. *
- * *
- * You should have received a copy of the GNU General Public License *
- * along with this software. If not see <http://www.gnu.org/licenses/>. *
- ***************************************************************************/
-
-#include <linux/hid.h>
-#include <linux/hid-debug.h>
-#include <linux/input.h>
-#include "hid-ids.h"
-#include "usbhid/usbhid.h"
-#include <linux/usb.h>
-
-#include <linux/fb.h>
-#include <linux/vmalloc.h>
-#include <linux/backlight.h>
-#include <linux/lcd.h>
-
-#include <linux/leds.h>
-
-#include <linux/seq_file.h>
-#include <linux/debugfs.h>
-
-#include <linux/completion.h>
-#include <linux/uaccess.h>
-#include <linux/module.h>
-
-#define PICOLCD_NAME "PicoLCD (graphic)"
-
-/* Report numbers */
-#define REPORT_ERROR_CODE 0x10 /* LCD: IN[16] */
-#define ERR_SUCCESS 0x00
-#define ERR_PARAMETER_MISSING 0x01
-#define ERR_DATA_MISSING 0x02
-#define ERR_BLOCK_READ_ONLY 0x03
-#define ERR_BLOCK_NOT_ERASABLE 0x04
-#define ERR_BLOCK_TOO_BIG 0x05
-#define ERR_SECTION_OVERFLOW 0x06
-#define ERR_INVALID_CMD_LEN 0x07
-#define ERR_INVALID_DATA_LEN 0x08
-#define REPORT_KEY_STATE 0x11 /* LCD: IN[2] */
-#define REPORT_IR_DATA 0x21 /* LCD: IN[63] */
-#define REPORT_EE_DATA 0x32 /* LCD: IN[63] */
-#define REPORT_MEMORY 0x41 /* LCD: IN[63] */
-#define REPORT_LED_STATE 0x81 /* LCD: OUT[1] */
-#define REPORT_BRIGHTNESS 0x91 /* LCD: OUT[1] */
-#define REPORT_CONTRAST 0x92 /* LCD: OUT[1] */
-#define REPORT_RESET 0x93 /* LCD: OUT[2] */
-#define REPORT_LCD_CMD 0x94 /* LCD: OUT[63] */
-#define REPORT_LCD_DATA 0x95 /* LCD: OUT[63] */
-#define REPORT_LCD_CMD_DATA 0x96 /* LCD: OUT[63] */
-#define REPORT_EE_READ 0xa3 /* LCD: OUT[63] */
-#define REPORT_EE_WRITE 0xa4 /* LCD: OUT[63] */
-#define REPORT_ERASE_MEMORY 0xb2 /* LCD: OUT[2] */
-#define REPORT_READ_MEMORY 0xb3 /* LCD: OUT[3] */
-#define REPORT_WRITE_MEMORY 0xb4 /* LCD: OUT[63] */
-#define REPORT_SPLASH_RESTART 0xc1 /* LCD: OUT[1] */
-#define REPORT_EXIT_KEYBOARD 0xef /* LCD: OUT[2] */
-#define REPORT_VERSION 0xf1 /* LCD: IN[2],OUT[1] Bootloader: IN[2],OUT[1] */
-#define REPORT_BL_ERASE_MEMORY 0xf2 /* Bootloader: IN[36],OUT[4] */
-#define REPORT_BL_READ_MEMORY 0xf3 /* Bootloader: IN[36],OUT[4] */
-#define REPORT_BL_WRITE_MEMORY 0xf4 /* Bootloader: IN[36],OUT[36] */
-#define REPORT_DEVID 0xf5 /* LCD: IN[5], OUT[1] Bootloader: IN[5],OUT[1] */
-#define REPORT_SPLASH_SIZE 0xf6 /* LCD: IN[4], OUT[1] */
-#define REPORT_HOOK_VERSION 0xf7 /* LCD: IN[2], OUT[1] */
-#define REPORT_EXIT_FLASHER 0xff /* Bootloader: OUT[2] */
-
-#ifdef CONFIG_HID_PICOLCD_FB
-/* Framebuffer
- *
- * The PicoLCD use a Topway LCD module of 256x64 pixel
- * This display area is tiled over 4 controllers with 8 tiles
- * each. Each tile has 8x64 pixel, each data byte representing
- * a 1-bit wide vertical line of the tile.
- *
- * The display can be updated at a tile granularity.
- *
- * Chip 1 Chip 2 Chip 3 Chip 4
- * +----------------+----------------+----------------+----------------+
- * | Tile 1 | Tile 1 | Tile 1 | Tile 1 |
- * +----------------+----------------+----------------+----------------+
- * | Tile 2 | Tile 2 | Tile 2 | Tile 2 |
- * +----------------+----------------+----------------+----------------+
- * ...
- * +----------------+----------------+----------------+----------------+
- * | Tile 8 | Tile 8 | Tile 8 | Tile 8 |
- * +----------------+----------------+----------------+----------------+
- */
-#define PICOLCDFB_NAME "picolcdfb"
-#define PICOLCDFB_WIDTH (256)
-#define PICOLCDFB_HEIGHT (64)
-#define PICOLCDFB_SIZE (PICOLCDFB_WIDTH * PICOLCDFB_HEIGHT / 8)
-
-#define PICOLCDFB_UPDATE_RATE_LIMIT 10
-#define PICOLCDFB_UPDATE_RATE_DEFAULT 2
-
-/* Framebuffer visual structures */
-static const struct fb_fix_screeninfo picolcdfb_fix = {
- .id = PICOLCDFB_NAME,
- .type = FB_TYPE_PACKED_PIXELS,
- .visual = FB_VISUAL_MONO01,
- .xpanstep = 0,
- .ypanstep = 0,
- .ywrapstep = 0,
- .line_length = PICOLCDFB_WIDTH / 8,
- .accel = FB_ACCEL_NONE,
-};
-
-static const struct fb_var_screeninfo picolcdfb_var = {
- .xres = PICOLCDFB_WIDTH,
- .yres = PICOLCDFB_HEIGHT,
- .xres_virtual = PICOLCDFB_WIDTH,
- .yres_virtual = PICOLCDFB_HEIGHT,
- .width = 103,
- .height = 26,
- .bits_per_pixel = 1,
- .grayscale = 1,
- .red = {
- .offset = 0,
- .length = 1,
- .msb_right = 0,
- },
- .green = {
- .offset = 0,
- .length = 1,
- .msb_right = 0,
- },
- .blue = {
- .offset = 0,
- .length = 1,
- .msb_right = 0,
- },
- .transp = {
- .offset = 0,
- .length = 0,
- .msb_right = 0,
- },
-};
-#endif /* CONFIG_HID_PICOLCD_FB */
-
-/* Input device
- *
- * The PicoLCD has an IR receiver header, a built-in keypad with 5 keys
- * and header for 4x4 key matrix. The built-in keys are part of the matrix.
- */
-static const unsigned short def_keymap[] = {
- KEY_RESERVED, /* none */
- KEY_BACK, /* col 4 + row 1 */
- KEY_HOMEPAGE, /* col 3 + row 1 */
- KEY_RESERVED, /* col 2 + row 1 */
- KEY_RESERVED, /* col 1 + row 1 */
- KEY_SCROLLUP, /* col 4 + row 2 */
- KEY_OK, /* col 3 + row 2 */
- KEY_SCROLLDOWN, /* col 2 + row 2 */
- KEY_RESERVED, /* col 1 + row 2 */
- KEY_RESERVED, /* col 4 + row 3 */
- KEY_RESERVED, /* col 3 + row 3 */
- KEY_RESERVED, /* col 2 + row 3 */
- KEY_RESERVED, /* col 1 + row 3 */
- KEY_RESERVED, /* col 4 + row 4 */
- KEY_RESERVED, /* col 3 + row 4 */
- KEY_RESERVED, /* col 2 + row 4 */
- KEY_RESERVED, /* col 1 + row 4 */
-};
-#define PICOLCD_KEYS ARRAY_SIZE(def_keymap)
-
-/* Description of in-progress IO operation, used for operations
- * that trigger response from device */
-struct picolcd_pending {
- struct hid_report *out_report;
- struct hid_report *in_report;
- struct completion ready;
- int raw_size;
- u8 raw_data[64];
-};
-
-/* Per device data structure */
-struct picolcd_data {
- struct hid_device *hdev;
-#ifdef CONFIG_DEBUG_FS
- struct dentry *debug_reset;
- struct dentry *debug_eeprom;
- struct dentry *debug_flash;
- struct mutex mutex_flash;
- int addr_sz;
-#endif
- u8 version[2];
- unsigned short opmode_delay;
- /* input stuff */
- u8 pressed_keys[2];
- struct input_dev *input_keys;
- struct input_dev *input_cir;
- unsigned short keycode[PICOLCD_KEYS];
-
-#ifdef CONFIG_HID_PICOLCD_FB
- /* Framebuffer stuff */
- u8 fb_update_rate;
- u8 fb_bpp;
- u8 fb_force;
- u8 *fb_vbitmap; /* local copy of what was sent to PicoLCD */
- u8 *fb_bitmap; /* framebuffer */
- struct fb_info *fb_info;
- struct fb_deferred_io fb_defio;
-#endif /* CONFIG_HID_PICOLCD_FB */
-#ifdef CONFIG_HID_PICOLCD_LCD
- struct lcd_device *lcd;
- u8 lcd_contrast;
-#endif /* CONFIG_HID_PICOLCD_LCD */
-#ifdef CONFIG_HID_PICOLCD_BACKLIGHT
- struct backlight_device *backlight;
- u8 lcd_brightness;
- u8 lcd_power;
-#endif /* CONFIG_HID_PICOLCD_BACKLIGHT */
-#ifdef CONFIG_HID_PICOLCD_LEDS
- /* LED stuff */
- u8 led_state;
- struct led_classdev *led[8];
-#endif /* CONFIG_HID_PICOLCD_LEDS */
-
- /* Housekeeping stuff */
- spinlock_t lock;
- struct mutex mutex;
- struct picolcd_pending *pending;
- int status;
-#define PICOLCD_BOOTLOADER 1
-#define PICOLCD_FAILED 2
-#define PICOLCD_READY_FB 4
-};
-
-
-/* Find a given report */
-#define picolcd_in_report(id, dev) picolcd_report(id, dev, HID_INPUT_REPORT)
-#define picolcd_out_report(id, dev) picolcd_report(id, dev, HID_OUTPUT_REPORT)
-
-static struct hid_report *picolcd_report(int id, struct hid_device *hdev, int dir)
-{
- struct list_head *feature_report_list = &hdev->report_enum[dir].report_list;
- struct hid_report *report = NULL;
-
- list_for_each_entry(report, feature_report_list, list) {
- if (report->id == id)
- return report;
- }
- hid_warn(hdev, "No report with id 0x%x found\n", id);
- return NULL;
-}
-
-#ifdef CONFIG_DEBUG_FS
-static void picolcd_debug_out_report(struct picolcd_data *data,
- struct hid_device *hdev, struct hid_report *report);
-#define usbhid_submit_report(a, b, c) \
- do { \
- picolcd_debug_out_report(hid_get_drvdata(a), a, b); \
- usbhid_submit_report(a, b, c); \
- } while (0)
-#endif
-
-/* Submit a report and wait for a reply from device - if device fades away
- * or does not respond in time, return NULL */
-static struct picolcd_pending *picolcd_send_and_wait(struct hid_device *hdev,
- int report_id, const u8 *raw_data, int size)
-{
- struct picolcd_data *data = hid_get_drvdata(hdev);
- struct picolcd_pending *work;
- struct hid_report *report = picolcd_out_report(report_id, hdev);
- unsigned long flags;
- int i, j, k;
-
- if (!report || !data)
- return NULL;
- if (data->status & PICOLCD_FAILED)
- return NULL;
- work = kzalloc(sizeof(*work), GFP_KERNEL);
- if (!work)
- return NULL;
-
- init_completion(&work->ready);
- work->out_report = report;
- work->in_report = NULL;
- work->raw_size = 0;
-
- mutex_lock(&data->mutex);
- spin_lock_irqsave(&data->lock, flags);
- for (i = k = 0; i < report->maxfield; i++)
- for (j = 0; j < report->field[i]->report_count; j++) {
- hid_set_field(report->field[i], j, k < size ? raw_data[k] : 0);
- k++;
- }
- data->pending = work;
- usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
- wait_for_completion_interruptible_timeout(&work->ready, HZ*2);
- spin_lock_irqsave(&data->lock, flags);
- data->pending = NULL;
- spin_unlock_irqrestore(&data->lock, flags);
- mutex_unlock(&data->mutex);
- return work;
-}
-
-#ifdef CONFIG_HID_PICOLCD_FB
-/* Send a given tile to PicoLCD */
-static int picolcd_fb_send_tile(struct hid_device *hdev, int chip, int tile)
-{
- struct picolcd_data *data = hid_get_drvdata(hdev);
- struct hid_report *report1 = picolcd_out_report(REPORT_LCD_CMD_DATA, hdev);
- struct hid_report *report2 = picolcd_out_report(REPORT_LCD_DATA, hdev);
- unsigned long flags;
- u8 *tdata;
- int i;
-
- if (!report1 || report1->maxfield != 1 || !report2 || report2->maxfield != 1)
- return -ENODEV;
-
- spin_lock_irqsave(&data->lock, flags);
- hid_set_field(report1->field[0], 0, chip << 2);
- hid_set_field(report1->field[0], 1, 0x02);
- hid_set_field(report1->field[0], 2, 0x00);
- hid_set_field(report1->field[0], 3, 0x00);
- hid_set_field(report1->field[0], 4, 0xb8 | tile);
- hid_set_field(report1->field[0], 5, 0x00);
- hid_set_field(report1->field[0], 6, 0x00);
- hid_set_field(report1->field[0], 7, 0x40);
- hid_set_field(report1->field[0], 8, 0x00);
- hid_set_field(report1->field[0], 9, 0x00);
- hid_set_field(report1->field[0], 10, 32);
-
- hid_set_field(report2->field[0], 0, (chip << 2) | 0x01);
- hid_set_field(report2->field[0], 1, 0x00);
- hid_set_field(report2->field[0], 2, 0x00);
- hid_set_field(report2->field[0], 3, 32);
-
- tdata = data->fb_vbitmap + (tile * 4 + chip) * 64;
- for (i = 0; i < 64; i++)
- if (i < 32)
- hid_set_field(report1->field[0], 11 + i, tdata[i]);
- else
- hid_set_field(report2->field[0], 4 + i - 32, tdata[i]);
-
- usbhid_submit_report(data->hdev, report1, USB_DIR_OUT);
- usbhid_submit_report(data->hdev, report2, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
- return 0;
-}
-
-/* Translate a single tile*/
-static int picolcd_fb_update_tile(u8 *vbitmap, const u8 *bitmap, int bpp,
- int chip, int tile)
-{
- int i, b, changed = 0;
- u8 tdata[64];
- u8 *vdata = vbitmap + (tile * 4 + chip) * 64;
-
- if (bpp == 1) {
- for (b = 7; b >= 0; b--) {
- const u8 *bdata = bitmap + tile * 256 + chip * 8 + b * 32;
- for (i = 0; i < 64; i++) {
- tdata[i] <<= 1;
- tdata[i] |= (bdata[i/8] >> (i % 8)) & 0x01;
- }
- }
- } else if (bpp == 8) {
- for (b = 7; b >= 0; b--) {
- const u8 *bdata = bitmap + (tile * 256 + chip * 8 + b * 32) * 8;
- for (i = 0; i < 64; i++) {
- tdata[i] <<= 1;
- tdata[i] |= (bdata[i] & 0x80) ? 0x01 : 0x00;
- }
- }
- } else {
- /* Oops, we should never get here! */
- WARN_ON(1);
- return 0;
- }
-
- for (i = 0; i < 64; i++)
- if (tdata[i] != vdata[i]) {
- changed = 1;
- vdata[i] = tdata[i];
- }
- return changed;
-}
-
-/* Reconfigure LCD display */
-static int picolcd_fb_reset(struct picolcd_data *data, int clear)
-{
- struct hid_report *report = picolcd_out_report(REPORT_LCD_CMD, data->hdev);
- int i, j;
- unsigned long flags;
- static const u8 mapcmd[8] = { 0x00, 0x02, 0x00, 0x64, 0x3f, 0x00, 0x64, 0xc0 };
-
- if (!report || report->maxfield != 1)
- return -ENODEV;
-
- spin_lock_irqsave(&data->lock, flags);
- for (i = 0; i < 4; i++) {
- for (j = 0; j < report->field[0]->maxusage; j++)
- if (j == 0)
- hid_set_field(report->field[0], j, i << 2);
- else if (j < sizeof(mapcmd))
- hid_set_field(report->field[0], j, mapcmd[j]);
- else
- hid_set_field(report->field[0], j, 0);
- usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
- }
-
- data->status |= PICOLCD_READY_FB;
- spin_unlock_irqrestore(&data->lock, flags);
-
- if (data->fb_bitmap) {
- if (clear) {
- memset(data->fb_vbitmap, 0, PICOLCDFB_SIZE);
- memset(data->fb_bitmap, 0, PICOLCDFB_SIZE*data->fb_bpp);
- }
- data->fb_force = 1;
- }
-
- /* schedule first output of framebuffer */
- if (data->fb_info)
- schedule_delayed_work(&data->fb_info->deferred_work, 0);
-
- return 0;
-}
-
-/* Update fb_vbitmap from the screen_base and send changed tiles to device */
-static void picolcd_fb_update(struct picolcd_data *data)
-{
- int chip, tile, n;
- unsigned long flags;
-
- if (!data)
- return;
-
- spin_lock_irqsave(&data->lock, flags);
- if (!(data->status & PICOLCD_READY_FB)) {
- spin_unlock_irqrestore(&data->lock, flags);
- picolcd_fb_reset(data, 0);
- } else {
- spin_unlock_irqrestore(&data->lock, flags);
- }
-
- /*
- * Translate the framebuffer into the format needed by the PicoLCD.
- * See display layout above.
- * Do this one tile after the other and push those tiles that changed.
- *
- * Wait for our IO to complete as otherwise we might flood the queue!
- */
- n = 0;
- for (chip = 0; chip < 4; chip++)
- for (tile = 0; tile < 8; tile++)
- if (picolcd_fb_update_tile(data->fb_vbitmap,
- data->fb_bitmap, data->fb_bpp, chip, tile) ||
- data->fb_force) {
- n += 2;
- if (!data->fb_info->par)
- return; /* device lost! */
- if (n >= HID_OUTPUT_FIFO_SIZE / 2) {
- usbhid_wait_io(data->hdev);
- n = 0;
- }
- picolcd_fb_send_tile(data->hdev, chip, tile);
- }
- data->fb_force = false;
- if (n)
- usbhid_wait_io(data->hdev);
-}
-
-/* Stub to call the system default and update the image on the picoLCD */
-static void picolcd_fb_fillrect(struct fb_info *info,
- const struct fb_fillrect *rect)
-{
- if (!info->par)
- return;
- sys_fillrect(info, rect);
-
- schedule_delayed_work(&info->deferred_work, 0);
-}
-
-/* Stub to call the system default and update the image on the picoLCD */
-static void picolcd_fb_copyarea(struct fb_info *info,
- const struct fb_copyarea *area)
-{
- if (!info->par)
- return;
- sys_copyarea(info, area);
-
- schedule_delayed_work(&info->deferred_work, 0);
-}
-
-/* Stub to call the system default and update the image on the picoLCD */
-static void picolcd_fb_imageblit(struct fb_info *info, const struct fb_image *image)
-{
- if (!info->par)
- return;
- sys_imageblit(info, image);
-
- schedule_delayed_work(&info->deferred_work, 0);
-}
-
-/*
- * this is the slow path from userspace. they can seek and write to
- * the fb. it's inefficient to do anything less than a full screen draw
- */
-static ssize_t picolcd_fb_write(struct fb_info *info, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- ssize_t ret;
- if (!info->par)
- return -ENODEV;
- ret = fb_sys_write(info, buf, count, ppos);
- if (ret >= 0)
- schedule_delayed_work(&info->deferred_work, 0);
- return ret;
-}
-
-static int picolcd_fb_blank(int blank, struct fb_info *info)
-{
- if (!info->par)
- return -ENODEV;
- /* We let fb notification do this for us via lcd/backlight device */
- return 0;
-}
-
-static void picolcd_fb_destroy(struct fb_info *info)
-{
- struct picolcd_data *data = info->par;
- u32 *ref_cnt = info->pseudo_palette;
- int may_release;
-
- info->par = NULL;
- if (data)
- data->fb_info = NULL;
- fb_deferred_io_cleanup(info);
-
- ref_cnt--;
- mutex_lock(&info->lock);
- (*ref_cnt)--;
- may_release = !*ref_cnt;
- mutex_unlock(&info->lock);
- if (may_release) {
- vfree((u8 *)info->fix.smem_start);
- framebuffer_release(info);
- }
-}
-
-static int picolcd_fb_check_var(struct fb_var_screeninfo *var, struct fb_info *info)
-{
- __u32 bpp = var->bits_per_pixel;
- __u32 activate = var->activate;
-
- /* only allow 1/8 bit depth (8-bit is grayscale) */
- *var = picolcdfb_var;
- var->activate = activate;
- if (bpp >= 8) {
- var->bits_per_pixel = 8;
- var->red.length = 8;
- var->green.length = 8;
- var->blue.length = 8;
- } else {
- var->bits_per_pixel = 1;
- var->red.length = 1;
- var->green.length = 1;
- var->blue.length = 1;
- }
- return 0;
-}
-
-static int picolcd_set_par(struct fb_info *info)
-{
- struct picolcd_data *data = info->par;
- u8 *tmp_fb, *o_fb;
- if (!data)
- return -ENODEV;
- if (info->var.bits_per_pixel == data->fb_bpp)
- return 0;
- /* switch between 1/8 bit depths */
- if (info->var.bits_per_pixel != 1 && info->var.bits_per_pixel != 8)
- return -EINVAL;
-
- o_fb = data->fb_bitmap;
- tmp_fb = kmalloc(PICOLCDFB_SIZE*info->var.bits_per_pixel, GFP_KERNEL);
- if (!tmp_fb)
- return -ENOMEM;
-
- /* translate FB content to new bits-per-pixel */
- if (info->var.bits_per_pixel == 1) {
- int i, b;
- for (i = 0; i < PICOLCDFB_SIZE; i++) {
- u8 p = 0;
- for (b = 0; b < 8; b++) {
- p <<= 1;
- p |= o_fb[i*8+b] ? 0x01 : 0x00;
- }
- tmp_fb[i] = p;
- }
- memcpy(o_fb, tmp_fb, PICOLCDFB_SIZE);
- info->fix.visual = FB_VISUAL_MONO01;
- info->fix.line_length = PICOLCDFB_WIDTH / 8;
- } else {
- int i;
- memcpy(tmp_fb, o_fb, PICOLCDFB_SIZE);
- for (i = 0; i < PICOLCDFB_SIZE * 8; i++)
- o_fb[i] = tmp_fb[i/8] & (0x01 << (7 - i % 8)) ? 0xff : 0x00;
- info->fix.visual = FB_VISUAL_DIRECTCOLOR;
- info->fix.line_length = PICOLCDFB_WIDTH;
- }
-
- kfree(tmp_fb);
- data->fb_bpp = info->var.bits_per_pixel;
- return 0;
-}
-
-/* Do refcounting on our FB and cleanup per worker if FB is
- * closed after unplug of our device
- * (fb_release holds info->lock and still touches info after
- * we return so we can't release it immediately.
- */
-struct picolcd_fb_cleanup_item {
- struct fb_info *info;
- struct picolcd_fb_cleanup_item *next;
-};
-static struct picolcd_fb_cleanup_item *fb_pending;
-static DEFINE_SPINLOCK(fb_pending_lock);
-
-static void picolcd_fb_do_cleanup(struct work_struct *data)
-{
- struct picolcd_fb_cleanup_item *item;
- unsigned long flags;
-
- do {
- spin_lock_irqsave(&fb_pending_lock, flags);
- item = fb_pending;
- fb_pending = item ? item->next : NULL;
- spin_unlock_irqrestore(&fb_pending_lock, flags);
-
- if (item) {
- u8 *fb = (u8 *)item->info->fix.smem_start;
- /* make sure we do not race against fb core when
- * releasing */
- mutex_lock(&item->info->lock);
- mutex_unlock(&item->info->lock);
- framebuffer_release(item->info);
- vfree(fb);
- }
- } while (item);
-}
-
-static DECLARE_WORK(picolcd_fb_cleanup, picolcd_fb_do_cleanup);
-
-static int picolcd_fb_open(struct fb_info *info, int u)
-{
- u32 *ref_cnt = info->pseudo_palette;
- ref_cnt--;
-
- (*ref_cnt)++;
- return 0;
-}
-
-static int picolcd_fb_release(struct fb_info *info, int u)
-{
- u32 *ref_cnt = info->pseudo_palette;
- ref_cnt--;
-
- (*ref_cnt)++;
- if (!*ref_cnt) {
- unsigned long flags;
- struct picolcd_fb_cleanup_item *item = (struct picolcd_fb_cleanup_item *)ref_cnt;
- item--;
- spin_lock_irqsave(&fb_pending_lock, flags);
- item->next = fb_pending;
- fb_pending = item;
- spin_unlock_irqrestore(&fb_pending_lock, flags);
- schedule_work(&picolcd_fb_cleanup);
- }
- return 0;
-}
-
-/* Note this can't be const because of struct fb_info definition */
-static struct fb_ops picolcdfb_ops = {
- .owner = THIS_MODULE,
- .fb_destroy = picolcd_fb_destroy,
- .fb_open = picolcd_fb_open,
- .fb_release = picolcd_fb_release,
- .fb_read = fb_sys_read,
- .fb_write = picolcd_fb_write,
- .fb_blank = picolcd_fb_blank,
- .fb_fillrect = picolcd_fb_fillrect,
- .fb_copyarea = picolcd_fb_copyarea,
- .fb_imageblit = picolcd_fb_imageblit,
- .fb_check_var = picolcd_fb_check_var,
- .fb_set_par = picolcd_set_par,
-};
-
-
-/* Callback from deferred IO workqueue */
-static void picolcd_fb_deferred_io(struct fb_info *info, struct list_head *pagelist)
-{
- picolcd_fb_update(info->par);
-}
-
-static const struct fb_deferred_io picolcd_fb_defio = {
- .delay = HZ / PICOLCDFB_UPDATE_RATE_DEFAULT,
- .deferred_io = picolcd_fb_deferred_io,
-};
-
-
-/*
- * The "fb_update_rate" sysfs attribute
- */
-static ssize_t picolcd_fb_update_rate_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct picolcd_data *data = dev_get_drvdata(dev);
- unsigned i, fb_update_rate = data->fb_update_rate;
- size_t ret = 0;
-
- for (i = 1; i <= PICOLCDFB_UPDATE_RATE_LIMIT; i++)
- if (ret >= PAGE_SIZE)
- break;
- else if (i == fb_update_rate)
- ret += snprintf(buf+ret, PAGE_SIZE-ret, "[%u] ", i);
- else
- ret += snprintf(buf+ret, PAGE_SIZE-ret, "%u ", i);
- if (ret > 0)
- buf[min(ret, (size_t)PAGE_SIZE)-1] = '\n';
- return ret;
-}
-
-static ssize_t picolcd_fb_update_rate_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t count)
-{
- struct picolcd_data *data = dev_get_drvdata(dev);
- int i;
- unsigned u;
-
- if (count < 1 || count > 10)
- return -EINVAL;
-
- i = sscanf(buf, "%u", &u);
- if (i != 1)
- return -EINVAL;
-
- if (u > PICOLCDFB_UPDATE_RATE_LIMIT)
- return -ERANGE;
- else if (u == 0)
- u = PICOLCDFB_UPDATE_RATE_DEFAULT;
-
- data->fb_update_rate = u;
- data->fb_defio.delay = HZ / data->fb_update_rate;
- return count;
-}
-
-static DEVICE_ATTR(fb_update_rate, 0666, picolcd_fb_update_rate_show,
- picolcd_fb_update_rate_store);
-
-/* initialize Framebuffer device */
-static int picolcd_init_framebuffer(struct picolcd_data *data)
-{
- struct device *dev = &data->hdev->dev;
- struct fb_info *info = NULL;
- int i, error = -ENOMEM;
- u8 *fb_vbitmap = NULL;
- u8 *fb_bitmap = NULL;
- u32 *palette;
-
- fb_bitmap = vmalloc(PICOLCDFB_SIZE*8);
- if (fb_bitmap == NULL) {
- dev_err(dev, "can't get a free page for framebuffer\n");
- goto err_nomem;
- }
-
- fb_vbitmap = kmalloc(PICOLCDFB_SIZE, GFP_KERNEL);
- if (fb_vbitmap == NULL) {
- dev_err(dev, "can't alloc vbitmap image buffer\n");
- goto err_nomem;
- }
-
- data->fb_update_rate = PICOLCDFB_UPDATE_RATE_DEFAULT;
- data->fb_defio = picolcd_fb_defio;
- /* The extra memory is:
- * - struct picolcd_fb_cleanup_item
- * - u32 for ref_count
- * - 256*u32 for pseudo_palette
- */
- info = framebuffer_alloc(257 * sizeof(u32) + sizeof(struct picolcd_fb_cleanup_item), dev);
- if (info == NULL) {
- dev_err(dev, "failed to allocate a framebuffer\n");
- goto err_nomem;
- }
-
- palette = info->par + sizeof(struct picolcd_fb_cleanup_item);
- *palette = 1;
- palette++;
- for (i = 0; i < 256; i++)
- palette[i] = i > 0 && i < 16 ? 0xff : 0;
- info->pseudo_palette = palette;
- info->fbdefio = &data->fb_defio;
- info->screen_base = (char __force __iomem *)fb_bitmap;
- info->fbops = &picolcdfb_ops;
- info->var = picolcdfb_var;
- info->fix = picolcdfb_fix;
- info->fix.smem_len = PICOLCDFB_SIZE*8;
- info->fix.smem_start = (unsigned long)fb_bitmap;
- info->par = data;
- info->flags = FBINFO_FLAG_DEFAULT;
-
- data->fb_vbitmap = fb_vbitmap;
- data->fb_bitmap = fb_bitmap;
- data->fb_bpp = picolcdfb_var.bits_per_pixel;
- error = picolcd_fb_reset(data, 1);
- if (error) {
- dev_err(dev, "failed to configure display\n");
- goto err_cleanup;
- }
- error = device_create_file(dev, &dev_attr_fb_update_rate);
- if (error) {
- dev_err(dev, "failed to create sysfs attributes\n");
- goto err_cleanup;
- }
- fb_deferred_io_init(info);
- data->fb_info = info;
- error = register_framebuffer(info);
- if (error) {
- dev_err(dev, "failed to register framebuffer\n");
- goto err_sysfs;
- }
- /* schedule first output of framebuffer */
- data->fb_force = 1;
- schedule_delayed_work(&info->deferred_work, 0);
- return 0;
-
-err_sysfs:
- fb_deferred_io_cleanup(info);
- device_remove_file(dev, &dev_attr_fb_update_rate);
-err_cleanup:
- data->fb_vbitmap = NULL;
- data->fb_bitmap = NULL;
- data->fb_bpp = 0;
- data->fb_info = NULL;
-
-err_nomem:
- framebuffer_release(info);
- vfree(fb_bitmap);
- kfree(fb_vbitmap);
- return error;
-}
-
-static void picolcd_exit_framebuffer(struct picolcd_data *data)
-{
- struct fb_info *info = data->fb_info;
- u8 *fb_vbitmap = data->fb_vbitmap;
-
- if (!info)
- return;
-
- info->par = NULL;
- device_remove_file(&data->hdev->dev, &dev_attr_fb_update_rate);
- unregister_framebuffer(info);
- data->fb_vbitmap = NULL;
- data->fb_bitmap = NULL;
- data->fb_bpp = 0;
- data->fb_info = NULL;
- kfree(fb_vbitmap);
-}
-
-#define picolcd_fbinfo(d) ((d)->fb_info)
-#else
-static inline int picolcd_fb_reset(struct picolcd_data *data, int clear)
-{
- return 0;
-}
-static inline int picolcd_init_framebuffer(struct picolcd_data *data)
-{
- return 0;
-}
-static inline void picolcd_exit_framebuffer(struct picolcd_data *data)
-{
-}
-#define picolcd_fbinfo(d) NULL
-#endif /* CONFIG_HID_PICOLCD_FB */
-
-#ifdef CONFIG_HID_PICOLCD_BACKLIGHT
-/*
- * backlight class device
- */
-static int picolcd_get_brightness(struct backlight_device *bdev)
-{
- struct picolcd_data *data = bl_get_data(bdev);
- return data->lcd_brightness;
-}
-
-static int picolcd_set_brightness(struct backlight_device *bdev)
-{
- struct picolcd_data *data = bl_get_data(bdev);
- struct hid_report *report = picolcd_out_report(REPORT_BRIGHTNESS, data->hdev);
- unsigned long flags;
-
- if (!report || report->maxfield != 1 || report->field[0]->report_count != 1)
- return -ENODEV;
-
- data->lcd_brightness = bdev->props.brightness & 0x0ff;
- data->lcd_power = bdev->props.power;
- spin_lock_irqsave(&data->lock, flags);
- hid_set_field(report->field[0], 0, data->lcd_power == FB_BLANK_UNBLANK ? data->lcd_brightness : 0);
- usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
- return 0;
-}
-
-static int picolcd_check_bl_fb(struct backlight_device *bdev, struct fb_info *fb)
-{
- return fb && fb == picolcd_fbinfo((struct picolcd_data *)bl_get_data(bdev));
-}
-
-static const struct backlight_ops picolcd_blops = {
- .update_status = picolcd_set_brightness,
- .get_brightness = picolcd_get_brightness,
- .check_fb = picolcd_check_bl_fb,
-};
-
-static int picolcd_init_backlight(struct picolcd_data *data, struct hid_report *report)
-{
- struct device *dev = &data->hdev->dev;
- struct backlight_device *bdev;
- struct backlight_properties props;
- if (!report)
- return -ENODEV;
- if (report->maxfield != 1 || report->field[0]->report_count != 1 ||
- report->field[0]->report_size != 8) {
- dev_err(dev, "unsupported BRIGHTNESS report");
- return -EINVAL;
- }
-
- memset(&props, 0, sizeof(props));
- props.type = BACKLIGHT_RAW;
- props.max_brightness = 0xff;
- bdev = backlight_device_register(dev_name(dev), dev, data,
- &picolcd_blops, &props);
- if (IS_ERR(bdev)) {
- dev_err(dev, "failed to register backlight\n");
- return PTR_ERR(bdev);
- }
- bdev->props.brightness = 0xff;
- data->lcd_brightness = 0xff;
- data->backlight = bdev;
- picolcd_set_brightness(bdev);
- return 0;
-}
-
-static void picolcd_exit_backlight(struct picolcd_data *data)
-{
- struct backlight_device *bdev = data->backlight;
-
- data->backlight = NULL;
- if (bdev)
- backlight_device_unregister(bdev);
-}
-
-static inline int picolcd_resume_backlight(struct picolcd_data *data)
-{
- if (!data->backlight)
- return 0;
- return picolcd_set_brightness(data->backlight);
-}
-
-#ifdef CONFIG_PM
-static void picolcd_suspend_backlight(struct picolcd_data *data)
-{
- int bl_power = data->lcd_power;
- if (!data->backlight)
- return;
-
- data->backlight->props.power = FB_BLANK_POWERDOWN;
- picolcd_set_brightness(data->backlight);
- data->lcd_power = data->backlight->props.power = bl_power;
-}
-#endif /* CONFIG_PM */
-#else
-static inline int picolcd_init_backlight(struct picolcd_data *data,
- struct hid_report *report)
-{
- return 0;
-}
-static inline void picolcd_exit_backlight(struct picolcd_data *data)
-{
-}
-static inline int picolcd_resume_backlight(struct picolcd_data *data)
-{
- return 0;
-}
-static inline void picolcd_suspend_backlight(struct picolcd_data *data)
-{
-}
-#endif /* CONFIG_HID_PICOLCD_BACKLIGHT */
-
-#ifdef CONFIG_HID_PICOLCD_LCD
-/*
- * lcd class device
- */
-static int picolcd_get_contrast(struct lcd_device *ldev)
-{
- struct picolcd_data *data = lcd_get_data(ldev);
- return data->lcd_contrast;
-}
-
-static int picolcd_set_contrast(struct lcd_device *ldev, int contrast)
-{
- struct picolcd_data *data = lcd_get_data(ldev);
- struct hid_report *report = picolcd_out_report(REPORT_CONTRAST, data->hdev);
- unsigned long flags;
-
- if (!report || report->maxfield != 1 || report->field[0]->report_count != 1)
- return -ENODEV;
-
- data->lcd_contrast = contrast & 0x0ff;
- spin_lock_irqsave(&data->lock, flags);
- hid_set_field(report->field[0], 0, data->lcd_contrast);
- usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
- return 0;
-}
-
-static int picolcd_check_lcd_fb(struct lcd_device *ldev, struct fb_info *fb)
-{
- return fb && fb == picolcd_fbinfo((struct picolcd_data *)lcd_get_data(ldev));
-}
-
-static struct lcd_ops picolcd_lcdops = {
- .get_contrast = picolcd_get_contrast,
- .set_contrast = picolcd_set_contrast,
- .check_fb = picolcd_check_lcd_fb,
-};
-
-static int picolcd_init_lcd(struct picolcd_data *data, struct hid_report *report)
-{
- struct device *dev = &data->hdev->dev;
- struct lcd_device *ldev;
-
- if (!report)
- return -ENODEV;
- if (report->maxfield != 1 || report->field[0]->report_count != 1 ||
- report->field[0]->report_size != 8) {
- dev_err(dev, "unsupported CONTRAST report");
- return -EINVAL;
- }
-
- ldev = lcd_device_register(dev_name(dev), dev, data, &picolcd_lcdops);
- if (IS_ERR(ldev)) {
- dev_err(dev, "failed to register LCD\n");
- return PTR_ERR(ldev);
- }
- ldev->props.max_contrast = 0x0ff;
- data->lcd_contrast = 0xe5;
- data->lcd = ldev;
- picolcd_set_contrast(ldev, 0xe5);
- return 0;
-}
-
-static void picolcd_exit_lcd(struct picolcd_data *data)
-{
- struct lcd_device *ldev = data->lcd;
-
- data->lcd = NULL;
- if (ldev)
- lcd_device_unregister(ldev);
-}
-
-static inline int picolcd_resume_lcd(struct picolcd_data *data)
-{
- if (!data->lcd)
- return 0;
- return picolcd_set_contrast(data->lcd, data->lcd_contrast);
-}
-#else
-static inline int picolcd_init_lcd(struct picolcd_data *data,
- struct hid_report *report)
-{
- return 0;
-}
-static inline void picolcd_exit_lcd(struct picolcd_data *data)
-{
-}
-static inline int picolcd_resume_lcd(struct picolcd_data *data)
-{
- return 0;
-}
-#endif /* CONFIG_HID_PICOLCD_LCD */
-
-#ifdef CONFIG_HID_PICOLCD_LEDS
-/**
- * LED class device
- */
-static void picolcd_leds_set(struct picolcd_data *data)
-{
- struct hid_report *report;
- unsigned long flags;
-
- if (!data->led[0])
- return;
- report = picolcd_out_report(REPORT_LED_STATE, data->hdev);
- if (!report || report->maxfield != 1 || report->field[0]->report_count != 1)
- return;
-
- spin_lock_irqsave(&data->lock, flags);
- hid_set_field(report->field[0], 0, data->led_state);
- usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
-}
-
-static void picolcd_led_set_brightness(struct led_classdev *led_cdev,
- enum led_brightness value)
-{
- struct device *dev;
- struct hid_device *hdev;
- struct picolcd_data *data;
- int i, state = 0;
-
- dev = led_cdev->dev->parent;
- hdev = container_of(dev, struct hid_device, dev);
- data = hid_get_drvdata(hdev);
- for (i = 0; i < 8; i++) {
- if (led_cdev != data->led[i])
- continue;
- state = (data->led_state >> i) & 1;
- if (value == LED_OFF && state) {
- data->led_state &= ~(1 << i);
- picolcd_leds_set(data);
- } else if (value != LED_OFF && !state) {
- data->led_state |= 1 << i;
- picolcd_leds_set(data);
- }
- break;
- }
-}
-
-static enum led_brightness picolcd_led_get_brightness(struct led_classdev *led_cdev)
-{
- struct device *dev;
- struct hid_device *hdev;
- struct picolcd_data *data;
- int i, value = 0;
-
- dev = led_cdev->dev->parent;
- hdev = container_of(dev, struct hid_device, dev);
- data = hid_get_drvdata(hdev);
- for (i = 0; i < 8; i++)
- if (led_cdev == data->led[i]) {
- value = (data->led_state >> i) & 1;
- break;
- }
- return value ? LED_FULL : LED_OFF;
-}
-
-static int picolcd_init_leds(struct picolcd_data *data, struct hid_report *report)
-{
- struct device *dev = &data->hdev->dev;
- struct led_classdev *led;
- size_t name_sz = strlen(dev_name(dev)) + 8;
- char *name;
- int i, ret = 0;
-
- if (!report)
- return -ENODEV;
- if (report->maxfield != 1 || report->field[0]->report_count != 1 ||
- report->field[0]->report_size != 8) {
- dev_err(dev, "unsupported LED_STATE report");
- return -EINVAL;
- }
-
- for (i = 0; i < 8; i++) {
- led = kzalloc(sizeof(struct led_classdev)+name_sz, GFP_KERNEL);
- if (!led) {
- dev_err(dev, "can't allocate memory for LED %d\n", i);
- ret = -ENOMEM;
- goto err;
- }
- name = (void *)(&led[1]);
- snprintf(name, name_sz, "%s::GPO%d", dev_name(dev), i);
- led->name = name;
- led->brightness = 0;
- led->max_brightness = 1;
- led->brightness_get = picolcd_led_get_brightness;
- led->brightness_set = picolcd_led_set_brightness;
-
- data->led[i] = led;
- ret = led_classdev_register(dev, data->led[i]);
- if (ret) {
- data->led[i] = NULL;
- kfree(led);
- dev_err(dev, "can't register LED %d\n", i);
- goto err;
- }
- }
- return 0;
-err:
- for (i = 0; i < 8; i++)
- if (data->led[i]) {
- led = data->led[i];
- data->led[i] = NULL;
- led_classdev_unregister(led);
- kfree(led);
- }
- return ret;
-}
-
-static void picolcd_exit_leds(struct picolcd_data *data)
-{
- struct led_classdev *led;
- int i;
-
- for (i = 0; i < 8; i++) {
- led = data->led[i];
- data->led[i] = NULL;
- if (!led)
- continue;
- led_classdev_unregister(led);
- kfree(led);
- }
-}
-
-#else
-static inline int picolcd_init_leds(struct picolcd_data *data,
- struct hid_report *report)
-{
- return 0;
-}
-static inline void picolcd_exit_leds(struct picolcd_data *data)
-{
-}
-static inline int picolcd_leds_set(struct picolcd_data *data)
-{
- return 0;
-}
-#endif /* CONFIG_HID_PICOLCD_LEDS */
-
-/*
- * input class device
- */
-static int picolcd_raw_keypad(struct picolcd_data *data,
- struct hid_report *report, u8 *raw_data, int size)
-{
- /*
- * Keypad event
- * First and second data bytes list currently pressed keys,
- * 0x00 means no key and at most 2 keys may be pressed at same time
- */
- int i, j;
-
- /* determine newly pressed keys */
- for (i = 0; i < size; i++) {
- unsigned int key_code;
- if (raw_data[i] == 0)
- continue;
- for (j = 0; j < sizeof(data->pressed_keys); j++)
- if (data->pressed_keys[j] == raw_data[i])
- goto key_already_down;
- for (j = 0; j < sizeof(data->pressed_keys); j++)
- if (data->pressed_keys[j] == 0) {
- data->pressed_keys[j] = raw_data[i];
- break;
- }
- input_event(data->input_keys, EV_MSC, MSC_SCAN, raw_data[i]);
- if (raw_data[i] < PICOLCD_KEYS)
- key_code = data->keycode[raw_data[i]];
- else
- key_code = KEY_UNKNOWN;
- if (key_code != KEY_UNKNOWN) {
- dbg_hid(PICOLCD_NAME " got key press for %u:%d",
- raw_data[i], key_code);
- input_report_key(data->input_keys, key_code, 1);
- }
- input_sync(data->input_keys);
-key_already_down:
- continue;
- }
-
- /* determine newly released keys */
- for (j = 0; j < sizeof(data->pressed_keys); j++) {
- unsigned int key_code;
- if (data->pressed_keys[j] == 0)
- continue;
- for (i = 0; i < size; i++)
- if (data->pressed_keys[j] == raw_data[i])
- goto key_still_down;
- input_event(data->input_keys, EV_MSC, MSC_SCAN, data->pressed_keys[j]);
- if (data->pressed_keys[j] < PICOLCD_KEYS)
- key_code = data->keycode[data->pressed_keys[j]];
- else
- key_code = KEY_UNKNOWN;
- if (key_code != KEY_UNKNOWN) {
- dbg_hid(PICOLCD_NAME " got key release for %u:%d",
- data->pressed_keys[j], key_code);
- input_report_key(data->input_keys, key_code, 0);
- }
- input_sync(data->input_keys);
- data->pressed_keys[j] = 0;
-key_still_down:
- continue;
- }
- return 1;
-}
-
-static int picolcd_raw_cir(struct picolcd_data *data,
- struct hid_report *report, u8 *raw_data, int size)
-{
- /* Need understanding of CIR data format to implement ... */
- return 1;
-}
-
-static int picolcd_check_version(struct hid_device *hdev)
-{
- struct picolcd_data *data = hid_get_drvdata(hdev);
- struct picolcd_pending *verinfo;
- int ret = 0;
-
- if (!data)
- return -ENODEV;
-
- verinfo = picolcd_send_and_wait(hdev, REPORT_VERSION, NULL, 0);
- if (!verinfo) {
- hid_err(hdev, "no version response from PicoLCD\n");
- return -ENODEV;
- }
-
- if (verinfo->raw_size == 2) {
- data->version[0] = verinfo->raw_data[1];
- data->version[1] = verinfo->raw_data[0];
- if (data->status & PICOLCD_BOOTLOADER) {
- hid_info(hdev, "PicoLCD, bootloader version %d.%d\n",
- verinfo->raw_data[1], verinfo->raw_data[0]);
- } else {
- hid_info(hdev, "PicoLCD, firmware version %d.%d\n",
- verinfo->raw_data[1], verinfo->raw_data[0]);
- }
- } else {
- hid_err(hdev, "confused, got unexpected version response from PicoLCD\n");
- ret = -EINVAL;
- }
- kfree(verinfo);
- return ret;
-}
-
-/*
- * Reset our device and wait for answer to VERSION request
- */
-static int picolcd_reset(struct hid_device *hdev)
-{
- struct picolcd_data *data = hid_get_drvdata(hdev);
- struct hid_report *report = picolcd_out_report(REPORT_RESET, hdev);
- unsigned long flags;
- int error;
-
- if (!data || !report || report->maxfield != 1)
- return -ENODEV;
-
- spin_lock_irqsave(&data->lock, flags);
- if (hdev->product == USB_DEVICE_ID_PICOLCD_BOOTLOADER)
- data->status |= PICOLCD_BOOTLOADER;
-
- /* perform the reset */
- hid_set_field(report->field[0], 0, 1);
- usbhid_submit_report(hdev, report, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
-
- error = picolcd_check_version(hdev);
- if (error)
- return error;
-
- picolcd_resume_lcd(data);
- picolcd_resume_backlight(data);
-#ifdef CONFIG_HID_PICOLCD_FB
- if (data->fb_info)
- schedule_delayed_work(&data->fb_info->deferred_work, 0);
-#endif /* CONFIG_HID_PICOLCD_FB */
-
- picolcd_leds_set(data);
- return 0;
-}
-
-/*
- * The "operation_mode" sysfs attribute
- */
-static ssize_t picolcd_operation_mode_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct picolcd_data *data = dev_get_drvdata(dev);
-
- if (data->status & PICOLCD_BOOTLOADER)
- return snprintf(buf, PAGE_SIZE, "[bootloader] lcd\n");
- else
- return snprintf(buf, PAGE_SIZE, "bootloader [lcd]\n");
-}
-
-static ssize_t picolcd_operation_mode_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t count)
-{
- struct picolcd_data *data = dev_get_drvdata(dev);
- struct hid_report *report = NULL;
- size_t cnt = count;
- int timeout = data->opmode_delay;
- unsigned long flags;
-
- if (cnt >= 3 && strncmp("lcd", buf, 3) == 0) {
- if (data->status & PICOLCD_BOOTLOADER)
- report = picolcd_out_report(REPORT_EXIT_FLASHER, data->hdev);
- buf += 3;
- cnt -= 3;
- } else if (cnt >= 10 && strncmp("bootloader", buf, 10) == 0) {
- if (!(data->status & PICOLCD_BOOTLOADER))
- report = picolcd_out_report(REPORT_EXIT_KEYBOARD, data->hdev);
- buf += 10;
- cnt -= 10;
- }
- if (!report)
- return -EINVAL;
-
- while (cnt > 0 && (buf[cnt-1] == '\n' || buf[cnt-1] == '\r'))
- cnt--;
- if (cnt != 0)
- return -EINVAL;
-
- spin_lock_irqsave(&data->lock, flags);
- hid_set_field(report->field[0], 0, timeout & 0xff);
- hid_set_field(report->field[0], 1, (timeout >> 8) & 0xff);
- usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
- spin_unlock_irqrestore(&data->lock, flags);
- return count;
-}
-
-static DEVICE_ATTR(operation_mode, 0644, picolcd_operation_mode_show,
- picolcd_operation_mode_store);
-
-/*
- * The "operation_mode_delay" sysfs attribute
- */
-static ssize_t picolcd_operation_mode_delay_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct picolcd_data *data = dev_get_drvdata(dev);
-
- return snprintf(buf, PAGE_SIZE, "%hu\n", data->opmode_delay);
-}
-
-static ssize_t picolcd_operation_mode_delay_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t count)
-{
- struct picolcd_data *data = dev_get_drvdata(dev);
- unsigned u;
- if (sscanf(buf, "%u", &u) != 1)
- return -EINVAL;
- if (u > 30000)
- return -EINVAL;
- else
- data->opmode_delay = u;
- return count;
-}
-
-static DEVICE_ATTR(operation_mode_delay, 0644, picolcd_operation_mode_delay_show,
- picolcd_operation_mode_delay_store);
-
-
-#ifdef CONFIG_DEBUG_FS
-/*
- * The "reset" file
- */
-static int picolcd_debug_reset_show(struct seq_file *f, void *p)
-{
- if (picolcd_fbinfo((struct picolcd_data *)f->private))
- seq_printf(f, "all fb\n");
- else
- seq_printf(f, "all\n");
- return 0;
-}
-
-static int picolcd_debug_reset_open(struct inode *inode, struct file *f)
-{
- return single_open(f, picolcd_debug_reset_show, inode->i_private);
-}
-
-static ssize_t picolcd_debug_reset_write(struct file *f, const char __user *user_buf,
- size_t count, loff_t *ppos)
-{
- struct picolcd_data *data = ((struct seq_file *)f->private_data)->private;
- char buf[32];
- size_t cnt = min(count, sizeof(buf)-1);
- if (copy_from_user(buf, user_buf, cnt))
- return -EFAULT;
-
- while (cnt > 0 && (buf[cnt-1] == ' ' || buf[cnt-1] == '\n'))
- cnt--;
- buf[cnt] = '\0';
- if (strcmp(buf, "all") == 0) {
- picolcd_reset(data->hdev);
- picolcd_fb_reset(data, 1);
- } else if (strcmp(buf, "fb") == 0) {
- picolcd_fb_reset(data, 1);
- } else {
- return -EINVAL;
- }
- return count;
-}
-
-static const struct file_operations picolcd_debug_reset_fops = {
- .owner = THIS_MODULE,
- .open = picolcd_debug_reset_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .write = picolcd_debug_reset_write,
- .release = single_release,
-};
-
-/*
- * The "eeprom" file
- */
-static ssize_t picolcd_debug_eeprom_read(struct file *f, char __user *u,
- size_t s, loff_t *off)
-{
- struct picolcd_data *data = f->private_data;
- struct picolcd_pending *resp;
- u8 raw_data[3];
- ssize_t ret = -EIO;
-
- if (s == 0)
- return -EINVAL;
- if (*off > 0x0ff)
- return 0;
-
- /* prepare buffer with info about what we want to read (addr & len) */
- raw_data[0] = *off & 0xff;
- raw_data[1] = (*off >> 8) & 0xff;
- raw_data[2] = s < 20 ? s : 20;
- if (*off + raw_data[2] > 0xff)
- raw_data[2] = 0x100 - *off;
- resp = picolcd_send_and_wait(data->hdev, REPORT_EE_READ, raw_data,
- sizeof(raw_data));
- if (!resp)
- return -EIO;
-
- if (resp->in_report && resp->in_report->id == REPORT_EE_DATA) {
- /* successful read :) */
- ret = resp->raw_data[2];
- if (ret > s)
- ret = s;
- if (copy_to_user(u, resp->raw_data+3, ret))
- ret = -EFAULT;
- else
- *off += ret;
- } /* anything else is some kind of IO error */
-
- kfree(resp);
- return ret;
-}
-
-static ssize_t picolcd_debug_eeprom_write(struct file *f, const char __user *u,
- size_t s, loff_t *off)
-{
- struct picolcd_data *data = f->private_data;
- struct picolcd_pending *resp;
- ssize_t ret = -EIO;
- u8 raw_data[23];
-
- if (s == 0)
- return -EINVAL;
- if (*off > 0x0ff)
- return -ENOSPC;
-
- memset(raw_data, 0, sizeof(raw_data));
- raw_data[0] = *off & 0xff;
- raw_data[1] = (*off >> 8) & 0xff;
- raw_data[2] = min((size_t)20, s);
- if (*off + raw_data[2] > 0xff)
- raw_data[2] = 0x100 - *off;
-
- if (copy_from_user(raw_data+3, u, min((u8)20, raw_data[2])))
- return -EFAULT;
- resp = picolcd_send_and_wait(data->hdev, REPORT_EE_WRITE, raw_data,
- sizeof(raw_data));
-
- if (!resp)
- return -EIO;
-
- if (resp->in_report && resp->in_report->id == REPORT_EE_DATA) {
- /* check if written data matches */
- if (memcmp(raw_data, resp->raw_data, 3+raw_data[2]) == 0) {
- *off += raw_data[2];
- ret = raw_data[2];
- }
- }
- kfree(resp);
- return ret;
-}
-
-/*
- * Notes:
- * - read/write happens in chunks of at most 20 bytes, it's up to userspace
- * to loop in order to get more data.
- * - on write errors on otherwise correct write request the bytes
- * that should have been written are in undefined state.
- */
-static const struct file_operations picolcd_debug_eeprom_fops = {
- .owner = THIS_MODULE,
- .open = simple_open,
- .read = picolcd_debug_eeprom_read,
- .write = picolcd_debug_eeprom_write,
- .llseek = generic_file_llseek,
-};
-
-/*
- * The "flash" file
- */
-/* record a flash address to buf (bounds check to be done by caller) */
-static int _picolcd_flash_setaddr(struct picolcd_data *data, u8 *buf, long off)
-{
- buf[0] = off & 0xff;
- buf[1] = (off >> 8) & 0xff;
- if (data->addr_sz == 3)
- buf[2] = (off >> 16) & 0xff;
- return data->addr_sz == 2 ? 2 : 3;
-}
-
-/* read a given size of data (bounds check to be done by caller) */
-static ssize_t _picolcd_flash_read(struct picolcd_data *data, int report_id,
- char __user *u, size_t s, loff_t *off)
-{
- struct picolcd_pending *resp;
- u8 raw_data[4];
- ssize_t ret = 0;
- int len_off, err = -EIO;
-
- while (s > 0) {
- err = -EIO;
- len_off = _picolcd_flash_setaddr(data, raw_data, *off);
- raw_data[len_off] = s > 32 ? 32 : s;
- resp = picolcd_send_and_wait(data->hdev, report_id, raw_data, len_off+1);
- if (!resp || !resp->in_report)
- goto skip;
- if (resp->in_report->id == REPORT_MEMORY ||
- resp->in_report->id == REPORT_BL_READ_MEMORY) {
- if (memcmp(raw_data, resp->raw_data, len_off+1) != 0)
- goto skip;
- if (copy_to_user(u+ret, resp->raw_data+len_off+1, raw_data[len_off])) {
- err = -EFAULT;
- goto skip;
- }
- *off += raw_data[len_off];
- s -= raw_data[len_off];
- ret += raw_data[len_off];
- err = 0;
- }
-skip:
- kfree(resp);
- if (err)
- return ret > 0 ? ret : err;
- }
- return ret;
-}
-
-static ssize_t picolcd_debug_flash_read(struct file *f, char __user *u,
- size_t s, loff_t *off)
-{
- struct picolcd_data *data = f->private_data;
-
- if (s == 0)
- return -EINVAL;
- if (*off > 0x05fff)
- return 0;
- if (*off + s > 0x05fff)
- s = 0x06000 - *off;
-
- if (data->status & PICOLCD_BOOTLOADER)
- return _picolcd_flash_read(data, REPORT_BL_READ_MEMORY, u, s, off);
- else
- return _picolcd_flash_read(data, REPORT_READ_MEMORY, u, s, off);
-}
-
-/* erase block aligned to 64bytes boundary */
-static ssize_t _picolcd_flash_erase64(struct picolcd_data *data, int report_id,
- loff_t *off)
-{
- struct picolcd_pending *resp;
- u8 raw_data[3];
- int len_off;
- ssize_t ret = -EIO;
-
- if (*off & 0x3f)
- return -EINVAL;
-
- len_off = _picolcd_flash_setaddr(data, raw_data, *off);
- resp = picolcd_send_and_wait(data->hdev, report_id, raw_data, len_off);
- if (!resp || !resp->in_report)
- goto skip;
- if (resp->in_report->id == REPORT_MEMORY ||
- resp->in_report->id == REPORT_BL_ERASE_MEMORY) {
- if (memcmp(raw_data, resp->raw_data, len_off) != 0)
- goto skip;
- ret = 0;
- }
-skip:
- kfree(resp);
- return ret;
-}
-
-/* write a given size of data (bounds check to be done by caller) */
-static ssize_t _picolcd_flash_write(struct picolcd_data *data, int report_id,
- const char __user *u, size_t s, loff_t *off)
-{
- struct picolcd_pending *resp;
- u8 raw_data[36];
- ssize_t ret = 0;
- int len_off, err = -EIO;
-
- while (s > 0) {
- err = -EIO;
- len_off = _picolcd_flash_setaddr(data, raw_data, *off);
- raw_data[len_off] = s > 32 ? 32 : s;
- if (copy_from_user(raw_data+len_off+1, u, raw_data[len_off])) {
- err = -EFAULT;
- break;
- }
- resp = picolcd_send_and_wait(data->hdev, report_id, raw_data,
- len_off+1+raw_data[len_off]);
- if (!resp || !resp->in_report)
- goto skip;
- if (resp->in_report->id == REPORT_MEMORY ||
- resp->in_report->id == REPORT_BL_WRITE_MEMORY) {
- if (memcmp(raw_data, resp->raw_data, len_off+1+raw_data[len_off]) != 0)
- goto skip;
- *off += raw_data[len_off];
- s -= raw_data[len_off];
- ret += raw_data[len_off];
- err = 0;
- }
-skip:
- kfree(resp);
- if (err)
- break;
- }
- return ret > 0 ? ret : err;
-}
-
-static ssize_t picolcd_debug_flash_write(struct file *f, const char __user *u,
- size_t s, loff_t *off)
-{
- struct picolcd_data *data = f->private_data;
- ssize_t err, ret = 0;
- int report_erase, report_write;
-
- if (s == 0)
- return -EINVAL;
- if (*off > 0x5fff)
- return -ENOSPC;
- if (s & 0x3f)
- return -EINVAL;
- if (*off & 0x3f)
- return -EINVAL;
-
- if (data->status & PICOLCD_BOOTLOADER) {
- report_erase = REPORT_BL_ERASE_MEMORY;
- report_write = REPORT_BL_WRITE_MEMORY;
- } else {
- report_erase = REPORT_ERASE_MEMORY;
- report_write = REPORT_WRITE_MEMORY;
- }
- mutex_lock(&data->mutex_flash);
- while (s > 0) {
- err = _picolcd_flash_erase64(data, report_erase, off);
- if (err)
- break;
- err = _picolcd_flash_write(data, report_write, u, 64, off);
- if (err < 0)
- break;
- ret += err;
- *off += err;
- s -= err;
- if (err != 64)
- break;
- }
- mutex_unlock(&data->mutex_flash);
- return ret > 0 ? ret : err;
-}
-
-/*
- * Notes:
- * - concurrent writing is prevented by mutex and all writes must be
- * n*64 bytes and 64-byte aligned, each write being preceded by an
- * ERASE which erases a 64byte block.
- * If less than requested was written or an error is returned for an
- * otherwise correct write request the next 64-byte block which should
- * have been written is in undefined state (mostly: original, erased,
- * (half-)written with write error)
- * - reading can happen without special restriction
- */
-static const struct file_operations picolcd_debug_flash_fops = {
- .owner = THIS_MODULE,
- .open = simple_open,
- .read = picolcd_debug_flash_read,
- .write = picolcd_debug_flash_write,
- .llseek = generic_file_llseek,
-};
-
-
-/*
- * Helper code for HID report level dumping/debugging
- */
-static const char *error_codes[] = {
- "success", "parameter missing", "data_missing", "block readonly",
- "block not erasable", "block too big", "section overflow",
- "invalid command length", "invalid data length",
-};
-
-static void dump_buff_as_hex(char *dst, size_t dst_sz, const u8 *data,
- const size_t data_len)
-{
- int i, j;
- for (i = j = 0; i < data_len && j + 3 < dst_sz; i++) {
- dst[j++] = hex_asc[(data[i] >> 4) & 0x0f];
- dst[j++] = hex_asc[data[i] & 0x0f];
- dst[j++] = ' ';
- }
- if (j < dst_sz) {
- dst[j--] = '\0';
- dst[j] = '\n';
- } else
- dst[j] = '\0';
-}
-
-static void picolcd_debug_out_report(struct picolcd_data *data,
- struct hid_device *hdev, struct hid_report *report)
-{
- u8 raw_data[70];
- int raw_size = (report->size >> 3) + 1;
- char *buff;
-#define BUFF_SZ 256
-
- /* Avoid unnecessary overhead if debugfs is disabled */
- if (list_empty(&hdev->debug_list))
- return;
-
- buff = kmalloc(BUFF_SZ, GFP_ATOMIC);
- if (!buff)
- return;
-
- snprintf(buff, BUFF_SZ, "\nout report %d (size %d) = ",
- report->id, raw_size);
- hid_debug_event(hdev, buff);
- if (raw_size + 5 > sizeof(raw_data)) {
- kfree(buff);
- hid_debug_event(hdev, " TOO BIG\n");
- return;
- } else {
- raw_data[0] = report->id;
- hid_output_report(report, raw_data);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data, raw_size);
- hid_debug_event(hdev, buff);
- }
-
- switch (report->id) {
- case REPORT_LED_STATE:
- /* 1 data byte with GPO state */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_LED_STATE", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tGPO state: 0x%02x\n", raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_BRIGHTNESS:
- /* 1 data byte with brightness */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_BRIGHTNESS", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tBrightness: 0x%02x\n", raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_CONTRAST:
- /* 1 data byte with contrast */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_CONTRAST", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tContrast: 0x%02x\n", raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_RESET:
- /* 2 data bytes with reset duration in ms */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_RESET", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tDuration: 0x%02x%02x (%dms)\n",
- raw_data[2], raw_data[1], raw_data[2] << 8 | raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_LCD_CMD:
- /* 63 data bytes with LCD commands */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_LCD_CMD", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- /* TODO: format decoding */
- break;
- case REPORT_LCD_DATA:
- /* 63 data bytes with LCD data */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_LCD_CMD", report->id, raw_size-1);
- /* TODO: format decoding */
- hid_debug_event(hdev, buff);
- break;
- case REPORT_LCD_CMD_DATA:
- /* 63 data bytes with LCD commands and data */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_LCD_CMD", report->id, raw_size-1);
- /* TODO: format decoding */
- hid_debug_event(hdev, buff);
- break;
- case REPORT_EE_READ:
- /* 3 data bytes with read area description */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_EE_READ", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_EE_WRITE:
- /* 3+1..20 data bytes with write area description */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_EE_WRITE", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
- hid_debug_event(hdev, buff);
- if (raw_data[3] == 0) {
- snprintf(buff, BUFF_SZ, "\tNo data\n");
- } else if (raw_data[3] + 4 <= raw_size) {
- snprintf(buff, BUFF_SZ, "\tData: ");
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
- } else {
- snprintf(buff, BUFF_SZ, "\tData overflowed\n");
- }
- hid_debug_event(hdev, buff);
- break;
- case REPORT_ERASE_MEMORY:
- case REPORT_BL_ERASE_MEMORY:
- /* 3 data bytes with pointer inside erase block */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_ERASE_MEMORY", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- switch (data->addr_sz) {
- case 2:
- snprintf(buff, BUFF_SZ, "\tAddress inside 64 byte block: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- break;
- case 3:
- snprintf(buff, BUFF_SZ, "\tAddress inside 64 byte block: 0x%02x%02x%02x\n",
- raw_data[3], raw_data[2], raw_data[1]);
- break;
- default:
- snprintf(buff, BUFF_SZ, "\tNot supported\n");
- }
- hid_debug_event(hdev, buff);
- break;
- case REPORT_READ_MEMORY:
- case REPORT_BL_READ_MEMORY:
- /* 4 data bytes with read area description */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_READ_MEMORY", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- switch (data->addr_sz) {
- case 2:
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
- break;
- case 3:
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x%02x\n",
- raw_data[3], raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[4]);
- break;
- default:
- snprintf(buff, BUFF_SZ, "\tNot supported\n");
- }
- hid_debug_event(hdev, buff);
- break;
- case REPORT_WRITE_MEMORY:
- case REPORT_BL_WRITE_MEMORY:
- /* 4+1..32 data bytes with write adrea description */
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_WRITE_MEMORY", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- switch (data->addr_sz) {
- case 2:
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
- hid_debug_event(hdev, buff);
- if (raw_data[3] == 0) {
- snprintf(buff, BUFF_SZ, "\tNo data\n");
- } else if (raw_data[3] + 4 <= raw_size) {
- snprintf(buff, BUFF_SZ, "\tData: ");
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
- } else {
- snprintf(buff, BUFF_SZ, "\tData overflowed\n");
- }
- break;
- case 3:
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x%02x\n",
- raw_data[3], raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[4]);
- hid_debug_event(hdev, buff);
- if (raw_data[4] == 0) {
- snprintf(buff, BUFF_SZ, "\tNo data\n");
- } else if (raw_data[4] + 5 <= raw_size) {
- snprintf(buff, BUFF_SZ, "\tData: ");
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+5, raw_data[4]);
- } else {
- snprintf(buff, BUFF_SZ, "\tData overflowed\n");
- }
- break;
- default:
- snprintf(buff, BUFF_SZ, "\tNot supported\n");
- }
- hid_debug_event(hdev, buff);
- break;
- case REPORT_SPLASH_RESTART:
- /* TODO */
- break;
- case REPORT_EXIT_KEYBOARD:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_EXIT_KEYBOARD", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tRestart delay: %dms (0x%02x%02x)\n",
- raw_data[1] | (raw_data[2] << 8),
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_VERSION:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_VERSION", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_DEVID:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_DEVID", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_SPLASH_SIZE:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_SPLASH_SIZE", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_HOOK_VERSION:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_HOOK_VERSION", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_EXIT_FLASHER:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "REPORT_VERSION", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tRestart delay: %dms (0x%02x%02x)\n",
- raw_data[1] | (raw_data[2] << 8),
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- default:
- snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
- "<unknown>", report->id, raw_size-1);
- hid_debug_event(hdev, buff);
- break;
- }
- wake_up_interruptible(&hdev->debug_wait);
- kfree(buff);
-}
-
-static void picolcd_debug_raw_event(struct picolcd_data *data,
- struct hid_device *hdev, struct hid_report *report,
- u8 *raw_data, int size)
-{
- char *buff;
-
-#define BUFF_SZ 256
- /* Avoid unnecessary overhead if debugfs is disabled */
- if (!hdev->debug_events)
- return;
-
- buff = kmalloc(BUFF_SZ, GFP_ATOMIC);
- if (!buff)
- return;
-
- switch (report->id) {
- case REPORT_ERROR_CODE:
- /* 2 data bytes with affected report and error code */
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_ERROR_CODE", report->id, size-1);
- hid_debug_event(hdev, buff);
- if (raw_data[2] < ARRAY_SIZE(error_codes))
- snprintf(buff, BUFF_SZ, "\tError code 0x%02x (%s) in reply to report 0x%02x\n",
- raw_data[2], error_codes[raw_data[2]], raw_data[1]);
- else
- snprintf(buff, BUFF_SZ, "\tError code 0x%02x in reply to report 0x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_KEY_STATE:
- /* 2 data bytes with key state */
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_KEY_STATE", report->id, size-1);
- hid_debug_event(hdev, buff);
- if (raw_data[1] == 0)
- snprintf(buff, BUFF_SZ, "\tNo key pressed\n");
- else if (raw_data[2] == 0)
- snprintf(buff, BUFF_SZ, "\tOne key pressed: 0x%02x (%d)\n",
- raw_data[1], raw_data[1]);
- else
- snprintf(buff, BUFF_SZ, "\tTwo keys pressed: 0x%02x (%d), 0x%02x (%d)\n",
- raw_data[1], raw_data[1], raw_data[2], raw_data[2]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_IR_DATA:
- /* Up to 20 byes of IR scancode data */
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_IR_DATA", report->id, size-1);
- hid_debug_event(hdev, buff);
- if (raw_data[1] == 0) {
- snprintf(buff, BUFF_SZ, "\tUnexpectedly 0 data length\n");
- hid_debug_event(hdev, buff);
- } else if (raw_data[1] + 1 <= size) {
- snprintf(buff, BUFF_SZ, "\tData length: %d\n\tIR Data: ",
- raw_data[1]-1);
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+2, raw_data[1]-1);
- hid_debug_event(hdev, buff);
- } else {
- snprintf(buff, BUFF_SZ, "\tOverflowing data length: %d\n",
- raw_data[1]-1);
- hid_debug_event(hdev, buff);
- }
- break;
- case REPORT_EE_DATA:
- /* Data buffer in response to REPORT_EE_READ or REPORT_EE_WRITE */
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_EE_DATA", report->id, size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
- hid_debug_event(hdev, buff);
- if (raw_data[3] == 0) {
- snprintf(buff, BUFF_SZ, "\tNo data\n");
- hid_debug_event(hdev, buff);
- } else if (raw_data[3] + 4 <= size) {
- snprintf(buff, BUFF_SZ, "\tData: ");
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
- hid_debug_event(hdev, buff);
- } else {
- snprintf(buff, BUFF_SZ, "\tData overflowed\n");
- hid_debug_event(hdev, buff);
- }
- break;
- case REPORT_MEMORY:
- /* Data buffer in response to REPORT_READ_MEMORY or REPORT_WRTIE_MEMORY */
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_MEMORY", report->id, size-1);
- hid_debug_event(hdev, buff);
- switch (data->addr_sz) {
- case 2:
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
- hid_debug_event(hdev, buff);
- if (raw_data[3] == 0) {
- snprintf(buff, BUFF_SZ, "\tNo data\n");
- } else if (raw_data[3] + 4 <= size) {
- snprintf(buff, BUFF_SZ, "\tData: ");
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
- } else {
- snprintf(buff, BUFF_SZ, "\tData overflowed\n");
- }
- break;
- case 3:
- snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x%02x\n",
- raw_data[3], raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[4]);
- hid_debug_event(hdev, buff);
- if (raw_data[4] == 0) {
- snprintf(buff, BUFF_SZ, "\tNo data\n");
- } else if (raw_data[4] + 5 <= size) {
- snprintf(buff, BUFF_SZ, "\tData: ");
- hid_debug_event(hdev, buff);
- dump_buff_as_hex(buff, BUFF_SZ, raw_data+5, raw_data[4]);
- } else {
- snprintf(buff, BUFF_SZ, "\tData overflowed\n");
- }
- break;
- default:
- snprintf(buff, BUFF_SZ, "\tNot supported\n");
- }
- hid_debug_event(hdev, buff);
- break;
- case REPORT_VERSION:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_VERSION", report->id, size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tFirmware version: %d.%d\n",
- raw_data[2], raw_data[1]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_BL_ERASE_MEMORY:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_BL_ERASE_MEMORY", report->id, size-1);
- hid_debug_event(hdev, buff);
- /* TODO */
- break;
- case REPORT_BL_READ_MEMORY:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_BL_READ_MEMORY", report->id, size-1);
- hid_debug_event(hdev, buff);
- /* TODO */
- break;
- case REPORT_BL_WRITE_MEMORY:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_BL_WRITE_MEMORY", report->id, size-1);
- hid_debug_event(hdev, buff);
- /* TODO */
- break;
- case REPORT_DEVID:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_DEVID", report->id, size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tSerial: 0x%02x%02x%02x%02x\n",
- raw_data[1], raw_data[2], raw_data[3], raw_data[4]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tType: 0x%02x\n",
- raw_data[5]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_SPLASH_SIZE:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_SPLASH_SIZE", report->id, size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tTotal splash space: %d\n",
- (raw_data[2] << 8) | raw_data[1]);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tUsed splash space: %d\n",
- (raw_data[4] << 8) | raw_data[3]);
- hid_debug_event(hdev, buff);
- break;
- case REPORT_HOOK_VERSION:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "REPORT_HOOK_VERSION", report->id, size-1);
- hid_debug_event(hdev, buff);
- snprintf(buff, BUFF_SZ, "\tFirmware version: %d.%d\n",
- raw_data[1], raw_data[2]);
- hid_debug_event(hdev, buff);
- break;
- default:
- snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
- "<unknown>", report->id, size-1);
- hid_debug_event(hdev, buff);
- break;
- }
- wake_up_interruptible(&hdev->debug_wait);
- kfree(buff);
-}
-
-static void picolcd_init_devfs(struct picolcd_data *data,
- struct hid_report *eeprom_r, struct hid_report *eeprom_w,
- struct hid_report *flash_r, struct hid_report *flash_w,
- struct hid_report *reset)
-{
- struct hid_device *hdev = data->hdev;
-
- mutex_init(&data->mutex_flash);
-
- /* reset */
- if (reset)
- data->debug_reset = debugfs_create_file("reset", 0600,
- hdev->debug_dir, data, &picolcd_debug_reset_fops);
-
- /* eeprom */
- if (eeprom_r || eeprom_w)
- data->debug_eeprom = debugfs_create_file("eeprom",
- (eeprom_w ? S_IWUSR : 0) | (eeprom_r ? S_IRUSR : 0),
- hdev->debug_dir, data, &picolcd_debug_eeprom_fops);
-
- /* flash */
- if (flash_r && flash_r->maxfield == 1 && flash_r->field[0]->report_size == 8)
- data->addr_sz = flash_r->field[0]->report_count - 1;
- else
- data->addr_sz = -1;
- if (data->addr_sz == 2 || data->addr_sz == 3) {
- data->debug_flash = debugfs_create_file("flash",
- (flash_w ? S_IWUSR : 0) | (flash_r ? S_IRUSR : 0),
- hdev->debug_dir, data, &picolcd_debug_flash_fops);
- } else if (flash_r || flash_w)
- hid_warn(hdev, "Unexpected FLASH access reports, please submit rdesc for review\n");
-}
-
-static void picolcd_exit_devfs(struct picolcd_data *data)
-{
- struct dentry *dent;
-
- dent = data->debug_reset;
- data->debug_reset = NULL;
- if (dent)
- debugfs_remove(dent);
- dent = data->debug_eeprom;
- data->debug_eeprom = NULL;
- if (dent)
- debugfs_remove(dent);
- dent = data->debug_flash;
- data->debug_flash = NULL;
- if (dent)
- debugfs_remove(dent);
- mutex_destroy(&data->mutex_flash);
-}
-#else
-static inline void picolcd_debug_raw_event(struct picolcd_data *data,
- struct hid_device *hdev, struct hid_report *report,
- u8 *raw_data, int size)
-{
-}
-static inline void picolcd_init_devfs(struct picolcd_data *data,
- struct hid_report *eeprom_r, struct hid_report *eeprom_w,
- struct hid_report *flash_r, struct hid_report *flash_w,
- struct hid_report *reset)
-{
-}
-static inline void picolcd_exit_devfs(struct picolcd_data *data)
-{
-}
-#endif /* CONFIG_DEBUG_FS */
-
-/*
- * Handle raw report as sent by device
- */
-static int picolcd_raw_event(struct hid_device *hdev,
- struct hid_report *report, u8 *raw_data, int size)
-{
- struct picolcd_data *data = hid_get_drvdata(hdev);
- unsigned long flags;
- int ret = 0;
-
- if (!data)
- return 1;
-
- if (report->id == REPORT_KEY_STATE) {
- if (data->input_keys)
- ret = picolcd_raw_keypad(data, report, raw_data+1, size-1);
- } else if (report->id == REPORT_IR_DATA) {
- if (data->input_cir)
- ret = picolcd_raw_cir(data, report, raw_data+1, size-1);
- } else {
- spin_lock_irqsave(&data->lock, flags);
- /*
- * We let the caller of picolcd_send_and_wait() check if the
- * report we got is one of the expected ones or not.
- */
- if (data->pending) {
- memcpy(data->pending->raw_data, raw_data+1, size-1);
- data->pending->raw_size = size-1;
- data->pending->in_report = report;
- complete(&data->pending->ready);
- }
- spin_unlock_irqrestore(&data->lock, flags);
- }
-
- picolcd_debug_raw_event(data, hdev, report, raw_data, size);
- return 1;
-}
-
-#ifdef CONFIG_PM
-static int picolcd_suspend(struct hid_device *hdev, pm_message_t message)
-{
- if (PMSG_IS_AUTO(message))
- return 0;
-
- picolcd_suspend_backlight(hid_get_drvdata(hdev));
- dbg_hid(PICOLCD_NAME " device ready for suspend\n");
- return 0;
-}
-
-static int picolcd_resume(struct hid_device *hdev)
-{
- int ret;
- ret = picolcd_resume_backlight(hid_get_drvdata(hdev));
- if (ret)
- dbg_hid(PICOLCD_NAME " restoring backlight failed: %d\n", ret);
- return 0;
-}
-
-static int picolcd_reset_resume(struct hid_device *hdev)
-{
- int ret;
- ret = picolcd_reset(hdev);
- if (ret)
- dbg_hid(PICOLCD_NAME " resetting our device failed: %d\n", ret);
- ret = picolcd_fb_reset(hid_get_drvdata(hdev), 0);
- if (ret)
- dbg_hid(PICOLCD_NAME " restoring framebuffer content failed: %d\n", ret);
- ret = picolcd_resume_lcd(hid_get_drvdata(hdev));
- if (ret)
- dbg_hid(PICOLCD_NAME " restoring lcd failed: %d\n", ret);
- ret = picolcd_resume_backlight(hid_get_drvdata(hdev));
- if (ret)
- dbg_hid(PICOLCD_NAME " restoring backlight failed: %d\n", ret);
- picolcd_leds_set(hid_get_drvdata(hdev));
- return 0;
-}
-#endif
-
-/* initialize keypad input device */
-static int picolcd_init_keys(struct picolcd_data *data,
- struct hid_report *report)
-{
- struct hid_device *hdev = data->hdev;
- struct input_dev *idev;
- int error, i;
-
- if (!report)
- return -ENODEV;
- if (report->maxfield != 1 || report->field[0]->report_count != 2 ||
- report->field[0]->report_size != 8) {
- hid_err(hdev, "unsupported KEY_STATE report\n");
- return -EINVAL;
- }
-
- idev = input_allocate_device();
- if (idev == NULL) {
- hid_err(hdev, "failed to allocate input device\n");
- return -ENOMEM;
- }
- input_set_drvdata(idev, hdev);
- memcpy(data->keycode, def_keymap, sizeof(def_keymap));
- idev->name = hdev->name;
- idev->phys = hdev->phys;
- idev->uniq = hdev->uniq;
- idev->id.bustype = hdev->bus;
- idev->id.vendor = hdev->vendor;
- idev->id.product = hdev->product;
- idev->id.version = hdev->version;
- idev->dev.parent = hdev->dev.parent;
- idev->keycode = &data->keycode;
- idev->keycodemax = PICOLCD_KEYS;
- idev->keycodesize = sizeof(data->keycode[0]);
- input_set_capability(idev, EV_MSC, MSC_SCAN);
- set_bit(EV_REP, idev->evbit);
- for (i = 0; i < PICOLCD_KEYS; i++)
- input_set_capability(idev, EV_KEY, data->keycode[i]);
- error = input_register_device(idev);
- if (error) {
- hid_err(hdev, "error registering the input device\n");
- input_free_device(idev);
- return error;
- }
- data->input_keys = idev;
- return 0;
-}
-
-static void picolcd_exit_keys(struct picolcd_data *data)
-{
- struct input_dev *idev = data->input_keys;
-
- data->input_keys = NULL;
- if (idev)
- input_unregister_device(idev);
-}
-
-/* initialize CIR input device */
-static inline int picolcd_init_cir(struct picolcd_data *data, struct hid_report *report)
-{
- /* support not implemented yet */
- return 0;
-}
-
-static inline void picolcd_exit_cir(struct picolcd_data *data)
-{
-}
-
-static int picolcd_probe_lcd(struct hid_device *hdev, struct picolcd_data *data)
-{
- int error;
-
- error = picolcd_check_version(hdev);
- if (error)
- return error;
-
- if (data->version[0] != 0 && data->version[1] != 3)
- hid_info(hdev, "Device with untested firmware revision, please submit /sys/kernel/debug/hid/%s/rdesc for this device.\n",
- dev_name(&hdev->dev));
-
- /* Setup keypad input device */
- error = picolcd_init_keys(data, picolcd_in_report(REPORT_KEY_STATE, hdev));
- if (error)
- goto err;
-
- /* Setup CIR input device */
- error = picolcd_init_cir(data, picolcd_in_report(REPORT_IR_DATA, hdev));
- if (error)
- goto err;
-
- /* Set up the framebuffer device */
- error = picolcd_init_framebuffer(data);
- if (error)
- goto err;
-
- /* Setup lcd class device */
- error = picolcd_init_lcd(data, picolcd_out_report(REPORT_CONTRAST, hdev));
- if (error)
- goto err;
-
- /* Setup backlight class device */
- error = picolcd_init_backlight(data, picolcd_out_report(REPORT_BRIGHTNESS, hdev));
- if (error)
- goto err;
-
- /* Setup the LED class devices */
- error = picolcd_init_leds(data, picolcd_out_report(REPORT_LED_STATE, hdev));
- if (error)
- goto err;
-
- picolcd_init_devfs(data, picolcd_out_report(REPORT_EE_READ, hdev),
- picolcd_out_report(REPORT_EE_WRITE, hdev),
- picolcd_out_report(REPORT_READ_MEMORY, hdev),
- picolcd_out_report(REPORT_WRITE_MEMORY, hdev),
- picolcd_out_report(REPORT_RESET, hdev));
- return 0;
-err:
- picolcd_exit_leds(data);
- picolcd_exit_backlight(data);
- picolcd_exit_lcd(data);
- picolcd_exit_framebuffer(data);
- picolcd_exit_cir(data);
- picolcd_exit_keys(data);
- return error;
-}
-
-static int picolcd_probe_bootloader(struct hid_device *hdev, struct picolcd_data *data)
-{
- int error;
-
- error = picolcd_check_version(hdev);
- if (error)
- return error;
-
- if (data->version[0] != 1 && data->version[1] != 0)
- hid_info(hdev, "Device with untested bootloader revision, please submit /sys/kernel/debug/hid/%s/rdesc for this device.\n",
- dev_name(&hdev->dev));
-
- picolcd_init_devfs(data, NULL, NULL,
- picolcd_out_report(REPORT_BL_READ_MEMORY, hdev),
- picolcd_out_report(REPORT_BL_WRITE_MEMORY, hdev), NULL);
- return 0;
-}
-
-static int picolcd_probe(struct hid_device *hdev,
- const struct hid_device_id *id)
-{
- struct picolcd_data *data;
- int error = -ENOMEM;
-
- dbg_hid(PICOLCD_NAME " hardware probe...\n");
-
- /*
- * Let's allocate the picolcd data structure, set some reasonable
- * defaults, and associate it with the device
- */
- data = kzalloc(sizeof(struct picolcd_data), GFP_KERNEL);
- if (data == NULL) {
- hid_err(hdev, "can't allocate space for Minibox PicoLCD device data\n");
- error = -ENOMEM;
- goto err_no_cleanup;
- }
-
- spin_lock_init(&data->lock);
- mutex_init(&data->mutex);
- data->hdev = hdev;
- data->opmode_delay = 5000;
- if (hdev->product == USB_DEVICE_ID_PICOLCD_BOOTLOADER)
- data->status |= PICOLCD_BOOTLOADER;
- hid_set_drvdata(hdev, data);
-
- /* Parse the device reports and start it up */
- error = hid_parse(hdev);
- if (error) {
- hid_err(hdev, "device report parse failed\n");
- goto err_cleanup_data;
- }
-
- error = hid_hw_start(hdev, 0);
- if (error) {
- hid_err(hdev, "hardware start failed\n");
- goto err_cleanup_data;
- }
-
- error = hid_hw_open(hdev);
- if (error) {
- hid_err(hdev, "failed to open input interrupt pipe for key and IR events\n");
- goto err_cleanup_hid_hw;
- }
-
- error = device_create_file(&hdev->dev, &dev_attr_operation_mode_delay);
- if (error) {
- hid_err(hdev, "failed to create sysfs attributes\n");
- goto err_cleanup_hid_ll;
- }
-
- error = device_create_file(&hdev->dev, &dev_attr_operation_mode);
- if (error) {
- hid_err(hdev, "failed to create sysfs attributes\n");
- goto err_cleanup_sysfs1;
- }
-
- if (data->status & PICOLCD_BOOTLOADER)
- error = picolcd_probe_bootloader(hdev, data);
- else
- error = picolcd_probe_lcd(hdev, data);
- if (error)
- goto err_cleanup_sysfs2;
-
- dbg_hid(PICOLCD_NAME " activated and initialized\n");
- return 0;
-
-err_cleanup_sysfs2:
- device_remove_file(&hdev->dev, &dev_attr_operation_mode);
-err_cleanup_sysfs1:
- device_remove_file(&hdev->dev, &dev_attr_operation_mode_delay);
-err_cleanup_hid_ll:
- hid_hw_close(hdev);
-err_cleanup_hid_hw:
- hid_hw_stop(hdev);
-err_cleanup_data:
- kfree(data);
-err_no_cleanup:
- hid_set_drvdata(hdev, NULL);
-
- return error;
-}
-
-static void picolcd_remove(struct hid_device *hdev)
-{
- struct picolcd_data *data = hid_get_drvdata(hdev);
- unsigned long flags;
-
- dbg_hid(PICOLCD_NAME " hardware remove...\n");
- spin_lock_irqsave(&data->lock, flags);
- data->status |= PICOLCD_FAILED;
- spin_unlock_irqrestore(&data->lock, flags);
-#ifdef CONFIG_HID_PICOLCD_FB
- /* short-circuit FB as early as possible in order to
- * avoid long delays if we host console.
- */
- if (data->fb_info)
- data->fb_info->par = NULL;
-#endif
-
- picolcd_exit_devfs(data);
- device_remove_file(&hdev->dev, &dev_attr_operation_mode);
- device_remove_file(&hdev->dev, &dev_attr_operation_mode_delay);
- hid_hw_close(hdev);
- hid_hw_stop(hdev);
- hid_set_drvdata(hdev, NULL);
-
- /* Shortcut potential pending reply that will never arrive */
- spin_lock_irqsave(&data->lock, flags);
- if (data->pending)
- complete(&data->pending->ready);
- spin_unlock_irqrestore(&data->lock, flags);
-
- /* Cleanup LED */
- picolcd_exit_leds(data);
- /* Clean up the framebuffer */
- picolcd_exit_backlight(data);
- picolcd_exit_lcd(data);
- picolcd_exit_framebuffer(data);
- /* Cleanup input */
- picolcd_exit_cir(data);
- picolcd_exit_keys(data);
-
- mutex_destroy(&data->mutex);
- /* Finally, clean up the picolcd data itself */
- kfree(data);
-}
-
-static const struct hid_device_id picolcd_devices[] = {
- { HID_USB_DEVICE(USB_VENDOR_ID_MICROCHIP, USB_DEVICE_ID_PICOLCD) },
- { HID_USB_DEVICE(USB_VENDOR_ID_MICROCHIP, USB_DEVICE_ID_PICOLCD_BOOTLOADER) },
- { }
-};
-MODULE_DEVICE_TABLE(hid, picolcd_devices);
-
-static struct hid_driver picolcd_driver = {
- .name = "hid-picolcd",
- .id_table = picolcd_devices,
- .probe = picolcd_probe,
- .remove = picolcd_remove,
- .raw_event = picolcd_raw_event,
-#ifdef CONFIG_PM
- .suspend = picolcd_suspend,
- .resume = picolcd_resume,
- .reset_resume = picolcd_reset_resume,
-#endif
-};
-
-static int __init picolcd_init(void)
-{
- return hid_register_driver(&picolcd_driver);
-}
-
-static void __exit picolcd_exit(void)
-{
- hid_unregister_driver(&picolcd_driver);
-#ifdef CONFIG_HID_PICOLCD_FB
- flush_work_sync(&picolcd_fb_cleanup);
- WARN_ON(fb_pending);
-#endif
-}
-
-module_init(picolcd_init);
-module_exit(picolcd_exit);
-MODULE_DESCRIPTION("Minibox graphics PicoLCD Driver");
-MODULE_LICENSE("GPL v2");
diff --git a/drivers/hid/hid-picolcd.h b/drivers/hid/hid-picolcd.h
new file mode 100644
index 0000000..020cef6
--- /dev/null
+++ b/drivers/hid/hid-picolcd.h
@@ -0,0 +1,309 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#define PICOLCD_NAME "PicoLCD (graphic)"
+
+/* Report numbers */
+#define REPORT_ERROR_CODE 0x10 /* LCD: IN[16] */
+#define ERR_SUCCESS 0x00
+#define ERR_PARAMETER_MISSING 0x01
+#define ERR_DATA_MISSING 0x02
+#define ERR_BLOCK_READ_ONLY 0x03
+#define ERR_BLOCK_NOT_ERASABLE 0x04
+#define ERR_BLOCK_TOO_BIG 0x05
+#define ERR_SECTION_OVERFLOW 0x06
+#define ERR_INVALID_CMD_LEN 0x07
+#define ERR_INVALID_DATA_LEN 0x08
+#define REPORT_KEY_STATE 0x11 /* LCD: IN[2] */
+#define REPORT_IR_DATA 0x21 /* LCD: IN[63] */
+#define REPORT_EE_DATA 0x32 /* LCD: IN[63] */
+#define REPORT_MEMORY 0x41 /* LCD: IN[63] */
+#define REPORT_LED_STATE 0x81 /* LCD: OUT[1] */
+#define REPORT_BRIGHTNESS 0x91 /* LCD: OUT[1] */
+#define REPORT_CONTRAST 0x92 /* LCD: OUT[1] */
+#define REPORT_RESET 0x93 /* LCD: OUT[2] */
+#define REPORT_LCD_CMD 0x94 /* LCD: OUT[63] */
+#define REPORT_LCD_DATA 0x95 /* LCD: OUT[63] */
+#define REPORT_LCD_CMD_DATA 0x96 /* LCD: OUT[63] */
+#define REPORT_EE_READ 0xa3 /* LCD: OUT[63] */
+#define REPORT_EE_WRITE 0xa4 /* LCD: OUT[63] */
+#define REPORT_ERASE_MEMORY 0xb2 /* LCD: OUT[2] */
+#define REPORT_READ_MEMORY 0xb3 /* LCD: OUT[3] */
+#define REPORT_WRITE_MEMORY 0xb4 /* LCD: OUT[63] */
+#define REPORT_SPLASH_RESTART 0xc1 /* LCD: OUT[1] */
+#define REPORT_EXIT_KEYBOARD 0xef /* LCD: OUT[2] */
+#define REPORT_VERSION 0xf1 /* LCD: IN[2],OUT[1] Bootloader: IN[2],OUT[1] */
+#define REPORT_BL_ERASE_MEMORY 0xf2 /* Bootloader: IN[36],OUT[4] */
+#define REPORT_BL_READ_MEMORY 0xf3 /* Bootloader: IN[36],OUT[4] */
+#define REPORT_BL_WRITE_MEMORY 0xf4 /* Bootloader: IN[36],OUT[36] */
+#define REPORT_DEVID 0xf5 /* LCD: IN[5], OUT[1] Bootloader: IN[5],OUT[1] */
+#define REPORT_SPLASH_SIZE 0xf6 /* LCD: IN[4], OUT[1] */
+#define REPORT_HOOK_VERSION 0xf7 /* LCD: IN[2], OUT[1] */
+#define REPORT_EXIT_FLASHER 0xff /* Bootloader: OUT[2] */
+
+/* Description of in-progress IO operation, used for operations
+ * that trigger response from device */
+struct picolcd_pending {
+ struct hid_report *out_report;
+ struct hid_report *in_report;
+ struct completion ready;
+ int raw_size;
+ u8 raw_data[64];
+};
+
+
+#define PICOLCD_KEYS 17
+
+/* Per device data structure */
+struct picolcd_data {
+ struct hid_device *hdev;
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debug_reset;
+ struct dentry *debug_eeprom;
+ struct dentry *debug_flash;
+ struct mutex mutex_flash;
+ int addr_sz;
+#endif
+ u8 version[2];
+ unsigned short opmode_delay;
+ /* input stuff */
+ u8 pressed_keys[2];
+ struct input_dev *input_keys;
+#ifdef CONFIG_HID_PICOLCD_CIR
+ struct rc_dev *rc_dev;
+#endif
+ unsigned short keycode[PICOLCD_KEYS];
+
+#ifdef CONFIG_HID_PICOLCD_FB
+ /* Framebuffer stuff */
+ struct fb_info *fb_info;
+#endif /* CONFIG_HID_PICOLCD_FB */
+#ifdef CONFIG_HID_PICOLCD_LCD
+ struct lcd_device *lcd;
+ u8 lcd_contrast;
+#endif /* CONFIG_HID_PICOLCD_LCD */
+#ifdef CONFIG_HID_PICOLCD_BACKLIGHT
+ struct backlight_device *backlight;
+ u8 lcd_brightness;
+ u8 lcd_power;
+#endif /* CONFIG_HID_PICOLCD_BACKLIGHT */
+#ifdef CONFIG_HID_PICOLCD_LEDS
+ /* LED stuff */
+ u8 led_state;
+ struct led_classdev *led[8];
+#endif /* CONFIG_HID_PICOLCD_LEDS */
+
+ /* Housekeeping stuff */
+ spinlock_t lock;
+ struct mutex mutex;
+ struct picolcd_pending *pending;
+ int status;
+#define PICOLCD_BOOTLOADER 1
+#define PICOLCD_FAILED 2
+#define PICOLCD_CIR_SHUN 4
+};
+
+#ifdef CONFIG_HID_PICOLCD_FB
+struct picolcd_fb_data {
+ /* Framebuffer stuff */
+ spinlock_t lock;
+ struct picolcd_data *picolcd;
+ u8 update_rate;
+ u8 bpp;
+ u8 force;
+ u8 ready;
+ u8 *vbitmap; /* local copy of what was sent to PicoLCD */
+ u8 *bitmap; /* framebuffer */
+};
+#endif /* CONFIG_HID_PICOLCD_FB */
+
+/* Find a given report */
+#define picolcd_in_report(id, dev) picolcd_report(id, dev, HID_INPUT_REPORT)
+#define picolcd_out_report(id, dev) picolcd_report(id, dev, HID_OUTPUT_REPORT)
+
+struct hid_report *picolcd_report(int id, struct hid_device *hdev, int dir);
+
+#ifdef CONFIG_DEBUG_FS
+void picolcd_debug_out_report(struct picolcd_data *data,
+ struct hid_device *hdev, struct hid_report *report);
+#define usbhid_submit_report(a, b, c) \
+ do { \
+ picolcd_debug_out_report(hid_get_drvdata(a), a, b); \
+ usbhid_submit_report(a, b, c); \
+ } while (0)
+
+void picolcd_debug_raw_event(struct picolcd_data *data,
+ struct hid_device *hdev, struct hid_report *report,
+ u8 *raw_data, int size);
+
+void picolcd_init_devfs(struct picolcd_data *data,
+ struct hid_report *eeprom_r, struct hid_report *eeprom_w,
+ struct hid_report *flash_r, struct hid_report *flash_w,
+ struct hid_report *reset);
+
+void picolcd_exit_devfs(struct picolcd_data *data);
+#else
+static inline void picolcd_debug_out_report(struct picolcd_data *data,
+ struct hid_device *hdev, struct hid_report *report)
+{
+}
+static inline void picolcd_debug_raw_event(struct picolcd_data *data,
+ struct hid_device *hdev, struct hid_report *report,
+ u8 *raw_data, int size)
+{
+}
+static inline void picolcd_init_devfs(struct picolcd_data *data,
+ struct hid_report *eeprom_r, struct hid_report *eeprom_w,
+ struct hid_report *flash_r, struct hid_report *flash_w,
+ struct hid_report *reset)
+{
+}
+static inline void picolcd_exit_devfs(struct picolcd_data *data)
+{
+}
+#endif /* CONFIG_DEBUG_FS */
+
+
+#ifdef CONFIG_HID_PICOLCD_FB
+int picolcd_fb_reset(struct picolcd_data *data, int clear);
+
+int picolcd_init_framebuffer(struct picolcd_data *data);
+
+void picolcd_exit_framebuffer(struct picolcd_data *data);
+
+void picolcd_fb_refresh(struct picolcd_data *data);
+#define picolcd_fbinfo(d) ((d)->fb_info)
+#else
+static inline int picolcd_fb_reset(struct picolcd_data *data, int clear)
+{
+ return 0;
+}
+static inline int picolcd_init_framebuffer(struct picolcd_data *data)
+{
+ return 0;
+}
+static inline void picolcd_exit_framebuffer(struct picolcd_data *data)
+{
+}
+static inline void picolcd_fb_refresh(struct picolcd_data *data)
+{
+}
+#define picolcd_fbinfo(d) NULL
+#endif /* CONFIG_HID_PICOLCD_FB */
+
+
+#ifdef CONFIG_HID_PICOLCD_BACKLIGHT
+int picolcd_init_backlight(struct picolcd_data *data,
+ struct hid_report *report);
+
+void picolcd_exit_backlight(struct picolcd_data *data);
+
+int picolcd_resume_backlight(struct picolcd_data *data);
+
+void picolcd_suspend_backlight(struct picolcd_data *data);
+#else
+static inline int picolcd_init_backlight(struct picolcd_data *data,
+ struct hid_report *report)
+{
+ return 0;
+}
+static inline void picolcd_exit_backlight(struct picolcd_data *data)
+{
+}
+static inline int picolcd_resume_backlight(struct picolcd_data *data)
+{
+ return 0;
+}
+static inline void picolcd_suspend_backlight(struct picolcd_data *data)
+{
+}
+
+#endif /* CONFIG_HID_PICOLCD_BACKLIGHT */
+
+
+#ifdef CONFIG_HID_PICOLCD_LCD
+int picolcd_init_lcd(struct picolcd_data *data,
+ struct hid_report *report);
+
+void picolcd_exit_lcd(struct picolcd_data *data);
+
+int picolcd_resume_lcd(struct picolcd_data *data);
+#else
+static inline int picolcd_init_lcd(struct picolcd_data *data,
+ struct hid_report *report)
+{
+ return 0;
+}
+static inline void picolcd_exit_lcd(struct picolcd_data *data)
+{
+}
+static inline int picolcd_resume_lcd(struct picolcd_data *data)
+{
+ return 0;
+}
+#endif /* CONFIG_HID_PICOLCD_LCD */
+
+
+#ifdef CONFIG_HID_PICOLCD_LEDS
+int picolcd_init_leds(struct picolcd_data *data,
+ struct hid_report *report);
+
+void picolcd_exit_leds(struct picolcd_data *data);
+
+void picolcd_leds_set(struct picolcd_data *data);
+#else
+static inline int picolcd_init_leds(struct picolcd_data *data,
+ struct hid_report *report)
+{
+ return 0;
+}
+static inline void picolcd_exit_leds(struct picolcd_data *data)
+{
+}
+static inline void picolcd_leds_set(struct picolcd_data *data)
+{
+}
+#endif /* CONFIG_HID_PICOLCD_LEDS */
+
+
+#ifdef CONFIG_HID_PICOLCD_CIR
+int picolcd_raw_cir(struct picolcd_data *data,
+ struct hid_report *report, u8 *raw_data, int size);
+
+int picolcd_init_cir(struct picolcd_data *data, struct hid_report *report);
+
+void picolcd_exit_cir(struct picolcd_data *data);
+#else
+static inline int picolcd_raw_cir(struct picolcd_data *data,
+ struct hid_report *report, u8 *raw_data, int size)
+{
+ return 1;
+}
+static inline int picolcd_init_cir(struct picolcd_data *data, struct hid_report *report)
+{
+ return 0;
+}
+static inline void picolcd_exit_cir(struct picolcd_data *data)
+{
+}
+#endif /* CONFIG_HID_PICOLCD_LIRC */
+
+int picolcd_reset(struct hid_device *hdev);
+struct picolcd_pending *picolcd_send_and_wait(struct hid_device *hdev,
+ int report_id, const u8 *raw_data, int size);
diff --git a/drivers/hid/hid-picolcd_backlight.c b/drivers/hid/hid-picolcd_backlight.c
new file mode 100644
index 0000000..b91f309
--- /dev/null
+++ b/drivers/hid/hid-picolcd_backlight.c
@@ -0,0 +1,122 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/backlight.h>
+
+#include "hid-picolcd.h"
+
+static int picolcd_get_brightness(struct backlight_device *bdev)
+{
+ struct picolcd_data *data = bl_get_data(bdev);
+ return data->lcd_brightness;
+}
+
+static int picolcd_set_brightness(struct backlight_device *bdev)
+{
+ struct picolcd_data *data = bl_get_data(bdev);
+ struct hid_report *report = picolcd_out_report(REPORT_BRIGHTNESS, data->hdev);
+ unsigned long flags;
+
+ if (!report || report->maxfield != 1 || report->field[0]->report_count != 1)
+ return -ENODEV;
+
+ data->lcd_brightness = bdev->props.brightness & 0x0ff;
+ data->lcd_power = bdev->props.power;
+ spin_lock_irqsave(&data->lock, flags);
+ hid_set_field(report->field[0], 0, data->lcd_power == FB_BLANK_UNBLANK ? data->lcd_brightness : 0);
+ if (!(data->status & PICOLCD_FAILED))
+ usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+ return 0;
+}
+
+static int picolcd_check_bl_fb(struct backlight_device *bdev, struct fb_info *fb)
+{
+ return fb && fb == picolcd_fbinfo((struct picolcd_data *)bl_get_data(bdev));
+}
+
+static const struct backlight_ops picolcd_blops = {
+ .update_status = picolcd_set_brightness,
+ .get_brightness = picolcd_get_brightness,
+ .check_fb = picolcd_check_bl_fb,
+};
+
+int picolcd_init_backlight(struct picolcd_data *data, struct hid_report *report)
+{
+ struct device *dev = &data->hdev->dev;
+ struct backlight_device *bdev;
+ struct backlight_properties props;
+ if (!report)
+ return -ENODEV;
+ if (report->maxfield != 1 || report->field[0]->report_count != 1 ||
+ report->field[0]->report_size != 8) {
+ dev_err(dev, "unsupported BRIGHTNESS report");
+ return -EINVAL;
+ }
+
+ memset(&props, 0, sizeof(props));
+ props.type = BACKLIGHT_RAW;
+ props.max_brightness = 0xff;
+ bdev = backlight_device_register(dev_name(dev), dev, data,
+ &picolcd_blops, &props);
+ if (IS_ERR(bdev)) {
+ dev_err(dev, "failed to register backlight\n");
+ return PTR_ERR(bdev);
+ }
+ bdev->props.brightness = 0xff;
+ data->lcd_brightness = 0xff;
+ data->backlight = bdev;
+ picolcd_set_brightness(bdev);
+ return 0;
+}
+
+void picolcd_exit_backlight(struct picolcd_data *data)
+{
+ struct backlight_device *bdev = data->backlight;
+
+ data->backlight = NULL;
+ if (bdev)
+ backlight_device_unregister(bdev);
+}
+
+int picolcd_resume_backlight(struct picolcd_data *data)
+{
+ if (!data->backlight)
+ return 0;
+ return picolcd_set_brightness(data->backlight);
+}
+
+#ifdef CONFIG_PM
+void picolcd_suspend_backlight(struct picolcd_data *data)
+{
+ int bl_power = data->lcd_power;
+ if (!data->backlight)
+ return;
+
+ data->backlight->props.power = FB_BLANK_POWERDOWN;
+ picolcd_set_brightness(data->backlight);
+ data->lcd_power = data->backlight->props.power = bl_power;
+}
+#endif /* CONFIG_PM */
+
diff --git a/drivers/hid/hid-picolcd_cir.c b/drivers/hid/hid-picolcd_cir.c
new file mode 100644
index 0000000..13ca919
--- /dev/null
+++ b/drivers/hid/hid-picolcd_cir.c
@@ -0,0 +1,152 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include <linux/hid-debug.h>
+#include <linux/input.h>
+#include "hid-ids.h"
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/vmalloc.h>
+#include <linux/backlight.h>
+#include <linux/lcd.h>
+
+#include <linux/leds.h>
+
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+
+#include <linux/completion.h>
+#include <linux/uaccess.h>
+#include <linux/module.h>
+#include <media/rc-core.h>
+
+#include "hid-picolcd.h"
+
+
+int picolcd_raw_cir(struct picolcd_data *data,
+ struct hid_report *report, u8 *raw_data, int size)
+{
+ unsigned long flags;
+ int i, w, sz;
+ DEFINE_IR_RAW_EVENT(rawir);
+
+ /* ignore if rc_dev is NULL or status is shunned */
+ spin_lock_irqsave(&data->lock, flags);
+ if (!data->rc_dev || (data->status & PICOLCD_CIR_SHUN)) {
+ spin_unlock_irqrestore(&data->lock, flags);
+ return 1;
+ }
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ /* PicoLCD USB packets contain 16-bit intervals in network order,
+ * with value negated for pulse. Intervals are in microseconds.
+ *
+ * Note: some userspace LIRC code for PicoLCD says negated values
+ * for space - is it a matter of IR chip? (pulse for my TSOP2236)
+ *
+ * In addition, the first interval seems to be around 15000 + base
+ * interval for non-first report of IR data - thus the quirk below
+ * to get RC_CODE to understand Sony and JVC remotes I have at hand
+ */
+ sz = size > 0 ? min((int)raw_data[0], size-1) : 0;
+ for (i = 0; i+1 < sz; i += 2) {
+ init_ir_raw_event(&rawir);
+ w = (raw_data[i] << 8) | (raw_data[i+1]);
+ rawir.pulse = !!(w & 0x8000);
+ rawir.duration = US_TO_NS(rawir.pulse ? (65536 - w) : w);
+ /* Quirk!! - see above */
+ if (i == 0 && rawir.duration > 15000000)
+ rawir.duration -= 15000000;
+ ir_raw_event_store(data->rc_dev, &rawir);
+ }
+ ir_raw_event_handle(data->rc_dev);
+
+ return 1;
+}
+
+static int picolcd_cir_open(struct rc_dev *dev)
+{
+ struct picolcd_data *data = dev->priv;
+ unsigned long flags;
+
+ spin_lock_irqsave(&data->lock, flags);
+ data->status &= ~PICOLCD_CIR_SHUN;
+ spin_unlock_irqrestore(&data->lock, flags);
+ return 0;
+}
+
+static void picolcd_cir_close(struct rc_dev *dev)
+{
+ struct picolcd_data *data = dev->priv;
+ unsigned long flags;
+
+ spin_lock_irqsave(&data->lock, flags);
+ data->status |= PICOLCD_CIR_SHUN;
+ spin_unlock_irqrestore(&data->lock, flags);
+}
+
+/* initialize CIR input device */
+int picolcd_init_cir(struct picolcd_data *data, struct hid_report *report)
+{
+ struct rc_dev *rdev;
+ int ret = 0;
+
+ rdev = rc_allocate_device();
+ if (!rdev)
+ return -ENOMEM;
+
+ rdev->priv = data;
+ rdev->driver_type = RC_DRIVER_IR_RAW;
+ rdev->allowed_protos = RC_TYPE_ALL;
+ rdev->open = picolcd_cir_open;
+ rdev->close = picolcd_cir_close;
+ rdev->input_name = data->hdev->name;
+ rdev->input_phys = data->hdev->phys;
+ rdev->input_id.bustype = data->hdev->bus;
+ rdev->input_id.vendor = data->hdev->vendor;
+ rdev->input_id.product = data->hdev->product;
+ rdev->input_id.version = data->hdev->version;
+ rdev->dev.parent = &data->hdev->dev;
+ rdev->driver_name = PICOLCD_NAME;
+ rdev->map_name = RC_MAP_RC6_MCE;
+ rdev->timeout = MS_TO_NS(100);
+ rdev->rx_resolution = US_TO_NS(1);
+
+ ret = rc_register_device(rdev);
+ if (ret)
+ goto err;
+ data->rc_dev = rdev;
+ return 0;
+
+err:
+ rc_free_device(rdev);
+ return ret;
+}
+
+void picolcd_exit_cir(struct picolcd_data *data)
+{
+ struct rc_dev *rdev = data->rc_dev;
+
+ data->rc_dev = NULL;
+ rc_unregister_device(rdev);
+}
+
diff --git a/drivers/hid/hid-picolcd_core.c b/drivers/hid/hid-picolcd_core.c
new file mode 100644
index 0000000..86df26e
--- /dev/null
+++ b/drivers/hid/hid-picolcd_core.c
@@ -0,0 +1,689 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include <linux/hid-debug.h>
+#include <linux/input.h>
+#include "hid-ids.h"
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/vmalloc.h>
+
+#include <linux/completion.h>
+#include <linux/uaccess.h>
+#include <linux/module.h>
+
+#include "hid-picolcd.h"
+
+
+/* Input device
+ *
+ * The PicoLCD has an IR receiver header, a built-in keypad with 5 keys
+ * and header for 4x4 key matrix. The built-in keys are part of the matrix.
+ */
+static const unsigned short def_keymap[PICOLCD_KEYS] = {
+ KEY_RESERVED, /* none */
+ KEY_BACK, /* col 4 + row 1 */
+ KEY_HOMEPAGE, /* col 3 + row 1 */
+ KEY_RESERVED, /* col 2 + row 1 */
+ KEY_RESERVED, /* col 1 + row 1 */
+ KEY_SCROLLUP, /* col 4 + row 2 */
+ KEY_OK, /* col 3 + row 2 */
+ KEY_SCROLLDOWN, /* col 2 + row 2 */
+ KEY_RESERVED, /* col 1 + row 2 */
+ KEY_RESERVED, /* col 4 + row 3 */
+ KEY_RESERVED, /* col 3 + row 3 */
+ KEY_RESERVED, /* col 2 + row 3 */
+ KEY_RESERVED, /* col 1 + row 3 */
+ KEY_RESERVED, /* col 4 + row 4 */
+ KEY_RESERVED, /* col 3 + row 4 */
+ KEY_RESERVED, /* col 2 + row 4 */
+ KEY_RESERVED, /* col 1 + row 4 */
+};
+
+
+/* Find a given report */
+struct hid_report *picolcd_report(int id, struct hid_device *hdev, int dir)
+{
+ struct list_head *feature_report_list = &hdev->report_enum[dir].report_list;
+ struct hid_report *report = NULL;
+
+ list_for_each_entry(report, feature_report_list, list) {
+ if (report->id == id)
+ return report;
+ }
+ hid_warn(hdev, "No report with id 0x%x found\n", id);
+ return NULL;
+}
+
+/* Submit a report and wait for a reply from device - if device fades away
+ * or does not respond in time, return NULL */
+struct picolcd_pending *picolcd_send_and_wait(struct hid_device *hdev,
+ int report_id, const u8 *raw_data, int size)
+{
+ struct picolcd_data *data = hid_get_drvdata(hdev);
+ struct picolcd_pending *work;
+ struct hid_report *report = picolcd_out_report(report_id, hdev);
+ unsigned long flags;
+ int i, j, k;
+
+ if (!report || !data)
+ return NULL;
+ if (data->status & PICOLCD_FAILED)
+ return NULL;
+ work = kzalloc(sizeof(*work), GFP_KERNEL);
+ if (!work)
+ return NULL;
+
+ init_completion(&work->ready);
+ work->out_report = report;
+ work->in_report = NULL;
+ work->raw_size = 0;
+
+ mutex_lock(&data->mutex);
+ spin_lock_irqsave(&data->lock, flags);
+ for (i = k = 0; i < report->maxfield; i++)
+ for (j = 0; j < report->field[i]->report_count; j++) {
+ hid_set_field(report->field[i], j, k < size ? raw_data[k] : 0);
+ k++;
+ }
+ if (data->status & PICOLCD_FAILED) {
+ kfree(work);
+ work = NULL;
+ } else {
+ data->pending = work;
+ usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+ wait_for_completion_interruptible_timeout(&work->ready, HZ*2);
+ spin_lock_irqsave(&data->lock, flags);
+ data->pending = NULL;
+ }
+ spin_unlock_irqrestore(&data->lock, flags);
+ mutex_unlock(&data->mutex);
+ return work;
+}
+
+/*
+ * input class device
+ */
+static int picolcd_raw_keypad(struct picolcd_data *data,
+ struct hid_report *report, u8 *raw_data, int size)
+{
+ /*
+ * Keypad event
+ * First and second data bytes list currently pressed keys,
+ * 0x00 means no key and at most 2 keys may be pressed at same time
+ */
+ int i, j;
+
+ /* determine newly pressed keys */
+ for (i = 0; i < size; i++) {
+ unsigned int key_code;
+ if (raw_data[i] == 0)
+ continue;
+ for (j = 0; j < sizeof(data->pressed_keys); j++)
+ if (data->pressed_keys[j] == raw_data[i])
+ goto key_already_down;
+ for (j = 0; j < sizeof(data->pressed_keys); j++)
+ if (data->pressed_keys[j] == 0) {
+ data->pressed_keys[j] = raw_data[i];
+ break;
+ }
+ input_event(data->input_keys, EV_MSC, MSC_SCAN, raw_data[i]);
+ if (raw_data[i] < PICOLCD_KEYS)
+ key_code = data->keycode[raw_data[i]];
+ else
+ key_code = KEY_UNKNOWN;
+ if (key_code != KEY_UNKNOWN) {
+ dbg_hid(PICOLCD_NAME " got key press for %u:%d",
+ raw_data[i], key_code);
+ input_report_key(data->input_keys, key_code, 1);
+ }
+ input_sync(data->input_keys);
+key_already_down:
+ continue;
+ }
+
+ /* determine newly released keys */
+ for (j = 0; j < sizeof(data->pressed_keys); j++) {
+ unsigned int key_code;
+ if (data->pressed_keys[j] == 0)
+ continue;
+ for (i = 0; i < size; i++)
+ if (data->pressed_keys[j] == raw_data[i])
+ goto key_still_down;
+ input_event(data->input_keys, EV_MSC, MSC_SCAN, data->pressed_keys[j]);
+ if (data->pressed_keys[j] < PICOLCD_KEYS)
+ key_code = data->keycode[data->pressed_keys[j]];
+ else
+ key_code = KEY_UNKNOWN;
+ if (key_code != KEY_UNKNOWN) {
+ dbg_hid(PICOLCD_NAME " got key release for %u:%d",
+ data->pressed_keys[j], key_code);
+ input_report_key(data->input_keys, key_code, 0);
+ }
+ input_sync(data->input_keys);
+ data->pressed_keys[j] = 0;
+key_still_down:
+ continue;
+ }
+ return 1;
+}
+
+static int picolcd_check_version(struct hid_device *hdev)
+{
+ struct picolcd_data *data = hid_get_drvdata(hdev);
+ struct picolcd_pending *verinfo;
+ int ret = 0;
+
+ if (!data)
+ return -ENODEV;
+
+ verinfo = picolcd_send_and_wait(hdev, REPORT_VERSION, NULL, 0);
+ if (!verinfo) {
+ hid_err(hdev, "no version response from PicoLCD\n");
+ return -ENODEV;
+ }
+
+ if (verinfo->raw_size == 2) {
+ data->version[0] = verinfo->raw_data[1];
+ data->version[1] = verinfo->raw_data[0];
+ if (data->status & PICOLCD_BOOTLOADER) {
+ hid_info(hdev, "PicoLCD, bootloader version %d.%d\n",
+ verinfo->raw_data[1], verinfo->raw_data[0]);
+ } else {
+ hid_info(hdev, "PicoLCD, firmware version %d.%d\n",
+ verinfo->raw_data[1], verinfo->raw_data[0]);
+ }
+ } else {
+ hid_err(hdev, "confused, got unexpected version response from PicoLCD\n");
+ ret = -EINVAL;
+ }
+ kfree(verinfo);
+ return ret;
+}
+
+/*
+ * Reset our device and wait for answer to VERSION request
+ */
+int picolcd_reset(struct hid_device *hdev)
+{
+ struct picolcd_data *data = hid_get_drvdata(hdev);
+ struct hid_report *report = picolcd_out_report(REPORT_RESET, hdev);
+ unsigned long flags;
+ int error;
+
+ if (!data || !report || report->maxfield != 1)
+ return -ENODEV;
+
+ spin_lock_irqsave(&data->lock, flags);
+ if (hdev->product == USB_DEVICE_ID_PICOLCD_BOOTLOADER)
+ data->status |= PICOLCD_BOOTLOADER;
+
+ /* perform the reset */
+ hid_set_field(report->field[0], 0, 1);
+ if (data->status & PICOLCD_FAILED) {
+ spin_unlock_irqrestore(&data->lock, flags);
+ return -ENODEV;
+ }
+ usbhid_submit_report(hdev, report, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ error = picolcd_check_version(hdev);
+ if (error)
+ return error;
+
+ picolcd_resume_lcd(data);
+ picolcd_resume_backlight(data);
+ picolcd_fb_refresh(data);
+ picolcd_leds_set(data);
+ return 0;
+}
+
+/*
+ * The "operation_mode" sysfs attribute
+ */
+static ssize_t picolcd_operation_mode_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct picolcd_data *data = dev_get_drvdata(dev);
+
+ if (data->status & PICOLCD_BOOTLOADER)
+ return snprintf(buf, PAGE_SIZE, "[bootloader] lcd\n");
+ else
+ return snprintf(buf, PAGE_SIZE, "bootloader [lcd]\n");
+}
+
+static ssize_t picolcd_operation_mode_store(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t count)
+{
+ struct picolcd_data *data = dev_get_drvdata(dev);
+ struct hid_report *report = NULL;
+ size_t cnt = count;
+ int timeout = data->opmode_delay;
+ unsigned long flags;
+
+ if (cnt >= 3 && strncmp("lcd", buf, 3) == 0) {
+ if (data->status & PICOLCD_BOOTLOADER)
+ report = picolcd_out_report(REPORT_EXIT_FLASHER, data->hdev);
+ buf += 3;
+ cnt -= 3;
+ } else if (cnt >= 10 && strncmp("bootloader", buf, 10) == 0) {
+ if (!(data->status & PICOLCD_BOOTLOADER))
+ report = picolcd_out_report(REPORT_EXIT_KEYBOARD, data->hdev);
+ buf += 10;
+ cnt -= 10;
+ }
+ if (!report)
+ return -EINVAL;
+
+ while (cnt > 0 && (buf[cnt-1] == '\n' || buf[cnt-1] == '\r'))
+ cnt--;
+ if (cnt != 0)
+ return -EINVAL;
+
+ spin_lock_irqsave(&data->lock, flags);
+ hid_set_field(report->field[0], 0, timeout & 0xff);
+ hid_set_field(report->field[0], 1, (timeout >> 8) & 0xff);
+ usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+ return count;
+}
+
+static DEVICE_ATTR(operation_mode, 0644, picolcd_operation_mode_show,
+ picolcd_operation_mode_store);
+
+/*
+ * The "operation_mode_delay" sysfs attribute
+ */
+static ssize_t picolcd_operation_mode_delay_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct picolcd_data *data = dev_get_drvdata(dev);
+
+ return snprintf(buf, PAGE_SIZE, "%hu\n", data->opmode_delay);
+}
+
+static ssize_t picolcd_operation_mode_delay_store(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t count)
+{
+ struct picolcd_data *data = dev_get_drvdata(dev);
+ unsigned u;
+ if (sscanf(buf, "%u", &u) != 1)
+ return -EINVAL;
+ if (u > 30000)
+ return -EINVAL;
+ else
+ data->opmode_delay = u;
+ return count;
+}
+
+static DEVICE_ATTR(operation_mode_delay, 0644, picolcd_operation_mode_delay_show,
+ picolcd_operation_mode_delay_store);
+
+/*
+ * Handle raw report as sent by device
+ */
+static int picolcd_raw_event(struct hid_device *hdev,
+ struct hid_report *report, u8 *raw_data, int size)
+{
+ struct picolcd_data *data = hid_get_drvdata(hdev);
+ unsigned long flags;
+ int ret = 0;
+
+ if (!data)
+ return 1;
+
+ if (report->id == REPORT_KEY_STATE) {
+ if (data->input_keys)
+ ret = picolcd_raw_keypad(data, report, raw_data+1, size-1);
+ } else if (report->id == REPORT_IR_DATA) {
+ ret = picolcd_raw_cir(data, report, raw_data+1, size-1);
+ } else {
+ spin_lock_irqsave(&data->lock, flags);
+ /*
+ * We let the caller of picolcd_send_and_wait() check if the
+ * report we got is one of the expected ones or not.
+ */
+ if (data->pending) {
+ memcpy(data->pending->raw_data, raw_data+1, size-1);
+ data->pending->raw_size = size-1;
+ data->pending->in_report = report;
+ complete(&data->pending->ready);
+ }
+ spin_unlock_irqrestore(&data->lock, flags);
+ }
+
+ picolcd_debug_raw_event(data, hdev, report, raw_data, size);
+ return 1;
+}
+
+#ifdef CONFIG_PM
+static int picolcd_suspend(struct hid_device *hdev, pm_message_t message)
+{
+ if (PMSG_IS_AUTO(message))
+ return 0;
+
+ picolcd_suspend_backlight(hid_get_drvdata(hdev));
+ dbg_hid(PICOLCD_NAME " device ready for suspend\n");
+ return 0;
+}
+
+static int picolcd_resume(struct hid_device *hdev)
+{
+ int ret;
+ ret = picolcd_resume_backlight(hid_get_drvdata(hdev));
+ if (ret)
+ dbg_hid(PICOLCD_NAME " restoring backlight failed: %d\n", ret);
+ return 0;
+}
+
+static int picolcd_reset_resume(struct hid_device *hdev)
+{
+ int ret;
+ ret = picolcd_reset(hdev);
+ if (ret)
+ dbg_hid(PICOLCD_NAME " resetting our device failed: %d\n", ret);
+ ret = picolcd_fb_reset(hid_get_drvdata(hdev), 0);
+ if (ret)
+ dbg_hid(PICOLCD_NAME " restoring framebuffer content failed: %d\n", ret);
+ ret = picolcd_resume_lcd(hid_get_drvdata(hdev));
+ if (ret)
+ dbg_hid(PICOLCD_NAME " restoring lcd failed: %d\n", ret);
+ ret = picolcd_resume_backlight(hid_get_drvdata(hdev));
+ if (ret)
+ dbg_hid(PICOLCD_NAME " restoring backlight failed: %d\n", ret);
+ picolcd_leds_set(hid_get_drvdata(hdev));
+ return 0;
+}
+#endif
+
+/* initialize keypad input device */
+static int picolcd_init_keys(struct picolcd_data *data,
+ struct hid_report *report)
+{
+ struct hid_device *hdev = data->hdev;
+ struct input_dev *idev;
+ int error, i;
+
+ if (!report)
+ return -ENODEV;
+ if (report->maxfield != 1 || report->field[0]->report_count != 2 ||
+ report->field[0]->report_size != 8) {
+ hid_err(hdev, "unsupported KEY_STATE report\n");
+ return -EINVAL;
+ }
+
+ idev = input_allocate_device();
+ if (idev == NULL) {
+ hid_err(hdev, "failed to allocate input device\n");
+ return -ENOMEM;
+ }
+ input_set_drvdata(idev, hdev);
+ memcpy(data->keycode, def_keymap, sizeof(def_keymap));
+ idev->name = hdev->name;
+ idev->phys = hdev->phys;
+ idev->uniq = hdev->uniq;
+ idev->id.bustype = hdev->bus;
+ idev->id.vendor = hdev->vendor;
+ idev->id.product = hdev->product;
+ idev->id.version = hdev->version;
+ idev->dev.parent = &hdev->dev;
+ idev->keycode = &data->keycode;
+ idev->keycodemax = PICOLCD_KEYS;
+ idev->keycodesize = sizeof(data->keycode[0]);
+ input_set_capability(idev, EV_MSC, MSC_SCAN);
+ set_bit(EV_REP, idev->evbit);
+ for (i = 0; i < PICOLCD_KEYS; i++)
+ input_set_capability(idev, EV_KEY, data->keycode[i]);
+ error = input_register_device(idev);
+ if (error) {
+ hid_err(hdev, "error registering the input device\n");
+ input_free_device(idev);
+ return error;
+ }
+ data->input_keys = idev;
+ return 0;
+}
+
+static void picolcd_exit_keys(struct picolcd_data *data)
+{
+ struct input_dev *idev = data->input_keys;
+
+ data->input_keys = NULL;
+ if (idev)
+ input_unregister_device(idev);
+}
+
+static int picolcd_probe_lcd(struct hid_device *hdev, struct picolcd_data *data)
+{
+ int error;
+
+ /* Setup keypad input device */
+ error = picolcd_init_keys(data, picolcd_in_report(REPORT_KEY_STATE, hdev));
+ if (error)
+ goto err;
+
+ /* Setup CIR input device */
+ error = picolcd_init_cir(data, picolcd_in_report(REPORT_IR_DATA, hdev));
+ if (error)
+ goto err;
+
+ /* Set up the framebuffer device */
+ error = picolcd_init_framebuffer(data);
+ if (error)
+ goto err;
+
+ /* Setup lcd class device */
+ error = picolcd_init_lcd(data, picolcd_out_report(REPORT_CONTRAST, hdev));
+ if (error)
+ goto err;
+
+ /* Setup backlight class device */
+ error = picolcd_init_backlight(data, picolcd_out_report(REPORT_BRIGHTNESS, hdev));
+ if (error)
+ goto err;
+
+ /* Setup the LED class devices */
+ error = picolcd_init_leds(data, picolcd_out_report(REPORT_LED_STATE, hdev));
+ if (error)
+ goto err;
+
+ picolcd_init_devfs(data, picolcd_out_report(REPORT_EE_READ, hdev),
+ picolcd_out_report(REPORT_EE_WRITE, hdev),
+ picolcd_out_report(REPORT_READ_MEMORY, hdev),
+ picolcd_out_report(REPORT_WRITE_MEMORY, hdev),
+ picolcd_out_report(REPORT_RESET, hdev));
+ return 0;
+err:
+ picolcd_exit_leds(data);
+ picolcd_exit_backlight(data);
+ picolcd_exit_lcd(data);
+ picolcd_exit_framebuffer(data);
+ picolcd_exit_cir(data);
+ picolcd_exit_keys(data);
+ return error;
+}
+
+static int picolcd_probe_bootloader(struct hid_device *hdev, struct picolcd_data *data)
+{
+ picolcd_init_devfs(data, NULL, NULL,
+ picolcd_out_report(REPORT_BL_READ_MEMORY, hdev),
+ picolcd_out_report(REPORT_BL_WRITE_MEMORY, hdev), NULL);
+ return 0;
+}
+
+static int picolcd_probe(struct hid_device *hdev,
+ const struct hid_device_id *id)
+{
+ struct picolcd_data *data;
+ int error = -ENOMEM;
+
+ dbg_hid(PICOLCD_NAME " hardware probe...\n");
+
+ /*
+ * Let's allocate the picolcd data structure, set some reasonable
+ * defaults, and associate it with the device
+ */
+ data = kzalloc(sizeof(struct picolcd_data), GFP_KERNEL);
+ if (data == NULL) {
+ hid_err(hdev, "can't allocate space for Minibox PicoLCD device data\n");
+ error = -ENOMEM;
+ goto err_no_cleanup;
+ }
+
+ spin_lock_init(&data->lock);
+ mutex_init(&data->mutex);
+ data->hdev = hdev;
+ data->opmode_delay = 5000;
+ if (hdev->product == USB_DEVICE_ID_PICOLCD_BOOTLOADER)
+ data->status |= PICOLCD_BOOTLOADER;
+ hid_set_drvdata(hdev, data);
+
+ /* Parse the device reports and start it up */
+ error = hid_parse(hdev);
+ if (error) {
+ hid_err(hdev, "device report parse failed\n");
+ goto err_cleanup_data;
+ }
+
+ error = hid_hw_start(hdev, 0);
+ if (error) {
+ hid_err(hdev, "hardware start failed\n");
+ goto err_cleanup_data;
+ }
+
+ error = hid_hw_open(hdev);
+ if (error) {
+ hid_err(hdev, "failed to open input interrupt pipe for key and IR events\n");
+ goto err_cleanup_hid_hw;
+ }
+
+ error = device_create_file(&hdev->dev, &dev_attr_operation_mode_delay);
+ if (error) {
+ hid_err(hdev, "failed to create sysfs attributes\n");
+ goto err_cleanup_hid_ll;
+ }
+
+ error = device_create_file(&hdev->dev, &dev_attr_operation_mode);
+ if (error) {
+ hid_err(hdev, "failed to create sysfs attributes\n");
+ goto err_cleanup_sysfs1;
+ }
+
+ if (data->status & PICOLCD_BOOTLOADER)
+ error = picolcd_probe_bootloader(hdev, data);
+ else
+ error = picolcd_probe_lcd(hdev, data);
+ if (error)
+ goto err_cleanup_sysfs2;
+
+ dbg_hid(PICOLCD_NAME " activated and initialized\n");
+ return 0;
+
+err_cleanup_sysfs2:
+ device_remove_file(&hdev->dev, &dev_attr_operation_mode);
+err_cleanup_sysfs1:
+ device_remove_file(&hdev->dev, &dev_attr_operation_mode_delay);
+err_cleanup_hid_ll:
+ hid_hw_close(hdev);
+err_cleanup_hid_hw:
+ hid_hw_stop(hdev);
+err_cleanup_data:
+ kfree(data);
+err_no_cleanup:
+ hid_set_drvdata(hdev, NULL);
+
+ return error;
+}
+
+static void picolcd_remove(struct hid_device *hdev)
+{
+ struct picolcd_data *data = hid_get_drvdata(hdev);
+ unsigned long flags;
+
+ dbg_hid(PICOLCD_NAME " hardware remove...\n");
+ spin_lock_irqsave(&data->lock, flags);
+ data->status |= PICOLCD_FAILED;
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ picolcd_exit_devfs(data);
+ device_remove_file(&hdev->dev, &dev_attr_operation_mode);
+ device_remove_file(&hdev->dev, &dev_attr_operation_mode_delay);
+ hid_hw_close(hdev);
+ hid_hw_stop(hdev);
+
+ /* Shortcut potential pending reply that will never arrive */
+ spin_lock_irqsave(&data->lock, flags);
+ if (data->pending)
+ complete(&data->pending->ready);
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ /* Cleanup LED */
+ picolcd_exit_leds(data);
+ /* Clean up the framebuffer */
+ picolcd_exit_backlight(data);
+ picolcd_exit_lcd(data);
+ picolcd_exit_framebuffer(data);
+ /* Cleanup input */
+ picolcd_exit_cir(data);
+ picolcd_exit_keys(data);
+
+ hid_set_drvdata(hdev, NULL);
+ mutex_destroy(&data->mutex);
+ /* Finally, clean up the picolcd data itself */
+ kfree(data);
+}
+
+static const struct hid_device_id picolcd_devices[] = {
+ { HID_USB_DEVICE(USB_VENDOR_ID_MICROCHIP, USB_DEVICE_ID_PICOLCD) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_MICROCHIP, USB_DEVICE_ID_PICOLCD_BOOTLOADER) },
+ { }
+};
+MODULE_DEVICE_TABLE(hid, picolcd_devices);
+
+static struct hid_driver picolcd_driver = {
+ .name = "hid-picolcd",
+ .id_table = picolcd_devices,
+ .probe = picolcd_probe,
+ .remove = picolcd_remove,
+ .raw_event = picolcd_raw_event,
+#ifdef CONFIG_PM
+ .suspend = picolcd_suspend,
+ .resume = picolcd_resume,
+ .reset_resume = picolcd_reset_resume,
+#endif
+};
+
+static int __init picolcd_init(void)
+{
+ return hid_register_driver(&picolcd_driver);
+}
+
+static void __exit picolcd_exit(void)
+{
+ hid_unregister_driver(&picolcd_driver);
+}
+
+module_init(picolcd_init);
+module_exit(picolcd_exit);
+MODULE_DESCRIPTION("Minibox graphics PicoLCD Driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/hid/hid-picolcd_debugfs.c b/drivers/hid/hid-picolcd_debugfs.c
new file mode 100644
index 0000000..4809aa1
--- /dev/null
+++ b/drivers/hid/hid-picolcd_debugfs.c
@@ -0,0 +1,899 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include <linux/hid-debug.h>
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+
+#include <linux/module.h>
+#include <linux/uaccess.h>
+
+#include "hid-picolcd.h"
+
+
+static int picolcd_debug_reset_show(struct seq_file *f, void *p)
+{
+ if (picolcd_fbinfo((struct picolcd_data *)f->private))
+ seq_printf(f, "all fb\n");
+ else
+ seq_printf(f, "all\n");
+ return 0;
+}
+
+static int picolcd_debug_reset_open(struct inode *inode, struct file *f)
+{
+ return single_open(f, picolcd_debug_reset_show, inode->i_private);
+}
+
+static ssize_t picolcd_debug_reset_write(struct file *f, const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct picolcd_data *data = ((struct seq_file *)f->private_data)->private;
+ char buf[32];
+ size_t cnt = min(count, sizeof(buf)-1);
+ if (copy_from_user(buf, user_buf, cnt))
+ return -EFAULT;
+
+ while (cnt > 0 && (buf[cnt-1] == ' ' || buf[cnt-1] == '\n'))
+ cnt--;
+ buf[cnt] = '\0';
+ if (strcmp(buf, "all") == 0) {
+ picolcd_reset(data->hdev);
+ picolcd_fb_reset(data, 1);
+ } else if (strcmp(buf, "fb") == 0) {
+ picolcd_fb_reset(data, 1);
+ } else {
+ return -EINVAL;
+ }
+ return count;
+}
+
+static const struct file_operations picolcd_debug_reset_fops = {
+ .owner = THIS_MODULE,
+ .open = picolcd_debug_reset_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .write = picolcd_debug_reset_write,
+ .release = single_release,
+};
+
+/*
+ * The "eeprom" file
+ */
+static ssize_t picolcd_debug_eeprom_read(struct file *f, char __user *u,
+ size_t s, loff_t *off)
+{
+ struct picolcd_data *data = f->private_data;
+ struct picolcd_pending *resp;
+ u8 raw_data[3];
+ ssize_t ret = -EIO;
+
+ if (s == 0)
+ return -EINVAL;
+ if (*off > 0x0ff)
+ return 0;
+
+ /* prepare buffer with info about what we want to read (addr & len) */
+ raw_data[0] = *off & 0xff;
+ raw_data[1] = (*off >> 8) & 0xff;
+ raw_data[2] = s < 20 ? s : 20;
+ if (*off + raw_data[2] > 0xff)
+ raw_data[2] = 0x100 - *off;
+ resp = picolcd_send_and_wait(data->hdev, REPORT_EE_READ, raw_data,
+ sizeof(raw_data));
+ if (!resp)
+ return -EIO;
+
+ if (resp->in_report && resp->in_report->id == REPORT_EE_DATA) {
+ /* successful read :) */
+ ret = resp->raw_data[2];
+ if (ret > s)
+ ret = s;
+ if (copy_to_user(u, resp->raw_data+3, ret))
+ ret = -EFAULT;
+ else
+ *off += ret;
+ } /* anything else is some kind of IO error */
+
+ kfree(resp);
+ return ret;
+}
+
+static ssize_t picolcd_debug_eeprom_write(struct file *f, const char __user *u,
+ size_t s, loff_t *off)
+{
+ struct picolcd_data *data = f->private_data;
+ struct picolcd_pending *resp;
+ ssize_t ret = -EIO;
+ u8 raw_data[23];
+
+ if (s == 0)
+ return -EINVAL;
+ if (*off > 0x0ff)
+ return -ENOSPC;
+
+ memset(raw_data, 0, sizeof(raw_data));
+ raw_data[0] = *off & 0xff;
+ raw_data[1] = (*off >> 8) & 0xff;
+ raw_data[2] = min_t(size_t, 20, s);
+ if (*off + raw_data[2] > 0xff)
+ raw_data[2] = 0x100 - *off;
+
+ if (copy_from_user(raw_data+3, u, min((u8)20, raw_data[2])))
+ return -EFAULT;
+ resp = picolcd_send_and_wait(data->hdev, REPORT_EE_WRITE, raw_data,
+ sizeof(raw_data));
+
+ if (!resp)
+ return -EIO;
+
+ if (resp->in_report && resp->in_report->id == REPORT_EE_DATA) {
+ /* check if written data matches */
+ if (memcmp(raw_data, resp->raw_data, 3+raw_data[2]) == 0) {
+ *off += raw_data[2];
+ ret = raw_data[2];
+ }
+ }
+ kfree(resp);
+ return ret;
+}
+
+/*
+ * Notes:
+ * - read/write happens in chunks of at most 20 bytes, it's up to userspace
+ * to loop in order to get more data.
+ * - on write errors on otherwise correct write request the bytes
+ * that should have been written are in undefined state.
+ */
+static const struct file_operations picolcd_debug_eeprom_fops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = picolcd_debug_eeprom_read,
+ .write = picolcd_debug_eeprom_write,
+ .llseek = generic_file_llseek,
+};
+
+/*
+ * The "flash" file
+ */
+/* record a flash address to buf (bounds check to be done by caller) */
+static int _picolcd_flash_setaddr(struct picolcd_data *data, u8 *buf, long off)
+{
+ buf[0] = off & 0xff;
+ buf[1] = (off >> 8) & 0xff;
+ if (data->addr_sz == 3)
+ buf[2] = (off >> 16) & 0xff;
+ return data->addr_sz == 2 ? 2 : 3;
+}
+
+/* read a given size of data (bounds check to be done by caller) */
+static ssize_t _picolcd_flash_read(struct picolcd_data *data, int report_id,
+ char __user *u, size_t s, loff_t *off)
+{
+ struct picolcd_pending *resp;
+ u8 raw_data[4];
+ ssize_t ret = 0;
+ int len_off, err = -EIO;
+
+ while (s > 0) {
+ err = -EIO;
+ len_off = _picolcd_flash_setaddr(data, raw_data, *off);
+ raw_data[len_off] = s > 32 ? 32 : s;
+ resp = picolcd_send_and_wait(data->hdev, report_id, raw_data, len_off+1);
+ if (!resp || !resp->in_report)
+ goto skip;
+ if (resp->in_report->id == REPORT_MEMORY ||
+ resp->in_report->id == REPORT_BL_READ_MEMORY) {
+ if (memcmp(raw_data, resp->raw_data, len_off+1) != 0)
+ goto skip;
+ if (copy_to_user(u+ret, resp->raw_data+len_off+1, raw_data[len_off])) {
+ err = -EFAULT;
+ goto skip;
+ }
+ *off += raw_data[len_off];
+ s -= raw_data[len_off];
+ ret += raw_data[len_off];
+ err = 0;
+ }
+skip:
+ kfree(resp);
+ if (err)
+ return ret > 0 ? ret : err;
+ }
+ return ret;
+}
+
+static ssize_t picolcd_debug_flash_read(struct file *f, char __user *u,
+ size_t s, loff_t *off)
+{
+ struct picolcd_data *data = f->private_data;
+
+ if (s == 0)
+ return -EINVAL;
+ if (*off > 0x05fff)
+ return 0;
+ if (*off + s > 0x05fff)
+ s = 0x06000 - *off;
+
+ if (data->status & PICOLCD_BOOTLOADER)
+ return _picolcd_flash_read(data, REPORT_BL_READ_MEMORY, u, s, off);
+ else
+ return _picolcd_flash_read(data, REPORT_READ_MEMORY, u, s, off);
+}
+
+/* erase block aligned to 64bytes boundary */
+static ssize_t _picolcd_flash_erase64(struct picolcd_data *data, int report_id,
+ loff_t *off)
+{
+ struct picolcd_pending *resp;
+ u8 raw_data[3];
+ int len_off;
+ ssize_t ret = -EIO;
+
+ if (*off & 0x3f)
+ return -EINVAL;
+
+ len_off = _picolcd_flash_setaddr(data, raw_data, *off);
+ resp = picolcd_send_and_wait(data->hdev, report_id, raw_data, len_off);
+ if (!resp || !resp->in_report)
+ goto skip;
+ if (resp->in_report->id == REPORT_MEMORY ||
+ resp->in_report->id == REPORT_BL_ERASE_MEMORY) {
+ if (memcmp(raw_data, resp->raw_data, len_off) != 0)
+ goto skip;
+ ret = 0;
+ }
+skip:
+ kfree(resp);
+ return ret;
+}
+
+/* write a given size of data (bounds check to be done by caller) */
+static ssize_t _picolcd_flash_write(struct picolcd_data *data, int report_id,
+ const char __user *u, size_t s, loff_t *off)
+{
+ struct picolcd_pending *resp;
+ u8 raw_data[36];
+ ssize_t ret = 0;
+ int len_off, err = -EIO;
+
+ while (s > 0) {
+ err = -EIO;
+ len_off = _picolcd_flash_setaddr(data, raw_data, *off);
+ raw_data[len_off] = s > 32 ? 32 : s;
+ if (copy_from_user(raw_data+len_off+1, u, raw_data[len_off])) {
+ err = -EFAULT;
+ break;
+ }
+ resp = picolcd_send_and_wait(data->hdev, report_id, raw_data,
+ len_off+1+raw_data[len_off]);
+ if (!resp || !resp->in_report)
+ goto skip;
+ if (resp->in_report->id == REPORT_MEMORY ||
+ resp->in_report->id == REPORT_BL_WRITE_MEMORY) {
+ if (memcmp(raw_data, resp->raw_data, len_off+1+raw_data[len_off]) != 0)
+ goto skip;
+ *off += raw_data[len_off];
+ s -= raw_data[len_off];
+ ret += raw_data[len_off];
+ err = 0;
+ }
+skip:
+ kfree(resp);
+ if (err)
+ break;
+ }
+ return ret > 0 ? ret : err;
+}
+
+static ssize_t picolcd_debug_flash_write(struct file *f, const char __user *u,
+ size_t s, loff_t *off)
+{
+ struct picolcd_data *data = f->private_data;
+ ssize_t err, ret = 0;
+ int report_erase, report_write;
+
+ if (s == 0)
+ return -EINVAL;
+ if (*off > 0x5fff)
+ return -ENOSPC;
+ if (s & 0x3f)
+ return -EINVAL;
+ if (*off & 0x3f)
+ return -EINVAL;
+
+ if (data->status & PICOLCD_BOOTLOADER) {
+ report_erase = REPORT_BL_ERASE_MEMORY;
+ report_write = REPORT_BL_WRITE_MEMORY;
+ } else {
+ report_erase = REPORT_ERASE_MEMORY;
+ report_write = REPORT_WRITE_MEMORY;
+ }
+ mutex_lock(&data->mutex_flash);
+ while (s > 0) {
+ err = _picolcd_flash_erase64(data, report_erase, off);
+ if (err)
+ break;
+ err = _picolcd_flash_write(data, report_write, u, 64, off);
+ if (err < 0)
+ break;
+ ret += err;
+ *off += err;
+ s -= err;
+ if (err != 64)
+ break;
+ }
+ mutex_unlock(&data->mutex_flash);
+ return ret > 0 ? ret : err;
+}
+
+/*
+ * Notes:
+ * - concurrent writing is prevented by mutex and all writes must be
+ * n*64 bytes and 64-byte aligned, each write being preceded by an
+ * ERASE which erases a 64byte block.
+ * If less than requested was written or an error is returned for an
+ * otherwise correct write request the next 64-byte block which should
+ * have been written is in undefined state (mostly: original, erased,
+ * (half-)written with write error)
+ * - reading can happen without special restriction
+ */
+static const struct file_operations picolcd_debug_flash_fops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = picolcd_debug_flash_read,
+ .write = picolcd_debug_flash_write,
+ .llseek = generic_file_llseek,
+};
+
+
+/*
+ * Helper code for HID report level dumping/debugging
+ */
+static const char * const error_codes[] = {
+ "success", "parameter missing", "data_missing", "block readonly",
+ "block not erasable", "block too big", "section overflow",
+ "invalid command length", "invalid data length",
+};
+
+static void dump_buff_as_hex(char *dst, size_t dst_sz, const u8 *data,
+ const size_t data_len)
+{
+ int i, j;
+ for (i = j = 0; i < data_len && j + 4 < dst_sz; i++) {
+ dst[j++] = hex_asc[(data[i] >> 4) & 0x0f];
+ dst[j++] = hex_asc[data[i] & 0x0f];
+ dst[j++] = ' ';
+ }
+ dst[j] = '\0';
+ if (j > 0)
+ dst[j-1] = '\n';
+ if (i < data_len && j > 2)
+ dst[j-2] = dst[j-3] = '.';
+}
+
+void picolcd_debug_out_report(struct picolcd_data *data,
+ struct hid_device *hdev, struct hid_report *report)
+{
+ u8 raw_data[70];
+ int raw_size = (report->size >> 3) + 1;
+ char *buff;
+#define BUFF_SZ 256
+
+ /* Avoid unnecessary overhead if debugfs is disabled */
+ if (list_empty(&hdev->debug_list))
+ return;
+
+ buff = kmalloc(BUFF_SZ, GFP_ATOMIC);
+ if (!buff)
+ return;
+
+ snprintf(buff, BUFF_SZ, "\nout report %d (size %d) = ",
+ report->id, raw_size);
+ hid_debug_event(hdev, buff);
+ if (raw_size + 5 > sizeof(raw_data)) {
+ kfree(buff);
+ hid_debug_event(hdev, " TOO BIG\n");
+ return;
+ } else {
+ raw_data[0] = report->id;
+ hid_output_report(report, raw_data);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data, raw_size);
+ hid_debug_event(hdev, buff);
+ }
+
+ switch (report->id) {
+ case REPORT_LED_STATE:
+ /* 1 data byte with GPO state */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_LED_STATE", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tGPO state: 0x%02x\n", raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_BRIGHTNESS:
+ /* 1 data byte with brightness */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_BRIGHTNESS", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tBrightness: 0x%02x\n", raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_CONTRAST:
+ /* 1 data byte with contrast */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_CONTRAST", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tContrast: 0x%02x\n", raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_RESET:
+ /* 2 data bytes with reset duration in ms */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_RESET", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tDuration: 0x%02x%02x (%dms)\n",
+ raw_data[2], raw_data[1], raw_data[2] << 8 | raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_LCD_CMD:
+ /* 63 data bytes with LCD commands */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_LCD_CMD", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ /* TODO: format decoding */
+ break;
+ case REPORT_LCD_DATA:
+ /* 63 data bytes with LCD data */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_LCD_CMD", report->id, raw_size-1);
+ /* TODO: format decoding */
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_LCD_CMD_DATA:
+ /* 63 data bytes with LCD commands and data */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_LCD_CMD", report->id, raw_size-1);
+ /* TODO: format decoding */
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_EE_READ:
+ /* 3 data bytes with read area description */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_EE_READ", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_EE_WRITE:
+ /* 3+1..20 data bytes with write area description */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_EE_WRITE", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
+ hid_debug_event(hdev, buff);
+ if (raw_data[3] == 0) {
+ snprintf(buff, BUFF_SZ, "\tNo data\n");
+ } else if (raw_data[3] + 4 <= raw_size) {
+ snprintf(buff, BUFF_SZ, "\tData: ");
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tData overflowed\n");
+ }
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_ERASE_MEMORY:
+ case REPORT_BL_ERASE_MEMORY:
+ /* 3 data bytes with pointer inside erase block */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_ERASE_MEMORY", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ switch (data->addr_sz) {
+ case 2:
+ snprintf(buff, BUFF_SZ, "\tAddress inside 64 byte block: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ break;
+ case 3:
+ snprintf(buff, BUFF_SZ, "\tAddress inside 64 byte block: 0x%02x%02x%02x\n",
+ raw_data[3], raw_data[2], raw_data[1]);
+ break;
+ default:
+ snprintf(buff, BUFF_SZ, "\tNot supported\n");
+ }
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_READ_MEMORY:
+ case REPORT_BL_READ_MEMORY:
+ /* 4 data bytes with read area description */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_READ_MEMORY", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ switch (data->addr_sz) {
+ case 2:
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
+ break;
+ case 3:
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x%02x\n",
+ raw_data[3], raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[4]);
+ break;
+ default:
+ snprintf(buff, BUFF_SZ, "\tNot supported\n");
+ }
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_WRITE_MEMORY:
+ case REPORT_BL_WRITE_MEMORY:
+ /* 4+1..32 data bytes with write adrea description */
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_WRITE_MEMORY", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ switch (data->addr_sz) {
+ case 2:
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
+ hid_debug_event(hdev, buff);
+ if (raw_data[3] == 0) {
+ snprintf(buff, BUFF_SZ, "\tNo data\n");
+ } else if (raw_data[3] + 4 <= raw_size) {
+ snprintf(buff, BUFF_SZ, "\tData: ");
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tData overflowed\n");
+ }
+ break;
+ case 3:
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x%02x\n",
+ raw_data[3], raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[4]);
+ hid_debug_event(hdev, buff);
+ if (raw_data[4] == 0) {
+ snprintf(buff, BUFF_SZ, "\tNo data\n");
+ } else if (raw_data[4] + 5 <= raw_size) {
+ snprintf(buff, BUFF_SZ, "\tData: ");
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+5, raw_data[4]);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tData overflowed\n");
+ }
+ break;
+ default:
+ snprintf(buff, BUFF_SZ, "\tNot supported\n");
+ }
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_SPLASH_RESTART:
+ /* TODO */
+ break;
+ case REPORT_EXIT_KEYBOARD:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_EXIT_KEYBOARD", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tRestart delay: %dms (0x%02x%02x)\n",
+ raw_data[1] | (raw_data[2] << 8),
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_VERSION:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_VERSION", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_DEVID:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_DEVID", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_SPLASH_SIZE:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_SPLASH_SIZE", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_HOOK_VERSION:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_HOOK_VERSION", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_EXIT_FLASHER:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "REPORT_VERSION", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tRestart delay: %dms (0x%02x%02x)\n",
+ raw_data[1] | (raw_data[2] << 8),
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ default:
+ snprintf(buff, BUFF_SZ, "out report %s (%d, size=%d)\n",
+ "<unknown>", report->id, raw_size-1);
+ hid_debug_event(hdev, buff);
+ break;
+ }
+ wake_up_interruptible(&hdev->debug_wait);
+ kfree(buff);
+}
+
+void picolcd_debug_raw_event(struct picolcd_data *data,
+ struct hid_device *hdev, struct hid_report *report,
+ u8 *raw_data, int size)
+{
+ char *buff;
+
+#define BUFF_SZ 256
+ /* Avoid unnecessary overhead if debugfs is disabled */
+ if (list_empty(&hdev->debug_list))
+ return;
+
+ buff = kmalloc(BUFF_SZ, GFP_ATOMIC);
+ if (!buff)
+ return;
+
+ switch (report->id) {
+ case REPORT_ERROR_CODE:
+ /* 2 data bytes with affected report and error code */
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_ERROR_CODE", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ if (raw_data[2] < ARRAY_SIZE(error_codes))
+ snprintf(buff, BUFF_SZ, "\tError code 0x%02x (%s) in reply to report 0x%02x\n",
+ raw_data[2], error_codes[raw_data[2]], raw_data[1]);
+ else
+ snprintf(buff, BUFF_SZ, "\tError code 0x%02x in reply to report 0x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_KEY_STATE:
+ /* 2 data bytes with key state */
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_KEY_STATE", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ if (raw_data[1] == 0)
+ snprintf(buff, BUFF_SZ, "\tNo key pressed\n");
+ else if (raw_data[2] == 0)
+ snprintf(buff, BUFF_SZ, "\tOne key pressed: 0x%02x (%d)\n",
+ raw_data[1], raw_data[1]);
+ else
+ snprintf(buff, BUFF_SZ, "\tTwo keys pressed: 0x%02x (%d), 0x%02x (%d)\n",
+ raw_data[1], raw_data[1], raw_data[2], raw_data[2]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_IR_DATA:
+ /* Up to 20 byes of IR scancode data */
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_IR_DATA", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ if (raw_data[1] == 0) {
+ snprintf(buff, BUFF_SZ, "\tUnexpectedly 0 data length\n");
+ hid_debug_event(hdev, buff);
+ } else if (raw_data[1] + 1 <= size) {
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n\tIR Data: ",
+ raw_data[1]);
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+2, raw_data[1]);
+ hid_debug_event(hdev, buff);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tOverflowing data length: %d\n",
+ raw_data[1]-1);
+ hid_debug_event(hdev, buff);
+ }
+ break;
+ case REPORT_EE_DATA:
+ /* Data buffer in response to REPORT_EE_READ or REPORT_EE_WRITE */
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_EE_DATA", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
+ hid_debug_event(hdev, buff);
+ if (raw_data[3] == 0) {
+ snprintf(buff, BUFF_SZ, "\tNo data\n");
+ hid_debug_event(hdev, buff);
+ } else if (raw_data[3] + 4 <= size) {
+ snprintf(buff, BUFF_SZ, "\tData: ");
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
+ hid_debug_event(hdev, buff);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tData overflowed\n");
+ hid_debug_event(hdev, buff);
+ }
+ break;
+ case REPORT_MEMORY:
+ /* Data buffer in response to REPORT_READ_MEMORY or REPORT_WRTIE_MEMORY */
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_MEMORY", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ switch (data->addr_sz) {
+ case 2:
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[3]);
+ hid_debug_event(hdev, buff);
+ if (raw_data[3] == 0) {
+ snprintf(buff, BUFF_SZ, "\tNo data\n");
+ } else if (raw_data[3] + 4 <= size) {
+ snprintf(buff, BUFF_SZ, "\tData: ");
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+4, raw_data[3]);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tData overflowed\n");
+ }
+ break;
+ case 3:
+ snprintf(buff, BUFF_SZ, "\tData address: 0x%02x%02x%02x\n",
+ raw_data[3], raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tData length: %d\n", raw_data[4]);
+ hid_debug_event(hdev, buff);
+ if (raw_data[4] == 0) {
+ snprintf(buff, BUFF_SZ, "\tNo data\n");
+ } else if (raw_data[4] + 5 <= size) {
+ snprintf(buff, BUFF_SZ, "\tData: ");
+ hid_debug_event(hdev, buff);
+ dump_buff_as_hex(buff, BUFF_SZ, raw_data+5, raw_data[4]);
+ } else {
+ snprintf(buff, BUFF_SZ, "\tData overflowed\n");
+ }
+ break;
+ default:
+ snprintf(buff, BUFF_SZ, "\tNot supported\n");
+ }
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_VERSION:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_VERSION", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tFirmware version: %d.%d\n",
+ raw_data[2], raw_data[1]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_BL_ERASE_MEMORY:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_BL_ERASE_MEMORY", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ /* TODO */
+ break;
+ case REPORT_BL_READ_MEMORY:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_BL_READ_MEMORY", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ /* TODO */
+ break;
+ case REPORT_BL_WRITE_MEMORY:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_BL_WRITE_MEMORY", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ /* TODO */
+ break;
+ case REPORT_DEVID:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_DEVID", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tSerial: 0x%02x%02x%02x%02x\n",
+ raw_data[1], raw_data[2], raw_data[3], raw_data[4]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tType: 0x%02x\n",
+ raw_data[5]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_SPLASH_SIZE:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_SPLASH_SIZE", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tTotal splash space: %d\n",
+ (raw_data[2] << 8) | raw_data[1]);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tUsed splash space: %d\n",
+ (raw_data[4] << 8) | raw_data[3]);
+ hid_debug_event(hdev, buff);
+ break;
+ case REPORT_HOOK_VERSION:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "REPORT_HOOK_VERSION", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ snprintf(buff, BUFF_SZ, "\tFirmware version: %d.%d\n",
+ raw_data[1], raw_data[2]);
+ hid_debug_event(hdev, buff);
+ break;
+ default:
+ snprintf(buff, BUFF_SZ, "report %s (%d, size=%d)\n",
+ "<unknown>", report->id, size-1);
+ hid_debug_event(hdev, buff);
+ break;
+ }
+ wake_up_interruptible(&hdev->debug_wait);
+ kfree(buff);
+}
+
+void picolcd_init_devfs(struct picolcd_data *data,
+ struct hid_report *eeprom_r, struct hid_report *eeprom_w,
+ struct hid_report *flash_r, struct hid_report *flash_w,
+ struct hid_report *reset)
+{
+ struct hid_device *hdev = data->hdev;
+
+ mutex_init(&data->mutex_flash);
+
+ /* reset */
+ if (reset)
+ data->debug_reset = debugfs_create_file("reset", 0600,
+ hdev->debug_dir, data, &picolcd_debug_reset_fops);
+
+ /* eeprom */
+ if (eeprom_r || eeprom_w)
+ data->debug_eeprom = debugfs_create_file("eeprom",
+ (eeprom_w ? S_IWUSR : 0) | (eeprom_r ? S_IRUSR : 0),
+ hdev->debug_dir, data, &picolcd_debug_eeprom_fops);
+
+ /* flash */
+ if (flash_r && flash_r->maxfield == 1 && flash_r->field[0]->report_size == 8)
+ data->addr_sz = flash_r->field[0]->report_count - 1;
+ else
+ data->addr_sz = -1;
+ if (data->addr_sz == 2 || data->addr_sz == 3) {
+ data->debug_flash = debugfs_create_file("flash",
+ (flash_w ? S_IWUSR : 0) | (flash_r ? S_IRUSR : 0),
+ hdev->debug_dir, data, &picolcd_debug_flash_fops);
+ } else if (flash_r || flash_w)
+ hid_warn(hdev, "Unexpected FLASH access reports, please submit rdesc for review\n");
+}
+
+void picolcd_exit_devfs(struct picolcd_data *data)
+{
+ struct dentry *dent;
+
+ dent = data->debug_reset;
+ data->debug_reset = NULL;
+ if (dent)
+ debugfs_remove(dent);
+ dent = data->debug_eeprom;
+ data->debug_eeprom = NULL;
+ if (dent)
+ debugfs_remove(dent);
+ dent = data->debug_flash;
+ data->debug_flash = NULL;
+ if (dent)
+ debugfs_remove(dent);
+ mutex_destroy(&data->mutex_flash);
+}
+
diff --git a/drivers/hid/hid-picolcd_fb.c b/drivers/hid/hid-picolcd_fb.c
new file mode 100644
index 0000000..0008a51
--- /dev/null
+++ b/drivers/hid/hid-picolcd_fb.c
@@ -0,0 +1,615 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include <linux/vmalloc.h>
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/module.h>
+
+#include "hid-picolcd.h"
+
+/* Framebuffer
+ *
+ * The PicoLCD use a Topway LCD module of 256x64 pixel
+ * This display area is tiled over 4 controllers with 8 tiles
+ * each. Each tile has 8x64 pixel, each data byte representing
+ * a 1-bit wide vertical line of the tile.
+ *
+ * The display can be updated at a tile granularity.
+ *
+ * Chip 1 Chip 2 Chip 3 Chip 4
+ * +----------------+----------------+----------------+----------------+
+ * | Tile 1 | Tile 1 | Tile 1 | Tile 1 |
+ * +----------------+----------------+----------------+----------------+
+ * | Tile 2 | Tile 2 | Tile 2 | Tile 2 |
+ * +----------------+----------------+----------------+----------------+
+ * ...
+ * +----------------+----------------+----------------+----------------+
+ * | Tile 8 | Tile 8 | Tile 8 | Tile 8 |
+ * +----------------+----------------+----------------+----------------+
+ */
+#define PICOLCDFB_NAME "picolcdfb"
+#define PICOLCDFB_WIDTH (256)
+#define PICOLCDFB_HEIGHT (64)
+#define PICOLCDFB_SIZE (PICOLCDFB_WIDTH * PICOLCDFB_HEIGHT / 8)
+
+#define PICOLCDFB_UPDATE_RATE_LIMIT 10
+#define PICOLCDFB_UPDATE_RATE_DEFAULT 2
+
+/* Framebuffer visual structures */
+static const struct fb_fix_screeninfo picolcdfb_fix = {
+ .id = PICOLCDFB_NAME,
+ .type = FB_TYPE_PACKED_PIXELS,
+ .visual = FB_VISUAL_MONO01,
+ .xpanstep = 0,
+ .ypanstep = 0,
+ .ywrapstep = 0,
+ .line_length = PICOLCDFB_WIDTH / 8,
+ .accel = FB_ACCEL_NONE,
+};
+
+static const struct fb_var_screeninfo picolcdfb_var = {
+ .xres = PICOLCDFB_WIDTH,
+ .yres = PICOLCDFB_HEIGHT,
+ .xres_virtual = PICOLCDFB_WIDTH,
+ .yres_virtual = PICOLCDFB_HEIGHT,
+ .width = 103,
+ .height = 26,
+ .bits_per_pixel = 1,
+ .grayscale = 1,
+ .red = {
+ .offset = 0,
+ .length = 1,
+ .msb_right = 0,
+ },
+ .green = {
+ .offset = 0,
+ .length = 1,
+ .msb_right = 0,
+ },
+ .blue = {
+ .offset = 0,
+ .length = 1,
+ .msb_right = 0,
+ },
+ .transp = {
+ .offset = 0,
+ .length = 0,
+ .msb_right = 0,
+ },
+};
+
+/* Send a given tile to PicoLCD */
+static int picolcd_fb_send_tile(struct picolcd_data *data, u8 *vbitmap,
+ int chip, int tile)
+{
+ struct hid_report *report1, *report2;
+ unsigned long flags;
+ u8 *tdata;
+ int i;
+
+ report1 = picolcd_out_report(REPORT_LCD_CMD_DATA, data->hdev);
+ if (!report1 || report1->maxfield != 1)
+ return -ENODEV;
+ report2 = picolcd_out_report(REPORT_LCD_DATA, data->hdev);
+ if (!report2 || report2->maxfield != 1)
+ return -ENODEV;
+
+ spin_lock_irqsave(&data->lock, flags);
+ if ((data->status & PICOLCD_FAILED)) {
+ spin_unlock_irqrestore(&data->lock, flags);
+ return -ENODEV;
+ }
+ hid_set_field(report1->field[0], 0, chip << 2);
+ hid_set_field(report1->field[0], 1, 0x02);
+ hid_set_field(report1->field[0], 2, 0x00);
+ hid_set_field(report1->field[0], 3, 0x00);
+ hid_set_field(report1->field[0], 4, 0xb8 | tile);
+ hid_set_field(report1->field[0], 5, 0x00);
+ hid_set_field(report1->field[0], 6, 0x00);
+ hid_set_field(report1->field[0], 7, 0x40);
+ hid_set_field(report1->field[0], 8, 0x00);
+ hid_set_field(report1->field[0], 9, 0x00);
+ hid_set_field(report1->field[0], 10, 32);
+
+ hid_set_field(report2->field[0], 0, (chip << 2) | 0x01);
+ hid_set_field(report2->field[0], 1, 0x00);
+ hid_set_field(report2->field[0], 2, 0x00);
+ hid_set_field(report2->field[0], 3, 32);
+
+ tdata = vbitmap + (tile * 4 + chip) * 64;
+ for (i = 0; i < 64; i++)
+ if (i < 32)
+ hid_set_field(report1->field[0], 11 + i, tdata[i]);
+ else
+ hid_set_field(report2->field[0], 4 + i - 32, tdata[i]);
+
+ usbhid_submit_report(data->hdev, report1, USB_DIR_OUT);
+ usbhid_submit_report(data->hdev, report2, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+ return 0;
+}
+
+/* Translate a single tile*/
+static int picolcd_fb_update_tile(u8 *vbitmap, const u8 *bitmap, int bpp,
+ int chip, int tile)
+{
+ int i, b, changed = 0;
+ u8 tdata[64];
+ u8 *vdata = vbitmap + (tile * 4 + chip) * 64;
+
+ if (bpp == 1) {
+ for (b = 7; b >= 0; b--) {
+ const u8 *bdata = bitmap + tile * 256 + chip * 8 + b * 32;
+ for (i = 0; i < 64; i++) {
+ tdata[i] <<= 1;
+ tdata[i] |= (bdata[i/8] >> (i % 8)) & 0x01;
+ }
+ }
+ } else if (bpp == 8) {
+ for (b = 7; b >= 0; b--) {
+ const u8 *bdata = bitmap + (tile * 256 + chip * 8 + b * 32) * 8;
+ for (i = 0; i < 64; i++) {
+ tdata[i] <<= 1;
+ tdata[i] |= (bdata[i] & 0x80) ? 0x01 : 0x00;
+ }
+ }
+ } else {
+ /* Oops, we should never get here! */
+ WARN_ON(1);
+ return 0;
+ }
+
+ for (i = 0; i < 64; i++)
+ if (tdata[i] != vdata[i]) {
+ changed = 1;
+ vdata[i] = tdata[i];
+ }
+ return changed;
+}
+
+void picolcd_fb_refresh(struct picolcd_data *data)
+{
+ if (data->fb_info)
+ schedule_delayed_work(&data->fb_info->deferred_work, 0);
+}
+
+/* Reconfigure LCD display */
+int picolcd_fb_reset(struct picolcd_data *data, int clear)
+{
+ struct hid_report *report = picolcd_out_report(REPORT_LCD_CMD, data->hdev);
+ struct picolcd_fb_data *fbdata = data->fb_info->par;
+ int i, j;
+ unsigned long flags;
+ static const u8 mapcmd[8] = { 0x00, 0x02, 0x00, 0x64, 0x3f, 0x00, 0x64, 0xc0 };
+
+ if (!report || report->maxfield != 1)
+ return -ENODEV;
+
+ spin_lock_irqsave(&data->lock, flags);
+ for (i = 0; i < 4; i++) {
+ for (j = 0; j < report->field[0]->maxusage; j++)
+ if (j == 0)
+ hid_set_field(report->field[0], j, i << 2);
+ else if (j < sizeof(mapcmd))
+ hid_set_field(report->field[0], j, mapcmd[j]);
+ else
+ hid_set_field(report->field[0], j, 0);
+ usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
+ }
+ spin_unlock_irqrestore(&data->lock, flags);
+
+ if (clear) {
+ memset(fbdata->vbitmap, 0, PICOLCDFB_SIZE);
+ memset(fbdata->bitmap, 0, PICOLCDFB_SIZE*fbdata->bpp);
+ }
+ fbdata->force = 1;
+
+ /* schedule first output of framebuffer */
+ if (fbdata->ready)
+ schedule_delayed_work(&data->fb_info->deferred_work, 0);
+ else
+ fbdata->ready = 1;
+
+ return 0;
+}
+
+/* Update fb_vbitmap from the screen_base and send changed tiles to device */
+static void picolcd_fb_update(struct fb_info *info)
+{
+ int chip, tile, n;
+ unsigned long flags;
+ struct picolcd_fb_data *fbdata = info->par;
+ struct picolcd_data *data;
+
+ mutex_lock(&info->lock);
+
+ spin_lock_irqsave(&fbdata->lock, flags);
+ if (!fbdata->ready && fbdata->picolcd)
+ picolcd_fb_reset(fbdata->picolcd, 0);
+ spin_unlock_irqrestore(&fbdata->lock, flags);
+
+ /*
+ * Translate the framebuffer into the format needed by the PicoLCD.
+ * See display layout above.
+ * Do this one tile after the other and push those tiles that changed.
+ *
+ * Wait for our IO to complete as otherwise we might flood the queue!
+ */
+ n = 0;
+ for (chip = 0; chip < 4; chip++)
+ for (tile = 0; tile < 8; tile++) {
+ if (!fbdata->force && !picolcd_fb_update_tile(
+ fbdata->vbitmap, fbdata->bitmap,
+ fbdata->bpp, chip, tile))
+ continue;
+ n += 2;
+ if (n >= HID_OUTPUT_FIFO_SIZE / 2) {
+ spin_lock_irqsave(&fbdata->lock, flags);
+ data = fbdata->picolcd;
+ spin_unlock_irqrestore(&fbdata->lock, flags);
+ mutex_unlock(&info->lock);
+ if (!data)
+ return;
+ usbhid_wait_io(data->hdev);
+ mutex_lock(&info->lock);
+ n = 0;
+ }
+ spin_lock_irqsave(&fbdata->lock, flags);
+ data = fbdata->picolcd;
+ spin_unlock_irqrestore(&fbdata->lock, flags);
+ if (!data || picolcd_fb_send_tile(data,
+ fbdata->vbitmap, chip, tile))
+ goto out;
+ }
+ fbdata->force = false;
+ if (n) {
+ spin_lock_irqsave(&fbdata->lock, flags);
+ data = fbdata->picolcd;
+ spin_unlock_irqrestore(&fbdata->lock, flags);
+ mutex_unlock(&info->lock);
+ if (data)
+ usbhid_wait_io(data->hdev);
+ return;
+ }
+out:
+ mutex_unlock(&info->lock);
+}
+
+/* Stub to call the system default and update the image on the picoLCD */
+static void picolcd_fb_fillrect(struct fb_info *info,
+ const struct fb_fillrect *rect)
+{
+ if (!info->par)
+ return;
+ sys_fillrect(info, rect);
+
+ schedule_delayed_work(&info->deferred_work, 0);
+}
+
+/* Stub to call the system default and update the image on the picoLCD */
+static void picolcd_fb_copyarea(struct fb_info *info,
+ const struct fb_copyarea *area)
+{
+ if (!info->par)
+ return;
+ sys_copyarea(info, area);
+
+ schedule_delayed_work(&info->deferred_work, 0);
+}
+
+/* Stub to call the system default and update the image on the picoLCD */
+static void picolcd_fb_imageblit(struct fb_info *info, const struct fb_image *image)
+{
+ if (!info->par)
+ return;
+ sys_imageblit(info, image);
+
+ schedule_delayed_work(&info->deferred_work, 0);
+}
+
+/*
+ * this is the slow path from userspace. they can seek and write to
+ * the fb. it's inefficient to do anything less than a full screen draw
+ */
+static ssize_t picolcd_fb_write(struct fb_info *info, const char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ ssize_t ret;
+ if (!info->par)
+ return -ENODEV;
+ ret = fb_sys_write(info, buf, count, ppos);
+ if (ret >= 0)
+ schedule_delayed_work(&info->deferred_work, 0);
+ return ret;
+}
+
+static int picolcd_fb_blank(int blank, struct fb_info *info)
+{
+ /* We let fb notification do this for us via lcd/backlight device */
+ return 0;
+}
+
+static void picolcd_fb_destroy(struct fb_info *info)
+{
+ struct picolcd_fb_data *fbdata = info->par;
+
+ /* make sure no work is deferred */
+ fb_deferred_io_cleanup(info);
+
+ /* No thridparty should ever unregister our framebuffer! */
+ WARN_ON(fbdata->picolcd != NULL);
+
+ vfree((u8 *)info->fix.smem_start);
+ framebuffer_release(info);
+}
+
+static int picolcd_fb_check_var(struct fb_var_screeninfo *var, struct fb_info *info)
+{
+ __u32 bpp = var->bits_per_pixel;
+ __u32 activate = var->activate;
+
+ /* only allow 1/8 bit depth (8-bit is grayscale) */
+ *var = picolcdfb_var;
+ var->activate = activate;
+ if (bpp >= 8) {
+ var->bits_per_pixel = 8;
+ var->red.length = 8;
+ var->green.length = 8;
+ var->blue.length = 8;
+ } else {
+ var->bits_per_pixel = 1;
+ var->red.length = 1;
+ var->green.length = 1;
+ var->blue.length = 1;
+ }
+ return 0;
+}
+
+static int picolcd_set_par(struct fb_info *info)
+{
+ struct picolcd_fb_data *fbdata = info->par;
+ u8 *tmp_fb, *o_fb;
+ if (info->var.bits_per_pixel == fbdata->bpp)
+ return 0;
+ /* switch between 1/8 bit depths */
+ if (info->var.bits_per_pixel != 1 && info->var.bits_per_pixel != 8)
+ return -EINVAL;
+
+ o_fb = fbdata->bitmap;
+ tmp_fb = kmalloc(PICOLCDFB_SIZE*info->var.bits_per_pixel, GFP_KERNEL);
+ if (!tmp_fb)
+ return -ENOMEM;
+
+ /* translate FB content to new bits-per-pixel */
+ if (info->var.bits_per_pixel == 1) {
+ int i, b;
+ for (i = 0; i < PICOLCDFB_SIZE; i++) {
+ u8 p = 0;
+ for (b = 0; b < 8; b++) {
+ p <<= 1;
+ p |= o_fb[i*8+b] ? 0x01 : 0x00;
+ }
+ tmp_fb[i] = p;
+ }
+ memcpy(o_fb, tmp_fb, PICOLCDFB_SIZE);
+ info->fix.visual = FB_VISUAL_MONO01;
+ info->fix.line_length = PICOLCDFB_WIDTH / 8;
+ } else {
+ int i;
+ memcpy(tmp_fb, o_fb, PICOLCDFB_SIZE);
+ for (i = 0; i < PICOLCDFB_SIZE * 8; i++)
+ o_fb[i] = tmp_fb[i/8] & (0x01 << (7 - i % 8)) ? 0xff : 0x00;
+ info->fix.visual = FB_VISUAL_DIRECTCOLOR;
+ info->fix.line_length = PICOLCDFB_WIDTH;
+ }
+
+ kfree(tmp_fb);
+ fbdata->bpp = info->var.bits_per_pixel;
+ return 0;
+}
+
+/* Note this can't be const because of struct fb_info definition */
+static struct fb_ops picolcdfb_ops = {
+ .owner = THIS_MODULE,
+ .fb_destroy = picolcd_fb_destroy,
+ .fb_read = fb_sys_read,
+ .fb_write = picolcd_fb_write,
+ .fb_blank = picolcd_fb_blank,
+ .fb_fillrect = picolcd_fb_fillrect,
+ .fb_copyarea = picolcd_fb_copyarea,
+ .fb_imageblit = picolcd_fb_imageblit,
+ .fb_check_var = picolcd_fb_check_var,
+ .fb_set_par = picolcd_set_par,
+};
+
+
+/* Callback from deferred IO workqueue */
+static void picolcd_fb_deferred_io(struct fb_info *info, struct list_head *pagelist)
+{
+ picolcd_fb_update(info);
+}
+
+static const struct fb_deferred_io picolcd_fb_defio = {
+ .delay = HZ / PICOLCDFB_UPDATE_RATE_DEFAULT,
+ .deferred_io = picolcd_fb_deferred_io,
+};
+
+
+/*
+ * The "fb_update_rate" sysfs attribute
+ */
+static ssize_t picolcd_fb_update_rate_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct picolcd_data *data = dev_get_drvdata(dev);
+ struct picolcd_fb_data *fbdata = data->fb_info->par;
+ unsigned i, fb_update_rate = fbdata->update_rate;
+ size_t ret = 0;
+
+ for (i = 1; i <= PICOLCDFB_UPDATE_RATE_LIMIT; i++)
+ if (ret >= PAGE_SIZE)
+ break;
+ else if (i == fb_update_rate)
+ ret += snprintf(buf+ret, PAGE_SIZE-ret, "[%u] ", i);
+ else
+ ret += snprintf(buf+ret, PAGE_SIZE-ret, "%u ", i);
+ if (ret > 0)
+ buf[min(ret, (size_t)PAGE_SIZE)-1] = '\n';
+ return ret;
+}
+
+static ssize_t picolcd_fb_update_rate_store(struct device *dev,
+ struct device_attribute *attr, const char *buf, size_t count)
+{
+ struct picolcd_data *data = dev_get_drvdata(dev);
+ struct picolcd_fb_data *fbdata = data->fb_info->par;
+ int i;
+ unsigned u;
+
+ if (count < 1 || count > 10)
+ return -EINVAL;
+
+ i = sscanf(buf, "%u", &u);
+ if (i != 1)
+ return -EINVAL;
+
+ if (u > PICOLCDFB_UPDATE_RATE_LIMIT)
+ return -ERANGE;
+ else if (u == 0)
+ u = PICOLCDFB_UPDATE_RATE_DEFAULT;
+
+ fbdata->update_rate = u;
+ data->fb_info->fbdefio->delay = HZ / fbdata->update_rate;
+ return count;
+}
+
+static DEVICE_ATTR(fb_update_rate, 0666, picolcd_fb_update_rate_show,
+ picolcd_fb_update_rate_store);
+
+/* initialize Framebuffer device */
+int picolcd_init_framebuffer(struct picolcd_data *data)
+{
+ struct device *dev = &data->hdev->dev;
+ struct fb_info *info = NULL;
+ struct picolcd_fb_data *fbdata = NULL;
+ int i, error = -ENOMEM;
+ u32 *palette;
+
+ /* The extra memory is:
+ * - 256*u32 for pseudo_palette
+ * - struct fb_deferred_io
+ */
+ info = framebuffer_alloc(256 * sizeof(u32) +
+ sizeof(struct fb_deferred_io) +
+ sizeof(struct picolcd_fb_data) +
+ PICOLCDFB_SIZE, dev);
+ if (info == NULL) {
+ dev_err(dev, "failed to allocate a framebuffer\n");
+ goto err_nomem;
+ }
+
+ info->fbdefio = info->par;
+ *info->fbdefio = picolcd_fb_defio;
+ info->par += sizeof(struct fb_deferred_io);
+ palette = info->par;
+ info->par += 256 * sizeof(u32);
+ for (i = 0; i < 256; i++)
+ palette[i] = i > 0 && i < 16 ? 0xff : 0;
+ info->pseudo_palette = palette;
+ info->fbops = &picolcdfb_ops;
+ info->var = picolcdfb_var;
+ info->fix = picolcdfb_fix;
+ info->fix.smem_len = PICOLCDFB_SIZE*8;
+ info->flags = FBINFO_FLAG_DEFAULT;
+
+ fbdata = info->par;
+ spin_lock_init(&fbdata->lock);
+ fbdata->picolcd = data;
+ fbdata->update_rate = PICOLCDFB_UPDATE_RATE_DEFAULT;
+ fbdata->bpp = picolcdfb_var.bits_per_pixel;
+ fbdata->force = 1;
+ fbdata->vbitmap = info->par + sizeof(struct picolcd_fb_data);
+ fbdata->bitmap = vmalloc(PICOLCDFB_SIZE*8);
+ if (fbdata->bitmap == NULL) {
+ dev_err(dev, "can't get a free page for framebuffer\n");
+ goto err_nomem;
+ }
+ info->screen_base = (char __force __iomem *)fbdata->bitmap;
+ info->fix.smem_start = (unsigned long)fbdata->bitmap;
+ memset(fbdata->vbitmap, 0xff, PICOLCDFB_SIZE);
+ data->fb_info = info;
+
+ error = picolcd_fb_reset(data, 1);
+ if (error) {
+ dev_err(dev, "failed to configure display\n");
+ goto err_cleanup;
+ }
+
+ error = device_create_file(dev, &dev_attr_fb_update_rate);
+ if (error) {
+ dev_err(dev, "failed to create sysfs attributes\n");
+ goto err_cleanup;
+ }
+
+ fb_deferred_io_init(info);
+ error = register_framebuffer(info);
+ if (error) {
+ dev_err(dev, "failed to register framebuffer\n");
+ goto err_sysfs;
+ }
+ return 0;
+
+err_sysfs:
+ device_remove_file(dev, &dev_attr_fb_update_rate);
+ fb_deferred_io_cleanup(info);
+err_cleanup:
+ data->fb_info = NULL;
+
+err_nomem:
+ if (fbdata)
+ vfree(fbdata->bitmap);
+ framebuffer_release(info);
+ return error;
+}
+
+void picolcd_exit_framebuffer(struct picolcd_data *data)
+{
+ struct fb_info *info = data->fb_info;
+ struct picolcd_fb_data *fbdata = info->par;
+ unsigned long flags;
+
+ device_remove_file(&data->hdev->dev, &dev_attr_fb_update_rate);
+
+ /* disconnect framebuffer from HID dev */
+ spin_lock_irqsave(&fbdata->lock, flags);
+ fbdata->picolcd = NULL;
+ spin_unlock_irqrestore(&fbdata->lock, flags);
+
+ /* make sure there is no running update - thus that fbdata->picolcd
+ * once obtained under lock is guaranteed not to get free() under
+ * the feet of the deferred work */
+ flush_delayed_work_sync(&info->deferred_work);
+
+ data->fb_info = NULL;
+ unregister_framebuffer(info);
+}
diff --git a/drivers/hid/hid-picolcd_lcd.c b/drivers/hid/hid-picolcd_lcd.c
new file mode 100644
index 0000000..2d0ddc5
--- /dev/null
+++ b/drivers/hid/hid-picolcd_lcd.c
@@ -0,0 +1,107 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/lcd.h>
+
+#include "hid-picolcd.h"
+
+/*
+ * lcd class device
+ */
+static int picolcd_get_contrast(struct lcd_device *ldev)
+{
+ struct picolcd_data *data = lcd_get_data(ldev);
+ return data->lcd_contrast;
+}
+
+static int picolcd_set_contrast(struct lcd_device *ldev, int contrast)
+{
+ struct picolcd_data *data = lcd_get_data(ldev);
+ struct hid_report *report = picolcd_out_report(REPORT_CONTRAST, data->hdev);
+ unsigned long flags;
+
+ if (!report || report->maxfield != 1 || report->field[0]->report_count != 1)
+ return -ENODEV;
+
+ data->lcd_contrast = contrast & 0x0ff;
+ spin_lock_irqsave(&data->lock, flags);
+ hid_set_field(report->field[0], 0, data->lcd_contrast);
+ if (!(data->status & PICOLCD_FAILED))
+ usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+ return 0;
+}
+
+static int picolcd_check_lcd_fb(struct lcd_device *ldev, struct fb_info *fb)
+{
+ return fb && fb == picolcd_fbinfo((struct picolcd_data *)lcd_get_data(ldev));
+}
+
+static struct lcd_ops picolcd_lcdops = {
+ .get_contrast = picolcd_get_contrast,
+ .set_contrast = picolcd_set_contrast,
+ .check_fb = picolcd_check_lcd_fb,
+};
+
+int picolcd_init_lcd(struct picolcd_data *data, struct hid_report *report)
+{
+ struct device *dev = &data->hdev->dev;
+ struct lcd_device *ldev;
+
+ if (!report)
+ return -ENODEV;
+ if (report->maxfield != 1 || report->field[0]->report_count != 1 ||
+ report->field[0]->report_size != 8) {
+ dev_err(dev, "unsupported CONTRAST report");
+ return -EINVAL;
+ }
+
+ ldev = lcd_device_register(dev_name(dev), dev, data, &picolcd_lcdops);
+ if (IS_ERR(ldev)) {
+ dev_err(dev, "failed to register LCD\n");
+ return PTR_ERR(ldev);
+ }
+ ldev->props.max_contrast = 0x0ff;
+ data->lcd_contrast = 0xe5;
+ data->lcd = ldev;
+ picolcd_set_contrast(ldev, 0xe5);
+ return 0;
+}
+
+void picolcd_exit_lcd(struct picolcd_data *data)
+{
+ struct lcd_device *ldev = data->lcd;
+
+ data->lcd = NULL;
+ if (ldev)
+ lcd_device_unregister(ldev);
+}
+
+int picolcd_resume_lcd(struct picolcd_data *data)
+{
+ if (!data->lcd)
+ return 0;
+ return picolcd_set_contrast(data->lcd, data->lcd_contrast);
+}
+
diff --git a/drivers/hid/hid-picolcd_leds.c b/drivers/hid/hid-picolcd_leds.c
new file mode 100644
index 0000000..28cb6a4
--- /dev/null
+++ b/drivers/hid/hid-picolcd_leds.c
@@ -0,0 +1,175 @@
+/***************************************************************************
+ * Copyright (C) 2010-2012 by Bruno Prémont <bonbons@linux-vserver.org> *
+ * *
+ * Based on Logitech G13 driver (v0.4) *
+ * Copyright (C) 2009 by Rick L. Vinyard, Jr. <rvinyard@cs.nmsu.edu> *
+ * *
+ * This program is free software: you can redistribute it and/or modify *
+ * it under the terms of the GNU General Public License as published by *
+ * the Free Software Foundation, version 2 of the License. *
+ * *
+ * This driver is distributed in the hope that it will be useful, but *
+ * WITHOUT ANY WARRANTY; without even the implied warranty of *
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
+ * General Public License for more details. *
+ * *
+ * You should have received a copy of the GNU General Public License *
+ * along with this software. If not see <http://www.gnu.org/licenses/>. *
+ ***************************************************************************/
+
+#include <linux/hid.h>
+#include <linux/hid-debug.h>
+#include <linux/input.h>
+#include "hid-ids.h"
+#include "usbhid/usbhid.h"
+#include <linux/usb.h>
+
+#include <linux/fb.h>
+#include <linux/vmalloc.h>
+#include <linux/backlight.h>
+#include <linux/lcd.h>
+
+#include <linux/leds.h>
+
+#include <linux/seq_file.h>
+#include <linux/debugfs.h>
+
+#include <linux/completion.h>
+#include <linux/uaccess.h>
+#include <linux/module.h>
+
+#include "hid-picolcd.h"
+
+
+void picolcd_leds_set(struct picolcd_data *data)
+{
+ struct hid_report *report;
+ unsigned long flags;
+
+ if (!data->led[0])
+ return;
+ report = picolcd_out_report(REPORT_LED_STATE, data->hdev);
+ if (!report || report->maxfield != 1 || report->field[0]->report_count != 1)
+ return;
+
+ spin_lock_irqsave(&data->lock, flags);
+ hid_set_field(report->field[0], 0, data->led_state);
+ if (!(data->status & PICOLCD_FAILED))
+ usbhid_submit_report(data->hdev, report, USB_DIR_OUT);
+ spin_unlock_irqrestore(&data->lock, flags);
+}
+
+static void picolcd_led_set_brightness(struct led_classdev *led_cdev,
+ enum led_brightness value)
+{
+ struct device *dev;
+ struct hid_device *hdev;
+ struct picolcd_data *data;
+ int i, state = 0;
+
+ dev = led_cdev->dev->parent;
+ hdev = container_of(dev, struct hid_device, dev);
+ data = hid_get_drvdata(hdev);
+ if (!data)
+ return;
+ for (i = 0; i < 8; i++) {
+ if (led_cdev != data->led[i])
+ continue;
+ state = (data->led_state >> i) & 1;
+ if (value == LED_OFF && state) {
+ data->led_state &= ~(1 << i);
+ picolcd_leds_set(data);
+ } else if (value != LED_OFF && !state) {
+ data->led_state |= 1 << i;
+ picolcd_leds_set(data);
+ }
+ break;
+ }
+}
+
+static enum led_brightness picolcd_led_get_brightness(struct led_classdev *led_cdev)
+{
+ struct device *dev;
+ struct hid_device *hdev;
+ struct picolcd_data *data;
+ int i, value = 0;
+
+ dev = led_cdev->dev->parent;
+ hdev = container_of(dev, struct hid_device, dev);
+ data = hid_get_drvdata(hdev);
+ for (i = 0; i < 8; i++)
+ if (led_cdev == data->led[i]) {
+ value = (data->led_state >> i) & 1;
+ break;
+ }
+ return value ? LED_FULL : LED_OFF;
+}
+
+int picolcd_init_leds(struct picolcd_data *data, struct hid_report *report)
+{
+ struct device *dev = &data->hdev->dev;
+ struct led_classdev *led;
+ size_t name_sz = strlen(dev_name(dev)) + 8;
+ char *name;
+ int i, ret = 0;
+
+ if (!report)
+ return -ENODEV;
+ if (report->maxfield != 1 || report->field[0]->report_count != 1 ||
+ report->field[0]->report_size != 8) {
+ dev_err(dev, "unsupported LED_STATE report");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < 8; i++) {
+ led = kzalloc(sizeof(struct led_classdev)+name_sz, GFP_KERNEL);
+ if (!led) {
+ dev_err(dev, "can't allocate memory for LED %d\n", i);
+ ret = -ENOMEM;
+ goto err;
+ }
+ name = (void *)(&led[1]);
+ snprintf(name, name_sz, "%s::GPO%d", dev_name(dev), i);
+ led->name = name;
+ led->brightness = 0;
+ led->max_brightness = 1;
+ led->brightness_get = picolcd_led_get_brightness;
+ led->brightness_set = picolcd_led_set_brightness;
+
+ data->led[i] = led;
+ ret = led_classdev_register(dev, data->led[i]);
+ if (ret) {
+ data->led[i] = NULL;
+ kfree(led);
+ dev_err(dev, "can't register LED %d\n", i);
+ goto err;
+ }
+ }
+ return 0;
+err:
+ for (i = 0; i < 8; i++)
+ if (data->led[i]) {
+ led = data->led[i];
+ data->led[i] = NULL;
+ led_classdev_unregister(led);
+ kfree(led);
+ }
+ return ret;
+}
+
+void picolcd_exit_leds(struct picolcd_data *data)
+{
+ struct led_classdev *led;
+ int i;
+
+ for (i = 0; i < 8; i++) {
+ led = data->led[i];
+ data->led[i] = NULL;
+ if (!led)
+ continue;
+ led_classdev_unregister(led);
+ kfree(led);
+ }
+}
+
+
diff --git a/drivers/hid/hid-primax.c b/drivers/hid/hid-primax.c
index 4d3c60d..c15adb0 100644
--- a/drivers/hid/hid-primax.c
+++ b/drivers/hid/hid-primax.c
@@ -64,29 +64,6 @@
return 0;
}
-static int px_probe(struct hid_device *hid, const struct hid_device_id *id)
-{
- int ret;
-
- ret = hid_parse(hid);
- if (ret) {
- hid_err(hid, "parse failed\n");
- goto fail;
- }
-
- ret = hid_hw_start(hid, HID_CONNECT_DEFAULT);
- if (ret)
- hid_err(hid, "hw start failed\n");
-
-fail:
- return ret;
-}
-
-static void px_remove(struct hid_device *hid)
-{
- hid_hw_stop(hid);
-}
-
static const struct hid_device_id px_devices[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_PRIMAX, USB_DEVICE_ID_PRIMAX_KEYBOARD) },
{ }
@@ -97,8 +74,6 @@
.name = "primax",
.id_table = px_devices,
.raw_event = px_raw_event,
- .probe = px_probe,
- .remove = px_remove,
};
static int __init px_init(void)
diff --git a/drivers/hid/hid-prodikeys.c b/drivers/hid/hid-prodikeys.c
index b71b77a..ec8ca33 100644
--- a/drivers/hid/hid-prodikeys.c
+++ b/drivers/hid/hid-prodikeys.c
@@ -105,7 +105,7 @@
struct device_attribute *attr, char *buf)
{
struct hid_device *hdev = container_of(dev, struct hid_device, dev);
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
dbg_hid("pcmidi sysfs read channel=%u\n", pk->pm->midi_channel);
@@ -118,7 +118,7 @@
struct device_attribute *attr, const char *buf, size_t count)
{
struct hid_device *hdev = container_of(dev, struct hid_device, dev);
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
unsigned channel = 0;
@@ -142,7 +142,7 @@
struct device_attribute *attr, char *buf)
{
struct hid_device *hdev = container_of(dev, struct hid_device, dev);
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
dbg_hid("pcmidi sysfs read sustain=%u\n", pk->pm->midi_sustain);
@@ -155,7 +155,7 @@
struct device_attribute *attr, const char *buf, size_t count)
{
struct hid_device *hdev = container_of(dev, struct hid_device, dev);
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
unsigned sustain = 0;
@@ -181,7 +181,7 @@
struct device_attribute *attr, char *buf)
{
struct hid_device *hdev = container_of(dev, struct hid_device, dev);
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
dbg_hid("pcmidi sysfs read octave=%d\n", pk->pm->midi_octave);
@@ -194,7 +194,7 @@
struct device_attribute *attr, const char *buf, size_t count)
{
struct hid_device *hdev = container_of(dev, struct hid_device, dev);
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
int octave = 0;
@@ -759,7 +759,7 @@
struct hid_field *field, struct hid_usage *usage,
unsigned long **bit, int *max)
{
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
struct pcmidi_snd *pm;
pm = pk->pm;
@@ -777,7 +777,7 @@
static int pk_raw_event(struct hid_device *hdev, struct hid_report *report,
u8 *data, int size)
{
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
int ret = 0;
if (1 == pk->pm->ifnum) {
@@ -858,7 +858,7 @@
static void pk_remove(struct hid_device *hdev)
{
- struct pk_device *pk = (struct pk_device *)hid_get_drvdata(hdev);
+ struct pk_device *pk = hid_get_drvdata(hdev);
struct pcmidi_snd *pm;
pm = pk->pm;
diff --git a/drivers/hid/hid-ps3remote.c b/drivers/hid/hid-ps3remote.c
new file mode 100644
index 0000000..03811e5
--- /dev/null
+++ b/drivers/hid/hid-ps3remote.c
@@ -0,0 +1,215 @@
+/*
+ * HID driver for Sony PS3 BD Remote Control
+ *
+ * Copyright (c) 2012 David Dillow <dave@thedillows.org>
+ * Based on a blend of the bluez fakehid user-space code by Marcel Holtmann
+ * and other kernel HID drivers.
+ */
+
+/*
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ */
+
+/* NOTE: in order for the Sony PS3 BD Remote Control to be found by
+ * a Bluetooth host, the key combination Start+Enter has to be kept pressed
+ * for about 7 seconds with the Bluetooth Host Controller in discovering mode.
+ *
+ * There will be no PIN request from the device.
+ */
+
+#include <linux/device.h>
+#include <linux/hid.h>
+#include <linux/module.h>
+
+#include "hid-ids.h"
+
+static __u8 ps3remote_rdesc[] = {
+ 0x05, 0x01, /* GUsagePage Generic Desktop */
+ 0x09, 0x05, /* LUsage 0x05 [Game Pad] */
+ 0xA1, 0x01, /* MCollection Application (mouse, keyboard) */
+
+ /* Use collection 1 for joypad buttons */
+ 0xA1, 0x02, /* MCollection Logical (interrelated data) */
+
+ /* Ignore the 1st byte, maybe it is used for a controller
+ * number but it's not needed for correct operation */
+ 0x75, 0x08, /* GReportSize 0x08 [8] */
+ 0x95, 0x01, /* GReportCount 0x01 [1] */
+ 0x81, 0x01, /* MInput 0x01 (Const[0] Arr[1] Abs[2]) */
+
+ /* Bytes from 2nd to 4th are a bitmap for joypad buttons, for these
+ * buttons multiple keypresses are allowed */
+ 0x05, 0x09, /* GUsagePage Button */
+ 0x19, 0x01, /* LUsageMinimum 0x01 [Button 1 (primary/trigger)] */
+ 0x29, 0x18, /* LUsageMaximum 0x18 [Button 24] */
+ 0x14, /* GLogicalMinimum [0] */
+ 0x25, 0x01, /* GLogicalMaximum 0x01 [1] */
+ 0x75, 0x01, /* GReportSize 0x01 [1] */
+ 0x95, 0x18, /* GReportCount 0x18 [24] */
+ 0x81, 0x02, /* MInput 0x02 (Data[0] Var[1] Abs[2]) */
+
+ 0xC0, /* MEndCollection */
+
+ /* Use collection 2 for remote control buttons */
+ 0xA1, 0x02, /* MCollection Logical (interrelated data) */
+
+ /* 5th byte is used for remote control buttons */
+ 0x05, 0x09, /* GUsagePage Button */
+ 0x18, /* LUsageMinimum [No button pressed] */
+ 0x29, 0xFE, /* LUsageMaximum 0xFE [Button 254] */
+ 0x14, /* GLogicalMinimum [0] */
+ 0x26, 0xFE, 0x00, /* GLogicalMaximum 0x00FE [254] */
+ 0x75, 0x08, /* GReportSize 0x08 [8] */
+ 0x95, 0x01, /* GReportCount 0x01 [1] */
+ 0x80, /* MInput */
+
+ /* Ignore bytes from 6th to 11th, 6th to 10th are always constant at
+ * 0xff and 11th is for press indication */
+ 0x75, 0x08, /* GReportSize 0x08 [8] */
+ 0x95, 0x06, /* GReportCount 0x06 [6] */
+ 0x81, 0x01, /* MInput 0x01 (Const[0] Arr[1] Abs[2]) */
+
+ /* 12th byte is for battery strength */
+ 0x05, 0x06, /* GUsagePage Generic Device Controls */
+ 0x09, 0x20, /* LUsage 0x20 [Battery Strength] */
+ 0x14, /* GLogicalMinimum [0] */
+ 0x25, 0x05, /* GLogicalMaximum 0x05 [5] */
+ 0x75, 0x08, /* GReportSize 0x08 [8] */
+ 0x95, 0x01, /* GReportCount 0x01 [1] */
+ 0x81, 0x02, /* MInput 0x02 (Data[0] Var[1] Abs[2]) */
+
+ 0xC0, /* MEndCollection */
+
+ 0xC0 /* MEndCollection [Game Pad] */
+};
+
+static const unsigned int ps3remote_keymap_joypad_buttons[] = {
+ [0x01] = KEY_SELECT,
+ [0x02] = BTN_THUMBL, /* L3 */
+ [0x03] = BTN_THUMBR, /* R3 */
+ [0x04] = BTN_START,
+ [0x05] = KEY_UP,
+ [0x06] = KEY_RIGHT,
+ [0x07] = KEY_DOWN,
+ [0x08] = KEY_LEFT,
+ [0x09] = BTN_TL2, /* L2 */
+ [0x0a] = BTN_TR2, /* R2 */
+ [0x0b] = BTN_TL, /* L1 */
+ [0x0c] = BTN_TR, /* R1 */
+ [0x0d] = KEY_OPTION, /* options/triangle */
+ [0x0e] = KEY_BACK, /* back/circle */
+ [0x0f] = BTN_0, /* cross */
+ [0x10] = KEY_SCREEN, /* view/square */
+ [0x11] = KEY_HOMEPAGE, /* PS button */
+ [0x14] = KEY_ENTER,
+};
+static const unsigned int ps3remote_keymap_remote_buttons[] = {
+ [0x00] = KEY_1,
+ [0x01] = KEY_2,
+ [0x02] = KEY_3,
+ [0x03] = KEY_4,
+ [0x04] = KEY_5,
+ [0x05] = KEY_6,
+ [0x06] = KEY_7,
+ [0x07] = KEY_8,
+ [0x08] = KEY_9,
+ [0x09] = KEY_0,
+ [0x0e] = KEY_ESC, /* return */
+ [0x0f] = KEY_CLEAR,
+ [0x16] = KEY_EJECTCD,
+ [0x1a] = KEY_MENU, /* top menu */
+ [0x28] = KEY_TIME,
+ [0x30] = KEY_PREVIOUS,
+ [0x31] = KEY_NEXT,
+ [0x32] = KEY_PLAY,
+ [0x33] = KEY_REWIND, /* scan back */
+ [0x34] = KEY_FORWARD, /* scan forward */
+ [0x38] = KEY_STOP,
+ [0x39] = KEY_PAUSE,
+ [0x40] = KEY_CONTEXT_MENU, /* pop up/menu */
+ [0x60] = KEY_FRAMEBACK, /* slow/step back */
+ [0x61] = KEY_FRAMEFORWARD, /* slow/step forward */
+ [0x63] = KEY_SUBTITLE,
+ [0x64] = KEY_AUDIO,
+ [0x65] = KEY_ANGLE,
+ [0x70] = KEY_INFO, /* display */
+ [0x80] = KEY_BLUE,
+ [0x81] = KEY_RED,
+ [0x82] = KEY_GREEN,
+ [0x83] = KEY_YELLOW,
+};
+
+static __u8 *ps3remote_fixup(struct hid_device *hdev, __u8 *rdesc,
+ unsigned int *rsize)
+{
+ *rsize = sizeof(ps3remote_rdesc);
+ return ps3remote_rdesc;
+}
+
+static int ps3remote_mapping(struct hid_device *hdev, struct hid_input *hi,
+ struct hid_field *field, struct hid_usage *usage,
+ unsigned long **bit, int *max)
+{
+ unsigned int key = usage->hid & HID_USAGE;
+
+ if ((usage->hid & HID_USAGE_PAGE) != HID_UP_BUTTON)
+ return -1;
+
+ switch (usage->collection_index) {
+ case 1:
+ if (key >= ARRAY_SIZE(ps3remote_keymap_joypad_buttons))
+ return -1;
+
+ key = ps3remote_keymap_joypad_buttons[key];
+ if (!key)
+ return -1;
+ break;
+ case 2:
+ if (key >= ARRAY_SIZE(ps3remote_keymap_remote_buttons))
+ return -1;
+
+ key = ps3remote_keymap_remote_buttons[key];
+ if (!key)
+ return -1;
+ break;
+ default:
+ return -1;
+ }
+
+ hid_map_usage_clear(hi, usage, bit, max, EV_KEY, key);
+ return 1;
+}
+
+static const struct hid_device_id ps3remote_devices[] = {
+ /* PS3 BD Remote Control */
+ { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_SONY, USB_DEVICE_ID_SONY_PS3_BDREMOTE) },
+ /* Logitech Harmony Adapter for PS3 */
+ { HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_HARMONY_PS3) },
+ { }
+};
+MODULE_DEVICE_TABLE(hid, ps3remote_devices);
+
+static struct hid_driver ps3remote_driver = {
+ .name = "ps3_remote",
+ .id_table = ps3remote_devices,
+ .report_fixup = ps3remote_fixup,
+ .input_mapping = ps3remote_mapping,
+};
+
+static int __init ps3remote_init(void)
+{
+ return hid_register_driver(&ps3remote_driver);
+}
+
+static void __exit ps3remote_exit(void)
+{
+ hid_unregister_driver(&ps3remote_driver);
+}
+
+module_init(ps3remote_init);
+module_exit(ps3remote_exit);
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("David Dillow <dave@thedillows.org>, Antonio Ospite <ospite@studenti.unina.it>");
diff --git a/drivers/hid/hid-samsung.c b/drivers/hid/hid-samsung.c
index 3c1fd8a..a5821d3 100644
--- a/drivers/hid/hid-samsung.c
+++ b/drivers/hid/hid-samsung.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
* Copyright (c) 2010 Don Prince <dhprince.devel@yahoo.co.uk>
*
diff --git a/drivers/hid/hid-sony.c b/drivers/hid/hid-sony.c
index 5cd25bd..7f33ebf 100644
--- a/drivers/hid/hid-sony.c
+++ b/drivers/hid/hid-sony.c
@@ -4,7 +4,6 @@
* Copyright (c) 1999 Andreas Gal
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
* Copyright (c) 2006-2008 Jiri Kosina
*/
diff --git a/drivers/hid/hid-sunplus.c b/drivers/hid/hid-sunplus.c
index d484a00..45b4b06 100644
--- a/drivers/hid/hid-sunplus.c
+++ b/drivers/hid/hid-sunplus.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby
*/
diff --git a/drivers/hid/hid-uclogic.c b/drivers/hid/hid-uclogic.c
index 3aba02b..2e56a1f 100644
--- a/drivers/hid/hid-uclogic.c
+++ b/drivers/hid/hid-uclogic.c
@@ -466,6 +466,86 @@
0xC0 /* End Collection */
};
+/*
+ * See TWHA60 description, device and HID report descriptors at
+ * http://sf.net/apps/mediawiki/digimend/?title=UC-Logic_Tablet_TWHA60
+ */
+
+/* Size of the original descriptors of TWHA60 tablet */
+#define TWHA60_RDESC_ORIG_SIZE0 254
+#define TWHA60_RDESC_ORIG_SIZE1 139
+
+/* Fixed TWHA60 report descriptor, interface 0 (stylus) */
+static __u8 twha60_rdesc_fixed0[] = {
+ 0x05, 0x0D, /* Usage Page (Digitizer), */
+ 0x09, 0x02, /* Usage (Pen), */
+ 0xA1, 0x01, /* Collection (Application), */
+ 0x85, 0x09, /* Report ID (9), */
+ 0x09, 0x20, /* Usage (Stylus), */
+ 0xA0, /* Collection (Physical), */
+ 0x75, 0x01, /* Report Size (1), */
+ 0x09, 0x42, /* Usage (Tip Switch), */
+ 0x09, 0x44, /* Usage (Barrel Switch), */
+ 0x09, 0x46, /* Usage (Tablet Pick), */
+ 0x14, /* Logical Minimum (0), */
+ 0x25, 0x01, /* Logical Maximum (1), */
+ 0x95, 0x03, /* Report Count (3), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0x95, 0x04, /* Report Count (4), */
+ 0x81, 0x01, /* Input (Constant), */
+ 0x09, 0x32, /* Usage (In Range), */
+ 0x95, 0x01, /* Report Count (1), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0x75, 0x10, /* Report Size (16), */
+ 0x95, 0x01, /* Report Count (1), */
+ 0x14, /* Logical Minimum (0), */
+ 0xA4, /* Push, */
+ 0x05, 0x01, /* Usage Page (Desktop), */
+ 0x55, 0xFD, /* Unit Exponent (-3), */
+ 0x65, 0x13, /* Unit (Inch), */
+ 0x34, /* Physical Minimum (0), */
+ 0x09, 0x30, /* Usage (X), */
+ 0x46, 0x10, 0x27, /* Physical Maximum (10000), */
+ 0x27, 0x3F, 0x9C,
+ 0x00, 0x00, /* Logical Maximum (39999), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0x09, 0x31, /* Usage (Y), */
+ 0x46, 0x6A, 0x18, /* Physical Maximum (6250), */
+ 0x26, 0xA7, 0x61, /* Logical Maximum (24999), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0xB4, /* Pop, */
+ 0x09, 0x30, /* Usage (Tip Pressure), */
+ 0x26, 0xFF, 0x03, /* Logical Maximum (1023), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0xC0, /* End Collection, */
+ 0xC0 /* End Collection */
+};
+
+/* Fixed TWHA60 report descriptor, interface 1 (frame buttons) */
+static __u8 twha60_rdesc_fixed1[] = {
+ 0x05, 0x01, /* Usage Page (Desktop), */
+ 0x09, 0x06, /* Usage (Keyboard), */
+ 0xA1, 0x01, /* Collection (Application), */
+ 0x85, 0x05, /* Report ID (5), */
+ 0x05, 0x07, /* Usage Page (Keyboard), */
+ 0x14, /* Logical Minimum (0), */
+ 0x25, 0x01, /* Logical Maximum (1), */
+ 0x75, 0x01, /* Report Size (1), */
+ 0x95, 0x08, /* Report Count (8), */
+ 0x81, 0x01, /* Input (Constant), */
+ 0x95, 0x0C, /* Report Count (12), */
+ 0x19, 0x3A, /* Usage Minimum (KB F1), */
+ 0x29, 0x45, /* Usage Maximum (KB F12), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0x95, 0x0C, /* Report Count (12), */
+ 0x19, 0x68, /* Usage Minimum (KB F13), */
+ 0x29, 0x73, /* Usage Maximum (KB F24), */
+ 0x81, 0x02, /* Input (Variable), */
+ 0x95, 0x08, /* Report Count (8), */
+ 0x81, 0x01, /* Input (Constant), */
+ 0xC0 /* End Collection */
+};
+
static __u8 *uclogic_report_fixup(struct hid_device *hdev, __u8 *rdesc,
unsigned int *rsize)
{
@@ -525,6 +605,22 @@
break;
}
break;
+ case USB_DEVICE_ID_UCLOGIC_TABLET_TWHA60:
+ switch (iface_num) {
+ case 0:
+ if (*rsize == TWHA60_RDESC_ORIG_SIZE0) {
+ rdesc = twha60_rdesc_fixed0;
+ *rsize = sizeof(twha60_rdesc_fixed0);
+ }
+ break;
+ case 1:
+ if (*rsize == TWHA60_RDESC_ORIG_SIZE1) {
+ rdesc = twha60_rdesc_fixed1;
+ *rsize = sizeof(twha60_rdesc_fixed1);
+ }
+ break;
+ }
+ break;
}
return rdesc;
@@ -543,6 +639,8 @@
USB_DEVICE_ID_UCLOGIC_TABLET_WP1062) },
{ HID_USB_DEVICE(USB_VENDOR_ID_UCLOGIC,
USB_DEVICE_ID_UCLOGIC_WIRELESS_TABLET_TWHL850) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_UCLOGIC,
+ USB_DEVICE_ID_UCLOGIC_TABLET_TWHA60) },
{ }
};
MODULE_DEVICE_TABLE(hid, uclogic_devices);
diff --git a/drivers/hid/hid-wacom.c b/drivers/hid/hid-wacom.c
index fe23a1e..2f60da9 100644
--- a/drivers/hid/hid-wacom.c
+++ b/drivers/hid/hid-wacom.c
@@ -5,7 +5,6 @@
* Copyright (c) 2000-2005 Vojtech Pavlik <vojtech@suse.cz>
* Copyright (c) 2005 Michael Haboustak <mike-@cinci.rr.com> for Concept2, Inc
* Copyright (c) 2006-2007 Jiri Kosina
- * Copyright (c) 2007 Paul Walmsley
* Copyright (c) 2008 Jiri Slaby <jirislaby@gmail.com>
* Copyright (c) 2006 Andrew Zabolotny <zap@homelink.ru>
* Copyright (c) 2009 Bastien Nocera <hadess@hadess.net>
@@ -33,6 +32,8 @@
#define PAD_DEVICE_ID 0x0F
#define WAC_CMD_LED_CONTROL 0x20
+#define WAC_CMD_ICON_START_STOP 0x21
+#define WAC_CMD_ICON_TRANSFER 0x26
struct wacom_data {
__u16 tool;
@@ -69,6 +70,91 @@
POWER_SUPPLY_PROP_SCOPE,
};
+static void wacom_scramble(__u8 *image)
+{
+ __u16 mask;
+ __u16 s1;
+ __u16 s2;
+ __u16 r1 ;
+ __u16 r2 ;
+ __u16 r;
+ __u8 buf[256];
+ int i, w, x, y, z;
+
+ for (x = 0; x < 32; x++) {
+ for (y = 0; y < 8; y++)
+ buf[(8 * x) + (7 - y)] = image[(8 * x) + y];
+ }
+
+ /* Change 76543210 into GECA6420 as required by Intuos4 WL
+ * HGFEDCBA HFDB7531
+ */
+ for (x = 0; x < 4; x++) {
+ for (y = 0; y < 4; y++) {
+ for (z = 0; z < 8; z++) {
+ mask = 0x0001;
+ r1 = 0;
+ r2 = 0;
+ i = (x << 6) + (y << 4) + z;
+ s1 = buf[i];
+ s2 = buf[i+8];
+ for (w = 0; w < 8; w++) {
+ r1 |= (s1 & mask);
+ r2 |= (s2 & mask);
+ s1 <<= 1;
+ s2 <<= 1;
+ mask <<= 2;
+ }
+ r = r1 | (r2 << 1);
+ i = (x << 6) + (y << 4) + (z << 1);
+ image[i] = 0xFF & r;
+ image[i+1] = (0xFF00 & r) >> 8;
+ }
+ }
+ }
+}
+
+static void wacom_set_image(struct hid_device *hdev, const char *image,
+ __u8 icon_no)
+{
+ __u8 rep_data[68];
+ __u8 p[256];
+ int ret, i, j;
+
+ for (i = 0; i < 256; i++)
+ p[i] = image[i];
+
+ rep_data[0] = WAC_CMD_ICON_START_STOP;
+ rep_data[1] = 0;
+ ret = hdev->hid_output_raw_report(hdev, rep_data, 2,
+ HID_FEATURE_REPORT);
+ if (ret < 0)
+ goto err;
+
+ rep_data[0] = WAC_CMD_ICON_TRANSFER;
+ rep_data[1] = icon_no & 0x07;
+
+ wacom_scramble(p);
+
+ for (i = 0; i < 4; i++) {
+ for (j = 0; j < 64; j++)
+ rep_data[j + 3] = p[(i << 6) + j];
+
+ rep_data[2] = i;
+ ret = hdev->hid_output_raw_report(hdev, rep_data, 67,
+ HID_FEATURE_REPORT);
+ }
+
+ rep_data[0] = WAC_CMD_ICON_START_STOP;
+ rep_data[1] = 0;
+
+ ret = hdev->hid_output_raw_report(hdev, rep_data, 2,
+ HID_FEATURE_REPORT);
+
+err:
+ return;
+}
+
static void wacom_leds_set_brightness(struct led_classdev *led_dev,
enum led_brightness value)
{
@@ -91,7 +177,10 @@
if (buf) {
buf[0] = WAC_CMD_LED_CONTROL;
buf[1] = led;
- buf[2] = value;
+ buf[2] = value >> 2;
+ buf[3] = value;
+ /* use fixed brightness for OLEDs */
+ buf[4] = 0x08;
hdev->hid_output_raw_report(hdev, buf, 9, HID_FEATURE_REPORT);
kfree(buf);
}
@@ -317,6 +406,34 @@
static DEVICE_ATTR(speed, S_IRUGO | S_IWUSR | S_IWGRP,
wacom_show_speed, wacom_store_speed);
+#define WACOM_STORE(OLED_ID) \
+static ssize_t wacom_oled##OLED_ID##_store(struct device *dev, \
+ struct device_attribute *attr, \
+ const char *buf, size_t count) \
+{ \
+ struct hid_device *hdev = container_of(dev, struct hid_device, \
+ dev); \
+ \
+ if (count != 256) \
+ return -EINVAL; \
+ \
+ wacom_set_image(hdev, buf, OLED_ID); \
+ \
+ return count; \
+} \
+ \
+static DEVICE_ATTR(oled##OLED_ID##_img, S_IWUSR | S_IWGRP, NULL, \
+ wacom_oled##OLED_ID##_store)
+
+WACOM_STORE(0);
+WACOM_STORE(1);
+WACOM_STORE(2);
+WACOM_STORE(3);
+WACOM_STORE(4);
+WACOM_STORE(5);
+WACOM_STORE(6);
+WACOM_STORE(7);
+
static int wacom_gr_parse_report(struct hid_device *hdev,
struct wacom_data *wdata,
struct input_dev *input, unsigned char *data)
@@ -717,17 +834,33 @@
hid_warn(hdev,
"can't create sysfs speed attribute err: %d\n", ret);
+#define OLED_INIT(OLED_ID) \
+ do { \
+ ret = device_create_file(&hdev->dev, \
+ &dev_attr_oled##OLED_ID##_img); \
+ if (ret) \
+ hid_warn(hdev, \
+ "can't create sysfs oled attribute, err: %d\n", ret);\
+ } while (0)
+
+OLED_INIT(0);
+OLED_INIT(1);
+OLED_INIT(2);
+OLED_INIT(3);
+OLED_INIT(4);
+OLED_INIT(5);
+OLED_INIT(6);
+OLED_INIT(7);
+
wdata->features = 0;
wacom_set_features(hdev, 1);
if (hdev->product == USB_DEVICE_ID_WACOM_INTUOS4_BLUETOOTH) {
sprintf(hdev->name, "%s", "Wacom Intuos4 WL");
ret = wacom_initialize_leds(hdev);
- if (ret) {
+ if (ret)
hid_warn(hdev,
"can't create led attribute, err: %d\n", ret);
- goto destroy_leds;
- }
}
wdata->battery.properties = wacom_battery_props;
@@ -740,8 +873,8 @@
ret = power_supply_register(&hdev->dev, &wdata->battery);
if (ret) {
- hid_warn(hdev, "can't create sysfs battery attribute, err: %d\n",
- ret);
+ hid_err(hdev, "can't create sysfs battery attribute, err: %d\n",
+ ret);
goto err_battery;
}
@@ -756,8 +889,8 @@
ret = power_supply_register(&hdev->dev, &wdata->ac);
if (ret) {
- hid_warn(hdev,
- "can't create ac battery attribute, err: %d\n", ret);
+ hid_err(hdev,
+ "can't create ac battery attribute, err: %d\n", ret);
goto err_ac;
}
@@ -767,10 +900,17 @@
err_ac:
power_supply_unregister(&wdata->battery);
err_battery:
+ wacom_destroy_leds(hdev);
+ device_remove_file(&hdev->dev, &dev_attr_oled0_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled1_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled2_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled3_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled4_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled5_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled6_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled7_img);
device_remove_file(&hdev->dev, &dev_attr_speed);
hid_hw_stop(hdev);
-destroy_leds:
- wacom_destroy_leds(hdev);
err_free:
kfree(wdata);
return ret;
@@ -781,6 +921,14 @@
struct wacom_data *wdata = hid_get_drvdata(hdev);
wacom_destroy_leds(hdev);
+ device_remove_file(&hdev->dev, &dev_attr_oled0_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled1_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled2_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled3_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled4_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled5_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled6_img);
+ device_remove_file(&hdev->dev, &dev_attr_oled7_img);
device_remove_file(&hdev->dev, &dev_attr_speed);
hid_hw_stop(hdev);
diff --git a/drivers/hid/hid-waltop.c b/drivers/hid/hid-waltop.c
index 745e4e9..bb536ab 100644
--- a/drivers/hid/hid-waltop.c
+++ b/drivers/hid/hid-waltop.c
@@ -638,28 +638,6 @@
0xC0 /* End Collection */
};
-static int waltop_probe(struct hid_device *hdev,
- const struct hid_device_id *id)
-{
- int ret;
-
- ret = hid_parse(hdev);
- if (ret) {
- hid_err(hdev, "parse failed\n");
- goto err;
- }
-
- ret = hid_hw_start(hdev, HID_CONNECT_DEFAULT);
- if (ret) {
- hid_err(hdev, "hw start failed\n");
- goto err;
- }
-
- return 0;
-err:
- return ret;
-}
-
static __u8 *waltop_report_fixup(struct hid_device *hdev, __u8 *rdesc,
unsigned int *rsize)
{
@@ -776,11 +754,6 @@
return 0;
}
-static void waltop_remove(struct hid_device *hdev)
-{
- hid_hw_stop(hdev);
-}
-
static const struct hid_device_id waltop_devices[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_WALTOP,
USB_DEVICE_ID_WALTOP_SLIM_TABLET_5_8_INCH) },
@@ -803,10 +776,8 @@
static struct hid_driver waltop_driver = {
.name = "waltop",
.id_table = waltop_devices,
- .probe = waltop_probe,
.report_fixup = waltop_report_fixup,
.raw_event = waltop_raw_event,
- .remove = waltop_remove,
};
static int __init waltop_init(void)
diff --git a/drivers/hid/hid-wiimote-ext.c b/drivers/hid/hid-wiimote-ext.c
index 0a1805c..bc85bf2 100644
--- a/drivers/hid/hid-wiimote-ext.c
+++ b/drivers/hid/hid-wiimote-ext.c
@@ -28,12 +28,14 @@
bool mp_plugged;
bool motionp;
__u8 ext_type;
+ __u16 calib[4][3];
};
enum wiiext_type {
WIIEXT_NONE, /* placeholder */
WIIEXT_CLASSIC, /* Nintendo classic controller */
WIIEXT_NUNCHUCK, /* Nintendo nunchuck controller */
+ WIIEXT_BALANCE_BOARD, /* Nintendo balance board controller */
};
enum wiiext_keys {
@@ -126,6 +128,7 @@
static __u8 ext_read(struct wiimote_ext *ext)
{
ssize_t ret;
+ __u8 buf[24], i, j, offs = 0;
__u8 rmem[2], wmem;
__u8 type = WIIEXT_NONE;
@@ -151,6 +154,28 @@
type = WIIEXT_NUNCHUCK;
else if (rmem[0] == 0x01 && rmem[1] == 0x01)
type = WIIEXT_CLASSIC;
+ else if (rmem[0] == 0x04 && rmem[1] == 0x02)
+ type = WIIEXT_BALANCE_BOARD;
+ }
+
+ /* get balance board calibration data */
+ if (type == WIIEXT_BALANCE_BOARD) {
+ ret = wiimote_cmd_read(ext->wdata, 0xa40024, buf, 12);
+ ret += wiimote_cmd_read(ext->wdata, 0xa40024 + 12,
+ buf + 12, 12);
+
+ if (ret != 24) {
+ type = WIIEXT_NONE;
+ } else {
+ for (i = 0; i < 3; i++) {
+ for (j = 0; j < 4; j++) {
+ ext->calib[j][i] = buf[offs];
+ ext->calib[j][i] <<= 8;
+ ext->calib[j][i] |= buf[offs + 1];
+ offs += 2;
+ }
+ }
+ }
}
wiimote_cmd_release(ext->wdata);
@@ -509,6 +534,71 @@
input_sync(ext->input);
}
+static void handler_balance_board(struct wiimote_ext *ext, const __u8 *payload)
+{
+ __s32 val[4], tmp;
+ unsigned int i;
+
+ /* Byte | 8 7 6 5 4 3 2 1 |
+ * -----+--------------------------+
+ * 1 | Top Right <15:8> |
+ * 2 | Top Right <7:0> |
+ * -----+--------------------------+
+ * 3 | Bottom Right <15:8> |
+ * 4 | Bottom Right <7:0> |
+ * -----+--------------------------+
+ * 5 | Top Left <15:8> |
+ * 6 | Top Left <7:0> |
+ * -----+--------------------------+
+ * 7 | Bottom Left <15:8> |
+ * 8 | Bottom Left <7:0> |
+ * -----+--------------------------+
+ *
+ * These values represent the weight-measurements of the Wii-balance
+ * board with 16bit precision.
+ *
+ * The balance-board is never reported interleaved with motionp.
+ */
+
+ val[0] = payload[0];
+ val[0] <<= 8;
+ val[0] |= payload[1];
+
+ val[1] = payload[2];
+ val[1] <<= 8;
+ val[1] |= payload[3];
+
+ val[2] = payload[4];
+ val[2] <<= 8;
+ val[2] |= payload[5];
+
+ val[3] = payload[6];
+ val[3] <<= 8;
+ val[3] |= payload[7];
+
+ /* apply calibration data */
+ for (i = 0; i < 4; i++) {
+ if (val[i] < ext->calib[i][1]) {
+ tmp = val[i] - ext->calib[i][0];
+ tmp *= 1700;
+ tmp /= ext->calib[i][1] - ext->calib[i][0];
+ } else {
+ tmp = val[i] - ext->calib[i][1];
+ tmp *= 1700;
+ tmp /= ext->calib[i][2] - ext->calib[i][1];
+ tmp += 1700;
+ }
+ val[i] = tmp;
+ }
+
+ input_report_abs(ext->input, ABS_HAT0X, val[0]);
+ input_report_abs(ext->input, ABS_HAT0Y, val[1]);
+ input_report_abs(ext->input, ABS_HAT1X, val[2]);
+ input_report_abs(ext->input, ABS_HAT1Y, val[3]);
+
+ input_sync(ext->input);
+}
+
/* call this with state.lock spinlock held */
void wiiext_handle(struct wiimote_data *wdata, const __u8 *payload)
{
@@ -523,6 +613,8 @@
handler_nunchuck(ext, payload);
} else if (ext->ext_type == WIIEXT_CLASSIC) {
handler_classic(ext, payload);
+ } else if (ext->ext_type == WIIEXT_BALANCE_BOARD) {
+ handler_balance_board(ext, payload);
}
}
@@ -551,6 +643,11 @@
return sprintf(buf, "motionp+classic\n");
else
return sprintf(buf, "classic\n");
+ } else if (type == WIIEXT_BALANCE_BOARD) {
+ if (motionp)
+ return sprintf(buf, "motionp+balanceboard\n");
+ else
+ return sprintf(buf, "balanceboard\n");
} else {
if (motionp)
return sprintf(buf, "motionp\n");
diff --git a/drivers/hid/hidraw.c b/drivers/hid/hidraw.c
index 3b6f7bf..17d15bb 100644
--- a/drivers/hid/hidraw.c
+++ b/drivers/hid/hidraw.c
@@ -42,6 +42,7 @@
static struct class *hidraw_class;
static struct hidraw *hidraw_table[HIDRAW_MAX_DEVICES];
static DEFINE_MUTEX(minors_lock);
+static void drop_ref(struct hidraw *hid, int exists_bit);
static ssize_t hidraw_read(struct file *file, char __user *buffer, size_t count, loff_t *ppos)
{
@@ -113,7 +114,7 @@
__u8 *buf;
int ret = 0;
- if (!hidraw_table[minor]) {
+ if (!hidraw_table[minor] || !hidraw_table[minor]->exist) {
ret = -ENODEV;
goto out;
}
@@ -261,7 +262,7 @@
}
mutex_lock(&minors_lock);
- if (!hidraw_table[minor]) {
+ if (!hidraw_table[minor] || !hidraw_table[minor]->exist) {
err = -ENODEV;
goto out_unlock;
}
@@ -298,36 +299,12 @@
static int hidraw_release(struct inode * inode, struct file * file)
{
unsigned int minor = iminor(inode);
- struct hidraw *dev;
struct hidraw_list *list = file->private_data;
- int ret;
- int i;
- mutex_lock(&minors_lock);
- if (!hidraw_table[minor]) {
- ret = -ENODEV;
- goto unlock;
- }
-
+ drop_ref(hidraw_table[minor], 0);
list_del(&list->node);
- dev = hidraw_table[minor];
- if (!--dev->open) {
- if (list->hidraw->exist) {
- hid_hw_power(dev->hid, PM_HINT_NORMAL);
- hid_hw_close(dev->hid);
- } else {
- kfree(list->hidraw);
- }
- }
-
- for (i = 0; i < HIDRAW_BUFFER_SIZE; ++i)
- kfree(list->buffer[i].value);
kfree(list);
- ret = 0;
-unlock:
- mutex_unlock(&minors_lock);
-
- return ret;
+ return 0;
}
static long hidraw_ioctl(struct file *file, unsigned int cmd,
@@ -529,21 +506,7 @@
void hidraw_disconnect(struct hid_device *hid)
{
struct hidraw *hidraw = hid->hidraw;
-
- mutex_lock(&minors_lock);
- hidraw->exist = 0;
-
- device_destroy(hidraw_class, MKDEV(hidraw_major, hidraw->minor));
-
- hidraw_table[hidraw->minor] = NULL;
-
- if (hidraw->open) {
- hid_hw_close(hid);
- wake_up_interruptible(&hidraw->wait);
- } else {
- kfree(hidraw);
- }
- mutex_unlock(&minors_lock);
+ drop_ref(hidraw, 1);
}
EXPORT_SYMBOL_GPL(hidraw_disconnect);
@@ -559,21 +522,28 @@
if (result < 0) {
pr_warn("can't get major number\n");
- result = 0;
goto out;
}
hidraw_class = class_create(THIS_MODULE, "hidraw");
if (IS_ERR(hidraw_class)) {
result = PTR_ERR(hidraw_class);
- unregister_chrdev(hidraw_major, "hidraw");
- goto out;
+ goto error_cdev;
}
cdev_init(&hidraw_cdev, &hidraw_ops);
- cdev_add(&hidraw_cdev, dev_id, HIDRAW_MAX_DEVICES);
+ result = cdev_add(&hidraw_cdev, dev_id, HIDRAW_MAX_DEVICES);
+ if (result < 0)
+ goto error_class;
+
out:
return result;
+
+error_class:
+ class_destroy(hidraw_class);
+error_cdev:
+ unregister_chrdev_region(dev_id, HIDRAW_MAX_DEVICES);
+ goto out;
}
void hidraw_exit(void)
@@ -585,3 +555,23 @@
unregister_chrdev_region(dev_id, HIDRAW_MAX_DEVICES);
}
+
+static void drop_ref(struct hidraw *hidraw, int exists_bit)
+{
+ mutex_lock(&minors_lock);
+ if (exists_bit) {
+ hid_hw_close(hidraw->hid);
+ hidraw->exist = 0;
+ if (hidraw->open)
+ wake_up_interruptible(&hidraw->wait);
+ } else {
+ --hidraw->open;
+ }
+
+ if (!hidraw->open && !hidraw->exist) {
+ device_destroy(hidraw_class, MKDEV(hidraw_major, hidraw->minor));
+ hidraw_table[hidraw->minor] = NULL;
+ kfree(hidraw);
+ }
+ mutex_unlock(&minors_lock);
+}
diff --git a/drivers/hid/usbhid/hid-core.c b/drivers/hid/usbhid/hid-core.c
index dedd8e4..8e0c4bf94 100644
--- a/drivers/hid/usbhid/hid-core.c
+++ b/drivers/hid/usbhid/hid-core.c
@@ -1415,20 +1415,20 @@
* configuration descriptors passed, we already know that
* the size of the HID report descriptor has not changed.
*/
- rdesc = kmalloc(hid->rsize, GFP_KERNEL);
+ rdesc = kmalloc(hid->dev_rsize, GFP_KERNEL);
if (!rdesc) {
dbg_hid("couldn't allocate rdesc memory (post_reset)\n");
return 1;
}
status = hid_get_class_descriptor(dev,
interface->desc.bInterfaceNumber,
- HID_DT_REPORT, rdesc, hid->rsize);
+ HID_DT_REPORT, rdesc, hid->dev_rsize);
if (status < 0) {
dbg_hid("reading report descriptor failed (post_reset)\n");
kfree(rdesc);
return 1;
}
- status = memcmp(rdesc, hid->rdesc, hid->rsize);
+ status = memcmp(rdesc, hid->dev_rdesc, hid->dev_rsize);
kfree(rdesc);
if (status != 0) {
dbg_hid("report descriptor changed\n");
diff --git a/drivers/hid/usbhid/hid-quirks.c b/drivers/hid/usbhid/hid-quirks.c
index 903eef3..11c7932 100644
--- a/drivers/hid/usbhid/hid-quirks.c
+++ b/drivers/hid/usbhid/hid-quirks.c
@@ -70,11 +70,13 @@
{ USB_VENDOR_ID_CH, USB_DEVICE_ID_CH_AXIS_295, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_DMI, USB_DEVICE_ID_DMI_ENC, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_ELO, USB_DEVICE_ID_ELO_TS2700, HID_QUIRK_NOGET },
+ { USB_VENDOR_ID_FREESCALE, USB_DEVICE_ID_FREESCALE_MX28, HID_QUIRK_NOGET },
+ { USB_VENDOR_ID_MGE, USB_DEVICE_ID_MGE_UPS, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_PIXART, USB_DEVICE_ID_PIXART_OPTICAL_TOUCH_SCREEN, HID_QUIRK_NO_INIT_REPORTS },
{ USB_VENDOR_ID_PIXART, USB_DEVICE_ID_PIXART_OPTICAL_TOUCH_SCREEN1, HID_QUIRK_NO_INIT_REPORTS },
{ USB_VENDOR_ID_PIXART, USB_DEVICE_ID_PIXART_OPTICAL_TOUCH_SCREEN2, HID_QUIRK_NO_INIT_REPORTS },
{ USB_VENDOR_ID_PRODIGE, USB_DEVICE_ID_PRODIGE_CORDLESS, HID_QUIRK_NOGET },
- { USB_VENDOR_ID_QUANTA, USB_DEVICE_ID_PIXART_IMAGING_INC_OPTICAL_TOUCH_SCREEN, HID_QUIRK_NOGET },
+ { USB_VENDOR_ID_QUANTA, USB_DEVICE_ID_QUANTA_OPTICAL_TOUCH_3001, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_QUANTA, USB_DEVICE_ID_QUANTA_OPTICAL_TOUCH_3008, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_SENNHEISER, USB_DEVICE_ID_SENNHEISER_BTD500USB, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_SUN, USB_DEVICE_ID_RARITAN_KVM_DONGLE, HID_QUIRK_NOGET },
diff --git a/drivers/hwmon/ad7314.c b/drivers/hwmon/ad7314.c
index cfec802..f915eb1 100644
--- a/drivers/hwmon/ad7314.c
+++ b/drivers/hwmon/ad7314.c
@@ -87,10 +87,18 @@
}
}
+static ssize_t ad7314_show_name(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ return sprintf(buf, "%s\n", to_spi_device(dev)->modalias);
+}
+
+static DEVICE_ATTR(name, S_IRUGO, ad7314_show_name, NULL);
static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO,
ad7314_show_temperature, NULL, 0);
static struct attribute *ad7314_attributes[] = {
+ &dev_attr_name.attr,
&sensor_dev_attr_temp1_input.dev_attr.attr,
NULL,
};
diff --git a/drivers/hwmon/ads7871.c b/drivers/hwmon/ads7871.c
index e65c6e4..7bf4ce3 100644
--- a/drivers/hwmon/ads7871.c
+++ b/drivers/hwmon/ads7871.c
@@ -139,6 +139,12 @@
}
}
+static ssize_t ads7871_show_name(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ return sprintf(buf, "%s\n", to_spi_device(dev)->modalias);
+}
+
static SENSOR_DEVICE_ATTR(in0_input, S_IRUGO, show_voltage, NULL, 0);
static SENSOR_DEVICE_ATTR(in1_input, S_IRUGO, show_voltage, NULL, 1);
static SENSOR_DEVICE_ATTR(in2_input, S_IRUGO, show_voltage, NULL, 2);
@@ -148,6 +154,8 @@
static SENSOR_DEVICE_ATTR(in6_input, S_IRUGO, show_voltage, NULL, 6);
static SENSOR_DEVICE_ATTR(in7_input, S_IRUGO, show_voltage, NULL, 7);
+static DEVICE_ATTR(name, S_IRUGO, ads7871_show_name, NULL);
+
static struct attribute *ads7871_attributes[] = {
&sensor_dev_attr_in0_input.dev_attr.attr,
&sensor_dev_attr_in1_input.dev_attr.attr,
@@ -157,6 +165,7 @@
&sensor_dev_attr_in5_input.dev_attr.attr,
&sensor_dev_attr_in6_input.dev_attr.attr,
&sensor_dev_attr_in7_input.dev_attr.attr,
+ &dev_attr_name.attr,
NULL
};
diff --git a/drivers/hwmon/applesmc.c b/drivers/hwmon/applesmc.c
index 2827088..8f3f6f2 100644
--- a/drivers/hwmon/applesmc.c
+++ b/drivers/hwmon/applesmc.c
@@ -53,10 +53,10 @@
#define APPLESMC_MAX_DATA_LENGTH 32
-/* wait up to 32 ms for a status change. */
+/* wait up to 128 ms for a status change. */
#define APPLESMC_MIN_WAIT 0x0010
#define APPLESMC_RETRY_WAIT 0x0100
-#define APPLESMC_MAX_WAIT 0x8000
+#define APPLESMC_MAX_WAIT 0x20000
#define APPLESMC_READ_CMD 0x10
#define APPLESMC_WRITE_CMD 0x11
diff --git a/drivers/hwmon/asus_atk0110.c b/drivers/hwmon/asus_atk0110.c
index 351d1f4..4ee5789 100644
--- a/drivers/hwmon/asus_atk0110.c
+++ b/drivers/hwmon/asus_atk0110.c
@@ -34,6 +34,12 @@
.matches = {
DMI_MATCH(DMI_BOARD_NAME, "SABERTOOTH X58")
}
+ }, {
+ /* Old interface reads the same sensor for fan0 and fan1 */
+ .ident = "Asus M5A78L",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_NAME, "M5A78L")
+ }
},
{ }
};
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 0fa356f..984a3f1 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -815,17 +815,20 @@
if (err)
goto exit;
+ get_online_cpus();
for_each_online_cpu(i)
get_core_online(i);
#ifndef CONFIG_HOTPLUG_CPU
if (list_empty(&pdev_list)) {
+ put_online_cpus();
err = -ENODEV;
goto exit_driver_unreg;
}
#endif
register_hotcpu_notifier(&coretemp_cpu_notifier);
+ put_online_cpus();
return 0;
#ifndef CONFIG_HOTPLUG_CPU
@@ -840,6 +843,7 @@
{
struct pdev_entry *p, *n;
+ get_online_cpus();
unregister_hotcpu_notifier(&coretemp_cpu_notifier);
mutex_lock(&pdev_list_mutex);
list_for_each_entry_safe(p, n, &pdev_list, list) {
@@ -848,6 +852,7 @@
kfree(p);
}
mutex_unlock(&pdev_list_mutex);
+ put_online_cpus();
platform_driver_unregister(&coretemp_driver);
}
diff --git a/drivers/hwmon/fam15h_power.c b/drivers/hwmon/fam15h_power.c
index 2764b78..af69073 100644
--- a/drivers/hwmon/fam15h_power.c
+++ b/drivers/hwmon/fam15h_power.c
@@ -129,12 +129,12 @@
* counter saturations resulting in bogus power readings.
* We correct this value ourselves to cope with older BIOSes.
*/
-static DEFINE_PCI_DEVICE_TABLE(affected_device) = {
+static const struct pci_device_id affected_device[] = {
{ PCI_VDEVICE(AMD, PCI_DEVICE_ID_AMD_15H_NB_F4) },
{ 0 }
};
-static void __devinit tweak_runavg_range(struct pci_dev *pdev)
+static void tweak_runavg_range(struct pci_dev *pdev)
{
u32 val;
@@ -158,6 +158,16 @@
REG_TDP_RUNNING_AVERAGE, val);
}
+#ifdef CONFIG_PM
+static int fam15h_power_resume(struct pci_dev *pdev)
+{
+ tweak_runavg_range(pdev);
+ return 0;
+}
+#else
+#define fam15h_power_resume NULL
+#endif
+
static void __devinit fam15h_power_init_data(struct pci_dev *f4,
struct fam15h_power_data *data)
{
@@ -256,6 +266,7 @@
.id_table = fam15h_power_id_table,
.probe = fam15h_power_probe,
.remove = __devexit_p(fam15h_power_remove),
+ .resume = fam15h_power_resume,
};
module_pci_driver(fam15h_power_driver);
diff --git a/drivers/hwmon/ina2xx.c b/drivers/hwmon/ina2xx.c
index 7f3f4a3..6021482 100644
--- a/drivers/hwmon/ina2xx.c
+++ b/drivers/hwmon/ina2xx.c
@@ -69,22 +69,6 @@
u16 regs[INA2XX_MAX_REGISTERS];
};
-int ina2xx_read_word(struct i2c_client *client, int reg)
-{
- int val = i2c_smbus_read_word_data(client, reg);
- if (unlikely(val < 0)) {
- dev_dbg(&client->dev,
- "Failed to read register: %d\n", reg);
- return val;
- }
- return be16_to_cpu(val);
-}
-
-void ina2xx_write_word(struct i2c_client *client, int reg, int data)
-{
- i2c_smbus_write_word_data(client, reg, cpu_to_be16(data));
-}
-
static struct ina2xx_data *ina2xx_update_device(struct device *dev)
{
struct i2c_client *client = to_i2c_client(dev);
@@ -102,7 +86,7 @@
/* Read all registers */
for (i = 0; i < data->registers; i++) {
- int rv = ina2xx_read_word(client, i);
+ int rv = i2c_smbus_read_word_swapped(client, i);
if (rv < 0) {
ret = ERR_PTR(rv);
goto abort;
@@ -279,22 +263,26 @@
switch (data->kind) {
case ina219:
/* device configuration */
- ina2xx_write_word(client, INA2XX_CONFIG, INA219_CONFIG_DEFAULT);
+ i2c_smbus_write_word_swapped(client, INA2XX_CONFIG,
+ INA219_CONFIG_DEFAULT);
/* set current LSB to 1mA, shunt is in uOhms */
/* (equation 13 in datasheet) */
- ina2xx_write_word(client, INA2XX_CALIBRATION, 40960000 / shunt);
+ i2c_smbus_write_word_swapped(client, INA2XX_CALIBRATION,
+ 40960000 / shunt);
dev_info(&client->dev,
"power monitor INA219 (Rshunt = %li uOhm)\n", shunt);
data->registers = INA219_REGISTERS;
break;
case ina226:
/* device configuration */
- ina2xx_write_word(client, INA2XX_CONFIG, INA226_CONFIG_DEFAULT);
+ i2c_smbus_write_word_swapped(client, INA2XX_CONFIG,
+ INA226_CONFIG_DEFAULT);
/* set current LSB to 1mA, shunt is in uOhms */
/* (equation 1 in datasheet)*/
- ina2xx_write_word(client, INA2XX_CALIBRATION, 5120000 / shunt);
+ i2c_smbus_write_word_swapped(client, INA2XX_CALIBRATION,
+ 5120000 / shunt);
dev_info(&client->dev,
"power monitor INA226 (Rshunt = %li uOhm)\n", shunt);
data->registers = INA226_REGISTERS;
diff --git a/drivers/hwmon/twl4030-madc-hwmon.c b/drivers/hwmon/twl4030-madc-hwmon.c
index 0018c7d..1a174f0 100644
--- a/drivers/hwmon/twl4030-madc-hwmon.c
+++ b/drivers/hwmon/twl4030-madc-hwmon.c
@@ -44,12 +44,13 @@
struct device_attribute *devattr, char *buf)
{
struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr);
- struct twl4030_madc_request req;
+ struct twl4030_madc_request req = {
+ .channels = 1 << attr->index,
+ .method = TWL4030_MADC_SW2,
+ .type = TWL4030_MADC_WAIT,
+ };
long val;
- req.channels = (1 << attr->index);
- req.method = TWL4030_MADC_SW2;
- req.func_cb = NULL;
val = twl4030_madc_conversion(&req);
if (val < 0)
return val;
diff --git a/drivers/hwmon/via-cputemp.c b/drivers/hwmon/via-cputemp.c
index ee4ebc1..2e56c6c 100644
--- a/drivers/hwmon/via-cputemp.c
+++ b/drivers/hwmon/via-cputemp.c
@@ -328,6 +328,7 @@
if (err)
goto exit;
+ get_online_cpus();
for_each_online_cpu(i) {
struct cpuinfo_x86 *c = &cpu_data(i);
@@ -347,12 +348,14 @@
#ifndef CONFIG_HOTPLUG_CPU
if (list_empty(&pdev_list)) {
+ put_online_cpus();
err = -ENODEV;
goto exit_driver_unreg;
}
#endif
register_hotcpu_notifier(&via_cputemp_cpu_notifier);
+ put_online_cpus();
return 0;
#ifndef CONFIG_HOTPLUG_CPU
@@ -367,6 +370,7 @@
{
struct pdev_entry *p, *n;
+ get_online_cpus();
unregister_hotcpu_notifier(&via_cputemp_cpu_notifier);
mutex_lock(&pdev_list_mutex);
list_for_each_entry_safe(p, n, &pdev_list, list) {
@@ -375,6 +379,7 @@
kfree(p);
}
mutex_unlock(&pdev_list_mutex);
+ put_online_cpus();
platform_driver_unregister(&via_cputemp_driver);
}
diff --git a/drivers/hwspinlock/hwspinlock_core.c b/drivers/hwspinlock/hwspinlock_core.c
index 1201a15..db713c0 100644
--- a/drivers/hwspinlock/hwspinlock_core.c
+++ b/drivers/hwspinlock/hwspinlock_core.c
@@ -552,7 +552,7 @@
*/
int hwspin_lock_free(struct hwspinlock *hwlock)
{
- struct device *dev = hwlock->bank->dev;
+ struct device *dev;
struct hwspinlock *tmp;
int ret;
@@ -561,6 +561,7 @@
return -EINVAL;
}
+ dev = hwlock->bank->dev;
mutex_lock(&hwspinlock_tree_lock);
/* make sure the hwspinlock is used */
diff --git a/drivers/i2c/algos/i2c-algo-pca.c b/drivers/i2c/algos/i2c-algo-pca.c
index 73133b1..6f5f98d 100644
--- a/drivers/i2c/algos/i2c-algo-pca.c
+++ b/drivers/i2c/algos/i2c-algo-pca.c
@@ -476,17 +476,17 @@
/* To avoid integer overflow, use clock/100 for calculations */
clock = pca_clock(pca_data) / 100;
- if (pca_data->i2c_clock > 10000) {
+ if (pca_data->i2c_clock > 1000000) {
mode = I2C_PCA_MODE_TURBO;
min_tlow = 14;
min_thi = 5;
raise_fall_time = 22; /* Raise 11e-8s, Fall 11e-8s */
- } else if (pca_data->i2c_clock > 4000) {
+ } else if (pca_data->i2c_clock > 400000) {
mode = I2C_PCA_MODE_FASTP;
min_tlow = 17;
min_thi = 9;
raise_fall_time = 22; /* Raise 11e-8s, Fall 11e-8s */
- } else if (pca_data->i2c_clock > 1000) {
+ } else if (pca_data->i2c_clock > 100000) {
mode = I2C_PCA_MODE_FAST;
min_tlow = 44;
min_thi = 20;
diff --git a/drivers/i2c/busses/Kconfig b/drivers/i2c/busses/Kconfig
index b4aaa1b..970a161 100644
--- a/drivers/i2c/busses/Kconfig
+++ b/drivers/i2c/busses/Kconfig
@@ -104,6 +104,7 @@
DH89xxCC (PCH)
Panther Point (PCH)
Lynx Point (PCH)
+ Lynx Point-LP (PCH)
This driver can also be built as a module. If so, the module
will be called i2c-i801.
@@ -354,9 +355,13 @@
devices such as DaVinci NIC.
For details please see http://www.ti.com/davinci
+config I2C_DESIGNWARE_CORE
+ tristate
+
config I2C_DESIGNWARE_PLATFORM
tristate "Synopsys DesignWare Platform"
depends on HAVE_CLK
+ select I2C_DESIGNWARE_CORE
help
If you say yes to this option, support will be included for the
Synopsys DesignWare I2C adapter. Only master mode is supported.
@@ -367,6 +372,7 @@
config I2C_DESIGNWARE_PCI
tristate "Synopsys DesignWare PCI"
depends on PCI
+ select I2C_DESIGNWARE_CORE
help
If you say yes to this option, support will be included for the
Synopsys DesignWare I2C adapter. Only master mode is supported.
diff --git a/drivers/i2c/busses/Makefile b/drivers/i2c/busses/Makefile
index ce3c2be..37c4182 100644
--- a/drivers/i2c/busses/Makefile
+++ b/drivers/i2c/busses/Makefile
@@ -33,10 +33,11 @@
obj-$(CONFIG_I2C_BLACKFIN_TWI) += i2c-bfin-twi.o
obj-$(CONFIG_I2C_CPM) += i2c-cpm.o
obj-$(CONFIG_I2C_DAVINCI) += i2c-davinci.o
+obj-$(CONFIG_I2C_DESIGNWARE_CORE) += i2c-designware-core.o
obj-$(CONFIG_I2C_DESIGNWARE_PLATFORM) += i2c-designware-platform.o
-i2c-designware-platform-objs := i2c-designware-platdrv.o i2c-designware-core.o
+i2c-designware-platform-objs := i2c-designware-platdrv.o
obj-$(CONFIG_I2C_DESIGNWARE_PCI) += i2c-designware-pci.o
-i2c-designware-pci-objs := i2c-designware-pcidrv.o i2c-designware-core.o
+i2c-designware-pci-objs := i2c-designware-pcidrv.o
obj-$(CONFIG_I2C_EG20T) += i2c-eg20t.o
obj-$(CONFIG_I2C_GPIO) += i2c-gpio.o
obj-$(CONFIG_I2C_HIGHLANDER) += i2c-highlander.o
diff --git a/drivers/i2c/busses/i2c-designware-core.c b/drivers/i2c/busses/i2c-designware-core.c
index 1e48bec..7b8ebbe 100644
--- a/drivers/i2c/busses/i2c-designware-core.c
+++ b/drivers/i2c/busses/i2c-designware-core.c
@@ -25,6 +25,7 @@
* ----------------------------------------------------------------------------
*
*/
+#include <linux/export.h>
#include <linux/clk.h>
#include <linux/errno.h>
#include <linux/err.h>
@@ -316,6 +317,7 @@
dw_writel(dev, dev->master_cfg , DW_IC_CON);
return 0;
}
+EXPORT_SYMBOL_GPL(i2c_dw_init);
/*
* Waiting for bus not busy
@@ -568,12 +570,14 @@
return ret;
}
+EXPORT_SYMBOL_GPL(i2c_dw_xfer);
u32 i2c_dw_func(struct i2c_adapter *adap)
{
struct dw_i2c_dev *dev = i2c_get_adapdata(adap);
return dev->functionality;
}
+EXPORT_SYMBOL_GPL(i2c_dw_func);
static u32 i2c_dw_read_clear_intrbits(struct dw_i2c_dev *dev)
{
@@ -678,17 +682,20 @@
return IRQ_HANDLED;
}
+EXPORT_SYMBOL_GPL(i2c_dw_isr);
void i2c_dw_enable(struct dw_i2c_dev *dev)
{
/* Enable the adapter */
dw_writel(dev, 1, DW_IC_ENABLE);
}
+EXPORT_SYMBOL_GPL(i2c_dw_enable);
u32 i2c_dw_is_enabled(struct dw_i2c_dev *dev)
{
return dw_readl(dev, DW_IC_ENABLE);
}
+EXPORT_SYMBOL_GPL(i2c_dw_is_enabled);
void i2c_dw_disable(struct dw_i2c_dev *dev)
{
@@ -699,18 +706,22 @@
dw_writel(dev, 0, DW_IC_INTR_MASK);
dw_readl(dev, DW_IC_CLR_INTR);
}
+EXPORT_SYMBOL_GPL(i2c_dw_disable);
void i2c_dw_clear_int(struct dw_i2c_dev *dev)
{
dw_readl(dev, DW_IC_CLR_INTR);
}
+EXPORT_SYMBOL_GPL(i2c_dw_clear_int);
void i2c_dw_disable_int(struct dw_i2c_dev *dev)
{
dw_writel(dev, 0, DW_IC_INTR_MASK);
}
+EXPORT_SYMBOL_GPL(i2c_dw_disable_int);
u32 i2c_dw_read_comp_param(struct dw_i2c_dev *dev)
{
return dw_readl(dev, DW_IC_COMP_PARAM_1);
}
+EXPORT_SYMBOL_GPL(i2c_dw_read_comp_param);
diff --git a/drivers/i2c/busses/i2c-i801.c b/drivers/i2c/busses/i2c-i801.c
index 898dcf9..33e9b0c 100644
--- a/drivers/i2c/busses/i2c-i801.c
+++ b/drivers/i2c/busses/i2c-i801.c
@@ -52,6 +52,7 @@
DH89xxCC (PCH) 0x2330 32 hard yes yes yes
Panther Point (PCH) 0x1e22 32 hard yes yes yes
Lynx Point (PCH) 0x8c22 32 hard yes yes yes
+ Lynx Point-LP (PCH) 0x9c22 32 hard yes yes yes
Features supported by this driver:
Software PEC no
@@ -155,6 +156,7 @@
#define PCI_DEVICE_ID_INTEL_DH89XXCC_SMBUS 0x2330
#define PCI_DEVICE_ID_INTEL_5_3400_SERIES_SMBUS 0x3b30
#define PCI_DEVICE_ID_INTEL_LYNXPOINT_SMBUS 0x8c22
+#define PCI_DEVICE_ID_INTEL_LYNXPOINT_LP_SMBUS 0x9c22
struct i801_priv {
struct i2c_adapter adapter;
@@ -771,6 +773,7 @@
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_DH89XXCC_SMBUS) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_PANTHERPOINT_SMBUS) },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_LYNXPOINT_SMBUS) },
+ { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_LYNXPOINT_LP_SMBUS) },
{ 0, }
};
diff --git a/drivers/i2c/busses/i2c-mxs.c b/drivers/i2c/busses/i2c-mxs.c
index 088c5c1..51f05b8 100644
--- a/drivers/i2c/busses/i2c-mxs.c
+++ b/drivers/i2c/busses/i2c-mxs.c
@@ -365,10 +365,6 @@
struct device_node *node = dev->of_node;
int ret;
- if (!node)
- return -EINVAL;
-
- i2c->speed = &mxs_i2c_95kHz_config;
ret = of_property_read_u32(node, "clock-frequency", &speed);
if (ret)
dev_warn(dev, "No I2C speed selected, using 100kHz\n");
@@ -419,10 +415,13 @@
return err;
i2c->dev = dev;
+ i2c->speed = &mxs_i2c_95kHz_config;
- err = mxs_i2c_get_ofdata(i2c);
- if (err)
- return err;
+ if (dev->of_node) {
+ err = mxs_i2c_get_ofdata(i2c);
+ if (err)
+ return err;
+ }
platform_set_drvdata(pdev, i2c);
diff --git a/drivers/i2c/busses/i2c-pnx.c b/drivers/i2c/busses/i2c-pnx.c
index 5d54416..8488bdd 100644
--- a/drivers/i2c/busses/i2c-pnx.c
+++ b/drivers/i2c/busses/i2c-pnx.c
@@ -48,8 +48,9 @@
mcntrl_afie = 0x00000002,
mcntrl_naie = 0x00000004,
mcntrl_drmie = 0x00000008,
- mcntrl_daie = 0x00000020,
- mcntrl_rffie = 0x00000040,
+ mcntrl_drsie = 0x00000010,
+ mcntrl_rffie = 0x00000020,
+ mcntrl_daie = 0x00000040,
mcntrl_tffie = 0x00000080,
mcntrl_reset = 0x00000100,
mcntrl_cdbmode = 0x00000400,
@@ -290,31 +291,37 @@
* or we didn't 'ask' for it yet.
*/
if (ioread32(I2C_REG_STS(alg_data)) & mstatus_rfe) {
- dev_dbg(&alg_data->adapter.dev,
- "%s(): Write dummy data to fill Rx-fifo...\n",
- __func__);
+ /* 'Asking' is done asynchronously, e.g. dummy TX of several
+ * bytes is done before the first actual RX arrives in FIFO.
+ * Therefore, ordered bytes (via TX) are counted separately.
+ */
+ if (alg_data->mif.order) {
+ dev_dbg(&alg_data->adapter.dev,
+ "%s(): Write dummy data to fill Rx-fifo...\n",
+ __func__);
- if (alg_data->mif.len == 1) {
- /* Last byte, do not acknowledge next rcv. */
- val |= stop_bit;
+ if (alg_data->mif.order == 1) {
+ /* Last byte, do not acknowledge next rcv. */
+ val |= stop_bit;
+
+ /*
+ * Enable interrupt RFDAIE (data in Rx fifo),
+ * and disable DRMIE (need data for Tx)
+ */
+ ctl = ioread32(I2C_REG_CTL(alg_data));
+ ctl |= mcntrl_rffie | mcntrl_daie;
+ ctl &= ~mcntrl_drmie;
+ iowrite32(ctl, I2C_REG_CTL(alg_data));
+ }
/*
- * Enable interrupt RFDAIE (data in Rx fifo),
- * and disable DRMIE (need data for Tx)
+ * Now we'll 'ask' for data:
+ * For each byte we want to receive, we must
+ * write a (dummy) byte to the Tx-FIFO.
*/
- ctl = ioread32(I2C_REG_CTL(alg_data));
- ctl |= mcntrl_rffie | mcntrl_daie;
- ctl &= ~mcntrl_drmie;
- iowrite32(ctl, I2C_REG_CTL(alg_data));
+ iowrite32(val, I2C_REG_TX(alg_data));
+ alg_data->mif.order--;
}
-
- /*
- * Now we'll 'ask' for data:
- * For each byte we want to receive, we must
- * write a (dummy) byte to the Tx-FIFO.
- */
- iowrite32(val, I2C_REG_TX(alg_data));
-
return 0;
}
@@ -514,6 +521,7 @@
alg_data->mif.buf = pmsg->buf;
alg_data->mif.len = pmsg->len;
+ alg_data->mif.order = pmsg->len;
alg_data->mif.mode = (pmsg->flags & I2C_M_RD) ?
I2C_SMBUS_READ : I2C_SMBUS_WRITE;
alg_data->mif.ret = 0;
@@ -566,6 +574,7 @@
/* Cleanup to be sure... */
alg_data->mif.buf = NULL;
alg_data->mif.len = 0;
+ alg_data->mif.order = 0;
dev_dbg(&alg_data->adapter.dev, "%s(): exiting, stat = %x\n",
__func__, ioread32(I2C_REG_STS(alg_data)));
diff --git a/drivers/i2c/i2c-core.c b/drivers/i2c/i2c-core.c
index 2efa56c..2091ae8f 100644
--- a/drivers/i2c/i2c-core.c
+++ b/drivers/i2c/i2c-core.c
@@ -637,6 +637,22 @@
}
/*
+ * This function is only needed for mutex_lock_nested, so it is never
+ * called unless locking correctness checking is enabled. Thus we
+ * make it inline to avoid a compiler warning. That's what gcc ends up
+ * doing anyway.
+ */
+static inline unsigned int i2c_adapter_depth(struct i2c_adapter *adapter)
+{
+ unsigned int depth = 0;
+
+ while ((adapter = i2c_parent_is_i2c_adapter(adapter)))
+ depth++;
+
+ return depth;
+}
+
+/*
* Let users instantiate I2C devices through sysfs. This can be used when
* platform initialization code doesn't contain the proper data for
* whatever reason. Also useful for drivers that do device detection and
@@ -726,7 +742,8 @@
/* Make sure the device was added through sysfs */
res = -ENOENT;
- mutex_lock(&adap->userspace_clients_lock);
+ mutex_lock_nested(&adap->userspace_clients_lock,
+ i2c_adapter_depth(adap));
list_for_each_entry_safe(client, next, &adap->userspace_clients,
detected) {
if (client->addr == addr) {
@@ -1073,7 +1090,8 @@
return res;
/* Remove devices instantiated from sysfs */
- mutex_lock(&adap->userspace_clients_lock);
+ mutex_lock_nested(&adap->userspace_clients_lock,
+ i2c_adapter_depth(adap));
list_for_each_entry_safe(client, next, &adap->userspace_clients,
detected) {
dev_dbg(&adap->dev, "Removing %s at 0x%x\n", client->name,
diff --git a/drivers/ide/ide-pm.c b/drivers/ide/ide-pm.c
index 9240609..8d1e32d 100644
--- a/drivers/ide/ide-pm.c
+++ b/drivers/ide/ide-pm.c
@@ -4,7 +4,7 @@
int generic_ide_suspend(struct device *dev, pm_message_t mesg)
{
- ide_drive_t *drive = dev_get_drvdata(dev);
+ ide_drive_t *drive = to_ide_device(dev);
ide_drive_t *pair = ide_get_pair_dev(drive);
ide_hwif_t *hwif = drive->hwif;
struct request *rq;
@@ -40,7 +40,7 @@
int generic_ide_resume(struct device *dev)
{
- ide_drive_t *drive = dev_get_drvdata(dev);
+ ide_drive_t *drive = to_ide_device(dev);
ide_drive_t *pair = ide_get_pair_dev(drive);
ide_hwif_t *hwif = drive->hwif;
struct request *rq;
diff --git a/drivers/iio/adc/at91_adc.c b/drivers/iio/adc/at91_adc.c
index f61780a..3bd5540 100644
--- a/drivers/iio/adc/at91_adc.c
+++ b/drivers/iio/adc/at91_adc.c
@@ -617,7 +617,7 @@
st->adc_clk = clk_get(&pdev->dev, "adc_op_clk");
if (IS_ERR(st->adc_clk)) {
dev_err(&pdev->dev, "Failed to get the ADC clock.\n");
- ret = PTR_ERR(st->clk);
+ ret = PTR_ERR(st->adc_clk);
goto error_disable_clk;
}
diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c
index 51f4206..6cfd4d8 100644
--- a/drivers/infiniband/hw/cxgb4/cm.c
+++ b/drivers/infiniband/hw/cxgb4/cm.c
@@ -1361,11 +1361,11 @@
struct tid_info *t = dev->rdev.lldi.tids;
ep = lookup_tid(t, tid);
- PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
if (!ep) {
printk(KERN_WARNING MOD "Abort rpl to freed endpoint\n");
return 0;
}
+ PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
mutex_lock(&ep->com.mutex);
switch (ep->com.state) {
case ABORTING:
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c
index 5358900..8615d7c 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.c
+++ b/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -42,6 +42,7 @@
*/
#include <linux/slab.h>
+#include <linux/smpboot.h>
#include "ehca_classes.h"
#include "ehca_irq.h"
@@ -652,7 +653,7 @@
ehca_process_eq((struct ehca_shca*)data, 1);
}
-static inline int find_next_online_cpu(struct ehca_comp_pool *pool)
+static int find_next_online_cpu(struct ehca_comp_pool *pool)
{
int cpu;
unsigned long flags;
@@ -662,17 +663,20 @@
ehca_dmp(cpu_online_mask, cpumask_size(), "");
spin_lock_irqsave(&pool->last_cpu_lock, flags);
- cpu = cpumask_next(pool->last_cpu, cpu_online_mask);
- if (cpu >= nr_cpu_ids)
- cpu = cpumask_first(cpu_online_mask);
- pool->last_cpu = cpu;
+ do {
+ cpu = cpumask_next(pool->last_cpu, cpu_online_mask);
+ if (cpu >= nr_cpu_ids)
+ cpu = cpumask_first(cpu_online_mask);
+ pool->last_cpu = cpu;
+ } while (!per_cpu_ptr(pool->cpu_comp_tasks, cpu)->active);
spin_unlock_irqrestore(&pool->last_cpu_lock, flags);
return cpu;
}
static void __queue_comp_task(struct ehca_cq *__cq,
- struct ehca_cpu_comp_task *cct)
+ struct ehca_cpu_comp_task *cct,
+ struct task_struct *thread)
{
unsigned long flags;
@@ -683,7 +687,7 @@
__cq->nr_callbacks++;
list_add_tail(&__cq->entry, &cct->cq_list);
cct->cq_jobs++;
- wake_up(&cct->wait_queue);
+ wake_up_process(thread);
} else
__cq->nr_callbacks++;
@@ -695,6 +699,7 @@
{
int cpu_id;
struct ehca_cpu_comp_task *cct;
+ struct task_struct *thread;
int cq_jobs;
unsigned long flags;
@@ -702,7 +707,8 @@
BUG_ON(!cpu_online(cpu_id));
cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu_id);
- BUG_ON(!cct);
+ thread = *per_cpu_ptr(pool->cpu_comp_threads, cpu_id);
+ BUG_ON(!cct || !thread);
spin_lock_irqsave(&cct->task_lock, flags);
cq_jobs = cct->cq_jobs;
@@ -710,28 +716,25 @@
if (cq_jobs > 0) {
cpu_id = find_next_online_cpu(pool);
cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu_id);
- BUG_ON(!cct);
+ thread = *per_cpu_ptr(pool->cpu_comp_threads, cpu_id);
+ BUG_ON(!cct || !thread);
}
-
- __queue_comp_task(__cq, cct);
+ __queue_comp_task(__cq, cct, thread);
}
static void run_comp_task(struct ehca_cpu_comp_task *cct)
{
struct ehca_cq *cq;
- unsigned long flags;
-
- spin_lock_irqsave(&cct->task_lock, flags);
while (!list_empty(&cct->cq_list)) {
cq = list_entry(cct->cq_list.next, struct ehca_cq, entry);
- spin_unlock_irqrestore(&cct->task_lock, flags);
+ spin_unlock_irq(&cct->task_lock);
comp_event_callback(cq);
if (atomic_dec_and_test(&cq->nr_events))
wake_up(&cq->wait_completion);
- spin_lock_irqsave(&cct->task_lock, flags);
+ spin_lock_irq(&cct->task_lock);
spin_lock(&cq->task_lock);
cq->nr_callbacks--;
if (!cq->nr_callbacks) {
@@ -740,159 +743,76 @@
}
spin_unlock(&cq->task_lock);
}
-
- spin_unlock_irqrestore(&cct->task_lock, flags);
}
-static int comp_task(void *__cct)
-{
- struct ehca_cpu_comp_task *cct = __cct;
- int cql_empty;
- DECLARE_WAITQUEUE(wait, current);
-
- set_current_state(TASK_INTERRUPTIBLE);
- while (!kthread_should_stop()) {
- add_wait_queue(&cct->wait_queue, &wait);
-
- spin_lock_irq(&cct->task_lock);
- cql_empty = list_empty(&cct->cq_list);
- spin_unlock_irq(&cct->task_lock);
- if (cql_empty)
- schedule();
- else
- __set_current_state(TASK_RUNNING);
-
- remove_wait_queue(&cct->wait_queue, &wait);
-
- spin_lock_irq(&cct->task_lock);
- cql_empty = list_empty(&cct->cq_list);
- spin_unlock_irq(&cct->task_lock);
- if (!cql_empty)
- run_comp_task(__cct);
-
- set_current_state(TASK_INTERRUPTIBLE);
- }
- __set_current_state(TASK_RUNNING);
-
- return 0;
-}
-
-static struct task_struct *create_comp_task(struct ehca_comp_pool *pool,
- int cpu)
-{
- struct ehca_cpu_comp_task *cct;
-
- cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
- spin_lock_init(&cct->task_lock);
- INIT_LIST_HEAD(&cct->cq_list);
- init_waitqueue_head(&cct->wait_queue);
- cct->task = kthread_create_on_node(comp_task, cct, cpu_to_node(cpu),
- "ehca_comp/%d", cpu);
-
- return cct->task;
-}
-
-static void destroy_comp_task(struct ehca_comp_pool *pool,
- int cpu)
-{
- struct ehca_cpu_comp_task *cct;
- struct task_struct *task;
- unsigned long flags_cct;
-
- cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
-
- spin_lock_irqsave(&cct->task_lock, flags_cct);
-
- task = cct->task;
- cct->task = NULL;
- cct->cq_jobs = 0;
-
- spin_unlock_irqrestore(&cct->task_lock, flags_cct);
-
- if (task)
- kthread_stop(task);
-}
-
-static void __cpuinit take_over_work(struct ehca_comp_pool *pool, int cpu)
+static void comp_task_park(unsigned int cpu)
{
struct ehca_cpu_comp_task *cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
+ struct ehca_cpu_comp_task *target;
+ struct task_struct *thread;
+ struct ehca_cq *cq, *tmp;
LIST_HEAD(list);
- struct ehca_cq *cq;
- unsigned long flags_cct;
- spin_lock_irqsave(&cct->task_lock, flags_cct);
-
+ spin_lock_irq(&cct->task_lock);
+ cct->cq_jobs = 0;
+ cct->active = 0;
list_splice_init(&cct->cq_list, &list);
+ spin_unlock_irq(&cct->task_lock);
- while (!list_empty(&list)) {
- cq = list_entry(cct->cq_list.next, struct ehca_cq, entry);
-
+ cpu = find_next_online_cpu(pool);
+ target = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
+ thread = *per_cpu_ptr(pool->cpu_comp_threads, cpu);
+ spin_lock_irq(&target->task_lock);
+ list_for_each_entry_safe(cq, tmp, &list, entry) {
list_del(&cq->entry);
- __queue_comp_task(cq, this_cpu_ptr(pool->cpu_comp_tasks));
+ __queue_comp_task(cq, target, thread);
}
-
- spin_unlock_irqrestore(&cct->task_lock, flags_cct);
-
+ spin_unlock_irq(&target->task_lock);
}
-static int __cpuinit comp_pool_callback(struct notifier_block *nfb,
- unsigned long action,
- void *hcpu)
+static void comp_task_stop(unsigned int cpu, bool online)
{
- unsigned int cpu = (unsigned long)hcpu;
- struct ehca_cpu_comp_task *cct;
+ struct ehca_cpu_comp_task *cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
- switch (action) {
- case CPU_UP_PREPARE:
- case CPU_UP_PREPARE_FROZEN:
- ehca_gen_dbg("CPU: %x (CPU_PREPARE)", cpu);
- if (!create_comp_task(pool, cpu)) {
- ehca_gen_err("Can't create comp_task for cpu: %x", cpu);
- return notifier_from_errno(-ENOMEM);
- }
- break;
- case CPU_UP_CANCELED:
- case CPU_UP_CANCELED_FROZEN:
- ehca_gen_dbg("CPU: %x (CPU_CANCELED)", cpu);
- cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
- kthread_bind(cct->task, cpumask_any(cpu_online_mask));
- destroy_comp_task(pool, cpu);
- break;
- case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
- ehca_gen_dbg("CPU: %x (CPU_ONLINE)", cpu);
- cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
- kthread_bind(cct->task, cpu);
- wake_up_process(cct->task);
- break;
- case CPU_DOWN_PREPARE:
- case CPU_DOWN_PREPARE_FROZEN:
- ehca_gen_dbg("CPU: %x (CPU_DOWN_PREPARE)", cpu);
- break;
- case CPU_DOWN_FAILED:
- case CPU_DOWN_FAILED_FROZEN:
- ehca_gen_dbg("CPU: %x (CPU_DOWN_FAILED)", cpu);
- break;
- case CPU_DEAD:
- case CPU_DEAD_FROZEN:
- ehca_gen_dbg("CPU: %x (CPU_DEAD)", cpu);
- destroy_comp_task(pool, cpu);
- take_over_work(pool, cpu);
- break;
- }
-
- return NOTIFY_OK;
+ spin_lock_irq(&cct->task_lock);
+ cct->cq_jobs = 0;
+ cct->active = 0;
+ WARN_ON(!list_empty(&cct->cq_list));
+ spin_unlock_irq(&cct->task_lock);
}
-static struct notifier_block comp_pool_callback_nb __cpuinitdata = {
- .notifier_call = comp_pool_callback,
- .priority = 0,
+static int comp_task_should_run(unsigned int cpu)
+{
+ struct ehca_cpu_comp_task *cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
+
+ return cct->cq_jobs;
+}
+
+static void comp_task(unsigned int cpu)
+{
+ struct ehca_cpu_comp_task *cct = this_cpu_ptr(pool->cpu_comp_tasks);
+ int cql_empty;
+
+ spin_lock_irq(&cct->task_lock);
+ cql_empty = list_empty(&cct->cq_list);
+ if (!cql_empty) {
+ __set_current_state(TASK_RUNNING);
+ run_comp_task(cct);
+ }
+ spin_unlock_irq(&cct->task_lock);
+}
+
+static struct smp_hotplug_thread comp_pool_threads = {
+ .thread_should_run = comp_task_should_run,
+ .thread_fn = comp_task,
+ .thread_comm = "ehca_comp/%u",
+ .cleanup = comp_task_stop,
+ .park = comp_task_park,
};
int ehca_create_comp_pool(void)
{
- int cpu;
- struct task_struct *task;
+ int cpu, ret = -ENOMEM;
if (!ehca_scaling_code)
return 0;
@@ -905,38 +825,46 @@
pool->last_cpu = cpumask_any(cpu_online_mask);
pool->cpu_comp_tasks = alloc_percpu(struct ehca_cpu_comp_task);
- if (pool->cpu_comp_tasks == NULL) {
- kfree(pool);
- return -EINVAL;
+ if (!pool->cpu_comp_tasks)
+ goto out_pool;
+
+ pool->cpu_comp_threads = alloc_percpu(struct task_struct *);
+ if (!pool->cpu_comp_threads)
+ goto out_tasks;
+
+ for_each_present_cpu(cpu) {
+ struct ehca_cpu_comp_task *cct;
+
+ cct = per_cpu_ptr(pool->cpu_comp_tasks, cpu);
+ spin_lock_init(&cct->task_lock);
+ INIT_LIST_HEAD(&cct->cq_list);
}
- for_each_online_cpu(cpu) {
- task = create_comp_task(pool, cpu);
- if (task) {
- kthread_bind(task, cpu);
- wake_up_process(task);
- }
- }
+ comp_pool_threads.store = pool->cpu_comp_threads;
+ ret = smpboot_register_percpu_thread(&comp_pool_threads);
+ if (ret)
+ goto out_threads;
- register_hotcpu_notifier(&comp_pool_callback_nb);
+ pr_info("eHCA scaling code enabled\n");
+ return ret;
- printk(KERN_INFO "eHCA scaling code enabled\n");
-
- return 0;
+out_threads:
+ free_percpu(pool->cpu_comp_threads);
+out_tasks:
+ free_percpu(pool->cpu_comp_tasks);
+out_pool:
+ kfree(pool);
+ return ret;
}
void ehca_destroy_comp_pool(void)
{
- int i;
-
if (!ehca_scaling_code)
return;
- unregister_hotcpu_notifier(&comp_pool_callback_nb);
+ smpboot_unregister_percpu_thread(&comp_pool_threads);
- for_each_online_cpu(i)
- destroy_comp_task(pool, i);
-
+ free_percpu(pool->cpu_comp_threads);
free_percpu(pool->cpu_comp_tasks);
kfree(pool);
}
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.h b/drivers/infiniband/hw/ehca/ehca_irq.h
index 3346cb0..5370199 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.h
+++ b/drivers/infiniband/hw/ehca/ehca_irq.h
@@ -58,15 +58,15 @@
void ehca_process_eq(struct ehca_shca *shca, int is_irq);
struct ehca_cpu_comp_task {
- wait_queue_head_t wait_queue;
struct list_head cq_list;
- struct task_struct *task;
spinlock_t task_lock;
int cq_jobs;
+ int active;
};
struct ehca_comp_pool {
- struct ehca_cpu_comp_task *cpu_comp_tasks;
+ struct ehca_cpu_comp_task __percpu *cpu_comp_tasks;
+ struct task_struct * __percpu *cpu_comp_threads;
int last_cpu;
spinlock_t last_cpu_lock;
};
diff --git a/drivers/infiniband/hw/mthca/mthca_reset.c b/drivers/infiniband/hw/mthca/mthca_reset.c
index 4fa3534..74c6a94 100644
--- a/drivers/infiniband/hw/mthca/mthca_reset.c
+++ b/drivers/infiniband/hw/mthca/mthca_reset.c
@@ -241,16 +241,16 @@
if (hca_pcie_cap) {
devctl = hca_header[(hca_pcie_cap + PCI_EXP_DEVCTL) / 4];
- if (pci_write_config_word(mdev->pdev, hca_pcie_cap + PCI_EXP_DEVCTL,
- devctl)) {
+ if (pcie_capability_write_word(mdev->pdev, PCI_EXP_DEVCTL,
+ devctl)) {
err = -ENODEV;
mthca_err(mdev, "Couldn't restore HCA PCI Express "
"Device Control register, aborting.\n");
goto out;
}
linkctl = hca_header[(hca_pcie_cap + PCI_EXP_LNKCTL) / 4];
- if (pci_write_config_word(mdev->pdev, hca_pcie_cap + PCI_EXP_LNKCTL,
- linkctl)) {
+ if (pcie_capability_write_word(mdev->pdev, PCI_EXP_LNKCTL,
+ linkctl)) {
err = -ENODEV;
mthca_err(mdev, "Couldn't restore HCA PCI Express "
"Link control register, aborting.\n");
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
index cb5b7f7..b29a424 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_verbs.c
@@ -2219,7 +2219,6 @@
u32 wqe_idx;
if (!qp->wqe_wr_id_tbl[tail].signaled) {
- expand = true; /* CQE cannot be consumed yet */
*polled = false; /* WC cannot be consumed yet */
} else {
ibwc->status = IB_WC_SUCCESS;
@@ -2227,10 +2226,11 @@
ibwc->qp = &qp->ibqp;
ocrdma_update_wc(qp, ibwc, tail);
*polled = true;
- wqe_idx = le32_to_cpu(cqe->wq.wqeidx) & OCRDMA_CQE_WQEIDX_MASK;
- if (tail != wqe_idx)
- expand = true; /* Coalesced CQE can't be consumed yet */
}
+ wqe_idx = le32_to_cpu(cqe->wq.wqeidx) & OCRDMA_CQE_WQEIDX_MASK;
+ if (tail != wqe_idx)
+ expand = true; /* Coalesced CQE can't be consumed yet */
+
ocrdma_hwq_inc_tail(&qp->sq);
return expand;
}
diff --git a/drivers/infiniband/hw/qib/qib.h b/drivers/infiniband/hw/qib/qib.h
index 7b1b8669..4d11575 100644
--- a/drivers/infiniband/hw/qib/qib.h
+++ b/drivers/infiniband/hw/qib/qib.h
@@ -87,7 +87,7 @@
};
extern struct qlogic_ib_stats qib_stats;
-extern struct pci_error_handlers qib_pci_err_handler;
+extern const struct pci_error_handlers qib_pci_err_handler;
extern struct pci_driver qib_driver;
#define QIB_CHIP_SWVERSION QIB_CHIP_VERS_MAJ
diff --git a/drivers/infiniband/hw/qib/qib_mad.c b/drivers/infiniband/hw/qib/qib_mad.c
index 19f1e6c..ccb1191 100644
--- a/drivers/infiniband/hw/qib/qib_mad.c
+++ b/drivers/infiniband/hw/qib/qib_mad.c
@@ -471,9 +471,10 @@
if (port_num != port) {
ibp = to_iport(ibdev, port_num);
ret = check_mkey(ibp, smp, 0);
- if (ret)
+ if (ret) {
ret = IB_MAD_RESULT_FAILURE;
goto bail;
+ }
}
}
diff --git a/drivers/infiniband/hw/qib/qib_pcie.c b/drivers/infiniband/hw/qib/qib_pcie.c
index 062c301..c574ec7 100644
--- a/drivers/infiniband/hw/qib/qib_pcie.c
+++ b/drivers/infiniband/hw/qib/qib_pcie.c
@@ -273,10 +273,9 @@
struct qib_msix_entry *entry)
{
u16 linkstat, speed;
- int pos = 0, pose, ret = 1;
+ int pos = 0, ret = 1;
- pose = pci_pcie_cap(dd->pcidev);
- if (!pose) {
+ if (!pci_is_pcie(dd->pcidev)) {
qib_dev_err(dd, "Can't find PCI Express capability!\n");
/* set up something... */
dd->lbus_width = 1;
@@ -298,7 +297,7 @@
if (!pos)
qib_enable_intx(dd->pcidev);
- pci_read_config_word(dd->pcidev, pose + PCI_EXP_LNKSTA, &linkstat);
+ pcie_capability_read_word(dd->pcidev, PCI_EXP_LNKSTA, &linkstat);
/*
* speed is bits 0-3, linkwidth is bits 4-8
* no defines for them in headers
@@ -516,7 +515,6 @@
{
int r;
struct pci_dev *parent;
- int ppos;
u16 devid;
u32 mask, bits, val;
@@ -529,8 +527,7 @@
qib_devinfo(dd->pcidev, "Parent not root\n");
return 1;
}
- ppos = pci_pcie_cap(parent);
- if (!ppos)
+ if (!pci_is_pcie(parent))
return 1;
if (parent->vendor != 0x8086)
return 1;
@@ -587,7 +584,6 @@
{
int ret = 1; /* Assume the worst */
struct pci_dev *parent;
- int ppos, epos;
u16 pcaps, pctl, ecaps, ectl;
int rc_sup, ep_sup;
int rc_cur, ep_cur;
@@ -598,19 +594,15 @@
qib_devinfo(dd->pcidev, "Parent not root\n");
goto bail;
}
- ppos = pci_pcie_cap(parent);
- if (ppos) {
- pci_read_config_word(parent, ppos + PCI_EXP_DEVCAP, &pcaps);
- pci_read_config_word(parent, ppos + PCI_EXP_DEVCTL, &pctl);
- } else
+
+ if (!pci_is_pcie(parent) || !pci_is_pcie(dd->pcidev))
goto bail;
+ pcie_capability_read_word(parent, PCI_EXP_DEVCAP, &pcaps);
+ pcie_capability_read_word(parent, PCI_EXP_DEVCTL, &pctl);
/* Find out supported and configured values for endpoint (us) */
- epos = pci_pcie_cap(dd->pcidev);
- if (epos) {
- pci_read_config_word(dd->pcidev, epos + PCI_EXP_DEVCAP, &ecaps);
- pci_read_config_word(dd->pcidev, epos + PCI_EXP_DEVCTL, &ectl);
- } else
- goto bail;
+ pcie_capability_read_word(dd->pcidev, PCI_EXP_DEVCAP, &ecaps);
+ pcie_capability_read_word(dd->pcidev, PCI_EXP_DEVCTL, &ectl);
+
ret = 0;
/* Find max payload supported by root, endpoint */
rc_sup = fld2val(pcaps, PCI_EXP_DEVCAP_PAYLOAD);
@@ -629,14 +621,14 @@
rc_cur = rc_sup;
pctl = (pctl & ~PCI_EXP_DEVCTL_PAYLOAD) |
val2fld(rc_cur, PCI_EXP_DEVCTL_PAYLOAD);
- pci_write_config_word(parent, ppos + PCI_EXP_DEVCTL, pctl);
+ pcie_capability_write_word(parent, PCI_EXP_DEVCTL, pctl);
}
/* If less than (allowed, supported), bump endpoint payload */
if (rc_sup > ep_cur) {
ep_cur = rc_sup;
ectl = (ectl & ~PCI_EXP_DEVCTL_PAYLOAD) |
val2fld(ep_cur, PCI_EXP_DEVCTL_PAYLOAD);
- pci_write_config_word(dd->pcidev, epos + PCI_EXP_DEVCTL, ectl);
+ pcie_capability_write_word(dd->pcidev, PCI_EXP_DEVCTL, ectl);
}
/*
@@ -654,13 +646,13 @@
rc_cur = rc_sup;
pctl = (pctl & ~PCI_EXP_DEVCTL_READRQ) |
val2fld(rc_cur, PCI_EXP_DEVCTL_READRQ);
- pci_write_config_word(parent, ppos + PCI_EXP_DEVCTL, pctl);
+ pcie_capability_write_word(parent, PCI_EXP_DEVCTL, pctl);
}
if (rc_sup > ep_cur) {
ep_cur = rc_sup;
ectl = (ectl & ~PCI_EXP_DEVCTL_READRQ) |
val2fld(ep_cur, PCI_EXP_DEVCTL_READRQ);
- pci_write_config_word(dd->pcidev, epos + PCI_EXP_DEVCTL, ectl);
+ pcie_capability_write_word(dd->pcidev, PCI_EXP_DEVCTL, ectl);
}
bail:
return ret;
@@ -753,7 +745,7 @@
qib_init(dd, 1); /* same as re-init after reset */
}
-struct pci_error_handlers qib_pci_err_handler = {
+const struct pci_error_handlers qib_pci_err_handler = {
.error_detected = qib_pci_error_detected,
.mmio_enabled = qib_pci_mmio_enabled,
.link_reset = qib_pci_link_reset,
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h
index ca43901..0af216d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -262,7 +262,10 @@
u16 max_coalesced_frames;
};
+struct ipoib_neigh_table;
+
struct ipoib_neigh_hash {
+ struct ipoib_neigh_table *ntbl;
struct ipoib_neigh __rcu **buckets;
struct rcu_head rcu;
u32 mask;
@@ -271,9 +274,9 @@
struct ipoib_neigh_table {
struct ipoib_neigh_hash __rcu *htbl;
- rwlock_t rwlock;
atomic_t entries;
struct completion flushed;
+ struct completion deleted;
};
/*
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 3e2085a..1e19b5a 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -546,15 +546,15 @@
struct ipoib_neigh *neigh;
unsigned long flags;
+ spin_lock_irqsave(&priv->lock, flags);
neigh = ipoib_neigh_alloc(daddr, dev);
if (!neigh) {
+ spin_unlock_irqrestore(&priv->lock, flags);
++dev->stats.tx_dropped;
dev_kfree_skb_any(skb);
return;
}
- spin_lock_irqsave(&priv->lock, flags);
-
path = __path_find(dev, daddr + 4);
if (!path) {
path = path_rec_create(dev, daddr + 4);
@@ -863,10 +863,10 @@
if (test_bit(IPOIB_STOP_NEIGH_GC, &priv->flags))
return;
- write_lock_bh(&ntbl->rwlock);
+ spin_lock_irqsave(&priv->lock, flags);
htbl = rcu_dereference_protected(ntbl->htbl,
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
if (!htbl)
goto out_unlock;
@@ -883,16 +883,14 @@
struct ipoib_neigh __rcu **np = &htbl->buckets[i];
while ((neigh = rcu_dereference_protected(*np,
- lockdep_is_held(&ntbl->rwlock))) != NULL) {
+ lockdep_is_held(&priv->lock))) != NULL) {
/* was the neigh idle for two GC periods */
if (time_after(neigh_obsolete, neigh->alive)) {
rcu_assign_pointer(*np,
rcu_dereference_protected(neigh->hnext,
- lockdep_is_held(&ntbl->rwlock)));
+ lockdep_is_held(&priv->lock)));
/* remove from path/mc list */
- spin_lock_irqsave(&priv->lock, flags);
list_del(&neigh->list);
- spin_unlock_irqrestore(&priv->lock, flags);
call_rcu(&neigh->rcu, ipoib_neigh_reclaim);
} else {
np = &neigh->hnext;
@@ -902,7 +900,7 @@
}
out_unlock:
- write_unlock_bh(&ntbl->rwlock);
+ spin_unlock_irqrestore(&priv->lock, flags);
}
static void ipoib_reap_neigh(struct work_struct *work)
@@ -947,10 +945,8 @@
struct ipoib_neigh *neigh;
u32 hash_val;
- write_lock_bh(&ntbl->rwlock);
-
htbl = rcu_dereference_protected(ntbl->htbl,
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
if (!htbl) {
neigh = NULL;
goto out_unlock;
@@ -961,10 +957,10 @@
*/
hash_val = ipoib_addr_hash(htbl, daddr);
for (neigh = rcu_dereference_protected(htbl->buckets[hash_val],
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
neigh != NULL;
neigh = rcu_dereference_protected(neigh->hnext,
- lockdep_is_held(&ntbl->rwlock))) {
+ lockdep_is_held(&priv->lock))) {
if (memcmp(daddr, neigh->daddr, INFINIBAND_ALEN) == 0) {
/* found, take one ref on behalf of the caller */
if (!atomic_inc_not_zero(&neigh->refcnt)) {
@@ -987,12 +983,11 @@
/* put in hash */
rcu_assign_pointer(neigh->hnext,
rcu_dereference_protected(htbl->buckets[hash_val],
- lockdep_is_held(&ntbl->rwlock)));
+ lockdep_is_held(&priv->lock)));
rcu_assign_pointer(htbl->buckets[hash_val], neigh);
atomic_inc(&ntbl->entries);
out_unlock:
- write_unlock_bh(&ntbl->rwlock);
return neigh;
}
@@ -1040,35 +1035,29 @@
struct ipoib_neigh *n;
u32 hash_val;
- write_lock_bh(&ntbl->rwlock);
-
htbl = rcu_dereference_protected(ntbl->htbl,
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
if (!htbl)
- goto out_unlock;
+ return;
hash_val = ipoib_addr_hash(htbl, neigh->daddr);
np = &htbl->buckets[hash_val];
for (n = rcu_dereference_protected(*np,
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
n != NULL;
n = rcu_dereference_protected(*np,
- lockdep_is_held(&ntbl->rwlock))) {
+ lockdep_is_held(&priv->lock))) {
if (n == neigh) {
/* found */
rcu_assign_pointer(*np,
rcu_dereference_protected(neigh->hnext,
- lockdep_is_held(&ntbl->rwlock)));
+ lockdep_is_held(&priv->lock)));
call_rcu(&neigh->rcu, ipoib_neigh_reclaim);
- goto out_unlock;
+ return;
} else {
np = &n->hnext;
}
}
-
-out_unlock:
- write_unlock_bh(&ntbl->rwlock);
-
}
static int ipoib_neigh_hash_init(struct ipoib_dev_priv *priv)
@@ -1080,7 +1069,6 @@
clear_bit(IPOIB_NEIGH_TBL_FLUSH, &priv->flags);
ntbl->htbl = NULL;
- rwlock_init(&ntbl->rwlock);
htbl = kzalloc(sizeof(*htbl), GFP_KERNEL);
if (!htbl)
return -ENOMEM;
@@ -1095,6 +1083,7 @@
htbl->mask = (size - 1);
htbl->buckets = buckets;
ntbl->htbl = htbl;
+ htbl->ntbl = ntbl;
atomic_set(&ntbl->entries, 0);
/* start garbage collection */
@@ -1111,9 +1100,11 @@
struct ipoib_neigh_hash,
rcu);
struct ipoib_neigh __rcu **buckets = htbl->buckets;
+ struct ipoib_neigh_table *ntbl = htbl->ntbl;
kfree(buckets);
kfree(htbl);
+ complete(&ntbl->deleted);
}
void ipoib_del_neighs_by_gid(struct net_device *dev, u8 *gid)
@@ -1125,10 +1116,10 @@
int i;
/* remove all neigh connected to a given path or mcast */
- write_lock_bh(&ntbl->rwlock);
+ spin_lock_irqsave(&priv->lock, flags);
htbl = rcu_dereference_protected(ntbl->htbl,
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
if (!htbl)
goto out_unlock;
@@ -1138,16 +1129,14 @@
struct ipoib_neigh __rcu **np = &htbl->buckets[i];
while ((neigh = rcu_dereference_protected(*np,
- lockdep_is_held(&ntbl->rwlock))) != NULL) {
+ lockdep_is_held(&priv->lock))) != NULL) {
/* delete neighs belong to this parent */
if (!memcmp(gid, neigh->daddr + 4, sizeof (union ib_gid))) {
rcu_assign_pointer(*np,
rcu_dereference_protected(neigh->hnext,
- lockdep_is_held(&ntbl->rwlock)));
+ lockdep_is_held(&priv->lock)));
/* remove from parent list */
- spin_lock_irqsave(&priv->lock, flags);
list_del(&neigh->list);
- spin_unlock_irqrestore(&priv->lock, flags);
call_rcu(&neigh->rcu, ipoib_neigh_reclaim);
} else {
np = &neigh->hnext;
@@ -1156,7 +1145,7 @@
}
}
out_unlock:
- write_unlock_bh(&ntbl->rwlock);
+ spin_unlock_irqrestore(&priv->lock, flags);
}
static void ipoib_flush_neighs(struct ipoib_dev_priv *priv)
@@ -1164,37 +1153,44 @@
struct ipoib_neigh_table *ntbl = &priv->ntbl;
struct ipoib_neigh_hash *htbl;
unsigned long flags;
- int i;
+ int i, wait_flushed = 0;
- write_lock_bh(&ntbl->rwlock);
+ init_completion(&priv->ntbl.flushed);
+
+ spin_lock_irqsave(&priv->lock, flags);
htbl = rcu_dereference_protected(ntbl->htbl,
- lockdep_is_held(&ntbl->rwlock));
+ lockdep_is_held(&priv->lock));
if (!htbl)
goto out_unlock;
+ wait_flushed = atomic_read(&priv->ntbl.entries);
+ if (!wait_flushed)
+ goto free_htbl;
+
for (i = 0; i < htbl->size; i++) {
struct ipoib_neigh *neigh;
struct ipoib_neigh __rcu **np = &htbl->buckets[i];
while ((neigh = rcu_dereference_protected(*np,
- lockdep_is_held(&ntbl->rwlock))) != NULL) {
+ lockdep_is_held(&priv->lock))) != NULL) {
rcu_assign_pointer(*np,
rcu_dereference_protected(neigh->hnext,
- lockdep_is_held(&ntbl->rwlock)));
+ lockdep_is_held(&priv->lock)));
/* remove from path/mc list */
- spin_lock_irqsave(&priv->lock, flags);
list_del(&neigh->list);
- spin_unlock_irqrestore(&priv->lock, flags);
call_rcu(&neigh->rcu, ipoib_neigh_reclaim);
}
}
+free_htbl:
rcu_assign_pointer(ntbl->htbl, NULL);
call_rcu(&htbl->rcu, neigh_hash_free_rcu);
out_unlock:
- write_unlock_bh(&ntbl->rwlock);
+ spin_unlock_irqrestore(&priv->lock, flags);
+ if (wait_flushed)
+ wait_for_completion(&priv->ntbl.flushed);
}
static void ipoib_neigh_hash_uninit(struct net_device *dev)
@@ -1203,7 +1199,7 @@
int stopped;
ipoib_dbg(priv, "ipoib_neigh_hash_uninit\n");
- init_completion(&priv->ntbl.flushed);
+ init_completion(&priv->ntbl.deleted);
set_bit(IPOIB_NEIGH_TBL_FLUSH, &priv->flags);
/* Stop GC if called at init fail need to cancel work */
@@ -1211,10 +1207,9 @@
if (!stopped)
cancel_delayed_work(&priv->neigh_reap_task);
- if (atomic_read(&priv->ntbl.entries)) {
- ipoib_flush_neighs(priv);
- wait_for_completion(&priv->ntbl.flushed);
- }
+ ipoib_flush_neighs(priv);
+
+ wait_for_completion(&priv->ntbl.deleted);
}
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index 13f4aa7..7536724 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -707,9 +707,7 @@
neigh = ipoib_neigh_get(dev, daddr);
spin_lock_irqsave(&priv->lock, flags);
if (!neigh) {
- spin_unlock_irqrestore(&priv->lock, flags);
neigh = ipoib_neigh_alloc(daddr, dev);
- spin_lock_irqsave(&priv->lock, flags);
if (neigh) {
kref_get(&mcast->ah->ref);
neigh->ah = mcast->ah;
diff --git a/drivers/input/evdev.c b/drivers/input/evdev.c
index 6c58bff..118d030 100644
--- a/drivers/input/evdev.c
+++ b/drivers/input/evdev.c
@@ -54,16 +54,9 @@
static struct evdev *evdev_table[EVDEV_MINORS];
static DEFINE_MUTEX(evdev_table_mutex);
-static void evdev_pass_event(struct evdev_client *client,
- struct input_event *event,
- ktime_t mono, ktime_t real)
+static void __pass_event(struct evdev_client *client,
+ const struct input_event *event)
{
- event->time = ktime_to_timeval(client->clkid == CLOCK_MONOTONIC ?
- mono : real);
-
- /* Interrupts are disabled, just acquire the lock. */
- spin_lock(&client->buffer_lock);
-
client->buffer[client->head++] = *event;
client->head &= client->bufsize - 1;
@@ -86,8 +79,63 @@
client->packet_head = client->head;
kill_fasync(&client->fasync, SIGIO, POLL_IN);
}
+}
+
+static void evdev_pass_values(struct evdev_client *client,
+ const struct input_value *vals, unsigned int count,
+ ktime_t mono, ktime_t real)
+{
+ struct evdev *evdev = client->evdev;
+ const struct input_value *v;
+ struct input_event event;
+ bool wakeup = false;
+
+ event.time = ktime_to_timeval(client->clkid == CLOCK_MONOTONIC ?
+ mono : real);
+
+ /* Interrupts are disabled, just acquire the lock. */
+ spin_lock(&client->buffer_lock);
+
+ for (v = vals; v != vals + count; v++) {
+ event.type = v->type;
+ event.code = v->code;
+ event.value = v->value;
+ __pass_event(client, &event);
+ if (v->type == EV_SYN && v->code == SYN_REPORT)
+ wakeup = true;
+ }
spin_unlock(&client->buffer_lock);
+
+ if (wakeup)
+ wake_up_interruptible(&evdev->wait);
+}
+
+/*
+ * Pass incoming events to all connected clients.
+ */
+static void evdev_events(struct input_handle *handle,
+ const struct input_value *vals, unsigned int count)
+{
+ struct evdev *evdev = handle->private;
+ struct evdev_client *client;
+ ktime_t time_mono, time_real;
+
+ time_mono = ktime_get();
+ time_real = ktime_sub(time_mono, ktime_get_monotonic_offset());
+
+ rcu_read_lock();
+
+ client = rcu_dereference(evdev->grab);
+
+ if (client)
+ evdev_pass_values(client, vals, count, time_mono, time_real);
+ else
+ list_for_each_entry_rcu(client, &evdev->client_list, node)
+ evdev_pass_values(client, vals, count,
+ time_mono, time_real);
+
+ rcu_read_unlock();
}
/*
@@ -96,32 +144,9 @@
static void evdev_event(struct input_handle *handle,
unsigned int type, unsigned int code, int value)
{
- struct evdev *evdev = handle->private;
- struct evdev_client *client;
- struct input_event event;
- ktime_t time_mono, time_real;
+ struct input_value vals[] = { { type, code, value } };
- time_mono = ktime_get();
- time_real = ktime_sub(time_mono, ktime_get_monotonic_offset());
-
- event.type = type;
- event.code = code;
- event.value = value;
-
- rcu_read_lock();
-
- client = rcu_dereference(evdev->grab);
-
- if (client)
- evdev_pass_event(client, &event, time_mono, time_real);
- else
- list_for_each_entry_rcu(client, &evdev->client_list, node)
- evdev_pass_event(client, &event, time_mono, time_real);
-
- rcu_read_unlock();
-
- if (type == EV_SYN && code == SYN_REPORT)
- wake_up_interruptible(&evdev->wait);
+ evdev_events(handle, vals, 1);
}
static int evdev_fasync(int fd, struct file *file, int on)
@@ -653,20 +678,22 @@
unsigned int size,
int __user *ip)
{
- const struct input_mt_slot *mt = dev->mt;
+ const struct input_mt *mt = dev->mt;
unsigned int code;
int max_slots;
int i;
if (get_user(code, &ip[0]))
return -EFAULT;
- if (!input_is_mt_value(code))
+ if (!mt || !input_is_mt_value(code))
return -EINVAL;
max_slots = (size - sizeof(__u32)) / sizeof(__s32);
- for (i = 0; i < dev->mtsize && i < max_slots; i++)
- if (put_user(input_mt_get_value(&mt[i], code), &ip[1 + i]))
+ for (i = 0; i < mt->num_slots && i < max_slots; i++) {
+ int value = input_mt_get_value(&mt->slots[i], code);
+ if (put_user(value, &ip[1 + i]))
return -EFAULT;
+ }
return 0;
}
@@ -1048,6 +1075,7 @@
static struct input_handler evdev_handler = {
.event = evdev_event,
+ .events = evdev_events,
.connect = evdev_connect,
.disconnect = evdev_disconnect,
.fops = &evdev_fops,
diff --git a/drivers/input/input-mt.c b/drivers/input/input-mt.c
index 70a16c7..c0ec7d4 100644
--- a/drivers/input/input-mt.c
+++ b/drivers/input/input-mt.c
@@ -14,6 +14,14 @@
#define TRKID_SGN ((TRKID_MAX + 1) >> 1)
+static void copy_abs(struct input_dev *dev, unsigned int dst, unsigned int src)
+{
+ if (dev->absinfo && test_bit(src, dev->absbit)) {
+ dev->absinfo[dst] = dev->absinfo[src];
+ dev->absbit[BIT_WORD(dst)] |= BIT_MASK(dst);
+ }
+}
+
/**
* input_mt_init_slots() - initialize MT input slots
* @dev: input device supporting MT events and finger tracking
@@ -25,29 +33,63 @@
* May be called repeatedly. Returns -EINVAL if attempting to
* reinitialize with a different number of slots.
*/
-int input_mt_init_slots(struct input_dev *dev, unsigned int num_slots)
+int input_mt_init_slots(struct input_dev *dev, unsigned int num_slots,
+ unsigned int flags)
{
+ struct input_mt *mt = dev->mt;
int i;
if (!num_slots)
return 0;
- if (dev->mt)
- return dev->mtsize != num_slots ? -EINVAL : 0;
+ if (mt)
+ return mt->num_slots != num_slots ? -EINVAL : 0;
- dev->mt = kcalloc(num_slots, sizeof(struct input_mt_slot), GFP_KERNEL);
- if (!dev->mt)
- return -ENOMEM;
+ mt = kzalloc(sizeof(*mt) + num_slots * sizeof(*mt->slots), GFP_KERNEL);
+ if (!mt)
+ goto err_mem;
- dev->mtsize = num_slots;
+ mt->num_slots = num_slots;
+ mt->flags = flags;
input_set_abs_params(dev, ABS_MT_SLOT, 0, num_slots - 1, 0, 0);
input_set_abs_params(dev, ABS_MT_TRACKING_ID, 0, TRKID_MAX, 0, 0);
- input_set_events_per_packet(dev, 6 * num_slots);
+
+ if (flags & (INPUT_MT_POINTER | INPUT_MT_DIRECT)) {
+ __set_bit(EV_KEY, dev->evbit);
+ __set_bit(BTN_TOUCH, dev->keybit);
+
+ copy_abs(dev, ABS_X, ABS_MT_POSITION_X);
+ copy_abs(dev, ABS_Y, ABS_MT_POSITION_Y);
+ copy_abs(dev, ABS_PRESSURE, ABS_MT_PRESSURE);
+ }
+ if (flags & INPUT_MT_POINTER) {
+ __set_bit(BTN_TOOL_FINGER, dev->keybit);
+ __set_bit(BTN_TOOL_DOUBLETAP, dev->keybit);
+ if (num_slots >= 3)
+ __set_bit(BTN_TOOL_TRIPLETAP, dev->keybit);
+ if (num_slots >= 4)
+ __set_bit(BTN_TOOL_QUADTAP, dev->keybit);
+ if (num_slots >= 5)
+ __set_bit(BTN_TOOL_QUINTTAP, dev->keybit);
+ __set_bit(INPUT_PROP_POINTER, dev->propbit);
+ }
+ if (flags & INPUT_MT_DIRECT)
+ __set_bit(INPUT_PROP_DIRECT, dev->propbit);
+ if (flags & INPUT_MT_TRACK) {
+ unsigned int n2 = num_slots * num_slots;
+ mt->red = kcalloc(n2, sizeof(*mt->red), GFP_KERNEL);
+ if (!mt->red)
+ goto err_mem;
+ }
/* Mark slots as 'unused' */
for (i = 0; i < num_slots; i++)
- input_mt_set_value(&dev->mt[i], ABS_MT_TRACKING_ID, -1);
+ input_mt_set_value(&mt->slots[i], ABS_MT_TRACKING_ID, -1);
+ dev->mt = mt;
return 0;
+err_mem:
+ kfree(mt);
+ return -ENOMEM;
}
EXPORT_SYMBOL(input_mt_init_slots);
@@ -60,11 +102,11 @@
*/
void input_mt_destroy_slots(struct input_dev *dev)
{
- kfree(dev->mt);
+ if (dev->mt) {
+ kfree(dev->mt->red);
+ kfree(dev->mt);
+ }
dev->mt = NULL;
- dev->mtsize = 0;
- dev->slot = 0;
- dev->trkid = 0;
}
EXPORT_SYMBOL(input_mt_destroy_slots);
@@ -83,18 +125,24 @@
void input_mt_report_slot_state(struct input_dev *dev,
unsigned int tool_type, bool active)
{
- struct input_mt_slot *mt;
+ struct input_mt *mt = dev->mt;
+ struct input_mt_slot *slot;
int id;
- if (!dev->mt || !active) {
+ if (!mt)
+ return;
+
+ slot = &mt->slots[mt->slot];
+ slot->frame = mt->frame;
+
+ if (!active) {
input_event(dev, EV_ABS, ABS_MT_TRACKING_ID, -1);
return;
}
- mt = &dev->mt[dev->slot];
- id = input_mt_get_value(mt, ABS_MT_TRACKING_ID);
- if (id < 0 || input_mt_get_value(mt, ABS_MT_TOOL_TYPE) != tool_type)
- id = input_mt_new_trkid(dev);
+ id = input_mt_get_value(slot, ABS_MT_TRACKING_ID);
+ if (id < 0 || input_mt_get_value(slot, ABS_MT_TOOL_TYPE) != tool_type)
+ id = input_mt_new_trkid(mt);
input_event(dev, EV_ABS, ABS_MT_TRACKING_ID, id);
input_event(dev, EV_ABS, ABS_MT_TOOL_TYPE, tool_type);
@@ -135,13 +183,19 @@
*/
void input_mt_report_pointer_emulation(struct input_dev *dev, bool use_count)
{
- struct input_mt_slot *oldest = NULL;
- int oldid = dev->trkid;
- int count = 0;
- int i;
+ struct input_mt *mt = dev->mt;
+ struct input_mt_slot *oldest;
+ int oldid, count, i;
- for (i = 0; i < dev->mtsize; ++i) {
- struct input_mt_slot *ps = &dev->mt[i];
+ if (!mt)
+ return;
+
+ oldest = 0;
+ oldid = mt->trkid;
+ count = 0;
+
+ for (i = 0; i < mt->num_slots; ++i) {
+ struct input_mt_slot *ps = &mt->slots[i];
int id = input_mt_get_value(ps, ABS_MT_TRACKING_ID);
if (id < 0)
@@ -160,13 +214,208 @@
if (oldest) {
int x = input_mt_get_value(oldest, ABS_MT_POSITION_X);
int y = input_mt_get_value(oldest, ABS_MT_POSITION_Y);
- int p = input_mt_get_value(oldest, ABS_MT_PRESSURE);
input_event(dev, EV_ABS, ABS_X, x);
input_event(dev, EV_ABS, ABS_Y, y);
- input_event(dev, EV_ABS, ABS_PRESSURE, p);
+
+ if (test_bit(ABS_MT_PRESSURE, dev->absbit)) {
+ int p = input_mt_get_value(oldest, ABS_MT_PRESSURE);
+ input_event(dev, EV_ABS, ABS_PRESSURE, p);
+ }
} else {
- input_event(dev, EV_ABS, ABS_PRESSURE, 0);
+ if (test_bit(ABS_MT_PRESSURE, dev->absbit))
+ input_event(dev, EV_ABS, ABS_PRESSURE, 0);
}
}
EXPORT_SYMBOL(input_mt_report_pointer_emulation);
+
+/**
+ * input_mt_sync_frame() - synchronize mt frame
+ * @dev: input device with allocated MT slots
+ *
+ * Close the frame and prepare the internal state for a new one.
+ * Depending on the flags, marks unused slots as inactive and performs
+ * pointer emulation.
+ */
+void input_mt_sync_frame(struct input_dev *dev)
+{
+ struct input_mt *mt = dev->mt;
+ struct input_mt_slot *s;
+
+ if (!mt)
+ return;
+
+ if (mt->flags & INPUT_MT_DROP_UNUSED) {
+ for (s = mt->slots; s != mt->slots + mt->num_slots; s++) {
+ if (s->frame == mt->frame)
+ continue;
+ input_mt_slot(dev, s - mt->slots);
+ input_event(dev, EV_ABS, ABS_MT_TRACKING_ID, -1);
+ }
+ }
+
+ input_mt_report_pointer_emulation(dev, (mt->flags & INPUT_MT_POINTER));
+
+ mt->frame++;
+}
+EXPORT_SYMBOL(input_mt_sync_frame);
+
+static int adjust_dual(int *begin, int step, int *end, int eq)
+{
+ int f, *p, s, c;
+
+ if (begin == end)
+ return 0;
+
+ f = *begin;
+ p = begin + step;
+ s = p == end ? f + 1 : *p;
+
+ for (; p != end; p += step)
+ if (*p < f)
+ s = f, f = *p;
+ else if (*p < s)
+ s = *p;
+
+ c = (f + s + 1) / 2;
+ if (c == 0 || (c > 0 && !eq))
+ return 0;
+ if (s < 0)
+ c *= 2;
+
+ for (p = begin; p != end; p += step)
+ *p -= c;
+
+ return (c < s && s <= 0) || (f >= 0 && f < c);
+}
+
+static void find_reduced_matrix(int *w, int nr, int nc, int nrc)
+{
+ int i, k, sum;
+
+ for (k = 0; k < nrc; k++) {
+ for (i = 0; i < nr; i++)
+ adjust_dual(w + i, nr, w + i + nrc, nr <= nc);
+ sum = 0;
+ for (i = 0; i < nrc; i += nr)
+ sum += adjust_dual(w + i, 1, w + i + nr, nc <= nr);
+ if (!sum)
+ break;
+ }
+}
+
+static int input_mt_set_matrix(struct input_mt *mt,
+ const struct input_mt_pos *pos, int num_pos)
+{
+ const struct input_mt_pos *p;
+ struct input_mt_slot *s;
+ int *w = mt->red;
+ int x, y;
+
+ for (s = mt->slots; s != mt->slots + mt->num_slots; s++) {
+ if (!input_mt_is_active(s))
+ continue;
+ x = input_mt_get_value(s, ABS_MT_POSITION_X);
+ y = input_mt_get_value(s, ABS_MT_POSITION_Y);
+ for (p = pos; p != pos + num_pos; p++) {
+ int dx = x - p->x, dy = y - p->y;
+ *w++ = dx * dx + dy * dy;
+ }
+ }
+
+ return w - mt->red;
+}
+
+static void input_mt_set_slots(struct input_mt *mt,
+ int *slots, int num_pos)
+{
+ struct input_mt_slot *s;
+ int *w = mt->red, *p;
+
+ for (p = slots; p != slots + num_pos; p++)
+ *p = -1;
+
+ for (s = mt->slots; s != mt->slots + mt->num_slots; s++) {
+ if (!input_mt_is_active(s))
+ continue;
+ for (p = slots; p != slots + num_pos; p++)
+ if (*w++ < 0)
+ *p = s - mt->slots;
+ }
+
+ for (s = mt->slots; s != mt->slots + mt->num_slots; s++) {
+ if (input_mt_is_active(s))
+ continue;
+ for (p = slots; p != slots + num_pos; p++)
+ if (*p < 0) {
+ *p = s - mt->slots;
+ break;
+ }
+ }
+}
+
+/**
+ * input_mt_assign_slots() - perform a best-match assignment
+ * @dev: input device with allocated MT slots
+ * @slots: the slot assignment to be filled
+ * @pos: the position array to match
+ * @num_pos: number of positions
+ *
+ * Performs a best match against the current contacts and returns
+ * the slot assignment list. New contacts are assigned to unused
+ * slots.
+ *
+ * Returns zero on success, or negative error in case of failure.
+ */
+int input_mt_assign_slots(struct input_dev *dev, int *slots,
+ const struct input_mt_pos *pos, int num_pos)
+{
+ struct input_mt *mt = dev->mt;
+ int nrc;
+
+ if (!mt || !mt->red)
+ return -ENXIO;
+ if (num_pos > mt->num_slots)
+ return -EINVAL;
+ if (num_pos < 1)
+ return 0;
+
+ nrc = input_mt_set_matrix(mt, pos, num_pos);
+ find_reduced_matrix(mt->red, num_pos, nrc / num_pos, nrc);
+ input_mt_set_slots(mt, slots, num_pos);
+
+ return 0;
+}
+EXPORT_SYMBOL(input_mt_assign_slots);
+
+/**
+ * input_mt_get_slot_by_key() - return slot matching key
+ * @dev: input device with allocated MT slots
+ * @key: the key of the sought slot
+ *
+ * Returns the slot of the given key, if it exists, otherwise
+ * set the key on the first unused slot and return.
+ *
+ * If no available slot can be found, -1 is returned.
+ */
+int input_mt_get_slot_by_key(struct input_dev *dev, int key)
+{
+ struct input_mt *mt = dev->mt;
+ struct input_mt_slot *s;
+
+ if (!mt)
+ return -1;
+
+ for (s = mt->slots; s != mt->slots + mt->num_slots; s++)
+ if (input_mt_is_active(s) && s->key == key)
+ return s - mt->slots;
+
+ for (s = mt->slots; s != mt->slots + mt->num_slots; s++)
+ if (!input_mt_is_active(s)) {
+ s->key = key;
+ return s - mt->slots;
+ }
+
+ return -1;
+}
+EXPORT_SYMBOL(input_mt_get_slot_by_key);
diff --git a/drivers/input/input.c b/drivers/input/input.c
index 8921c61..5244f3d 100644
--- a/drivers/input/input.c
+++ b/drivers/input/input.c
@@ -47,6 +47,8 @@
static struct input_handler *input_table[8];
+static const struct input_value input_value_sync = { EV_SYN, SYN_REPORT, 1 };
+
static inline int is_event_supported(unsigned int code,
unsigned long *bm, unsigned int max)
{
@@ -69,79 +71,6 @@
return value;
}
-/*
- * Pass event first through all filters and then, if event has not been
- * filtered out, through all open handles. This function is called with
- * dev->event_lock held and interrupts disabled.
- */
-static void input_pass_event(struct input_dev *dev,
- unsigned int type, unsigned int code, int value)
-{
- struct input_handler *handler;
- struct input_handle *handle;
-
- rcu_read_lock();
-
- handle = rcu_dereference(dev->grab);
- if (handle)
- handle->handler->event(handle, type, code, value);
- else {
- bool filtered = false;
-
- list_for_each_entry_rcu(handle, &dev->h_list, d_node) {
- if (!handle->open)
- continue;
-
- handler = handle->handler;
- if (!handler->filter) {
- if (filtered)
- break;
-
- handler->event(handle, type, code, value);
-
- } else if (handler->filter(handle, type, code, value))
- filtered = true;
- }
- }
-
- rcu_read_unlock();
-}
-
-/*
- * Generate software autorepeat event. Note that we take
- * dev->event_lock here to avoid racing with input_event
- * which may cause keys get "stuck".
- */
-static void input_repeat_key(unsigned long data)
-{
- struct input_dev *dev = (void *) data;
- unsigned long flags;
-
- spin_lock_irqsave(&dev->event_lock, flags);
-
- if (test_bit(dev->repeat_key, dev->key) &&
- is_event_supported(dev->repeat_key, dev->keybit, KEY_MAX)) {
-
- input_pass_event(dev, EV_KEY, dev->repeat_key, 2);
-
- if (dev->sync) {
- /*
- * Only send SYN_REPORT if we are not in a middle
- * of driver parsing a new hardware packet.
- * Otherwise assume that the driver will send
- * SYN_REPORT once it's done.
- */
- input_pass_event(dev, EV_SYN, SYN_REPORT, 1);
- }
-
- if (dev->rep[REP_PERIOD])
- mod_timer(&dev->timer, jiffies +
- msecs_to_jiffies(dev->rep[REP_PERIOD]));
- }
-
- spin_unlock_irqrestore(&dev->event_lock, flags);
-}
-
static void input_start_autorepeat(struct input_dev *dev, int code)
{
if (test_bit(EV_REP, dev->evbit) &&
@@ -158,14 +87,128 @@
del_timer(&dev->timer);
}
+/*
+ * Pass event first through all filters and then, if event has not been
+ * filtered out, through all open handles. This function is called with
+ * dev->event_lock held and interrupts disabled.
+ */
+static unsigned int input_to_handler(struct input_handle *handle,
+ struct input_value *vals, unsigned int count)
+{
+ struct input_handler *handler = handle->handler;
+ struct input_value *end = vals;
+ struct input_value *v;
+
+ for (v = vals; v != vals + count; v++) {
+ if (handler->filter &&
+ handler->filter(handle, v->type, v->code, v->value))
+ continue;
+ if (end != v)
+ *end = *v;
+ end++;
+ }
+
+ count = end - vals;
+ if (!count)
+ return 0;
+
+ if (handler->events)
+ handler->events(handle, vals, count);
+ else if (handler->event)
+ for (v = vals; v != end; v++)
+ handler->event(handle, v->type, v->code, v->value);
+
+ return count;
+}
+
+/*
+ * Pass values first through all filters and then, if event has not been
+ * filtered out, through all open handles. This function is called with
+ * dev->event_lock held and interrupts disabled.
+ */
+static void input_pass_values(struct input_dev *dev,
+ struct input_value *vals, unsigned int count)
+{
+ struct input_handle *handle;
+ struct input_value *v;
+
+ if (!count)
+ return;
+
+ rcu_read_lock();
+
+ handle = rcu_dereference(dev->grab);
+ if (handle) {
+ count = input_to_handler(handle, vals, count);
+ } else {
+ list_for_each_entry_rcu(handle, &dev->h_list, d_node)
+ if (handle->open)
+ count = input_to_handler(handle, vals, count);
+ }
+
+ rcu_read_unlock();
+
+ add_input_randomness(vals->type, vals->code, vals->value);
+
+ /* trigger auto repeat for key events */
+ for (v = vals; v != vals + count; v++) {
+ if (v->type == EV_KEY && v->value != 2) {
+ if (v->value)
+ input_start_autorepeat(dev, v->code);
+ else
+ input_stop_autorepeat(dev);
+ }
+ }
+}
+
+static void input_pass_event(struct input_dev *dev,
+ unsigned int type, unsigned int code, int value)
+{
+ struct input_value vals[] = { { type, code, value } };
+
+ input_pass_values(dev, vals, ARRAY_SIZE(vals));
+}
+
+/*
+ * Generate software autorepeat event. Note that we take
+ * dev->event_lock here to avoid racing with input_event
+ * which may cause keys get "stuck".
+ */
+static void input_repeat_key(unsigned long data)
+{
+ struct input_dev *dev = (void *) data;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->event_lock, flags);
+
+ if (test_bit(dev->repeat_key, dev->key) &&
+ is_event_supported(dev->repeat_key, dev->keybit, KEY_MAX)) {
+ struct input_value vals[] = {
+ { EV_KEY, dev->repeat_key, 2 },
+ input_value_sync
+ };
+
+ input_pass_values(dev, vals, ARRAY_SIZE(vals));
+
+ if (dev->rep[REP_PERIOD])
+ mod_timer(&dev->timer, jiffies +
+ msecs_to_jiffies(dev->rep[REP_PERIOD]));
+ }
+
+ spin_unlock_irqrestore(&dev->event_lock, flags);
+}
+
#define INPUT_IGNORE_EVENT 0
#define INPUT_PASS_TO_HANDLERS 1
#define INPUT_PASS_TO_DEVICE 2
+#define INPUT_SLOT 4
+#define INPUT_FLUSH 8
#define INPUT_PASS_TO_ALL (INPUT_PASS_TO_HANDLERS | INPUT_PASS_TO_DEVICE)
static int input_handle_abs_event(struct input_dev *dev,
unsigned int code, int *pval)
{
+ struct input_mt *mt = dev->mt;
bool is_mt_event;
int *pold;
@@ -174,8 +217,8 @@
* "Stage" the event; we'll flush it later, when we
* get actual touch data.
*/
- if (*pval >= 0 && *pval < dev->mtsize)
- dev->slot = *pval;
+ if (mt && *pval >= 0 && *pval < mt->num_slots)
+ mt->slot = *pval;
return INPUT_IGNORE_EVENT;
}
@@ -184,9 +227,8 @@
if (!is_mt_event) {
pold = &dev->absinfo[code].value;
- } else if (dev->mt) {
- struct input_mt_slot *mtslot = &dev->mt[dev->slot];
- pold = &mtslot->abs[code - ABS_MT_FIRST];
+ } else if (mt) {
+ pold = &mt->slots[mt->slot].abs[code - ABS_MT_FIRST];
} else {
/*
* Bypass filtering for multi-touch events when
@@ -205,16 +247,16 @@
}
/* Flush pending "slot" event */
- if (is_mt_event && dev->slot != input_abs_get_val(dev, ABS_MT_SLOT)) {
- input_abs_set_val(dev, ABS_MT_SLOT, dev->slot);
- input_pass_event(dev, EV_ABS, ABS_MT_SLOT, dev->slot);
+ if (is_mt_event && mt && mt->slot != input_abs_get_val(dev, ABS_MT_SLOT)) {
+ input_abs_set_val(dev, ABS_MT_SLOT, mt->slot);
+ return INPUT_PASS_TO_HANDLERS | INPUT_SLOT;
}
return INPUT_PASS_TO_HANDLERS;
}
-static void input_handle_event(struct input_dev *dev,
- unsigned int type, unsigned int code, int value)
+static int input_get_disposition(struct input_dev *dev,
+ unsigned int type, unsigned int code, int value)
{
int disposition = INPUT_IGNORE_EVENT;
@@ -227,37 +269,34 @@
break;
case SYN_REPORT:
- if (!dev->sync) {
- dev->sync = true;
- disposition = INPUT_PASS_TO_HANDLERS;
- }
+ disposition = INPUT_PASS_TO_HANDLERS | INPUT_FLUSH;
break;
case SYN_MT_REPORT:
- dev->sync = false;
disposition = INPUT_PASS_TO_HANDLERS;
break;
}
break;
case EV_KEY:
- if (is_event_supported(code, dev->keybit, KEY_MAX) &&
- !!test_bit(code, dev->key) != value) {
+ if (is_event_supported(code, dev->keybit, KEY_MAX)) {
- if (value != 2) {
- __change_bit(code, dev->key);
- if (value)
- input_start_autorepeat(dev, code);
- else
- input_stop_autorepeat(dev);
+ /* auto-repeat bypasses state updates */
+ if (value == 2) {
+ disposition = INPUT_PASS_TO_HANDLERS;
+ break;
}
- disposition = INPUT_PASS_TO_HANDLERS;
+ if (!!test_bit(code, dev->key) != !!value) {
+
+ __change_bit(code, dev->key);
+ disposition = INPUT_PASS_TO_HANDLERS;
+ }
}
break;
case EV_SW:
if (is_event_supported(code, dev->swbit, SW_MAX) &&
- !!test_bit(code, dev->sw) != value) {
+ !!test_bit(code, dev->sw) != !!value) {
__change_bit(code, dev->sw);
disposition = INPUT_PASS_TO_HANDLERS;
@@ -284,7 +323,7 @@
case EV_LED:
if (is_event_supported(code, dev->ledbit, LED_MAX) &&
- !!test_bit(code, dev->led) != value) {
+ !!test_bit(code, dev->led) != !!value) {
__change_bit(code, dev->led);
disposition = INPUT_PASS_TO_ALL;
@@ -317,14 +356,48 @@
break;
}
- if (disposition != INPUT_IGNORE_EVENT && type != EV_SYN)
- dev->sync = false;
+ return disposition;
+}
+
+static void input_handle_event(struct input_dev *dev,
+ unsigned int type, unsigned int code, int value)
+{
+ int disposition;
+
+ disposition = input_get_disposition(dev, type, code, value);
if ((disposition & INPUT_PASS_TO_DEVICE) && dev->event)
dev->event(dev, type, code, value);
- if (disposition & INPUT_PASS_TO_HANDLERS)
- input_pass_event(dev, type, code, value);
+ if (!dev->vals)
+ return;
+
+ if (disposition & INPUT_PASS_TO_HANDLERS) {
+ struct input_value *v;
+
+ if (disposition & INPUT_SLOT) {
+ v = &dev->vals[dev->num_vals++];
+ v->type = EV_ABS;
+ v->code = ABS_MT_SLOT;
+ v->value = dev->mt->slot;
+ }
+
+ v = &dev->vals[dev->num_vals++];
+ v->type = type;
+ v->code = code;
+ v->value = value;
+ }
+
+ if (disposition & INPUT_FLUSH) {
+ if (dev->num_vals >= 2)
+ input_pass_values(dev, dev->vals, dev->num_vals);
+ dev->num_vals = 0;
+ } else if (dev->num_vals >= dev->max_vals - 2) {
+ dev->vals[dev->num_vals++] = input_value_sync;
+ input_pass_values(dev, dev->vals, dev->num_vals);
+ dev->num_vals = 0;
+ }
+
}
/**
@@ -352,7 +425,6 @@
if (is_event_supported(type, dev->evbit, EV_MAX)) {
spin_lock_irqsave(&dev->event_lock, flags);
- add_input_randomness(type, code, value);
input_handle_event(dev, type, code, value);
spin_unlock_irqrestore(&dev->event_lock, flags);
}
@@ -831,10 +903,12 @@
if (test_bit(EV_KEY, dev->evbit) &&
!is_event_supported(old_keycode, dev->keybit, KEY_MAX) &&
__test_and_clear_bit(old_keycode, dev->key)) {
+ struct input_value vals[] = {
+ { EV_KEY, old_keycode, 0 },
+ input_value_sync
+ };
- input_pass_event(dev, EV_KEY, old_keycode, 0);
- if (dev->sync)
- input_pass_event(dev, EV_SYN, SYN_REPORT, 1);
+ input_pass_values(dev, vals, ARRAY_SIZE(vals));
}
out:
@@ -1416,6 +1490,7 @@
input_ff_destroy(dev);
input_mt_destroy_slots(dev);
kfree(dev->absinfo);
+ kfree(dev->vals);
kfree(dev);
module_put(THIS_MODULE);
@@ -1751,8 +1826,8 @@
int i;
unsigned int events;
- if (dev->mtsize) {
- mt_slots = dev->mtsize;
+ if (dev->mt) {
+ mt_slots = dev->mt->num_slots;
} else if (test_bit(ABS_MT_TRACKING_ID, dev->absbit)) {
mt_slots = dev->absinfo[ABS_MT_TRACKING_ID].maximum -
dev->absinfo[ABS_MT_TRACKING_ID].minimum + 1,
@@ -1778,6 +1853,9 @@
if (test_bit(i, dev->relbit))
events++;
+ /* Make room for KEY and MSC events */
+ events += 7;
+
return events;
}
@@ -1816,6 +1894,7 @@
{
static atomic_t input_no = ATOMIC_INIT(0);
struct input_handler *handler;
+ unsigned int packet_size;
const char *path;
int error;
@@ -1828,9 +1907,14 @@
/* Make sure that bitmasks not mentioned in dev->evbit are clean. */
input_cleanse_bitmasks(dev);
- if (!dev->hint_events_per_packet)
- dev->hint_events_per_packet =
- input_estimate_events_per_packet(dev);
+ packet_size = input_estimate_events_per_packet(dev);
+ if (dev->hint_events_per_packet < packet_size)
+ dev->hint_events_per_packet = packet_size;
+
+ dev->max_vals = max(dev->hint_events_per_packet, packet_size) + 2;
+ dev->vals = kcalloc(dev->max_vals, sizeof(*dev->vals), GFP_KERNEL);
+ if (!dev->vals)
+ return -ENOMEM;
/*
* If delay and period are pre-set by the driver, then autorepeating
diff --git a/drivers/input/keyboard/imx_keypad.c b/drivers/input/keyboard/imx_keypad.c
index ff4c0a8..cdc2526 100644
--- a/drivers/input/keyboard/imx_keypad.c
+++ b/drivers/input/keyboard/imx_keypad.c
@@ -358,6 +358,7 @@
/* Inhibit KDI and KRI interrupts. */
reg_val = readw(keypad->mmio_base + KPSR);
reg_val &= ~(KBD_STAT_KRIE | KBD_STAT_KDIE);
+ reg_val |= KBD_STAT_KPKR | KBD_STAT_KPKD;
writew(reg_val, keypad->mmio_base + KPSR);
/* Colums as open drain and disable all rows */
@@ -515,7 +516,9 @@
input_set_drvdata(input_dev, keypad);
/* Ensure that the keypad will stay dormant until opened */
+ clk_prepare_enable(keypad->clk);
imx_keypad_inhibit(keypad);
+ clk_disable_unprepare(keypad->clk);
error = request_irq(irq, imx_keypad_irq_handler, 0,
pdev->name, keypad);
diff --git a/drivers/input/misc/ab8500-ponkey.c b/drivers/input/misc/ab8500-ponkey.c
index f06231b..84ec691 100644
--- a/drivers/input/misc/ab8500-ponkey.c
+++ b/drivers/input/misc/ab8500-ponkey.c
@@ -74,8 +74,8 @@
ponkey->idev = input;
ponkey->ab8500 = ab8500;
- ponkey->irq_dbf = ab8500_irq_get_virq(ab8500, irq_dbf);
- ponkey->irq_dbr = ab8500_irq_get_virq(ab8500, irq_dbr);
+ ponkey->irq_dbf = irq_dbf;
+ ponkey->irq_dbr = irq_dbr;
input->name = "AB8500 POn(PowerOn) Key";
input->dev.parent = &pdev->dev;
diff --git a/drivers/input/misc/uinput.c b/drivers/input/misc/uinput.c
index 7360568..6b17975 100644
--- a/drivers/input/misc/uinput.c
+++ b/drivers/input/misc/uinput.c
@@ -405,7 +405,7 @@
goto exit;
if (test_bit(ABS_MT_SLOT, dev->absbit)) {
int nslot = input_abs_get_max(dev, ABS_MT_SLOT) + 1;
- input_mt_init_slots(dev, nslot);
+ input_mt_init_slots(dev, nslot, 0);
} else if (test_bit(ABS_MT_POSITION_X, dev->absbit)) {
input_set_events_per_packet(dev, 60);
}
diff --git a/drivers/input/mouse/alps.c b/drivers/input/mouse/alps.c
index 4a1347e..cf5af1f 100644
--- a/drivers/input/mouse/alps.c
+++ b/drivers/input/mouse/alps.c
@@ -1620,7 +1620,7 @@
case ALPS_PROTO_V3:
case ALPS_PROTO_V4:
set_bit(INPUT_PROP_SEMI_MT, dev1->propbit);
- input_mt_init_slots(dev1, 2);
+ input_mt_init_slots(dev1, 2, 0);
input_set_abs_params(dev1, ABS_MT_POSITION_X, 0, ALPS_V3_X_MAX, 0, 0);
input_set_abs_params(dev1, ABS_MT_POSITION_Y, 0, ALPS_V3_Y_MAX, 0, 0);
diff --git a/drivers/input/mouse/bcm5974.c b/drivers/input/mouse/bcm5974.c
index d528c23..3a78f23 100644
--- a/drivers/input/mouse/bcm5974.c
+++ b/drivers/input/mouse/bcm5974.c
@@ -40,6 +40,7 @@
#include <linux/usb/input.h>
#include <linux/hid.h>
#include <linux/mutex.h>
+#include <linux/input/mt.h>
#define USB_VENDOR_ID_APPLE 0x05ac
@@ -183,26 +184,26 @@
__le16 abs_y; /* absolute y coodinate */
__le16 rel_x; /* relative x coodinate */
__le16 rel_y; /* relative y coodinate */
- __le16 size_major; /* finger size, major axis? */
- __le16 size_minor; /* finger size, minor axis? */
+ __le16 tool_major; /* tool area, major axis */
+ __le16 tool_minor; /* tool area, minor axis */
__le16 orientation; /* 16384 when point, else 15 bit angle */
- __le16 force_major; /* trackpad force, major axis? */
- __le16 force_minor; /* trackpad force, minor axis? */
+ __le16 touch_major; /* touch area, major axis */
+ __le16 touch_minor; /* touch area, minor axis */
__le16 unused[3]; /* zeros */
__le16 multi; /* one finger: varies, more fingers: constant */
} __attribute__((packed,aligned(2)));
/* trackpad finger data size, empirically at least ten fingers */
+#define MAX_FINGERS 16
#define SIZEOF_FINGER sizeof(struct tp_finger)
-#define SIZEOF_ALL_FINGERS (16 * SIZEOF_FINGER)
+#define SIZEOF_ALL_FINGERS (MAX_FINGERS * SIZEOF_FINGER)
#define MAX_FINGER_ORIENTATION 16384
/* device-specific parameters */
struct bcm5974_param {
- int dim; /* logical dimension */
- int fuzz; /* logical noise value */
- int devmin; /* device minimum reading */
- int devmax; /* device maximum reading */
+ int snratio; /* signal-to-noise ratio */
+ int min; /* device minimum reading */
+ int max; /* device maximum reading */
};
/* device-specific configuration */
@@ -219,6 +220,7 @@
struct bcm5974_param w; /* finger width limits */
struct bcm5974_param x; /* horizontal limits */
struct bcm5974_param y; /* vertical limits */
+ struct bcm5974_param o; /* orientation limits */
};
/* logical device structure */
@@ -234,23 +236,16 @@
struct bt_data *bt_data; /* button transferred data */
struct urb *tp_urb; /* trackpad usb request block */
u8 *tp_data; /* trackpad transferred data */
- int fingers; /* number of fingers on trackpad */
+ const struct tp_finger *index[MAX_FINGERS]; /* finger index data */
+ struct input_mt_pos pos[MAX_FINGERS]; /* position array */
+ int slots[MAX_FINGERS]; /* slot assignments */
};
-/* logical dimensions */
-#define DIM_PRESSURE 256 /* maximum finger pressure */
-#define DIM_WIDTH 16 /* maximum finger width */
-#define DIM_X 1280 /* maximum trackpad x value */
-#define DIM_Y 800 /* maximum trackpad y value */
-
/* logical signal quality */
#define SN_PRESSURE 45 /* pressure signal-to-noise ratio */
-#define SN_WIDTH 100 /* width signal-to-noise ratio */
+#define SN_WIDTH 25 /* width signal-to-noise ratio */
#define SN_COORD 250 /* coordinate signal-to-noise ratio */
-
-/* pressure thresholds */
-#define PRESSURE_LOW (2 * DIM_PRESSURE / SN_PRESSURE)
-#define PRESSURE_HIGH (3 * PRESSURE_LOW)
+#define SN_ORIENT 10 /* orientation signal-to-noise ratio */
/* device constants */
static const struct bcm5974_config bcm5974_config_table[] = {
@@ -261,10 +256,11 @@
0,
0x84, sizeof(struct bt_data),
0x81, TYPE1, FINGER_TYPE1, FINGER_TYPE1 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 256 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4824, 5342 },
- { DIM_Y, DIM_Y / SN_COORD, -172, 5820 }
+ { SN_PRESSURE, 0, 256 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4824, 5342 },
+ { SN_COORD, -172, 5820 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING2_ANSI,
@@ -273,10 +269,11 @@
0,
0x84, sizeof(struct bt_data),
0x81, TYPE1, FINGER_TYPE1, FINGER_TYPE1 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 256 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4824, 4824 },
- { DIM_Y, DIM_Y / SN_COORD, -172, 4290 }
+ { SN_PRESSURE, 0, 256 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4824, 4824 },
+ { SN_COORD, -172, 4290 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING3_ANSI,
@@ -285,10 +282,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4460, 5166 },
- { DIM_Y, DIM_Y / SN_COORD, -75, 6700 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4460, 5166 },
+ { SN_COORD, -75, 6700 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING4_ANSI,
@@ -297,10 +295,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4620, 5140 },
- { DIM_Y, DIM_Y / SN_COORD, -150, 6600 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4620, 5140 },
+ { SN_COORD, -150, 6600 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING4A_ANSI,
@@ -309,10 +308,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4616, 5112 },
- { DIM_Y, DIM_Y / SN_COORD, -142, 5234 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4616, 5112 },
+ { SN_COORD, -142, 5234 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING5_ANSI,
@@ -321,10 +321,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4415, 5050 },
- { DIM_Y, DIM_Y / SN_COORD, -55, 6680 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4415, 5050 },
+ { SN_COORD, -55, 6680 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING6_ANSI,
@@ -333,10 +334,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4620, 5140 },
- { DIM_Y, DIM_Y / SN_COORD, -150, 6600 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4620, 5140 },
+ { SN_COORD, -150, 6600 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING5A_ANSI,
@@ -345,10 +347,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4750, 5280 },
- { DIM_Y, DIM_Y / SN_COORD, -150, 6730 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4750, 5280 },
+ { SN_COORD, -150, 6730 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING6A_ANSI,
@@ -357,10 +360,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4620, 5140 },
- { DIM_Y, DIM_Y / SN_COORD, -150, 6600 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4620, 5140 },
+ { SN_COORD, -150, 6600 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{
USB_DEVICE_ID_APPLE_WELLSPRING7_ANSI,
@@ -369,10 +373,11 @@
HAS_INTEGRATED_BUTTON,
0x84, sizeof(struct bt_data),
0x81, TYPE2, FINGER_TYPE2, FINGER_TYPE2 + SIZEOF_ALL_FINGERS,
- { DIM_PRESSURE, DIM_PRESSURE / SN_PRESSURE, 0, 300 },
- { DIM_WIDTH, DIM_WIDTH / SN_WIDTH, 0, 2048 },
- { DIM_X, DIM_X / SN_COORD, -4750, 5280 },
- { DIM_Y, DIM_Y / SN_COORD, -150, 6730 }
+ { SN_PRESSURE, 0, 300 },
+ { SN_WIDTH, 0, 2048 },
+ { SN_COORD, -4750, 5280 },
+ { SN_COORD, -150, 6730 },
+ { SN_ORIENT, -MAX_FINGER_ORIENTATION, MAX_FINGER_ORIENTATION }
},
{}
};
@@ -396,18 +401,11 @@
return (signed short)le16_to_cpu(x);
}
-/* scale device data to logical dimensions (asserts devmin < devmax) */
-static inline int int2scale(const struct bcm5974_param *p, int x)
+static void set_abs(struct input_dev *input, unsigned int code,
+ const struct bcm5974_param *p)
{
- return x * p->dim / (p->devmax - p->devmin);
-}
-
-/* all logical value ranges are [0,dim). */
-static inline int int2bound(const struct bcm5974_param *p, int x)
-{
- int s = int2scale(p, x);
-
- return clamp_val(s, 0, p->dim - 1);
+ int fuzz = p->snratio ? (p->max - p->min) / p->snratio : 0;
+ input_set_abs_params(input, code, p->min, p->max, fuzz, 0);
}
/* setup which logical events to report */
@@ -416,48 +414,30 @@
{
__set_bit(EV_ABS, input_dev->evbit);
- input_set_abs_params(input_dev, ABS_PRESSURE,
- 0, cfg->p.dim, cfg->p.fuzz, 0);
- input_set_abs_params(input_dev, ABS_TOOL_WIDTH,
- 0, cfg->w.dim, cfg->w.fuzz, 0);
- input_set_abs_params(input_dev, ABS_X,
- 0, cfg->x.dim, cfg->x.fuzz, 0);
- input_set_abs_params(input_dev, ABS_Y,
- 0, cfg->y.dim, cfg->y.fuzz, 0);
+ /* for synaptics only */
+ input_set_abs_params(input_dev, ABS_PRESSURE, 0, 256, 5, 0);
+ input_set_abs_params(input_dev, ABS_TOOL_WIDTH, 0, 16, 0, 0);
/* finger touch area */
- input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
- cfg->w.devmin, cfg->w.devmax, 0, 0);
- input_set_abs_params(input_dev, ABS_MT_TOUCH_MINOR,
- cfg->w.devmin, cfg->w.devmax, 0, 0);
+ set_abs(input_dev, ABS_MT_TOUCH_MAJOR, &cfg->w);
+ set_abs(input_dev, ABS_MT_TOUCH_MINOR, &cfg->w);
/* finger approach area */
- input_set_abs_params(input_dev, ABS_MT_WIDTH_MAJOR,
- cfg->w.devmin, cfg->w.devmax, 0, 0);
- input_set_abs_params(input_dev, ABS_MT_WIDTH_MINOR,
- cfg->w.devmin, cfg->w.devmax, 0, 0);
+ set_abs(input_dev, ABS_MT_WIDTH_MAJOR, &cfg->w);
+ set_abs(input_dev, ABS_MT_WIDTH_MINOR, &cfg->w);
/* finger orientation */
- input_set_abs_params(input_dev, ABS_MT_ORIENTATION,
- -MAX_FINGER_ORIENTATION,
- MAX_FINGER_ORIENTATION, 0, 0);
+ set_abs(input_dev, ABS_MT_ORIENTATION, &cfg->o);
/* finger position */
- input_set_abs_params(input_dev, ABS_MT_POSITION_X,
- cfg->x.devmin, cfg->x.devmax, 0, 0);
- input_set_abs_params(input_dev, ABS_MT_POSITION_Y,
- cfg->y.devmin, cfg->y.devmax, 0, 0);
+ set_abs(input_dev, ABS_MT_POSITION_X, &cfg->x);
+ set_abs(input_dev, ABS_MT_POSITION_Y, &cfg->y);
__set_bit(EV_KEY, input_dev->evbit);
- __set_bit(BTN_TOUCH, input_dev->keybit);
- __set_bit(BTN_TOOL_FINGER, input_dev->keybit);
- __set_bit(BTN_TOOL_DOUBLETAP, input_dev->keybit);
- __set_bit(BTN_TOOL_TRIPLETAP, input_dev->keybit);
- __set_bit(BTN_TOOL_QUADTAP, input_dev->keybit);
__set_bit(BTN_LEFT, input_dev->keybit);
- __set_bit(INPUT_PROP_POINTER, input_dev->propbit);
if (cfg->caps & HAS_INTEGRATED_BUTTON)
__set_bit(INPUT_PROP_BUTTONPAD, input_dev->propbit);
- input_set_events_per_packet(input_dev, 60);
+ input_mt_init_slots(input_dev, MAX_FINGERS,
+ INPUT_MT_POINTER | INPUT_MT_DROP_UNUSED | INPUT_MT_TRACK);
}
/* report button data as logical button state */
@@ -477,24 +457,44 @@
return 0;
}
-static void report_finger_data(struct input_dev *input,
- const struct bcm5974_config *cfg,
+static void report_finger_data(struct input_dev *input, int slot,
+ const struct input_mt_pos *pos,
const struct tp_finger *f)
{
+ input_mt_slot(input, slot);
+ input_mt_report_slot_state(input, MT_TOOL_FINGER, true);
+
input_report_abs(input, ABS_MT_TOUCH_MAJOR,
- raw2int(f->force_major) << 1);
+ raw2int(f->touch_major) << 1);
input_report_abs(input, ABS_MT_TOUCH_MINOR,
- raw2int(f->force_minor) << 1);
+ raw2int(f->touch_minor) << 1);
input_report_abs(input, ABS_MT_WIDTH_MAJOR,
- raw2int(f->size_major) << 1);
+ raw2int(f->tool_major) << 1);
input_report_abs(input, ABS_MT_WIDTH_MINOR,
- raw2int(f->size_minor) << 1);
+ raw2int(f->tool_minor) << 1);
input_report_abs(input, ABS_MT_ORIENTATION,
MAX_FINGER_ORIENTATION - raw2int(f->orientation));
- input_report_abs(input, ABS_MT_POSITION_X, raw2int(f->abs_x));
- input_report_abs(input, ABS_MT_POSITION_Y,
- cfg->y.devmin + cfg->y.devmax - raw2int(f->abs_y));
- input_mt_sync(input);
+ input_report_abs(input, ABS_MT_POSITION_X, pos->x);
+ input_report_abs(input, ABS_MT_POSITION_Y, pos->y);
+}
+
+static void report_synaptics_data(struct input_dev *input,
+ const struct bcm5974_config *cfg,
+ const struct tp_finger *f, int raw_n)
+{
+ int abs_p = 0, abs_w = 0;
+
+ if (raw_n) {
+ int p = raw2int(f->touch_major);
+ int w = raw2int(f->tool_major);
+ if (p > 0 && raw2int(f->origin)) {
+ abs_p = clamp_val(256 * p / cfg->p.max, 0, 255);
+ abs_w = clamp_val(16 * w / cfg->w.max, 0, 15);
+ }
+ }
+
+ input_report_abs(input, ABS_PRESSURE, abs_p);
+ input_report_abs(input, ABS_TOOL_WIDTH, abs_w);
}
/* report trackpad data as logical trackpad state */
@@ -503,9 +503,7 @@
const struct bcm5974_config *c = &dev->cfg;
const struct tp_finger *f;
struct input_dev *input = dev->input;
- int raw_p, raw_w, raw_x, raw_y, raw_n, i;
- int ptest, origin, ibt = 0, nmin = 0, nmax = 0;
- int abs_p = 0, abs_w = 0, abs_x = 0, abs_y = 0;
+ int raw_n, i, n = 0;
if (size < c->tp_offset || (size - c->tp_offset) % SIZEOF_FINGER != 0)
return -EIO;
@@ -514,76 +512,29 @@
f = (const struct tp_finger *)(dev->tp_data + c->tp_offset);
raw_n = (size - c->tp_offset) / SIZEOF_FINGER;
- /* always track the first finger; when detached, start over */
- if (raw_n) {
-
- /* report raw trackpad data */
- for (i = 0; i < raw_n; i++)
- report_finger_data(input, c, &f[i]);
-
- raw_p = raw2int(f->force_major);
- raw_w = raw2int(f->size_major);
- raw_x = raw2int(f->abs_x);
- raw_y = raw2int(f->abs_y);
-
- dprintk(9,
- "bcm5974: "
- "raw: p: %+05d w: %+05d x: %+05d y: %+05d n: %d\n",
- raw_p, raw_w, raw_x, raw_y, raw_n);
-
- ptest = int2bound(&c->p, raw_p);
- origin = raw2int(f->origin);
-
- /* while tracking finger still valid, count all fingers */
- if (ptest > PRESSURE_LOW && origin) {
- abs_p = ptest;
- abs_w = int2bound(&c->w, raw_w);
- abs_x = int2bound(&c->x, raw_x - c->x.devmin);
- abs_y = int2bound(&c->y, c->y.devmax - raw_y);
- while (raw_n--) {
- ptest = int2bound(&c->p,
- raw2int(f->force_major));
- if (ptest > PRESSURE_LOW)
- nmax++;
- if (ptest > PRESSURE_HIGH)
- nmin++;
- f++;
- }
- }
+ for (i = 0; i < raw_n; i++) {
+ if (raw2int(f[i].touch_major) == 0)
+ continue;
+ dev->pos[n].x = raw2int(f[i].abs_x);
+ dev->pos[n].y = c->y.min + c->y.max - raw2int(f[i].abs_y);
+ dev->index[n++] = &f[i];
}
- /* set the integrated button if applicable */
- if (c->tp_type == TYPE2)
- ibt = raw2int(dev->tp_data[BUTTON_TYPE2]);
+ input_mt_assign_slots(input, dev->slots, dev->pos, n);
- if (dev->fingers < nmin)
- dev->fingers = nmin;
- if (dev->fingers > nmax)
- dev->fingers = nmax;
+ for (i = 0; i < n; i++)
+ report_finger_data(input, dev->slots[i],
+ &dev->pos[i], dev->index[i]);
- input_report_key(input, BTN_TOUCH, dev->fingers > 0);
- input_report_key(input, BTN_TOOL_FINGER, dev->fingers == 1);
- input_report_key(input, BTN_TOOL_DOUBLETAP, dev->fingers == 2);
- input_report_key(input, BTN_TOOL_TRIPLETAP, dev->fingers == 3);
- input_report_key(input, BTN_TOOL_QUADTAP, dev->fingers > 3);
+ input_mt_sync_frame(input);
- input_report_abs(input, ABS_PRESSURE, abs_p);
- input_report_abs(input, ABS_TOOL_WIDTH, abs_w);
-
- if (abs_p) {
- input_report_abs(input, ABS_X, abs_x);
- input_report_abs(input, ABS_Y, abs_y);
-
- dprintk(8,
- "bcm5974: abs: p: %+05d w: %+05d x: %+05d y: %+05d "
- "nmin: %d nmax: %d n: %d ibt: %d\n", abs_p, abs_w,
- abs_x, abs_y, nmin, nmax, dev->fingers, ibt);
-
- }
+ report_synaptics_data(input, c, f, raw_n);
/* type 2 reports button events via ibt only */
- if (c->tp_type == TYPE2)
+ if (c->tp_type == TYPE2) {
+ int ibt = raw2int(dev->tp_data[BUTTON_TYPE2]);
input_report_key(input, BTN_LEFT, ibt);
+ }
input_sync(input);
@@ -742,9 +693,11 @@
goto err_out;
}
- error = usb_submit_urb(dev->bt_urb, GFP_KERNEL);
- if (error)
- goto err_reset_mode;
+ if (dev->bt_urb) {
+ error = usb_submit_urb(dev->bt_urb, GFP_KERNEL);
+ if (error)
+ goto err_reset_mode;
+ }
error = usb_submit_urb(dev->tp_urb, GFP_KERNEL);
if (error)
@@ -868,19 +821,23 @@
mutex_init(&dev->pm_mutex);
/* setup urbs */
- dev->bt_urb = usb_alloc_urb(0, GFP_KERNEL);
- if (!dev->bt_urb)
- goto err_free_devs;
+ if (cfg->tp_type == TYPE1) {
+ dev->bt_urb = usb_alloc_urb(0, GFP_KERNEL);
+ if (!dev->bt_urb)
+ goto err_free_devs;
+ }
dev->tp_urb = usb_alloc_urb(0, GFP_KERNEL);
if (!dev->tp_urb)
goto err_free_bt_urb;
- dev->bt_data = usb_alloc_coherent(dev->udev,
+ if (dev->bt_urb) {
+ dev->bt_data = usb_alloc_coherent(dev->udev,
dev->cfg.bt_datalen, GFP_KERNEL,
&dev->bt_urb->transfer_dma);
- if (!dev->bt_data)
- goto err_free_urb;
+ if (!dev->bt_data)
+ goto err_free_urb;
+ }
dev->tp_data = usb_alloc_coherent(dev->udev,
dev->cfg.tp_datalen, GFP_KERNEL,
@@ -888,10 +845,11 @@
if (!dev->tp_data)
goto err_free_bt_buffer;
- usb_fill_int_urb(dev->bt_urb, udev,
- usb_rcvintpipe(udev, cfg->bt_ep),
- dev->bt_data, dev->cfg.bt_datalen,
- bcm5974_irq_button, dev, 1);
+ if (dev->bt_urb)
+ usb_fill_int_urb(dev->bt_urb, udev,
+ usb_rcvintpipe(udev, cfg->bt_ep),
+ dev->bt_data, dev->cfg.bt_datalen,
+ bcm5974_irq_button, dev, 1);
usb_fill_int_urb(dev->tp_urb, udev,
usb_rcvintpipe(udev, cfg->tp_ep),
@@ -929,8 +887,9 @@
usb_free_coherent(dev->udev, dev->cfg.tp_datalen,
dev->tp_data, dev->tp_urb->transfer_dma);
err_free_bt_buffer:
- usb_free_coherent(dev->udev, dev->cfg.bt_datalen,
- dev->bt_data, dev->bt_urb->transfer_dma);
+ if (dev->bt_urb)
+ usb_free_coherent(dev->udev, dev->cfg.bt_datalen,
+ dev->bt_data, dev->bt_urb->transfer_dma);
err_free_urb:
usb_free_urb(dev->tp_urb);
err_free_bt_urb:
@@ -951,8 +910,9 @@
input_unregister_device(dev->input);
usb_free_coherent(dev->udev, dev->cfg.tp_datalen,
dev->tp_data, dev->tp_urb->transfer_dma);
- usb_free_coherent(dev->udev, dev->cfg.bt_datalen,
- dev->bt_data, dev->bt_urb->transfer_dma);
+ if (dev->bt_urb)
+ usb_free_coherent(dev->udev, dev->cfg.bt_datalen,
+ dev->bt_data, dev->bt_urb->transfer_dma);
usb_free_urb(dev->tp_urb);
usb_free_urb(dev->bt_urb);
kfree(dev);
diff --git a/drivers/input/mouse/elantech.c b/drivers/input/mouse/elantech.c
index 4790110..1e8e42f 100644
--- a/drivers/input/mouse/elantech.c
+++ b/drivers/input/mouse/elantech.c
@@ -1004,7 +1004,7 @@
input_set_abs_params(dev, ABS_TOOL_WIDTH, ETP_WMIN_V2,
ETP_WMAX_V2, 0, 0);
}
- input_mt_init_slots(dev, 2);
+ input_mt_init_slots(dev, 2, 0);
input_set_abs_params(dev, ABS_MT_POSITION_X, x_min, x_max, 0, 0);
input_set_abs_params(dev, ABS_MT_POSITION_Y, y_min, y_max, 0, 0);
break;
@@ -1035,7 +1035,7 @@
input_set_abs_params(dev, ABS_TOOL_WIDTH, ETP_WMIN_V2,
ETP_WMAX_V2, 0, 0);
/* Multitouch capable pad, up to 5 fingers. */
- input_mt_init_slots(dev, ETP_MAX_FINGERS);
+ input_mt_init_slots(dev, ETP_MAX_FINGERS, 0);
input_set_abs_params(dev, ABS_MT_POSITION_X, x_min, x_max, 0, 0);
input_set_abs_params(dev, ABS_MT_POSITION_Y, y_min, y_max, 0, 0);
input_abs_set_res(dev, ABS_MT_POSITION_X, x_res);
diff --git a/drivers/input/mouse/sentelic.c b/drivers/input/mouse/sentelic.c
index 3f5649f..e582922 100644
--- a/drivers/input/mouse/sentelic.c
+++ b/drivers/input/mouse/sentelic.c
@@ -721,6 +721,17 @@
switch (psmouse->packet[0] >> FSP_PKT_TYPE_SHIFT) {
case FSP_PKT_TYPE_ABS:
+
+ if ((packet[0] == 0x48 || packet[0] == 0x49) &&
+ packet[1] == 0 && packet[2] == 0) {
+ /*
+ * Ignore coordinate noise when finger leaving the
+ * surface, otherwise cursor may jump to upper-left
+ * corner.
+ */
+ packet[3] &= 0xf0;
+ }
+
abs_x = GET_ABS_X(packet);
abs_y = GET_ABS_Y(packet);
@@ -960,7 +971,7 @@
input_set_abs_params(dev, ABS_X, 0, abs_x, 0, 0);
input_set_abs_params(dev, ABS_Y, 0, abs_y, 0, 0);
- input_mt_init_slots(dev, 2);
+ input_mt_init_slots(dev, 2, 0);
input_set_abs_params(dev, ABS_MT_POSITION_X, 0, abs_x, 0, 0);
input_set_abs_params(dev, ABS_MT_POSITION_Y, 0, abs_y, 0, 0);
}
diff --git a/drivers/input/mouse/synaptics.c b/drivers/input/mouse/synaptics.c
index 14eaece..37033ad 100644
--- a/drivers/input/mouse/synaptics.c
+++ b/drivers/input/mouse/synaptics.c
@@ -1232,7 +1232,7 @@
input_set_abs_params(dev, ABS_PRESSURE, 0, 255, 0, 0);
if (SYN_CAP_IMAGE_SENSOR(priv->ext_cap_0c)) {
- input_mt_init_slots(dev, 2);
+ input_mt_init_slots(dev, 2, 0);
set_abs_position_params(dev, priv, ABS_MT_POSITION_X,
ABS_MT_POSITION_Y);
/* Image sensors can report per-contact pressure */
@@ -1244,7 +1244,7 @@
} else if (SYN_CAP_ADV_GESTURE(priv->ext_cap_0c)) {
/* Non-image sensors with AGM use semi-mt */
__set_bit(INPUT_PROP_SEMI_MT, dev->propbit);
- input_mt_init_slots(dev, 2);
+ input_mt_init_slots(dev, 2, 0);
set_abs_position_params(dev, priv, ABS_MT_POSITION_X,
ABS_MT_POSITION_Y);
}
diff --git a/drivers/input/serio/ambakmi.c b/drivers/input/serio/ambakmi.c
index 2ffd110..2e77246 100644
--- a/drivers/input/serio/ambakmi.c
+++ b/drivers/input/serio/ambakmi.c
@@ -72,7 +72,7 @@
unsigned int divisor;
int ret;
- ret = clk_enable(kmi->clk);
+ ret = clk_prepare_enable(kmi->clk);
if (ret)
goto out;
@@ -92,7 +92,7 @@
return 0;
clk_disable:
- clk_disable(kmi->clk);
+ clk_disable_unprepare(kmi->clk);
out:
return ret;
}
@@ -104,7 +104,7 @@
writeb(0, KMICR);
free_irq(kmi->irq, kmi);
- clk_disable(kmi->clk);
+ clk_disable_unprepare(kmi->clk);
}
static int __devinit amba_kmi_probe(struct amba_device *dev,
diff --git a/drivers/input/serio/i8042-x86ia64io.h b/drivers/input/serio/i8042-x86ia64io.h
index 5ec774d..d6cc77a 100644
--- a/drivers/input/serio/i8042-x86ia64io.h
+++ b/drivers/input/serio/i8042-x86ia64io.h
@@ -177,6 +177,20 @@
},
},
{
+ /* Gigabyte T1005 - defines wrong chassis type ("Other") */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "GIGABYTE"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "T1005"),
+ },
+ },
+ {
+ /* Gigabyte T1005M/P - defines wrong chassis type ("Other") */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "GIGABYTE"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "T1005M/P"),
+ },
+ },
+ {
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
DMI_MATCH(DMI_PRODUCT_NAME, "HP Pavilion dv9700"),
@@ -321,6 +335,12 @@
},
{
.matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "TOSHIBA"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "SATELLITE C850D"),
+ },
+ },
+ {
+ .matches = {
DMI_MATCH(DMI_SYS_VENDOR, "ALIENWARE"),
DMI_MATCH(DMI_PRODUCT_NAME, "Sentia"),
},
diff --git a/drivers/input/tablet/wacom_wac.c b/drivers/input/tablet/wacom_wac.c
index 0020419..2a81ce3 100644
--- a/drivers/input/tablet/wacom_wac.c
+++ b/drivers/input/tablet/wacom_wac.c
@@ -1530,7 +1530,7 @@
__set_bit(BTN_TOOL_TRIPLETAP, input_dev->keybit);
__set_bit(BTN_TOOL_QUADTAP, input_dev->keybit);
- input_mt_init_slots(input_dev, features->touch_max);
+ input_mt_init_slots(input_dev, features->touch_max, 0);
input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
0, 255, 0, 0);
@@ -1575,7 +1575,7 @@
case TABLETPC2FG:
if (features->device_type == BTN_TOOL_FINGER) {
- input_mt_init_slots(input_dev, features->touch_max);
+ input_mt_init_slots(input_dev, features->touch_max, 0);
input_set_abs_params(input_dev, ABS_MT_TOOL_TYPE,
0, MT_TOOL_MAX, 0, 0);
input_set_abs_params(input_dev, ABS_MT_POSITION_X,
@@ -1631,7 +1631,7 @@
__set_bit(BTN_TOOL_FINGER, input_dev->keybit);
__set_bit(BTN_TOOL_DOUBLETAP, input_dev->keybit);
- input_mt_init_slots(input_dev, features->touch_max);
+ input_mt_init_slots(input_dev, features->touch_max, 0);
if (features->pktlen == WACOM_PKGLEN_BBTOUCH3) {
__set_bit(BTN_TOOL_TRIPLETAP,
@@ -1848,7 +1848,10 @@
{ "Wacom Intuos5 M", WACOM_PKGLEN_INTUOS, 44704, 27940, 2047,
63, INTUOS5, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
static const struct wacom_features wacom_features_0xF4 =
- { "Wacom Cintiq 24HD", WACOM_PKGLEN_INTUOS, 104480, 65600, 2047,
+ { "Wacom Cintiq 24HD", WACOM_PKGLEN_INTUOS, 104480, 65600, 2047,
+ 63, WACOM_24HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
+static const struct wacom_features wacom_features_0xF8 =
+ { "Wacom Cintiq 24HD touch", WACOM_PKGLEN_INTUOS, 104480, 65600, 2047,
63, WACOM_24HD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
static const struct wacom_features wacom_features_0x3F =
{ "Wacom Cintiq 21UX", WACOM_PKGLEN_INTUOS, 87200, 65600, 1023,
@@ -2091,6 +2094,7 @@
{ USB_DEVICE_WACOM(0xEF) },
{ USB_DEVICE_WACOM(0x47) },
{ USB_DEVICE_WACOM(0xF4) },
+ { USB_DEVICE_WACOM(0xF8) },
{ USB_DEVICE_WACOM(0xFA) },
{ USB_DEVICE_LENOVO(0x6004) },
{ }
diff --git a/drivers/input/touchscreen/atmel_mxt_ts.c b/drivers/input/touchscreen/atmel_mxt_ts.c
index 4623cc6..e92615d 100644
--- a/drivers/input/touchscreen/atmel_mxt_ts.c
+++ b/drivers/input/touchscreen/atmel_mxt_ts.c
@@ -1152,7 +1152,7 @@
/* For multi touch */
num_mt_slots = data->T9_reportid_max - data->T9_reportid_min + 1;
- error = input_mt_init_slots(input_dev, num_mt_slots);
+ error = input_mt_init_slots(input_dev, num_mt_slots, 0);
if (error)
goto err_free_object;
input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
diff --git a/drivers/input/touchscreen/cyttsp_core.c b/drivers/input/touchscreen/cyttsp_core.c
index f030d9e..8e60437 100644
--- a/drivers/input/touchscreen/cyttsp_core.c
+++ b/drivers/input/touchscreen/cyttsp_core.c
@@ -571,7 +571,7 @@
input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
0, CY_MAXZ, 0, 0);
- input_mt_init_slots(input_dev, CY_MAX_ID);
+ input_mt_init_slots(input_dev, CY_MAX_ID, 0);
error = request_threaded_irq(ts->irq, NULL, cyttsp_irq,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
diff --git a/drivers/input/touchscreen/edt-ft5x06.c b/drivers/input/touchscreen/edt-ft5x06.c
index 9afc777..099d144a 100644
--- a/drivers/input/touchscreen/edt-ft5x06.c
+++ b/drivers/input/touchscreen/edt-ft5x06.c
@@ -566,9 +566,12 @@
}
read = min_t(size_t, count, tsdata->raw_bufsize - *off);
- error = copy_to_user(buf, tsdata->raw_buffer + *off, read);
- if (!error)
- *off += read;
+ if (copy_to_user(buf, tsdata->raw_buffer + *off, read)) {
+ error = -EFAULT;
+ goto out;
+ }
+
+ *off += read;
out:
mutex_unlock(&tsdata->mutex);
return error ?: read;
@@ -602,6 +605,7 @@
{
if (tsdata->debug_dir)
debugfs_remove_recursive(tsdata->debug_dir);
+ kfree(tsdata->raw_buffer);
}
#else
@@ -778,7 +782,7 @@
0, tsdata->num_x * 64 - 1, 0, 0);
input_set_abs_params(input, ABS_MT_POSITION_Y,
0, tsdata->num_y * 64 - 1, 0, 0);
- error = input_mt_init_slots(input, MAX_SUPPORT_POINTS);
+ error = input_mt_init_slots(input, MAX_SUPPORT_POINTS, 0);
if (error) {
dev_err(&client->dev, "Unable to init MT slots.\n");
goto err_free_mem;
@@ -843,7 +847,6 @@
if (gpio_is_valid(pdata->reset_pin))
gpio_free(pdata->reset_pin);
- kfree(tsdata->raw_buffer);
kfree(tsdata);
return 0;
diff --git a/drivers/input/touchscreen/egalax_ts.c b/drivers/input/touchscreen/egalax_ts.c
index 70524dd..c1e3460 100644
--- a/drivers/input/touchscreen/egalax_ts.c
+++ b/drivers/input/touchscreen/egalax_ts.c
@@ -204,7 +204,7 @@
ABS_MT_POSITION_X, 0, EGALAX_MAX_X, 0, 0);
input_set_abs_params(input_dev,
ABS_MT_POSITION_X, 0, EGALAX_MAX_Y, 0, 0);
- input_mt_init_slots(input_dev, MAX_SUPPORT_POINTS);
+ input_mt_init_slots(input_dev, MAX_SUPPORT_POINTS, 0);
input_set_drvdata(input_dev, ts);
diff --git a/drivers/input/touchscreen/ili210x.c b/drivers/input/touchscreen/ili210x.c
index c004417..4ac6976 100644
--- a/drivers/input/touchscreen/ili210x.c
+++ b/drivers/input/touchscreen/ili210x.c
@@ -252,7 +252,7 @@
input_set_abs_params(input, ABS_Y, 0, ymax, 0, 0);
/* Multi touch */
- input_mt_init_slots(input, MAX_TOUCHES);
+ input_mt_init_slots(input, MAX_TOUCHES, 0);
input_set_abs_params(input, ABS_MT_POSITION_X, 0, xmax, 0, 0);
input_set_abs_params(input, ABS_MT_POSITION_Y, 0, ymax, 0, 0);
diff --git a/drivers/input/touchscreen/mms114.c b/drivers/input/touchscreen/mms114.c
index 49c44bb..560cf09 100644
--- a/drivers/input/touchscreen/mms114.c
+++ b/drivers/input/touchscreen/mms114.c
@@ -404,7 +404,7 @@
input_set_abs_params(input_dev, ABS_Y, 0, data->pdata->y_size, 0, 0);
/* For multi touch */
- input_mt_init_slots(input_dev, MMS114_MAX_TOUCH);
+ input_mt_init_slots(input_dev, MMS114_MAX_TOUCH, 0);
input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
0, MMS114_MAX_AREA, 0, 0);
input_set_abs_params(input_dev, ABS_MT_POSITION_X,
diff --git a/drivers/input/touchscreen/penmount.c b/drivers/input/touchscreen/penmount.c
index 4ccde45..b49f0b8 100644
--- a/drivers/input/touchscreen/penmount.c
+++ b/drivers/input/touchscreen/penmount.c
@@ -264,7 +264,7 @@
input_set_abs_params(pm->dev, ABS_Y, 0, max_y, 0, 0);
if (pm->maxcontacts > 1) {
- input_mt_init_slots(pm->dev, pm->maxcontacts);
+ input_mt_init_slots(pm->dev, pm->maxcontacts, 0);
input_set_abs_params(pm->dev,
ABS_MT_POSITION_X, 0, max_x, 0, 0);
input_set_abs_params(pm->dev,
diff --git a/drivers/input/touchscreen/usbtouchscreen.c b/drivers/input/touchscreen/usbtouchscreen.c
index e32709e..721fdb3 100644
--- a/drivers/input/touchscreen/usbtouchscreen.c
+++ b/drivers/input/touchscreen/usbtouchscreen.c
@@ -304,6 +304,45 @@
#define EGALAX_PKT_TYPE_REPT 0x80
#define EGALAX_PKT_TYPE_DIAG 0x0A
+static int egalax_init(struct usbtouch_usb *usbtouch)
+{
+ int ret, i;
+ unsigned char *buf;
+ struct usb_device *udev = interface_to_usbdev(usbtouch->interface);
+
+ /*
+ * An eGalax diagnostic packet kicks the device into using the right
+ * protocol. We send a "check active" packet. The response will be
+ * read later and ignored.
+ */
+
+ buf = kmalloc(3, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ buf[0] = EGALAX_PKT_TYPE_DIAG;
+ buf[1] = 1; /* length */
+ buf[2] = 'A'; /* command - check active */
+
+ for (i = 0; i < 3; i++) {
+ ret = usb_control_msg(udev, usb_sndctrlpipe(udev, 0),
+ 0,
+ USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
+ 0, 0, buf, 3,
+ USB_CTRL_SET_TIMEOUT);
+ if (ret >= 0) {
+ ret = 0;
+ break;
+ }
+ if (ret != -EPIPE)
+ break;
+ }
+
+ kfree(buf);
+
+ return ret;
+}
+
static int egalax_read_data(struct usbtouch_usb *dev, unsigned char *pkt)
{
if ((pkt[0] & EGALAX_PKT_TYPE_MASK) != EGALAX_PKT_TYPE_REPT)
@@ -1056,6 +1095,7 @@
.process_pkt = usbtouch_process_multi,
.get_pkt_len = egalax_get_pkt_len,
.read_data = egalax_read_data,
+ .init = egalax_init,
},
#endif
diff --git a/drivers/input/touchscreen/wacom_w8001.c b/drivers/input/touchscreen/wacom_w8001.c
index 8f9ad2f..9a83be6 100644
--- a/drivers/input/touchscreen/wacom_w8001.c
+++ b/drivers/input/touchscreen/wacom_w8001.c
@@ -471,7 +471,7 @@
case 5:
w8001->pktlen = W8001_PKTLEN_TOUCH2FG;
- input_mt_init_slots(dev, 2);
+ input_mt_init_slots(dev, 2, 0);
input_set_abs_params(dev, ABS_MT_POSITION_X,
0, touch.x, 0, 0);
input_set_abs_params(dev, ABS_MT_POSITION_Y,
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index b64502d..e89daf1 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -266,7 +266,7 @@
static int iommu_init_device(struct device *dev)
{
- struct pci_dev *dma_pdev, *pdev = to_pci_dev(dev);
+ struct pci_dev *dma_pdev = NULL, *pdev = to_pci_dev(dev);
struct iommu_dev_data *dev_data;
struct iommu_group *group;
u16 alias;
@@ -293,7 +293,9 @@
dev_data->alias_data = alias_data;
dma_pdev = pci_get_bus_and_slot(alias >> 8, alias & 0xff);
- } else
+ }
+
+ if (dma_pdev == NULL)
dma_pdev = pci_dev_get(pdev);
/* Account for quirked devices */
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2297ec1..db820d7 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2351,7 +2351,7 @@
return 0;
if (pdev->class >> 8 == PCI_CLASS_BRIDGE_PCI)
return 0;
- } else if (pdev->pcie_type == PCI_EXP_TYPE_PCI_BRIDGE)
+ } else if (pci_pcie_type(pdev) == PCI_EXP_TYPE_PCI_BRIDGE)
return 0;
/*
@@ -3546,10 +3546,10 @@
struct pci_dev *bridge = bus->self;
if (!bridge || !pci_is_pcie(bridge) ||
- bridge->pcie_type == PCI_EXP_TYPE_PCI_BRIDGE)
+ pci_pcie_type(bridge) == PCI_EXP_TYPE_PCI_BRIDGE)
return 0;
- if (bridge->pcie_type == PCI_EXP_TYPE_ROOT_PORT) {
+ if (pci_pcie_type(bridge) == PCI_EXP_TYPE_ROOT_PORT) {
for (i = 0; i < atsru->devices_cnt; i++)
if (atsru->devices[i] == bridge)
return 1;
diff --git a/drivers/isdn/hardware/mISDN/avmfritz.c b/drivers/isdn/hardware/mISDN/avmfritz.c
index fa6ca47..dceaec8 100644
--- a/drivers/isdn/hardware/mISDN/avmfritz.c
+++ b/drivers/isdn/hardware/mISDN/avmfritz.c
@@ -857,8 +857,9 @@
switch (cmd) {
case CLOSE_CHANNEL:
test_and_clear_bit(FLG_OPEN, &bch->Flags);
+ cancel_work_sync(&bch->workq);
spin_lock_irqsave(&fc->lock, flags);
- mISDN_freebchannel(bch);
+ mISDN_clear_bchannel(bch);
modehdlc(bch, ISDN_P_NONE);
spin_unlock_irqrestore(&fc->lock, flags);
ch->protocol = ISDN_P_NONE;
diff --git a/drivers/isdn/hardware/mISDN/hfcmulti.c b/drivers/isdn/hardware/mISDN/hfcmulti.c
index 5e402cf2..f027942 100644
--- a/drivers/isdn/hardware/mISDN/hfcmulti.c
+++ b/drivers/isdn/hardware/mISDN/hfcmulti.c
@@ -5059,6 +5059,7 @@
printk(KERN_INFO
"HFC-E1 #%d has overlapping B-channels on fragment #%d\n",
E1_cnt + 1, pt);
+ kfree(hc);
return -EINVAL;
}
maskcheck |= hc->bmask[pt];
@@ -5086,6 +5087,7 @@
if ((poll >> 1) > sizeof(hc->silence_data)) {
printk(KERN_ERR "HFCMULTI error: silence_data too small, "
"please fix\n");
+ kfree(hc);
return -EINVAL;
}
for (i = 0; i < (poll >> 1); i++)
diff --git a/drivers/isdn/hardware/mISDN/mISDNipac.c b/drivers/isdn/hardware/mISDN/mISDNipac.c
index 752e082..ccd7d85 100644
--- a/drivers/isdn/hardware/mISDN/mISDNipac.c
+++ b/drivers/isdn/hardware/mISDN/mISDNipac.c
@@ -1406,8 +1406,9 @@
switch (cmd) {
case CLOSE_CHANNEL:
test_and_clear_bit(FLG_OPEN, &bch->Flags);
+ cancel_work_sync(&bch->workq);
spin_lock_irqsave(hx->ip->hwlock, flags);
- mISDN_freebchannel(bch);
+ mISDN_clear_bchannel(bch);
hscx_mode(hx, ISDN_P_NONE);
spin_unlock_irqrestore(hx->ip->hwlock, flags);
ch->protocol = ISDN_P_NONE;
diff --git a/drivers/isdn/hardware/mISDN/mISDNisar.c b/drivers/isdn/hardware/mISDN/mISDNisar.c
index be5973d..182ecf0 100644
--- a/drivers/isdn/hardware/mISDN/mISDNisar.c
+++ b/drivers/isdn/hardware/mISDN/mISDNisar.c
@@ -1588,8 +1588,9 @@
switch (cmd) {
case CLOSE_CHANNEL:
test_and_clear_bit(FLG_OPEN, &bch->Flags);
+ cancel_work_sync(&bch->workq);
spin_lock_irqsave(ich->is->hwlock, flags);
- mISDN_freebchannel(bch);
+ mISDN_clear_bchannel(bch);
modeisar(ich, ISDN_P_NONE);
spin_unlock_irqrestore(ich->is->hwlock, flags);
ch->protocol = ISDN_P_NONE;
diff --git a/drivers/isdn/hardware/mISDN/netjet.c b/drivers/isdn/hardware/mISDN/netjet.c
index c3e3e76..9bcade5 100644
--- a/drivers/isdn/hardware/mISDN/netjet.c
+++ b/drivers/isdn/hardware/mISDN/netjet.c
@@ -812,8 +812,9 @@
switch (cmd) {
case CLOSE_CHANNEL:
test_and_clear_bit(FLG_OPEN, &bch->Flags);
+ cancel_work_sync(&bch->workq);
spin_lock_irqsave(&card->lock, flags);
- mISDN_freebchannel(bch);
+ mISDN_clear_bchannel(bch);
mode_tiger(bc, ISDN_P_NONE);
spin_unlock_irqrestore(&card->lock, flags);
ch->protocol = ISDN_P_NONE;
diff --git a/drivers/isdn/hardware/mISDN/w6692.c b/drivers/isdn/hardware/mISDN/w6692.c
index 26a86b8..335fe64 100644
--- a/drivers/isdn/hardware/mISDN/w6692.c
+++ b/drivers/isdn/hardware/mISDN/w6692.c
@@ -1054,8 +1054,9 @@
switch (cmd) {
case CLOSE_CHANNEL:
test_and_clear_bit(FLG_OPEN, &bch->Flags);
+ cancel_work_sync(&bch->workq);
spin_lock_irqsave(&card->lock, flags);
- mISDN_freebchannel(bch);
+ mISDN_clear_bchannel(bch);
w6692_mode(bc, ISDN_P_NONE);
spin_unlock_irqrestore(&card->lock, flags);
ch->protocol = ISDN_P_NONE;
diff --git a/drivers/isdn/mISDN/hwchannel.c b/drivers/isdn/mISDN/hwchannel.c
index ef34fd4..2602be2 100644
--- a/drivers/isdn/mISDN/hwchannel.c
+++ b/drivers/isdn/mISDN/hwchannel.c
@@ -148,17 +148,16 @@
ch->next_minlen = ch->init_minlen;
ch->maxlen = ch->init_maxlen;
ch->next_maxlen = ch->init_maxlen;
+ skb_queue_purge(&ch->rqueue);
+ ch->rcount = 0;
}
EXPORT_SYMBOL(mISDN_clear_bchannel);
-int
+void
mISDN_freebchannel(struct bchannel *ch)
{
+ cancel_work_sync(&ch->workq);
mISDN_clear_bchannel(ch);
- skb_queue_purge(&ch->rqueue);
- ch->rcount = 0;
- flush_work_sync(&ch->workq);
- return 0;
}
EXPORT_SYMBOL(mISDN_freebchannel);
diff --git a/drivers/lguest/x86/core.c b/drivers/lguest/x86/core.c
index 39809035..4af12e1 100644
--- a/drivers/lguest/x86/core.c
+++ b/drivers/lguest/x86/core.c
@@ -203,8 +203,8 @@
* we set it now, so we can trap and pass that trap to the Guest if it
* uses the FPU.
*/
- if (cpu->ts)
- unlazy_fpu(current);
+ if (cpu->ts && user_has_fpu())
+ stts();
/*
* SYSENTER is an optimized way of doing system calls. We can't allow
@@ -234,6 +234,10 @@
if (boot_cpu_has(X86_FEATURE_SEP))
wrmsr(MSR_IA32_SYSENTER_CS, __KERNEL_CS, 0);
+ /* Clear the host TS bit if it was set above. */
+ if (cpu->ts && user_has_fpu())
+ clts();
+
/*
* If the Guest page faulted, then the cr2 register will tell us the
* bad virtual address. We have to grab this now, because once we
@@ -249,7 +253,7 @@
* a different CPU. So all the critical stuff should be done
* before this.
*/
- else if (cpu->regs->trapnum == 7)
+ else if (cpu->regs->trapnum == 7 && !user_has_fpu())
math_state_restore();
}
diff --git a/drivers/md/dm-mpath.c b/drivers/md/dm-mpath.c
index d8abb90..034233e 100644
--- a/drivers/md/dm-mpath.c
+++ b/drivers/md/dm-mpath.c
@@ -1555,6 +1555,7 @@
unsigned long arg)
{
struct multipath *m = ti->private;
+ struct pgpath *pgpath;
struct block_device *bdev;
fmode_t mode;
unsigned long flags;
@@ -1570,12 +1571,14 @@
if (!m->current_pgpath)
__choose_pgpath(m, 0);
- if (m->current_pgpath) {
- bdev = m->current_pgpath->path.dev->bdev;
- mode = m->current_pgpath->path.dev->mode;
+ pgpath = m->current_pgpath;
+
+ if (pgpath) {
+ bdev = pgpath->path.dev->bdev;
+ mode = pgpath->path.dev->mode;
}
- if (m->queue_io)
+ if ((pgpath && m->queue_io) || (!pgpath && m->queue_if_no_path))
r = -EAGAIN;
else if (!bdev)
r = -EIO;
diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index f900690..100368e 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -1212,6 +1212,41 @@
return &t->targets[(KEYS_PER_NODE * n) + k];
}
+static int count_device(struct dm_target *ti, struct dm_dev *dev,
+ sector_t start, sector_t len, void *data)
+{
+ unsigned *num_devices = data;
+
+ (*num_devices)++;
+
+ return 0;
+}
+
+/*
+ * Check whether a table has no data devices attached using each
+ * target's iterate_devices method.
+ * Returns false if the result is unknown because a target doesn't
+ * support iterate_devices.
+ */
+bool dm_table_has_no_data_devices(struct dm_table *table)
+{
+ struct dm_target *uninitialized_var(ti);
+ unsigned i = 0, num_devices = 0;
+
+ while (i < dm_table_get_num_targets(table)) {
+ ti = dm_table_get_target(table, i++);
+
+ if (!ti->type->iterate_devices)
+ return false;
+
+ ti->type->iterate_devices(ti, count_device, &num_devices);
+ if (num_devices)
+ return false;
+ }
+
+ return true;
+}
+
/*
* Establish the new table's queue_limits and validate them.
*/
@@ -1354,17 +1389,25 @@
return q && blk_queue_nonrot(q);
}
-static bool dm_table_is_nonrot(struct dm_table *t)
+static int device_is_not_random(struct dm_target *ti, struct dm_dev *dev,
+ sector_t start, sector_t len, void *data)
+{
+ struct request_queue *q = bdev_get_queue(dev->bdev);
+
+ return q && !blk_queue_add_random(q);
+}
+
+static bool dm_table_all_devices_attribute(struct dm_table *t,
+ iterate_devices_callout_fn func)
{
struct dm_target *ti;
unsigned i = 0;
- /* Ensure that all underlying device are non-rotational. */
while (i < dm_table_get_num_targets(t)) {
ti = dm_table_get_target(t, i++);
if (!ti->type->iterate_devices ||
- !ti->type->iterate_devices(ti, device_is_nonrot, NULL))
+ !ti->type->iterate_devices(ti, func, NULL))
return 0;
}
@@ -1396,7 +1439,8 @@
if (!dm_table_discard_zeroes_data(t))
q->limits.discard_zeroes_data = 0;
- if (dm_table_is_nonrot(t))
+ /* Ensure that all underlying devices are non-rotational. */
+ if (dm_table_all_devices_attribute(t, device_is_nonrot))
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, q);
else
queue_flag_clear_unlocked(QUEUE_FLAG_NONROT, q);
@@ -1404,6 +1448,15 @@
dm_table_set_integrity(t);
/*
+ * Determine whether or not this queue's I/O timings contribute
+ * to the entropy pool, Only request-based targets use this.
+ * Clear QUEUE_FLAG_ADD_RANDOM if any underlying device does not
+ * have it set.
+ */
+ if (blk_queue_add_random(q) && dm_table_all_devices_attribute(t, device_is_not_random))
+ queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, q);
+
+ /*
* QUEUE_FLAG_STACKABLE must be set after all queue settings are
* visible to other CPUs because, once the flag is set, incoming bios
* are processed by request-based dm, which refers to the queue
diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
index af1fc3b..c29410a 100644
--- a/drivers/md/dm-thin.c
+++ b/drivers/md/dm-thin.c
@@ -509,9 +509,9 @@
struct pool_features {
enum pool_mode mode;
- unsigned zero_new_blocks:1;
- unsigned discard_enabled:1;
- unsigned discard_passdown:1;
+ bool zero_new_blocks:1;
+ bool discard_enabled:1;
+ bool discard_passdown:1;
};
struct thin_c;
@@ -580,7 +580,8 @@
struct dm_target_callbacks callbacks;
dm_block_t low_water_blocks;
- struct pool_features pf;
+ struct pool_features requested_pf; /* Features requested during table load */
+ struct pool_features adjusted_pf; /* Features used after adjusting for constituent devices */
};
/*
@@ -1839,6 +1840,47 @@
/*----------------------------------------------------------------
* Binding of control targets to a pool object
*--------------------------------------------------------------*/
+static bool data_dev_supports_discard(struct pool_c *pt)
+{
+ struct request_queue *q = bdev_get_queue(pt->data_dev->bdev);
+
+ return q && blk_queue_discard(q);
+}
+
+/*
+ * If discard_passdown was enabled verify that the data device
+ * supports discards. Disable discard_passdown if not.
+ */
+static void disable_passdown_if_not_supported(struct pool_c *pt)
+{
+ struct pool *pool = pt->pool;
+ struct block_device *data_bdev = pt->data_dev->bdev;
+ struct queue_limits *data_limits = &bdev_get_queue(data_bdev)->limits;
+ sector_t block_size = pool->sectors_per_block << SECTOR_SHIFT;
+ const char *reason = NULL;
+ char buf[BDEVNAME_SIZE];
+
+ if (!pt->adjusted_pf.discard_passdown)
+ return;
+
+ if (!data_dev_supports_discard(pt))
+ reason = "discard unsupported";
+
+ else if (data_limits->max_discard_sectors < pool->sectors_per_block)
+ reason = "max discard sectors smaller than a block";
+
+ else if (data_limits->discard_granularity > block_size)
+ reason = "discard granularity larger than a block";
+
+ else if (block_size & (data_limits->discard_granularity - 1))
+ reason = "discard granularity not a factor of block size";
+
+ if (reason) {
+ DMWARN("Data device (%s) %s: Disabling discard passdown.", bdevname(data_bdev, buf), reason);
+ pt->adjusted_pf.discard_passdown = false;
+ }
+}
+
static int bind_control_target(struct pool *pool, struct dm_target *ti)
{
struct pool_c *pt = ti->private;
@@ -1847,31 +1889,16 @@
* We want to make sure that degraded pools are never upgraded.
*/
enum pool_mode old_mode = pool->pf.mode;
- enum pool_mode new_mode = pt->pf.mode;
+ enum pool_mode new_mode = pt->adjusted_pf.mode;
if (old_mode > new_mode)
new_mode = old_mode;
pool->ti = ti;
pool->low_water_blocks = pt->low_water_blocks;
- pool->pf = pt->pf;
- set_pool_mode(pool, new_mode);
+ pool->pf = pt->adjusted_pf;
- /*
- * If discard_passdown was enabled verify that the data device
- * supports discards. Disable discard_passdown if not; otherwise
- * -EOPNOTSUPP will be returned.
- */
- /* FIXME: pull this out into a sep fn. */
- if (pt->pf.discard_passdown) {
- struct request_queue *q = bdev_get_queue(pt->data_dev->bdev);
- if (!q || !blk_queue_discard(q)) {
- char buf[BDEVNAME_SIZE];
- DMWARN("Discard unsupported by data device (%s): Disabling discard passdown.",
- bdevname(pt->data_dev->bdev, buf));
- pool->pf.discard_passdown = 0;
- }
- }
+ set_pool_mode(pool, new_mode);
return 0;
}
@@ -1889,9 +1916,9 @@
static void pool_features_init(struct pool_features *pf)
{
pf->mode = PM_WRITE;
- pf->zero_new_blocks = 1;
- pf->discard_enabled = 1;
- pf->discard_passdown = 1;
+ pf->zero_new_blocks = true;
+ pf->discard_enabled = true;
+ pf->discard_passdown = true;
}
static void __pool_destroy(struct pool *pool)
@@ -2119,13 +2146,13 @@
argc--;
if (!strcasecmp(arg_name, "skip_block_zeroing"))
- pf->zero_new_blocks = 0;
+ pf->zero_new_blocks = false;
else if (!strcasecmp(arg_name, "ignore_discard"))
- pf->discard_enabled = 0;
+ pf->discard_enabled = false;
else if (!strcasecmp(arg_name, "no_discard_passdown"))
- pf->discard_passdown = 0;
+ pf->discard_passdown = false;
else if (!strcasecmp(arg_name, "read_only"))
pf->mode = PM_READ_ONLY;
@@ -2259,8 +2286,9 @@
pt->metadata_dev = metadata_dev;
pt->data_dev = data_dev;
pt->low_water_blocks = low_water_blocks;
- pt->pf = pf;
+ pt->adjusted_pf = pt->requested_pf = pf;
ti->num_flush_requests = 1;
+
/*
* Only need to enable discards if the pool should pass
* them down to the data device. The thin device's discard
@@ -2268,12 +2296,14 @@
*/
if (pf.discard_enabled && pf.discard_passdown) {
ti->num_discard_requests = 1;
+
/*
* Setting 'discards_supported' circumvents the normal
* stacking of discard limits (this keeps the pool and
* thin devices' discard limits consistent).
*/
ti->discards_supported = true;
+ ti->discard_zeroes_data_unsupported = true;
}
ti->private = pt;
@@ -2703,7 +2733,7 @@
format_dev_t(buf2, pt->data_dev->bdev->bd_dev),
(unsigned long)pool->sectors_per_block,
(unsigned long long)pt->low_water_blocks);
- emit_flags(&pt->pf, result, sz, maxlen);
+ emit_flags(&pt->requested_pf, result, sz, maxlen);
break;
}
@@ -2732,20 +2762,21 @@
return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
}
-static void set_discard_limits(struct pool *pool, struct queue_limits *limits)
+static void set_discard_limits(struct pool_c *pt, struct queue_limits *limits)
{
- /*
- * FIXME: these limits may be incompatible with the pool's data device
- */
+ struct pool *pool = pt->pool;
+ struct queue_limits *data_limits;
+
limits->max_discard_sectors = pool->sectors_per_block;
/*
- * This is just a hint, and not enforced. We have to cope with
- * bios that cover a block partially. A discard that spans a block
- * boundary is not sent to this target.
+ * discard_granularity is just a hint, and not enforced.
*/
- limits->discard_granularity = pool->sectors_per_block << SECTOR_SHIFT;
- limits->discard_zeroes_data = pool->pf.zero_new_blocks;
+ if (pt->adjusted_pf.discard_passdown) {
+ data_limits = &bdev_get_queue(pt->data_dev->bdev)->limits;
+ limits->discard_granularity = data_limits->discard_granularity;
+ } else
+ limits->discard_granularity = pool->sectors_per_block << SECTOR_SHIFT;
}
static void pool_io_hints(struct dm_target *ti, struct queue_limits *limits)
@@ -2755,15 +2786,25 @@
blk_limits_io_min(limits, 0);
blk_limits_io_opt(limits, pool->sectors_per_block << SECTOR_SHIFT);
- if (pool->pf.discard_enabled)
- set_discard_limits(pool, limits);
+
+ /*
+ * pt->adjusted_pf is a staging area for the actual features to use.
+ * They get transferred to the live pool in bind_control_target()
+ * called from pool_preresume().
+ */
+ if (!pt->adjusted_pf.discard_enabled)
+ return;
+
+ disable_passdown_if_not_supported(pt);
+
+ set_discard_limits(pt, limits);
}
static struct target_type pool_target = {
.name = "thin-pool",
.features = DM_TARGET_SINGLETON | DM_TARGET_ALWAYS_WRITEABLE |
DM_TARGET_IMMUTABLE,
- .version = {1, 3, 0},
+ .version = {1, 4, 0},
.module = THIS_MODULE,
.ctr = pool_ctr,
.dtr = pool_dtr,
@@ -3042,19 +3083,19 @@
return 0;
}
+/*
+ * A thin device always inherits its queue limits from its pool.
+ */
static void thin_io_hints(struct dm_target *ti, struct queue_limits *limits)
{
struct thin_c *tc = ti->private;
- struct pool *pool = tc->pool;
- blk_limits_io_min(limits, 0);
- blk_limits_io_opt(limits, pool->sectors_per_block << SECTOR_SHIFT);
- set_discard_limits(pool, limits);
+ *limits = bdev_get_queue(tc->pool_dev->bdev)->limits;
}
static struct target_type thin_target = {
.name = "thin",
- .version = {1, 3, 0},
+ .version = {1, 4, 0},
.module = THIS_MODULE,
.ctr = thin_ctr,
.dtr = thin_dtr,
diff --git a/drivers/md/dm-verity.c b/drivers/md/dm-verity.c
index 254d192..892ae27 100644
--- a/drivers/md/dm-verity.c
+++ b/drivers/md/dm-verity.c
@@ -718,8 +718,8 @@
v->hash_dev_block_bits = ffs(num) - 1;
if (sscanf(argv[5], "%llu%c", &num_ll, &dummy) != 1 ||
- num_ll << (v->data_dev_block_bits - SECTOR_SHIFT) !=
- (sector_t)num_ll << (v->data_dev_block_bits - SECTOR_SHIFT)) {
+ (sector_t)(num_ll << (v->data_dev_block_bits - SECTOR_SHIFT))
+ >> (v->data_dev_block_bits - SECTOR_SHIFT) != num_ll) {
ti->error = "Invalid data blocks";
r = -EINVAL;
goto bad;
@@ -733,8 +733,8 @@
}
if (sscanf(argv[6], "%llu%c", &num_ll, &dummy) != 1 ||
- num_ll << (v->hash_dev_block_bits - SECTOR_SHIFT) !=
- (sector_t)num_ll << (v->hash_dev_block_bits - SECTOR_SHIFT)) {
+ (sector_t)(num_ll << (v->hash_dev_block_bits - SECTOR_SHIFT))
+ >> (v->hash_dev_block_bits - SECTOR_SHIFT) != num_ll) {
ti->error = "Invalid hash start";
r = -EINVAL;
goto bad;
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 4e09b6f..67ffa39 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -865,10 +865,14 @@
{
int r = error;
struct dm_rq_target_io *tio = clone->end_io_data;
- dm_request_endio_fn rq_end_io = tio->ti->type->rq_end_io;
+ dm_request_endio_fn rq_end_io = NULL;
- if (mapped && rq_end_io)
- r = rq_end_io(tio->ti, clone, error, &tio->info);
+ if (tio->ti) {
+ rq_end_io = tio->ti->type->rq_end_io;
+
+ if (mapped && rq_end_io)
+ r = rq_end_io(tio->ti, clone, error, &tio->info);
+ }
if (r <= 0)
/* The target wants to complete the I/O */
@@ -1588,15 +1592,6 @@
int r, requeued = 0;
struct dm_rq_target_io *tio = clone->end_io_data;
- /*
- * Hold the md reference here for the in-flight I/O.
- * We can't rely on the reference count by device opener,
- * because the device may be closed during the request completion
- * when all bios are completed.
- * See the comment in rq_completed() too.
- */
- dm_get(md);
-
tio->ti = ti;
r = ti->type->map_rq(ti, clone, &tio->info);
switch (r) {
@@ -1628,6 +1623,26 @@
return requeued;
}
+static struct request *dm_start_request(struct mapped_device *md, struct request *orig)
+{
+ struct request *clone;
+
+ blk_start_request(orig);
+ clone = orig->special;
+ atomic_inc(&md->pending[rq_data_dir(clone)]);
+
+ /*
+ * Hold the md reference here for the in-flight I/O.
+ * We can't rely on the reference count by device opener,
+ * because the device may be closed during the request completion
+ * when all bios are completed.
+ * See the comment in rq_completed() too.
+ */
+ dm_get(md);
+
+ return clone;
+}
+
/*
* q->request_fn for request-based dm.
* Called with the queue lock held.
@@ -1657,14 +1672,21 @@
pos = blk_rq_pos(rq);
ti = dm_table_find_target(map, pos);
- BUG_ON(!dm_target_is_valid(ti));
+ if (!dm_target_is_valid(ti)) {
+ /*
+ * Must perform setup, that dm_done() requires,
+ * before calling dm_kill_unmapped_request
+ */
+ DMERR_LIMIT("request attempted access beyond the end of device");
+ clone = dm_start_request(md, rq);
+ dm_kill_unmapped_request(clone, -EIO);
+ continue;
+ }
if (ti->type->busy && ti->type->busy(ti))
goto delay_and_out;
- blk_start_request(rq);
- clone = rq->special;
- atomic_inc(&md->pending[rq_data_dir(clone)]);
+ clone = dm_start_request(md, rq);
spin_unlock(q->queue_lock);
if (map_request(ti, clone, md))
@@ -1684,8 +1706,6 @@
blk_delay_queue(q, HZ / 10);
out:
dm_table_put(map);
-
- return;
}
int dm_underlying_device_busy(struct request_queue *q)
@@ -2409,7 +2429,7 @@
*/
struct dm_table *dm_swap_table(struct mapped_device *md, struct dm_table *table)
{
- struct dm_table *map = ERR_PTR(-EINVAL);
+ struct dm_table *live_map, *map = ERR_PTR(-EINVAL);
struct queue_limits limits;
int r;
@@ -2419,6 +2439,19 @@
if (!dm_suspended_md(md))
goto out;
+ /*
+ * If the new table has no data devices, retain the existing limits.
+ * This helps multipath with queue_if_no_path if all paths disappear,
+ * then new I/O is queued based on these limits, and then some paths
+ * reappear.
+ */
+ if (dm_table_has_no_data_devices(table)) {
+ live_map = dm_get_live_table(md);
+ if (live_map)
+ limits = md->queue->limits;
+ dm_table_put(live_map);
+ }
+
r = dm_calculate_queue_limits(table, &limits);
if (r) {
map = ERR_PTR(r);
diff --git a/drivers/md/dm.h b/drivers/md/dm.h
index 52eef49..6a99fef 100644
--- a/drivers/md/dm.h
+++ b/drivers/md/dm.h
@@ -54,6 +54,7 @@
void (*fn)(void *), void *context);
struct dm_target *dm_table_get_target(struct dm_table *t, unsigned int index);
struct dm_target *dm_table_find_target(struct dm_table *t, sector_t sector);
+bool dm_table_has_no_data_devices(struct dm_table *table);
int dm_calculate_queue_limits(struct dm_table *table,
struct queue_limits *limits);
void dm_table_set_restrictions(struct dm_table *t, struct request_queue *q,
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 3f6203a..308e87b 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7619,6 +7619,8 @@
}
}
}
+ if (removed)
+ set_bit(MD_CHANGE_DEVS, &mddev->flags);
return spares;
}
@@ -7632,9 +7634,11 @@
!test_bit(MD_RECOVERY_REQUESTED, &mddev->recovery)) {
/* success...*/
/* activate any spares */
- if (mddev->pers->spare_active(mddev))
+ if (mddev->pers->spare_active(mddev)) {
sysfs_notify(&mddev->kobj, NULL,
"degraded");
+ set_bit(MD_CHANGE_DEVS, &mddev->flags);
+ }
}
if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) &&
mddev->pers->finish_reshape)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 1c2eb38..0138a72 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -1512,14 +1512,16 @@
do {
int n = conf->copies;
int cnt = 0;
+ int this = first;
while (n--) {
- if (conf->mirrors[first].rdev &&
- first != ignore)
+ if (conf->mirrors[this].rdev &&
+ this != ignore)
cnt++;
- first = (first+1) % geo->raid_disks;
+ this = (this+1) % geo->raid_disks;
}
if (cnt == 0)
return 0;
+ first = (first + geo->near_copies) % geo->raid_disks;
} while (first != 0);
return 1;
}
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index adda94d..0689173 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -393,6 +393,8 @@
degraded = 0;
for (i = 0; i < conf->previous_raid_disks; i++) {
struct md_rdev *rdev = rcu_dereference(conf->disks[i].rdev);
+ if (rdev && test_bit(Faulty, &rdev->flags))
+ rdev = rcu_dereference(conf->disks[i].replacement);
if (!rdev || test_bit(Faulty, &rdev->flags))
degraded++;
else if (test_bit(In_sync, &rdev->flags))
@@ -417,6 +419,8 @@
degraded2 = 0;
for (i = 0; i < conf->raid_disks; i++) {
struct md_rdev *rdev = rcu_dereference(conf->disks[i].rdev);
+ if (rdev && test_bit(Faulty, &rdev->flags))
+ rdev = rcu_dereference(conf->disks[i].replacement);
if (!rdev || test_bit(Faulty, &rdev->flags))
degraded2++;
else if (test_bit(In_sync, &rdev->flags))
@@ -1587,6 +1591,7 @@
#ifdef CONFIG_MULTICORE_RAID456
init_waitqueue_head(&nsh->ops.wait_for_ops);
#endif
+ spin_lock_init(&nsh->stripe_lock);
list_add(&nsh->lru, &newstripes);
}
@@ -4192,7 +4197,7 @@
finish_wait(&conf->wait_for_overlap, &w);
set_bit(STRIPE_HANDLE, &sh->state);
clear_bit(STRIPE_DELAYED, &sh->state);
- if ((bi->bi_rw & REQ_NOIDLE) &&
+ if ((bi->bi_rw & REQ_SYNC) &&
!test_and_set_bit(STRIPE_PREREAD_ACTIVE, &sh->state))
atomic_inc(&conf->preread_active_stripes);
release_stripe_plug(mddev, sh);
diff --git a/drivers/media/dvb/ngene/ngene-cards.c b/drivers/media/dvb/ngene/ngene-cards.c
index 72ee8de..0a497be 100644
--- a/drivers/media/dvb/ngene/ngene-cards.c
+++ b/drivers/media/dvb/ngene/ngene-cards.c
@@ -524,7 +524,7 @@
printk(KERN_INFO DEVICE_NAME ": resume\n");
}
-static struct pci_error_handlers ngene_errors = {
+static const struct pci_error_handlers ngene_errors = {
.error_detected = ngene_error_detected,
.link_reset = ngene_link_reset,
.slot_reset = ngene_slot_reset,
diff --git a/drivers/mfd/88pm800.c b/drivers/mfd/88pm800.c
index b67a301..ce229ea 100644
--- a/drivers/mfd/88pm800.c
+++ b/drivers/mfd/88pm800.c
@@ -470,7 +470,8 @@
ret =
mfd_add_devices(chip->dev, 0, &onkey_devs[0],
- ARRAY_SIZE(onkey_devs), &onkey_resources[0], 0);
+ ARRAY_SIZE(onkey_devs), &onkey_resources[0], 0,
+ NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add onkey subdev\n");
goto out_dev;
@@ -481,7 +482,7 @@
rtc_devs[0].platform_data = pdata->rtc;
rtc_devs[0].pdata_size = sizeof(struct pm80x_rtc_pdata);
ret = mfd_add_devices(chip->dev, 0, &rtc_devs[0],
- ARRAY_SIZE(rtc_devs), NULL, 0);
+ ARRAY_SIZE(rtc_devs), NULL, 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add rtc subdev\n");
goto out_dev;
diff --git a/drivers/mfd/88pm805.c b/drivers/mfd/88pm805.c
index 6146583..c20a311 100644
--- a/drivers/mfd/88pm805.c
+++ b/drivers/mfd/88pm805.c
@@ -216,7 +216,8 @@
}
ret = mfd_add_devices(chip->dev, 0, &codec_devs[0],
- ARRAY_SIZE(codec_devs), &codec_resources[0], 0);
+ ARRAY_SIZE(codec_devs), &codec_resources[0], 0,
+ NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add codec subdev\n");
goto out_codec;
diff --git a/drivers/mfd/88pm860x-core.c b/drivers/mfd/88pm860x-core.c
index d09918c..b73f033 100644
--- a/drivers/mfd/88pm860x-core.c
+++ b/drivers/mfd/88pm860x-core.c
@@ -637,7 +637,7 @@
bk_devs[i].resources = &bk_resources[j];
ret = mfd_add_devices(chip->dev, 0,
&bk_devs[i], 1,
- &bk_resources[j], 0);
+ &bk_resources[j], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add "
"backlight subdev\n");
@@ -672,7 +672,7 @@
led_devs[i].resources = &led_resources[j],
ret = mfd_add_devices(chip->dev, 0,
&led_devs[i], 1,
- &led_resources[j], 0);
+ &led_resources[j], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add "
"led subdev\n");
@@ -709,7 +709,7 @@
regulator_devs[i].resources = ®ulator_resources[seq];
ret = mfd_add_devices(chip->dev, 0, ®ulator_devs[i], 1,
- ®ulator_resources[seq], 0);
+ ®ulator_resources[seq], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add regulator subdev\n");
goto out;
@@ -733,7 +733,7 @@
rtc_devs[0].resources = &rtc_resources[0];
ret = mfd_add_devices(chip->dev, 0, &rtc_devs[0],
ARRAY_SIZE(rtc_devs), &rtc_resources[0],
- chip->irq_base);
+ chip->irq_base, NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add rtc subdev\n");
}
@@ -752,7 +752,7 @@
touch_devs[0].resources = &touch_resources[0];
ret = mfd_add_devices(chip->dev, 0, &touch_devs[0],
ARRAY_SIZE(touch_devs), &touch_resources[0],
- chip->irq_base);
+ chip->irq_base, NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add touch subdev\n");
}
@@ -770,7 +770,7 @@
power_devs[0].num_resources = ARRAY_SIZE(battery_resources);
power_devs[0].resources = &battery_resources[0],
ret = mfd_add_devices(chip->dev, 0, &power_devs[0], 1,
- &battery_resources[0], chip->irq_base);
+ &battery_resources[0], chip->irq_base, NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add battery subdev\n");
@@ -779,7 +779,7 @@
power_devs[1].num_resources = ARRAY_SIZE(charger_resources);
power_devs[1].resources = &charger_resources[0],
ret = mfd_add_devices(chip->dev, 0, &power_devs[1], 1,
- &charger_resources[0], chip->irq_base);
+ &charger_resources[0], chip->irq_base, NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add charger subdev\n");
@@ -788,7 +788,7 @@
power_devs[2].num_resources = ARRAY_SIZE(preg_resources);
power_devs[2].resources = &preg_resources[0],
ret = mfd_add_devices(chip->dev, 0, &power_devs[2], 1,
- &preg_resources[0], chip->irq_base);
+ &preg_resources[0], chip->irq_base, NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add preg subdev\n");
}
@@ -802,7 +802,7 @@
onkey_devs[0].resources = &onkey_resources[0],
ret = mfd_add_devices(chip->dev, 0, &onkey_devs[0],
ARRAY_SIZE(onkey_devs), &onkey_resources[0],
- chip->irq_base);
+ chip->irq_base, NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add onkey subdev\n");
}
@@ -815,7 +815,8 @@
codec_devs[0].num_resources = ARRAY_SIZE(codec_resources);
codec_devs[0].resources = &codec_resources[0],
ret = mfd_add_devices(chip->dev, 0, &codec_devs[0],
- ARRAY_SIZE(codec_devs), &codec_resources[0], 0);
+ ARRAY_SIZE(codec_devs), &codec_resources[0], 0,
+ NULL);
if (ret < 0)
dev_err(chip->dev, "Failed to add codec subdev\n");
}
diff --git a/drivers/mfd/aat2870-core.c b/drivers/mfd/aat2870-core.c
index 44a3fdb..f1beb49 100644
--- a/drivers/mfd/aat2870-core.c
+++ b/drivers/mfd/aat2870-core.c
@@ -424,7 +424,7 @@
}
ret = mfd_add_devices(aat2870->dev, 0, aat2870_devs,
- ARRAY_SIZE(aat2870_devs), NULL, 0);
+ ARRAY_SIZE(aat2870_devs), NULL, 0, NULL);
if (ret != 0) {
dev_err(aat2870->dev, "Failed to add subdev: %d\n", ret);
goto out_disable;
diff --git a/drivers/mfd/ab3100-core.c b/drivers/mfd/ab3100-core.c
index 78fca29..01781ae 100644
--- a/drivers/mfd/ab3100-core.c
+++ b/drivers/mfd/ab3100-core.c
@@ -946,7 +946,7 @@
}
err = mfd_add_devices(&client->dev, 0, ab3100_devs,
- ARRAY_SIZE(ab3100_devs), NULL, 0);
+ ARRAY_SIZE(ab3100_devs), NULL, 0, NULL);
ab3100_setup_debugfs(ab3100);
diff --git a/drivers/mfd/ab8500-core.c b/drivers/mfd/ab8500-core.c
index 626b4ec..47adf80 100644
--- a/drivers/mfd/ab8500-core.c
+++ b/drivers/mfd/ab8500-core.c
@@ -1418,25 +1418,25 @@
ret = mfd_add_devices(ab8500->dev, 0, abx500_common_devs,
ARRAY_SIZE(abx500_common_devs), NULL,
- ab8500->irq_base);
+ ab8500->irq_base, ab8500->domain);
if (ret)
goto out_freeirq;
if (is_ab9540(ab8500))
ret = mfd_add_devices(ab8500->dev, 0, ab9540_devs,
ARRAY_SIZE(ab9540_devs), NULL,
- ab8500->irq_base);
+ ab8500->irq_base, ab8500->domain);
else
ret = mfd_add_devices(ab8500->dev, 0, ab8500_devs,
ARRAY_SIZE(ab8500_devs), NULL,
- ab8500->irq_base);
+ ab8500->irq_base, ab8500->domain);
if (ret)
goto out_freeirq;
if (is_ab9540(ab8500) || is_ab8505(ab8500))
ret = mfd_add_devices(ab8500->dev, 0, ab9540_ab8505_devs,
ARRAY_SIZE(ab9540_ab8505_devs), NULL,
- ab8500->irq_base);
+ ab8500->irq_base, ab8500->domain);
if (ret)
goto out_freeirq;
@@ -1444,7 +1444,7 @@
/* Add battery management devices */
ret = mfd_add_devices(ab8500->dev, 0, ab8500_bm_devs,
ARRAY_SIZE(ab8500_bm_devs), NULL,
- ab8500->irq_base);
+ ab8500->irq_base, ab8500->domain);
if (ret)
dev_err(ab8500->dev, "error adding bm devices\n");
}
diff --git a/drivers/mfd/ab8500-gpadc.c b/drivers/mfd/ab8500-gpadc.c
index 866f959..29d72a2 100644
--- a/drivers/mfd/ab8500-gpadc.c
+++ b/drivers/mfd/ab8500-gpadc.c
@@ -342,7 +342,7 @@
/*
* Delay might be needed for ABB8500 cut 3.0, if not, remove
- * when hardware will be availible
+ * when hardware will be available
*/
msleep(1);
break;
diff --git a/drivers/mfd/arizona-core.c b/drivers/mfd/arizona-core.c
index c7983e8..1b48f20 100644
--- a/drivers/mfd/arizona-core.c
+++ b/drivers/mfd/arizona-core.c
@@ -316,7 +316,7 @@
}
ret = mfd_add_devices(arizona->dev, -1, early_devs,
- ARRAY_SIZE(early_devs), NULL, 0);
+ ARRAY_SIZE(early_devs), NULL, 0, NULL);
if (ret != 0) {
dev_err(dev, "Failed to add early children: %d\n", ret);
return ret;
@@ -516,11 +516,11 @@
switch (arizona->type) {
case WM5102:
ret = mfd_add_devices(arizona->dev, -1, wm5102_devs,
- ARRAY_SIZE(wm5102_devs), NULL, 0);
+ ARRAY_SIZE(wm5102_devs), NULL, 0, NULL);
break;
case WM5110:
ret = mfd_add_devices(arizona->dev, -1, wm5110_devs,
- ARRAY_SIZE(wm5102_devs), NULL, 0);
+ ARRAY_SIZE(wm5102_devs), NULL, 0, NULL);
break;
}
diff --git a/drivers/mfd/asic3.c b/drivers/mfd/asic3.c
index 683e18a..62f0883 100644
--- a/drivers/mfd/asic3.c
+++ b/drivers/mfd/asic3.c
@@ -913,14 +913,14 @@
if (pdata->clock_rate) {
ds1wm_pdata.clock_rate = pdata->clock_rate;
ret = mfd_add_devices(&pdev->dev, pdev->id,
- &asic3_cell_ds1wm, 1, mem, asic->irq_base);
+ &asic3_cell_ds1wm, 1, mem, asic->irq_base, NULL);
if (ret < 0)
goto out;
}
if (mem_sdio && (irq >= 0)) {
ret = mfd_add_devices(&pdev->dev, pdev->id,
- &asic3_cell_mmc, 1, mem_sdio, irq);
+ &asic3_cell_mmc, 1, mem_sdio, irq, NULL);
if (ret < 0)
goto out;
}
@@ -934,7 +934,7 @@
asic3_cell_leds[i].pdata_size = sizeof(pdata->leds[i]);
}
ret = mfd_add_devices(&pdev->dev, 0,
- asic3_cell_leds, ASIC3_NUM_LEDS, NULL, 0);
+ asic3_cell_leds, ASIC3_NUM_LEDS, NULL, 0, NULL);
}
out:
diff --git a/drivers/mfd/cs5535-mfd.c b/drivers/mfd/cs5535-mfd.c
index 3419e72..2b28213 100644
--- a/drivers/mfd/cs5535-mfd.c
+++ b/drivers/mfd/cs5535-mfd.c
@@ -149,7 +149,7 @@
}
err = mfd_add_devices(&pdev->dev, -1, cs5535_mfd_cells,
- ARRAY_SIZE(cs5535_mfd_cells), NULL, 0);
+ ARRAY_SIZE(cs5535_mfd_cells), NULL, 0, NULL);
if (err) {
dev_err(&pdev->dev, "MFD add devices failed: %d\n", err);
goto err_disable;
diff --git a/drivers/mfd/da9052-core.c b/drivers/mfd/da9052-core.c
index 2544910..a0a62b2 100644
--- a/drivers/mfd/da9052-core.c
+++ b/drivers/mfd/da9052-core.c
@@ -803,7 +803,7 @@
dev_err(da9052->dev, "DA9052 ADC IRQ failed ret=%d\n", ret);
ret = mfd_add_devices(da9052->dev, -1, da9052_subdev_info,
- ARRAY_SIZE(da9052_subdev_info), NULL, 0);
+ ARRAY_SIZE(da9052_subdev_info), NULL, 0, NULL);
if (ret)
goto err;
diff --git a/drivers/mfd/davinci_voicecodec.c b/drivers/mfd/davinci_voicecodec.c
index 4e2af2c..45e83a6 100644
--- a/drivers/mfd/davinci_voicecodec.c
+++ b/drivers/mfd/davinci_voicecodec.c
@@ -129,7 +129,7 @@
cell->pdata_size = sizeof(*davinci_vc);
ret = mfd_add_devices(&pdev->dev, pdev->id, davinci_vc->cells,
- DAVINCI_VC_CELLS, NULL, 0);
+ DAVINCI_VC_CELLS, NULL, 0, NULL);
if (ret != 0) {
dev_err(&pdev->dev, "fail to register client devices\n");
goto fail4;
diff --git a/drivers/mfd/db8500-prcmu.c b/drivers/mfd/db8500-prcmu.c
index 7040a00..6b67edb 100644
--- a/drivers/mfd/db8500-prcmu.c
+++ b/drivers/mfd/db8500-prcmu.c
@@ -418,6 +418,9 @@
static atomic_t ac_wake_req_state = ATOMIC_INIT(0);
+/* Functions definition */
+static void compute_armss_rate(void);
+
/* Spinlocks */
static DEFINE_SPINLOCK(prcmu_lock);
static DEFINE_SPINLOCK(clkout_lock);
@@ -517,6 +520,7 @@
}
};
+
/*
* Used by MCDE to setup all necessary PRCMU registers
*/
@@ -1013,6 +1017,7 @@
(mb1_transfer.ack.arm_opp != opp))
r = -EIO;
+ compute_armss_rate();
mutex_unlock(&mb1_transfer.lock);
return r;
@@ -1612,6 +1617,7 @@
if ((branch == PLL_FIX) || ((branch == PLL_DIV) &&
(val & PRCM_PLL_FREQ_DIV2EN) &&
((reg == PRCM_PLLSOC0_FREQ) ||
+ (reg == PRCM_PLLARM_FREQ) ||
(reg == PRCM_PLLDDR_FREQ))))
div *= 2;
@@ -1661,6 +1667,39 @@
else
return 0;
}
+static unsigned long latest_armss_rate;
+static unsigned long armss_rate(void)
+{
+ return latest_armss_rate;
+}
+
+static void compute_armss_rate(void)
+{
+ u32 r;
+ unsigned long rate;
+
+ r = readl(PRCM_ARM_CHGCLKREQ);
+
+ if (r & PRCM_ARM_CHGCLKREQ_PRCM_ARM_CHGCLKREQ) {
+ /* External ARMCLKFIX clock */
+
+ rate = pll_rate(PRCM_PLLDDR_FREQ, ROOT_CLOCK_RATE, PLL_FIX);
+
+ /* Check PRCM_ARM_CHGCLKREQ divider */
+ if (!(r & PRCM_ARM_CHGCLKREQ_PRCM_ARM_DIVSEL))
+ rate /= 2;
+
+ /* Check PRCM_ARMCLKFIX_MGT divider */
+ r = readl(PRCM_ARMCLKFIX_MGT);
+ r &= PRCM_CLK_MGT_CLKPLLDIV_MASK;
+ rate /= r;
+
+ } else {/* ARM PLL */
+ rate = pll_rate(PRCM_PLLARM_FREQ, ROOT_CLOCK_RATE, PLL_DIV);
+ }
+
+ latest_armss_rate = rate;
+}
static unsigned long dsiclk_rate(u8 n)
{
@@ -1707,6 +1746,8 @@
return pll_rate(PRCM_PLLSOC0_FREQ, ROOT_CLOCK_RATE, PLL_RAW);
else if (clock == PRCMU_PLLSOC1)
return pll_rate(PRCM_PLLSOC1_FREQ, ROOT_CLOCK_RATE, PLL_RAW);
+ else if (clock == PRCMU_ARMSS)
+ return armss_rate();
else if (clock == PRCMU_PLLDDR)
return pll_rate(PRCM_PLLDDR_FREQ, ROOT_CLOCK_RATE, PLL_RAW);
else if (clock == PRCMU_PLLDSI)
@@ -2693,6 +2734,7 @@
handle_simple_irq);
set_irq_flags(irq, IRQF_VALID);
}
+ compute_armss_rate();
}
static void __init init_prcm_registers(void)
@@ -3010,7 +3052,7 @@
prcmu_config_esram0_deep_sleep(ESRAM0_DEEP_SLEEP_STATE_RET);
err = mfd_add_devices(&pdev->dev, 0, db8500_prcmu_devs,
- ARRAY_SIZE(db8500_prcmu_devs), NULL, 0);
+ ARRAY_SIZE(db8500_prcmu_devs), NULL, 0, NULL);
if (err) {
pr_err("prcmu: Failed to add subdevices\n");
return err;
diff --git a/drivers/mfd/dbx500-prcmu-regs.h b/drivers/mfd/dbx500-prcmu-regs.h
index 23108a6..79c76eb 100644
--- a/drivers/mfd/dbx500-prcmu-regs.h
+++ b/drivers/mfd/dbx500-prcmu-regs.h
@@ -61,7 +61,8 @@
#define PRCM_PLLARM_LOCKP_PRCM_PLLARM_LOCKP3 0x2
#define PRCM_ARM_CHGCLKREQ (_PRCMU_BASE + 0x114)
-#define PRCM_ARM_CHGCLKREQ_PRCM_ARM_CHGCLKREQ 0x1
+#define PRCM_ARM_CHGCLKREQ_PRCM_ARM_CHGCLKREQ BIT(0)
+#define PRCM_ARM_CHGCLKREQ_PRCM_ARM_DIVSEL BIT(16)
#define PRCM_PLLARM_ENABLE (_PRCMU_BASE + 0x98)
#define PRCM_PLLARM_ENABLE_PRCM_PLLARM_ENABLE 0x1
@@ -140,6 +141,7 @@
/* PRCMU clock/PLL/reset registers */
#define PRCM_PLLSOC0_FREQ (_PRCMU_BASE + 0x080)
#define PRCM_PLLSOC1_FREQ (_PRCMU_BASE + 0x084)
+#define PRCM_PLLARM_FREQ (_PRCMU_BASE + 0x088)
#define PRCM_PLLDDR_FREQ (_PRCMU_BASE + 0x08C)
#define PRCM_PLL_FREQ_D_SHIFT 0
#define PRCM_PLL_FREQ_D_MASK BITS(0, 7)
diff --git a/drivers/mfd/htc-pasic3.c b/drivers/mfd/htc-pasic3.c
index 04c7093..9e5453d 100644
--- a/drivers/mfd/htc-pasic3.c
+++ b/drivers/mfd/htc-pasic3.c
@@ -168,7 +168,7 @@
/* the first 5 PASIC3 registers control the DS1WM */
ds1wm_resources[0].end = (5 << asic->bus_shift) - 1;
ret = mfd_add_devices(&pdev->dev, pdev->id,
- &ds1wm_cell, 1, r, irq);
+ &ds1wm_cell, 1, r, irq, NULL);
if (ret < 0)
dev_warn(dev, "failed to register DS1WM\n");
}
@@ -176,7 +176,8 @@
if (pdata && pdata->led_pdata) {
led_cell.platform_data = pdata->led_pdata;
led_cell.pdata_size = sizeof(struct pasic3_leds_machinfo);
- ret = mfd_add_devices(&pdev->dev, pdev->id, &led_cell, 1, r, 0);
+ ret = mfd_add_devices(&pdev->dev, pdev->id, &led_cell, 1, r,
+ 0, NULL);
if (ret < 0)
dev_warn(dev, "failed to register LED device\n");
}
diff --git a/drivers/mfd/intel_msic.c b/drivers/mfd/intel_msic.c
index 59df558..266bdc5b 100644
--- a/drivers/mfd/intel_msic.c
+++ b/drivers/mfd/intel_msic.c
@@ -344,13 +344,13 @@
continue;
ret = mfd_add_devices(&pdev->dev, -1, &msic_devs[i], 1, NULL,
- pdata->irq[i]);
+ pdata->irq[i], NULL);
if (ret)
goto fail;
}
ret = mfd_add_devices(&pdev->dev, 0, msic_other_devs,
- ARRAY_SIZE(msic_other_devs), NULL, 0);
+ ARRAY_SIZE(msic_other_devs), NULL, 0, NULL);
if (ret)
goto fail;
diff --git a/drivers/mfd/janz-cmodio.c b/drivers/mfd/janz-cmodio.c
index 2ea9998..965c480 100644
--- a/drivers/mfd/janz-cmodio.c
+++ b/drivers/mfd/janz-cmodio.c
@@ -147,7 +147,7 @@
}
return mfd_add_devices(&pdev->dev, 0, priv->cells,
- num_probed, NULL, pdev->irq);
+ num_probed, NULL, pdev->irq, NULL);
}
/*
diff --git a/drivers/mfd/jz4740-adc.c b/drivers/mfd/jz4740-adc.c
index 87662a1..c6b6d7d 100644
--- a/drivers/mfd/jz4740-adc.c
+++ b/drivers/mfd/jz4740-adc.c
@@ -287,7 +287,8 @@
writeb(0xff, adc->base + JZ_REG_ADC_CTRL);
ret = mfd_add_devices(&pdev->dev, 0, jz4740_adc_cells,
- ARRAY_SIZE(jz4740_adc_cells), mem_base, irq_base);
+ ARRAY_SIZE(jz4740_adc_cells), mem_base,
+ irq_base, NULL);
if (ret < 0)
goto err_clk_put;
diff --git a/drivers/mfd/lm3533-core.c b/drivers/mfd/lm3533-core.c
index 0b2879b..24212f4 100644
--- a/drivers/mfd/lm3533-core.c
+++ b/drivers/mfd/lm3533-core.c
@@ -393,7 +393,8 @@
lm3533_als_devs[0].platform_data = pdata->als;
lm3533_als_devs[0].pdata_size = sizeof(*pdata->als);
- ret = mfd_add_devices(lm3533->dev, 0, lm3533_als_devs, 1, NULL, 0);
+ ret = mfd_add_devices(lm3533->dev, 0, lm3533_als_devs, 1, NULL,
+ 0, NULL);
if (ret) {
dev_err(lm3533->dev, "failed to add ALS device\n");
return ret;
@@ -422,7 +423,7 @@
}
ret = mfd_add_devices(lm3533->dev, 0, lm3533_bl_devs,
- pdata->num_backlights, NULL, 0);
+ pdata->num_backlights, NULL, 0, NULL);
if (ret) {
dev_err(lm3533->dev, "failed to add backlight devices\n");
return ret;
@@ -451,7 +452,7 @@
}
ret = mfd_add_devices(lm3533->dev, 0, lm3533_led_devs,
- pdata->num_leds, NULL, 0);
+ pdata->num_leds, NULL, 0, NULL);
if (ret) {
dev_err(lm3533->dev, "failed to add LED devices\n");
return ret;
diff --git a/drivers/mfd/lpc_ich.c b/drivers/mfd/lpc_ich.c
index 027cc8f..092ad4b 100644
--- a/drivers/mfd/lpc_ich.c
+++ b/drivers/mfd/lpc_ich.c
@@ -750,7 +750,7 @@
lpc_ich_finalize_cell(&lpc_ich_cells[LPC_GPIO], id);
ret = mfd_add_devices(&dev->dev, -1, &lpc_ich_cells[LPC_GPIO],
- 1, NULL, 0);
+ 1, NULL, 0, NULL);
gpio_done:
if (acpi_conflict)
@@ -765,7 +765,6 @@
u32 base_addr_cfg;
u32 base_addr;
int ret;
- bool acpi_conflict = false;
struct resource *res;
/* Setup power management base register */
@@ -780,20 +779,11 @@
res = wdt_io_res(ICH_RES_IO_TCO);
res->start = base_addr + ACPIBASE_TCO_OFF;
res->end = base_addr + ACPIBASE_TCO_END;
- ret = acpi_check_resource_conflict(res);
- if (ret) {
- acpi_conflict = true;
- goto wdt_done;
- }
res = wdt_io_res(ICH_RES_IO_SMI);
res->start = base_addr + ACPIBASE_SMI_OFF;
res->end = base_addr + ACPIBASE_SMI_END;
- ret = acpi_check_resource_conflict(res);
- if (ret) {
- acpi_conflict = true;
- goto wdt_done;
- }
+
lpc_ich_enable_acpi_space(dev);
/*
@@ -813,21 +803,13 @@
res = wdt_mem_res(ICH_RES_MEM_GCS);
res->start = base_addr + ACPIBASE_GCS_OFF;
res->end = base_addr + ACPIBASE_GCS_END;
- ret = acpi_check_resource_conflict(res);
- if (ret) {
- acpi_conflict = true;
- goto wdt_done;
- }
}
lpc_ich_finalize_cell(&lpc_ich_cells[LPC_WDT], id);
ret = mfd_add_devices(&dev->dev, -1, &lpc_ich_cells[LPC_WDT],
- 1, NULL, 0);
+ 1, NULL, 0, NULL);
wdt_done:
- if (acpi_conflict)
- pr_warn("Resource conflict(s) found affecting %s\n",
- lpc_ich_cells[LPC_WDT].name);
return ret;
}
diff --git a/drivers/mfd/lpc_sch.c b/drivers/mfd/lpc_sch.c
index 9f20abc..f6b9c5c 100644
--- a/drivers/mfd/lpc_sch.c
+++ b/drivers/mfd/lpc_sch.c
@@ -127,7 +127,8 @@
lpc_sch_cells[i].id = id->device;
ret = mfd_add_devices(&dev->dev, 0,
- lpc_sch_cells, ARRAY_SIZE(lpc_sch_cells), NULL, 0);
+ lpc_sch_cells, ARRAY_SIZE(lpc_sch_cells), NULL,
+ 0, NULL);
if (ret)
goto out_dev;
@@ -153,7 +154,8 @@
tunnelcreek_cells[i].id = id->device;
ret = mfd_add_devices(&dev->dev, 0, tunnelcreek_cells,
- ARRAY_SIZE(tunnelcreek_cells), NULL, 0);
+ ARRAY_SIZE(tunnelcreek_cells), NULL,
+ 0, NULL);
}
return ret;
diff --git a/drivers/mfd/max77686.c b/drivers/mfd/max77686.c
index c03e12b..d9e24c8 100644
--- a/drivers/mfd/max77686.c
+++ b/drivers/mfd/max77686.c
@@ -126,7 +126,7 @@
max77686_irq_init(max77686);
ret = mfd_add_devices(max77686->dev, -1, max77686_devs,
- ARRAY_SIZE(max77686_devs), NULL, 0);
+ ARRAY_SIZE(max77686_devs), NULL, 0, NULL);
if (ret < 0)
goto err_mfd;
diff --git a/drivers/mfd/max77693-irq.c b/drivers/mfd/max77693-irq.c
index 2b40356..1029d01 100644
--- a/drivers/mfd/max77693-irq.c
+++ b/drivers/mfd/max77693-irq.c
@@ -137,6 +137,9 @@
const struct max77693_irq_data *irq_data =
irq_to_max77693_irq(max77693, data->irq);
+ if (irq_data->group >= MAX77693_IRQ_GROUP_NR)
+ return;
+
if (irq_data->group >= MUIC_INT1 && irq_data->group <= MUIC_INT3)
max77693->irq_masks_cur[irq_data->group] &= ~irq_data->mask;
else
@@ -149,6 +152,9 @@
const struct max77693_irq_data *irq_data =
irq_to_max77693_irq(max77693, data->irq);
+ if (irq_data->group >= MAX77693_IRQ_GROUP_NR)
+ return;
+
if (irq_data->group >= MUIC_INT1 && irq_data->group <= MUIC_INT3)
max77693->irq_masks_cur[irq_data->group] |= irq_data->mask;
else
@@ -200,7 +206,7 @@
if (irq_src & MAX77693_IRQSRC_MUIC)
/* MUIC INT1 ~ INT3 */
- max77693_bulk_read(max77693->regmap, MAX77693_MUIC_REG_INT1,
+ max77693_bulk_read(max77693->regmap_muic, MAX77693_MUIC_REG_INT1,
MAX77693_NUM_IRQ_MUIC_REGS, &irq_reg[MUIC_INT1]);
/* Apply masking */
@@ -255,7 +261,8 @@
{
struct irq_domain *domain;
int i;
- int ret;
+ int ret = 0;
+ u8 intsrc_mask;
mutex_init(&max77693->irqlock);
@@ -287,19 +294,38 @@
&max77693_irq_domain_ops, max77693);
if (!domain) {
dev_err(max77693->dev, "could not create irq domain\n");
- return -ENODEV;
+ ret = -ENODEV;
+ goto err_irq;
}
max77693->irq_domain = domain;
+ /* Unmask max77693 interrupt */
+ ret = max77693_read_reg(max77693->regmap,
+ MAX77693_PMIC_REG_INTSRC_MASK, &intsrc_mask);
+ if (ret < 0) {
+ dev_err(max77693->dev, "fail to read PMIC register\n");
+ goto err_irq;
+ }
+
+ intsrc_mask &= ~(MAX77693_IRQSRC_CHG);
+ intsrc_mask &= ~(MAX77693_IRQSRC_FLASH);
+ intsrc_mask &= ~(MAX77693_IRQSRC_MUIC);
+ ret = max77693_write_reg(max77693->regmap,
+ MAX77693_PMIC_REG_INTSRC_MASK, intsrc_mask);
+ if (ret < 0) {
+ dev_err(max77693->dev, "fail to write PMIC register\n");
+ goto err_irq;
+ }
+
ret = request_threaded_irq(max77693->irq, NULL, max77693_irq_thread,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
"max77693-irq", max77693);
-
if (ret)
dev_err(max77693->dev, "Failed to request IRQ %d: %d\n",
max77693->irq, ret);
- return 0;
+err_irq:
+ return ret;
}
void max77693_irq_exit(struct max77693_dev *max77693)
diff --git a/drivers/mfd/max77693.c b/drivers/mfd/max77693.c
index a1811cb..cc5155e 100644
--- a/drivers/mfd/max77693.c
+++ b/drivers/mfd/max77693.c
@@ -152,6 +152,20 @@
max77693->haptic = i2c_new_dummy(i2c->adapter, I2C_ADDR_HAPTIC);
i2c_set_clientdata(max77693->haptic, max77693);
+ /*
+ * Initialize register map for MUIC device because use regmap-muic
+ * instance of MUIC device when irq of max77693 is initialized
+ * before call max77693-muic probe() function.
+ */
+ max77693->regmap_muic = devm_regmap_init_i2c(max77693->muic,
+ &max77693_regmap_config);
+ if (IS_ERR(max77693->regmap_muic)) {
+ ret = PTR_ERR(max77693->regmap_muic);
+ dev_err(max77693->dev,
+ "failed to allocate register map: %d\n", ret);
+ goto err_regmap;
+ }
+
ret = max77693_irq_init(max77693);
if (ret < 0)
goto err_irq;
@@ -159,7 +173,7 @@
pm_runtime_set_active(max77693->dev);
ret = mfd_add_devices(max77693->dev, -1, max77693_devs,
- ARRAY_SIZE(max77693_devs), NULL, 0);
+ ARRAY_SIZE(max77693_devs), NULL, 0, NULL);
if (ret < 0)
goto err_mfd;
diff --git a/drivers/mfd/max8925-core.c b/drivers/mfd/max8925-core.c
index 825a7f0..ee53757 100644
--- a/drivers/mfd/max8925-core.c
+++ b/drivers/mfd/max8925-core.c
@@ -598,7 +598,7 @@
ret = mfd_add_devices(chip->dev, 0, &rtc_devs[0],
ARRAY_SIZE(rtc_devs),
- &rtc_resources[0], chip->irq_base);
+ &rtc_resources[0], chip->irq_base, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add rtc subdev\n");
goto out;
@@ -606,7 +606,7 @@
ret = mfd_add_devices(chip->dev, 0, &onkey_devs[0],
ARRAY_SIZE(onkey_devs),
- &onkey_resources[0], 0);
+ &onkey_resources[0], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add onkey subdev\n");
goto out_dev;
@@ -615,7 +615,7 @@
if (pdata) {
ret = mfd_add_devices(chip->dev, 0, ®ulator_devs[0],
ARRAY_SIZE(regulator_devs),
- ®ulator_resources[0], 0);
+ ®ulator_resources[0], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add regulator subdev\n");
goto out_dev;
@@ -625,7 +625,7 @@
if (pdata && pdata->backlight) {
ret = mfd_add_devices(chip->dev, 0, &backlight_devs[0],
ARRAY_SIZE(backlight_devs),
- &backlight_resources[0], 0);
+ &backlight_resources[0], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add backlight subdev\n");
goto out_dev;
@@ -635,7 +635,7 @@
if (pdata && pdata->power) {
ret = mfd_add_devices(chip->dev, 0, &power_devs[0],
ARRAY_SIZE(power_devs),
- &power_supply_resources[0], 0);
+ &power_supply_resources[0], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add power supply "
"subdev\n");
@@ -646,7 +646,7 @@
if (pdata && pdata->touch) {
ret = mfd_add_devices(chip->dev, 0, &touch_devs[0],
ARRAY_SIZE(touch_devs),
- &touch_resources[0], 0);
+ &touch_resources[0], 0, NULL);
if (ret < 0) {
dev_err(chip->dev, "Failed to add touch subdev\n");
goto out_dev;
diff --git a/drivers/mfd/max8997.c b/drivers/mfd/max8997.c
index 10b629c..f1235170 100644
--- a/drivers/mfd/max8997.c
+++ b/drivers/mfd/max8997.c
@@ -160,7 +160,7 @@
mfd_add_devices(max8997->dev, -1, max8997_devs,
ARRAY_SIZE(max8997_devs),
- NULL, 0);
+ NULL, 0, NULL);
/*
* TODO: enable others (flash, muic, rtc, battery, ...) and
diff --git a/drivers/mfd/max8998.c b/drivers/mfd/max8998.c
index 6ef56d2..d7218cc 100644
--- a/drivers/mfd/max8998.c
+++ b/drivers/mfd/max8998.c
@@ -161,13 +161,13 @@
switch (id->driver_data) {
case TYPE_LP3974:
ret = mfd_add_devices(max8998->dev, -1,
- lp3974_devs, ARRAY_SIZE(lp3974_devs),
- NULL, 0);
+ lp3974_devs, ARRAY_SIZE(lp3974_devs),
+ NULL, 0, NULL);
break;
case TYPE_MAX8998:
ret = mfd_add_devices(max8998->dev, -1,
- max8998_devs, ARRAY_SIZE(max8998_devs),
- NULL, 0);
+ max8998_devs, ARRAY_SIZE(max8998_devs),
+ NULL, 0, NULL);
break;
default:
ret = -EINVAL;
diff --git a/drivers/mfd/mc13xxx-core.c b/drivers/mfd/mc13xxx-core.c
index b801dc7..1ec79b5 100644
--- a/drivers/mfd/mc13xxx-core.c
+++ b/drivers/mfd/mc13xxx-core.c
@@ -612,7 +612,7 @@
if (!cell.name)
return -ENOMEM;
- return mfd_add_devices(mc13xxx->dev, -1, &cell, 1, NULL, 0);
+ return mfd_add_devices(mc13xxx->dev, -1, &cell, 1, NULL, 0, NULL);
}
static int mc13xxx_add_subdevice(struct mc13xxx *mc13xxx, const char *format)
diff --git a/drivers/mfd/mfd-core.c b/drivers/mfd/mfd-core.c
index 0c3a01c..f8b7771 100644
--- a/drivers/mfd/mfd-core.c
+++ b/drivers/mfd/mfd-core.c
@@ -74,12 +74,11 @@
static int mfd_add_device(struct device *parent, int id,
const struct mfd_cell *cell,
struct resource *mem_base,
- int irq_base)
+ int irq_base, struct irq_domain *domain)
{
struct resource *res;
struct platform_device *pdev;
struct device_node *np = NULL;
- struct irq_domain *domain = NULL;
int ret = -ENOMEM;
int r;
@@ -97,7 +96,6 @@
for_each_child_of_node(parent->of_node, np) {
if (of_device_is_compatible(np, cell->of_compatible)) {
pdev->dev.of_node = np;
- domain = irq_find_host(parent->of_node);
break;
}
}
@@ -177,7 +175,7 @@
int mfd_add_devices(struct device *parent, int id,
struct mfd_cell *cells, int n_devs,
struct resource *mem_base,
- int irq_base)
+ int irq_base, struct irq_domain *domain)
{
int i;
int ret = 0;
@@ -191,7 +189,8 @@
for (i = 0; i < n_devs; i++) {
atomic_set(&cnts[i], 0);
cells[i].usage_count = &cnts[i];
- ret = mfd_add_device(parent, id, cells + i, mem_base, irq_base);
+ ret = mfd_add_device(parent, id, cells + i, mem_base,
+ irq_base, domain);
if (ret)
break;
}
@@ -247,7 +246,8 @@
for (i = 0; i < n_clones; i++) {
cell_entry.name = clones[i];
/* don't give up if a single call fails; just report error */
- if (mfd_add_device(pdev->dev.parent, -1, &cell_entry, NULL, 0))
+ if (mfd_add_device(pdev->dev.parent, -1, &cell_entry, NULL, 0,
+ NULL))
dev_err(dev, "failed to create platform device '%s'\n",
clones[i]);
}
diff --git a/drivers/mfd/palmas.c b/drivers/mfd/palmas.c
index c4a69f1..a345f9bb 100644
--- a/drivers/mfd/palmas.c
+++ b/drivers/mfd/palmas.c
@@ -453,7 +453,8 @@
ret = mfd_add_devices(palmas->dev, -1,
children, ARRAY_SIZE(palmas_children),
- NULL, regmap_irq_chip_get_base(palmas->irq_data));
+ NULL, regmap_irq_chip_get_base(palmas->irq_data),
+ NULL);
kfree(children);
if (ret < 0)
diff --git a/drivers/mfd/rc5t583.c b/drivers/mfd/rc5t583.c
index cdc1df7..ff61efc 100644
--- a/drivers/mfd/rc5t583.c
+++ b/drivers/mfd/rc5t583.c
@@ -281,7 +281,7 @@
if (i2c->irq) {
ret = rc5t583_irq_init(rc5t583, i2c->irq, pdata->irq_base);
- /* Still continue with waring if irq init fails */
+ /* Still continue with warning, if irq init fails */
if (ret)
dev_warn(&i2c->dev, "IRQ init failed: %d\n", ret);
else
@@ -289,7 +289,7 @@
}
ret = mfd_add_devices(rc5t583->dev, -1, rc5t583_subdevs,
- ARRAY_SIZE(rc5t583_subdevs), NULL, 0);
+ ARRAY_SIZE(rc5t583_subdevs), NULL, 0, NULL);
if (ret) {
dev_err(&i2c->dev, "add mfd devices failed: %d\n", ret);
goto err_add_devs;
diff --git a/drivers/mfd/rdc321x-southbridge.c b/drivers/mfd/rdc321x-southbridge.c
index 685d61e..fbabc3cb 100644
--- a/drivers/mfd/rdc321x-southbridge.c
+++ b/drivers/mfd/rdc321x-southbridge.c
@@ -1,5 +1,5 @@
/*
- * RDC321x MFD southbrige driver
+ * RDC321x MFD southbridge driver
*
* Copyright (C) 2007-2010 Florian Fainelli <florian@openwrt.org>
* Copyright (C) 2010 Bernhard Loos <bernhardloos@googlemail.com>
@@ -87,7 +87,8 @@
rdc321x_wdt_pdata.sb_pdev = pdev;
return mfd_add_devices(&pdev->dev, -1,
- rdc321x_sb_cells, ARRAY_SIZE(rdc321x_sb_cells), NULL, 0);
+ rdc321x_sb_cells, ARRAY_SIZE(rdc321x_sb_cells),
+ NULL, 0, NULL);
}
static void __devexit rdc321x_sb_remove(struct pci_dev *pdev)
diff --git a/drivers/mfd/sec-core.c b/drivers/mfd/sec-core.c
index 2988efd..49d361a 100644
--- a/drivers/mfd/sec-core.c
+++ b/drivers/mfd/sec-core.c
@@ -141,19 +141,19 @@
switch (sec_pmic->device_type) {
case S5M8751X:
ret = mfd_add_devices(sec_pmic->dev, -1, s5m8751_devs,
- ARRAY_SIZE(s5m8751_devs), NULL, 0);
+ ARRAY_SIZE(s5m8751_devs), NULL, 0, NULL);
break;
case S5M8763X:
ret = mfd_add_devices(sec_pmic->dev, -1, s5m8763_devs,
- ARRAY_SIZE(s5m8763_devs), NULL, 0);
+ ARRAY_SIZE(s5m8763_devs), NULL, 0, NULL);
break;
case S5M8767X:
ret = mfd_add_devices(sec_pmic->dev, -1, s5m8767_devs,
- ARRAY_SIZE(s5m8767_devs), NULL, 0);
+ ARRAY_SIZE(s5m8767_devs), NULL, 0, NULL);
break;
case S2MPS11X:
ret = mfd_add_devices(sec_pmic->dev, -1, s2mps11_devs,
- ARRAY_SIZE(s2mps11_devs), NULL, 0);
+ ARRAY_SIZE(s2mps11_devs), NULL, 0, NULL);
break;
default:
/* If this happens the probe function is problem */
diff --git a/drivers/mfd/sta2x11-mfd.c b/drivers/mfd/sta2x11-mfd.c
index d31fed0..d35da68 100644
--- a/drivers/mfd/sta2x11-mfd.c
+++ b/drivers/mfd/sta2x11-mfd.c
@@ -407,7 +407,7 @@
sta2x11_mfd_bar0,
ARRAY_SIZE(sta2x11_mfd_bar0),
&pdev->resource[0],
- 0);
+ 0, NULL);
if (err) {
dev_err(&pdev->dev, "mfd_add_devices[0] failed: %d\n", err);
goto err_disable;
@@ -417,7 +417,7 @@
sta2x11_mfd_bar1,
ARRAY_SIZE(sta2x11_mfd_bar1),
&pdev->resource[1],
- 0);
+ 0, NULL);
if (err) {
dev_err(&pdev->dev, "mfd_add_devices[1] failed: %d\n", err);
goto err_disable;
diff --git a/drivers/mfd/stmpe.c b/drivers/mfd/stmpe.c
index 2dd8d49..c94f521 100644
--- a/drivers/mfd/stmpe.c
+++ b/drivers/mfd/stmpe.c
@@ -962,7 +962,7 @@
struct mfd_cell *cell, int irq)
{
return mfd_add_devices(stmpe->dev, stmpe->pdata->id, cell, 1,
- NULL, stmpe->irq_base + irq);
+ NULL, stmpe->irq_base + irq, NULL);
}
static int __devinit stmpe_devices_init(struct stmpe *stmpe)
diff --git a/drivers/mfd/t7l66xb.c b/drivers/mfd/t7l66xb.c
index 2d9e879..b32940e 100644
--- a/drivers/mfd/t7l66xb.c
+++ b/drivers/mfd/t7l66xb.c
@@ -388,7 +388,7 @@
ret = mfd_add_devices(&dev->dev, dev->id,
t7l66xb_cells, ARRAY_SIZE(t7l66xb_cells),
- iomem, t7l66xb->irq_base);
+ iomem, t7l66xb->irq_base, NULL);
if (!ret)
return 0;
diff --git a/drivers/mfd/tc3589x.c b/drivers/mfd/tc3589x.c
index 048bf05..b56ba6b 100644
--- a/drivers/mfd/tc3589x.c
+++ b/drivers/mfd/tc3589x.c
@@ -262,8 +262,8 @@
if (blocks & TC3589x_BLOCK_GPIO) {
ret = mfd_add_devices(tc3589x->dev, -1, tc3589x_dev_gpio,
- ARRAY_SIZE(tc3589x_dev_gpio), NULL,
- tc3589x->irq_base);
+ ARRAY_SIZE(tc3589x_dev_gpio), NULL,
+ tc3589x->irq_base, NULL);
if (ret) {
dev_err(tc3589x->dev, "failed to add gpio child\n");
return ret;
@@ -273,8 +273,8 @@
if (blocks & TC3589x_BLOCK_KEYPAD) {
ret = mfd_add_devices(tc3589x->dev, -1, tc3589x_dev_keypad,
- ARRAY_SIZE(tc3589x_dev_keypad), NULL,
- tc3589x->irq_base);
+ ARRAY_SIZE(tc3589x_dev_keypad), NULL,
+ tc3589x->irq_base, NULL);
if (ret) {
dev_err(tc3589x->dev, "failed to keypad child\n");
return ret;
diff --git a/drivers/mfd/tc6387xb.c b/drivers/mfd/tc6387xb.c
index d20a284..413c891 100644
--- a/drivers/mfd/tc6387xb.c
+++ b/drivers/mfd/tc6387xb.c
@@ -192,7 +192,7 @@
printk(KERN_INFO "Toshiba tc6387xb initialised\n");
ret = mfd_add_devices(&dev->dev, dev->id, tc6387xb_cells,
- ARRAY_SIZE(tc6387xb_cells), iomem, irq);
+ ARRAY_SIZE(tc6387xb_cells), iomem, irq, NULL);
if (!ret)
return 0;
diff --git a/drivers/mfd/tc6393xb.c b/drivers/mfd/tc6393xb.c
index 9612264..dcab026 100644
--- a/drivers/mfd/tc6393xb.c
+++ b/drivers/mfd/tc6393xb.c
@@ -700,8 +700,8 @@
tc6393xb_cells[TC6393XB_CELL_FB].pdata_size = sizeof(*tcpd->fb_data);
ret = mfd_add_devices(&dev->dev, dev->id,
- tc6393xb_cells, ARRAY_SIZE(tc6393xb_cells),
- iomem, tcpd->irq_base);
+ tc6393xb_cells, ARRAY_SIZE(tc6393xb_cells),
+ iomem, tcpd->irq_base, NULL);
if (!ret)
return 0;
diff --git a/drivers/mfd/ti-ssp.c b/drivers/mfd/ti-ssp.c
index 4fb0e6c..7c3675a 100644
--- a/drivers/mfd/ti-ssp.c
+++ b/drivers/mfd/ti-ssp.c
@@ -412,7 +412,7 @@
cells[id].data_size = data->pdata_size;
}
- error = mfd_add_devices(dev, 0, cells, 2, NULL, 0);
+ error = mfd_add_devices(dev, 0, cells, 2, NULL, 0, NULL);
if (error < 0) {
dev_err(dev, "cannot add mfd cells\n");
goto error_enable;
diff --git a/drivers/mfd/timberdale.c b/drivers/mfd/timberdale.c
index a447f4e..cccc626 100644
--- a/drivers/mfd/timberdale.c
+++ b/drivers/mfd/timberdale.c
@@ -757,25 +757,25 @@
err = mfd_add_devices(&dev->dev, -1,
timberdale_cells_bar0_cfg0,
ARRAY_SIZE(timberdale_cells_bar0_cfg0),
- &dev->resource[0], msix_entries[0].vector);
+ &dev->resource[0], msix_entries[0].vector, NULL);
break;
case TIMB_HW_VER1:
err = mfd_add_devices(&dev->dev, -1,
timberdale_cells_bar0_cfg1,
ARRAY_SIZE(timberdale_cells_bar0_cfg1),
- &dev->resource[0], msix_entries[0].vector);
+ &dev->resource[0], msix_entries[0].vector, NULL);
break;
case TIMB_HW_VER2:
err = mfd_add_devices(&dev->dev, -1,
timberdale_cells_bar0_cfg2,
ARRAY_SIZE(timberdale_cells_bar0_cfg2),
- &dev->resource[0], msix_entries[0].vector);
+ &dev->resource[0], msix_entries[0].vector, NULL);
break;
case TIMB_HW_VER3:
err = mfd_add_devices(&dev->dev, -1,
timberdale_cells_bar0_cfg3,
ARRAY_SIZE(timberdale_cells_bar0_cfg3),
- &dev->resource[0], msix_entries[0].vector);
+ &dev->resource[0], msix_entries[0].vector, NULL);
break;
default:
dev_err(&dev->dev, "Uknown IP setup: %d.%d.%d\n",
@@ -792,7 +792,7 @@
err = mfd_add_devices(&dev->dev, 0,
timberdale_cells_bar1, ARRAY_SIZE(timberdale_cells_bar1),
- &dev->resource[1], msix_entries[0].vector);
+ &dev->resource[1], msix_entries[0].vector, NULL);
if (err) {
dev_err(&dev->dev, "mfd_add_devices failed: %d\n", err);
goto err_mfd2;
@@ -803,7 +803,7 @@
((priv->fw.config & TIMB_HW_VER_MASK) == TIMB_HW_VER3)) {
err = mfd_add_devices(&dev->dev, 1, timberdale_cells_bar2,
ARRAY_SIZE(timberdale_cells_bar2),
- &dev->resource[2], msix_entries[0].vector);
+ &dev->resource[2], msix_entries[0].vector, NULL);
if (err) {
dev_err(&dev->dev, "mfd_add_devices failed: %d\n", err);
goto err_mfd2;
diff --git a/drivers/mfd/tps6105x.c b/drivers/mfd/tps6105x.c
index a293b97..14051bd 100644
--- a/drivers/mfd/tps6105x.c
+++ b/drivers/mfd/tps6105x.c
@@ -188,7 +188,7 @@
}
ret = mfd_add_devices(&client->dev, 0, tps6105x_cells,
- ARRAY_SIZE(tps6105x_cells), NULL, 0);
+ ARRAY_SIZE(tps6105x_cells), NULL, 0, NULL);
if (ret)
goto fail;
diff --git a/drivers/mfd/tps6507x.c b/drivers/mfd/tps6507x.c
index 33ba772..1b20349 100644
--- a/drivers/mfd/tps6507x.c
+++ b/drivers/mfd/tps6507x.c
@@ -100,7 +100,7 @@
ret = mfd_add_devices(tps6507x->dev, -1,
tps6507x_devs, ARRAY_SIZE(tps6507x_devs),
- NULL, 0);
+ NULL, 0, NULL);
if (ret < 0)
goto err;
diff --git a/drivers/mfd/tps65090.c b/drivers/mfd/tps65090.c
index 80e24f4..50fd87c 100644
--- a/drivers/mfd/tps65090.c
+++ b/drivers/mfd/tps65090.c
@@ -292,7 +292,7 @@
}
ret = mfd_add_devices(tps65090->dev, -1, tps65090s,
- ARRAY_SIZE(tps65090s), NULL, 0);
+ ARRAY_SIZE(tps65090s), NULL, 0, NULL);
if (ret) {
dev_err(&client->dev, "add mfd devices failed with err: %d\n",
ret);
diff --git a/drivers/mfd/tps65217.c b/drivers/mfd/tps65217.c
index 61c097a..a95e942 100644
--- a/drivers/mfd/tps65217.c
+++ b/drivers/mfd/tps65217.c
@@ -24,11 +24,18 @@
#include <linux/slab.h>
#include <linux/regmap.h>
#include <linux/err.h>
-#include <linux/regulator/of_regulator.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/mfd/core.h>
#include <linux/mfd/tps65217.h>
+static struct mfd_cell tps65217s[] = {
+ {
+ .name = "tps65217-pmic",
+ },
+};
+
/**
* tps65217_reg_read: Read a single tps65217 register.
*
@@ -133,83 +140,48 @@
}
EXPORT_SYMBOL_GPL(tps65217_clear_bits);
-#ifdef CONFIG_OF
-static struct of_regulator_match reg_matches[] = {
- { .name = "dcdc1", .driver_data = (void *)TPS65217_DCDC_1 },
- { .name = "dcdc2", .driver_data = (void *)TPS65217_DCDC_2 },
- { .name = "dcdc3", .driver_data = (void *)TPS65217_DCDC_3 },
- { .name = "ldo1", .driver_data = (void *)TPS65217_LDO_1 },
- { .name = "ldo2", .driver_data = (void *)TPS65217_LDO_2 },
- { .name = "ldo3", .driver_data = (void *)TPS65217_LDO_3 },
- { .name = "ldo4", .driver_data = (void *)TPS65217_LDO_4 },
-};
-
-static struct tps65217_board *tps65217_parse_dt(struct i2c_client *client)
-{
- struct device_node *node = client->dev.of_node;
- struct tps65217_board *pdata;
- struct device_node *regs;
- int count = ARRAY_SIZE(reg_matches);
- int ret, i;
-
- regs = of_find_node_by_name(node, "regulators");
- if (!regs)
- return NULL;
-
- ret = of_regulator_match(&client->dev, regs, reg_matches, count);
- of_node_put(regs);
- if ((ret < 0) || (ret > count))
- return NULL;
-
- count = ret;
- pdata = devm_kzalloc(&client->dev, count * sizeof(*pdata), GFP_KERNEL);
- if (!pdata)
- return NULL;
-
- for (i = 0; i < count; i++) {
- if (!reg_matches[i].init_data || !reg_matches[i].of_node)
- continue;
-
- pdata->tps65217_init_data[i] = reg_matches[i].init_data;
- pdata->of_node[i] = reg_matches[i].of_node;
- }
-
- return pdata;
-}
-
-static struct of_device_id tps65217_of_match[] = {
- { .compatible = "ti,tps65217", },
- { },
-};
-#else
-static struct tps65217_board *tps65217_parse_dt(struct i2c_client *client)
-{
- return NULL;
-}
-#endif
-
static struct regmap_config tps65217_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
};
+static const struct of_device_id tps65217_of_match[] = {
+ { .compatible = "ti,tps65217", .data = (void *)TPS65217 },
+ { /* sentinel */ },
+};
+
static int __devinit tps65217_probe(struct i2c_client *client,
const struct i2c_device_id *ids)
{
struct tps65217 *tps;
- struct regulator_init_data *reg_data;
- struct tps65217_board *pdata = client->dev.platform_data;
- int i, ret;
unsigned int version;
+ unsigned int chip_id = ids->driver_data;
+ const struct of_device_id *match;
+ int ret;
- if (!pdata && client->dev.of_node)
- pdata = tps65217_parse_dt(client);
+ if (client->dev.of_node) {
+ match = of_match_device(tps65217_of_match, &client->dev);
+ if (!match) {
+ dev_err(&client->dev,
+ "Failed to find matching dt id\n");
+ return -EINVAL;
+ }
+ chip_id = (unsigned int)match->data;
+ }
+
+ if (!chip_id) {
+ dev_err(&client->dev, "id is null.\n");
+ return -ENODEV;
+ }
tps = devm_kzalloc(&client->dev, sizeof(*tps), GFP_KERNEL);
if (!tps)
return -ENOMEM;
- tps->pdata = pdata;
+ i2c_set_clientdata(client, tps);
+ tps->dev = &client->dev;
+ tps->id = chip_id;
+
tps->regmap = devm_regmap_init_i2c(client, &tps65217_regmap_config);
if (IS_ERR(tps->regmap)) {
ret = PTR_ERR(tps->regmap);
@@ -218,8 +190,12 @@
return ret;
}
- i2c_set_clientdata(client, tps);
- tps->dev = &client->dev;
+ ret = mfd_add_devices(tps->dev, -1, tps65217s,
+ ARRAY_SIZE(tps65217s), NULL, 0, NULL);
+ if (ret < 0) {
+ dev_err(tps->dev, "mfd_add_devices failed: %d\n", ret);
+ return ret;
+ }
ret = tps65217_reg_read(tps, TPS65217_REG_CHIPID, &version);
if (ret < 0) {
@@ -232,41 +208,21 @@
(version & TPS65217_CHIPID_CHIP_MASK) >> 4,
version & TPS65217_CHIPID_REV_MASK);
- for (i = 0; i < TPS65217_NUM_REGULATOR; i++) {
- struct platform_device *pdev;
-
- pdev = platform_device_alloc("tps65217-pmic", i);
- if (!pdev) {
- dev_err(tps->dev, "Cannot create regulator %d\n", i);
- continue;
- }
-
- pdev->dev.parent = tps->dev;
- pdev->dev.of_node = pdata->of_node[i];
- reg_data = pdata->tps65217_init_data[i];
- platform_device_add_data(pdev, reg_data, sizeof(*reg_data));
- tps->regulator_pdev[i] = pdev;
-
- platform_device_add(pdev);
- }
-
return 0;
}
static int __devexit tps65217_remove(struct i2c_client *client)
{
struct tps65217 *tps = i2c_get_clientdata(client);
- int i;
- for (i = 0; i < TPS65217_NUM_REGULATOR; i++)
- platform_device_unregister(tps->regulator_pdev[i]);
+ mfd_remove_devices(tps->dev);
return 0;
}
static const struct i2c_device_id tps65217_id_table[] = {
- {"tps65217", 0xF0},
- {/* end of list */}
+ {"tps65217", TPS65217},
+ { /* sentinel */ }
};
MODULE_DEVICE_TABLE(i2c, tps65217_id_table);
diff --git a/drivers/mfd/tps6586x.c b/drivers/mfd/tps6586x.c
index 353c348..345960ca 100644
--- a/drivers/mfd/tps6586x.c
+++ b/drivers/mfd/tps6586x.c
@@ -25,6 +25,7 @@
#include <linux/i2c.h>
#include <linux/regmap.h>
#include <linux/regulator/of_regulator.h>
+#include <linux/regulator/machine.h>
#include <linux/mfd/core.h>
#include <linux/mfd/tps6586x.h>
@@ -346,6 +347,7 @@
#ifdef CONFIG_OF
static struct of_regulator_match tps6586x_matches[] = {
+ { .name = "sys", .driver_data = (void *)TPS6586X_ID_SYS },
{ .name = "sm0", .driver_data = (void *)TPS6586X_ID_SM_0 },
{ .name = "sm1", .driver_data = (void *)TPS6586X_ID_SM_1 },
{ .name = "sm2", .driver_data = (void *)TPS6586X_ID_SM_2 },
@@ -369,6 +371,7 @@
struct tps6586x_platform_data *pdata;
struct tps6586x_subdev_info *devs;
struct device_node *regs;
+ const char *sys_rail_name = NULL;
unsigned int count;
unsigned int i, j;
int err;
@@ -391,12 +394,22 @@
return NULL;
for (i = 0, j = 0; i < num && j < count; i++) {
+ struct regulator_init_data *reg_idata;
+
if (!tps6586x_matches[i].init_data)
continue;
+ reg_idata = tps6586x_matches[i].init_data;
devs[j].name = "tps6586x-regulator";
devs[j].platform_data = tps6586x_matches[i].init_data;
devs[j].id = (int)tps6586x_matches[i].driver_data;
+ if (devs[j].id == TPS6586X_ID_SYS)
+ sys_rail_name = reg_idata->constraints.name;
+
+ if ((devs[j].id == TPS6586X_ID_LDO_5) ||
+ (devs[j].id == TPS6586X_ID_LDO_RTC))
+ reg_idata->supply_regulator = sys_rail_name;
+
devs[j].of_node = tps6586x_matches[i].of_node;
j++;
}
@@ -493,7 +506,8 @@
}
ret = mfd_add_devices(tps6586x->dev, -1,
- tps6586x_cell, ARRAY_SIZE(tps6586x_cell), NULL, 0);
+ tps6586x_cell, ARRAY_SIZE(tps6586x_cell),
+ NULL, 0, NULL);
if (ret < 0) {
dev_err(&client->dev, "mfd_add_devices failed: %d\n", ret);
goto err_mfd_add;
diff --git a/drivers/mfd/tps65910.c b/drivers/mfd/tps65910.c
index 1c56379..d3ce4d5 100644
--- a/drivers/mfd/tps65910.c
+++ b/drivers/mfd/tps65910.c
@@ -254,7 +254,7 @@
ret = mfd_add_devices(tps65910->dev, -1,
tps65910s, ARRAY_SIZE(tps65910s),
- NULL, 0);
+ NULL, 0, NULL);
if (ret < 0) {
dev_err(&i2c->dev, "mfd_add_devices failed: %d\n", ret);
return ret;
diff --git a/drivers/mfd/tps65911-comparator.c b/drivers/mfd/tps65911-comparator.c
index 5a62e6b..0b6e361 100644
--- a/drivers/mfd/tps65911-comparator.c
+++ b/drivers/mfd/tps65911-comparator.c
@@ -136,7 +136,7 @@
ret = comp_threshold_set(tps65910, COMP2, pdata->vmbch2_threshold);
if (ret < 0) {
- dev_err(&pdev->dev, "cannot set COMP2 theshold\n");
+ dev_err(&pdev->dev, "cannot set COMP2 threshold\n");
return ret;
}
diff --git a/drivers/mfd/tps65912-core.c b/drivers/mfd/tps65912-core.c
index 74fd8cb..4658b5b 100644
--- a/drivers/mfd/tps65912-core.c
+++ b/drivers/mfd/tps65912-core.c
@@ -146,7 +146,7 @@
ret = mfd_add_devices(tps65912->dev, -1,
tps65912s, ARRAY_SIZE(tps65912s),
- NULL, 0);
+ NULL, 0, NULL);
if (ret < 0)
goto err;
diff --git a/drivers/mfd/twl4030-audio.c b/drivers/mfd/twl4030-audio.c
index 838ce4e..77c9acb 100644
--- a/drivers/mfd/twl4030-audio.c
+++ b/drivers/mfd/twl4030-audio.c
@@ -223,7 +223,7 @@
if (childs)
ret = mfd_add_devices(&pdev->dev, pdev->id, audio->cells,
- childs, NULL, 0);
+ childs, NULL, 0, NULL);
else {
dev_err(&pdev->dev, "No platform data found for childs\n");
ret = -ENODEV;
diff --git a/drivers/mfd/twl6040-core.c b/drivers/mfd/twl6040-core.c
index b0fad0f..3dca5c1 100644
--- a/drivers/mfd/twl6040-core.c
+++ b/drivers/mfd/twl6040-core.c
@@ -632,7 +632,7 @@
}
ret = mfd_add_devices(&client->dev, -1, twl6040->cells, children,
- NULL, 0);
+ NULL, 0, NULL);
if (ret)
goto mfd_err;
diff --git a/drivers/mfd/vx855.c b/drivers/mfd/vx855.c
index 872aff2..b9a636d 100644
--- a/drivers/mfd/vx855.c
+++ b/drivers/mfd/vx855.c
@@ -102,7 +102,7 @@
vx855_gpio_resources[1].end = vx855_gpio_resources[1].start + 3;
ret = mfd_add_devices(&pdev->dev, -1, vx855_cells, ARRAY_SIZE(vx855_cells),
- NULL, 0);
+ NULL, 0, NULL);
/* we always return -ENODEV here in order to enable other
* drivers like old, not-yet-platform_device ported i2c-viapro */
diff --git a/drivers/mfd/wl1273-core.c b/drivers/mfd/wl1273-core.c
index f39b756..86e0e43 100644
--- a/drivers/mfd/wl1273-core.c
+++ b/drivers/mfd/wl1273-core.c
@@ -241,7 +241,7 @@
__func__, children);
r = mfd_add_devices(&client->dev, -1, core->cells,
- children, NULL, 0);
+ children, NULL, 0, NULL);
if (r)
goto err;
diff --git a/drivers/mfd/wm831x-core.c b/drivers/mfd/wm831x-core.c
index 946698f..3017310 100644
--- a/drivers/mfd/wm831x-core.c
+++ b/drivers/mfd/wm831x-core.c
@@ -1813,27 +1813,27 @@
case WM8310:
ret = mfd_add_devices(wm831x->dev, wm831x_num,
wm8310_devs, ARRAY_SIZE(wm8310_devs),
- NULL, 0);
+ NULL, 0, NULL);
break;
case WM8311:
ret = mfd_add_devices(wm831x->dev, wm831x_num,
wm8311_devs, ARRAY_SIZE(wm8311_devs),
- NULL, 0);
+ NULL, 0, NULL);
if (!pdata || !pdata->disable_touch)
mfd_add_devices(wm831x->dev, wm831x_num,
touch_devs, ARRAY_SIZE(touch_devs),
- NULL, 0);
+ NULL, 0, NULL);
break;
case WM8312:
ret = mfd_add_devices(wm831x->dev, wm831x_num,
wm8312_devs, ARRAY_SIZE(wm8312_devs),
- NULL, 0);
+ NULL, 0, NULL);
if (!pdata || !pdata->disable_touch)
mfd_add_devices(wm831x->dev, wm831x_num,
touch_devs, ARRAY_SIZE(touch_devs),
- NULL, 0);
+ NULL, 0, NULL);
break;
case WM8320:
@@ -1842,7 +1842,7 @@
case WM8326:
ret = mfd_add_devices(wm831x->dev, wm831x_num,
wm8320_devs, ARRAY_SIZE(wm8320_devs),
- NULL, 0);
+ NULL, 0, NULL);
break;
default:
@@ -1867,7 +1867,7 @@
if (ret & WM831X_XTAL_ENA) {
ret = mfd_add_devices(wm831x->dev, wm831x_num,
rtc_devs, ARRAY_SIZE(rtc_devs),
- NULL, 0);
+ NULL, 0, NULL);
if (ret != 0) {
dev_err(wm831x->dev, "Failed to add RTC: %d\n", ret);
goto err_irq;
@@ -1880,7 +1880,7 @@
/* Treat errors as non-critical */
ret = mfd_add_devices(wm831x->dev, wm831x_num, backlight_devs,
ARRAY_SIZE(backlight_devs), NULL,
- 0);
+ 0, NULL);
if (ret < 0)
dev_err(wm831x->dev, "Failed to add backlight: %d\n",
ret);
diff --git a/drivers/mfd/wm8400-core.c b/drivers/mfd/wm8400-core.c
index 4b7d378..639ca35 100644
--- a/drivers/mfd/wm8400-core.c
+++ b/drivers/mfd/wm8400-core.c
@@ -70,7 +70,7 @@
.pdata_size = sizeof(*wm8400),
};
- return mfd_add_devices(wm8400->dev, -1, &cell, 1, NULL, 0);
+ return mfd_add_devices(wm8400->dev, -1, &cell, 1, NULL, 0, NULL);
}
/*
diff --git a/drivers/mfd/wm8994-core.c b/drivers/mfd/wm8994-core.c
index eec74aa..2febf88 100644
--- a/drivers/mfd/wm8994-core.c
+++ b/drivers/mfd/wm8994-core.c
@@ -414,7 +414,7 @@
ret = mfd_add_devices(wm8994->dev, -1,
wm8994_regulator_devs,
ARRAY_SIZE(wm8994_regulator_devs),
- NULL, 0);
+ NULL, 0, NULL);
if (ret != 0) {
dev_err(wm8994->dev, "Failed to add children: %d\n", ret);
goto err;
@@ -648,7 +648,7 @@
ret = mfd_add_devices(wm8994->dev, -1,
wm8994_devs, ARRAY_SIZE(wm8994_devs),
- NULL, 0);
+ NULL, 0, NULL);
if (ret != 0) {
dev_err(wm8994->dev, "Failed to add children: %d\n", ret);
goto err_irq;
diff --git a/drivers/mfd/wm8994-irq.c b/drivers/mfd/wm8994-irq.c
index 0aac4af..a050e56 100644
--- a/drivers/mfd/wm8994-irq.c
+++ b/drivers/mfd/wm8994-irq.c
@@ -135,6 +135,7 @@
.status_base = WM8994_INTERRUPT_STATUS_1,
.mask_base = WM8994_INTERRUPT_STATUS_1_MASK,
.ack_base = WM8994_INTERRUPT_STATUS_1,
+ .runtime_pm = true,
};
int wm8994_irq_init(struct wm8994 *wm8994)
diff --git a/drivers/mmc/card/block.c b/drivers/mmc/card/block.c
index f1c84de..172a768 100644
--- a/drivers/mmc/card/block.c
+++ b/drivers/mmc/card/block.c
@@ -1411,7 +1411,8 @@
/* complete ongoing async transfer before issuing discard */
if (card->host->areq)
mmc_blk_issue_rw_rq(mq, NULL);
- if (req->cmd_flags & REQ_SECURE)
+ if (req->cmd_flags & REQ_SECURE &&
+ !(card->quirks & MMC_QUIRK_SEC_ERASE_TRIM_BROKEN))
ret = mmc_blk_issue_secdiscard_rq(mq, req);
else
ret = mmc_blk_issue_discard_rq(mq, req);
@@ -1716,6 +1717,7 @@
#define CID_MANFID_SANDISK 0x2
#define CID_MANFID_TOSHIBA 0x11
#define CID_MANFID_MICRON 0x13
+#define CID_MANFID_SAMSUNG 0x15
static const struct mmc_fixup blk_fixups[] =
{
@@ -1752,6 +1754,28 @@
MMC_FIXUP(CID_NAME_ANY, CID_MANFID_MICRON, 0x200, add_quirk_mmc,
MMC_QUIRK_LONG_READ_TIME),
+ /*
+ * On these Samsung MoviNAND parts, performing secure erase or
+ * secure trim can result in unrecoverable corruption due to a
+ * firmware bug.
+ */
+ MMC_FIXUP("M8G2FA", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("MAG4FA", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("MBG8FA", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("MCGAFA", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("VAL00M", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("VYL00M", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("KYL00M", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+ MMC_FIXUP("VZL00M", CID_MANFID_SAMSUNG, CID_OEMID_ANY, add_quirk_mmc,
+ MMC_QUIRK_SEC_ERASE_TRIM_BROKEN),
+
END_FIXUP
};
diff --git a/drivers/mmc/core/sdio.c b/drivers/mmc/core/sdio.c
index d4619e2..2273ce6 100644
--- a/drivers/mmc/core/sdio.c
+++ b/drivers/mmc/core/sdio.c
@@ -641,7 +641,7 @@
/*
* If the host and card support UHS-I mode request the card
* to switch to 1.8V signaling level. No 1.8v signalling if
- * UHS mode is not enabled to maintain compatibilty and some
+ * UHS mode is not enabled to maintain compatibility and some
* systems that claim 1.8v signalling in fact do not support
* it.
*/
diff --git a/drivers/mmc/host/at91_mci.c b/drivers/mmc/host/at91_mci.c
index efdb81d..74bed0f 100644
--- a/drivers/mmc/host/at91_mci.c
+++ b/drivers/mmc/host/at91_mci.c
@@ -356,7 +356,7 @@
}
/*
- * Update bytes tranfered count during a write operation
+ * Update bytes transfered count during a write operation
*/
static void at91_mci_update_bytes_xfered(struct at91mci_host *host)
{
diff --git a/drivers/mmc/host/atmel-mci.c b/drivers/mmc/host/atmel-mci.c
index 322412c..852d5fb 100644
--- a/drivers/mmc/host/atmel-mci.c
+++ b/drivers/mmc/host/atmel-mci.c
@@ -81,6 +81,7 @@
bool has_bad_data_ordering;
bool need_reset_after_xfer;
bool need_blksz_mul_4;
+ bool need_notbusy_for_read_ops;
};
struct atmel_mci_dma {
@@ -1021,7 +1022,7 @@
}
/*
- * Stop data transfer because error(s) occured.
+ * Stop data transfer because error(s) occurred.
*/
static void atmci_stop_transfer_pdc(struct atmel_mci *host)
{
@@ -1625,7 +1626,8 @@
__func__);
atmci_set_completed(host, EVENT_XFER_COMPLETE);
- if (host->data->flags & MMC_DATA_WRITE) {
+ if (host->caps.need_notbusy_for_read_ops ||
+ (host->data->flags & MMC_DATA_WRITE)) {
atmci_writel(host, ATMCI_IER, ATMCI_NOTBUSY);
state = STATE_WAITING_NOTBUSY;
} else if (host->mrq->stop) {
@@ -2218,6 +2220,7 @@
host->caps.has_bad_data_ordering = 1;
host->caps.need_reset_after_xfer = 1;
host->caps.need_blksz_mul_4 = 1;
+ host->caps.need_notbusy_for_read_ops = 0;
/* keep only major version number */
switch (version & 0xf00) {
@@ -2238,6 +2241,7 @@
case 0x200:
host->caps.has_rwproof = 1;
host->caps.need_blksz_mul_4 = 0;
+ host->caps.need_notbusy_for_read_ops = 1;
case 0x100:
host->caps.has_bad_data_ordering = 0;
host->caps.need_reset_after_xfer = 0;
diff --git a/drivers/mmc/host/bfin_sdh.c b/drivers/mmc/host/bfin_sdh.c
index 0366617..a17dd73 100644
--- a/drivers/mmc/host/bfin_sdh.c
+++ b/drivers/mmc/host/bfin_sdh.c
@@ -49,13 +49,6 @@
#define bfin_write_SDH_CFG bfin_write_RSI_CFG
#endif
-struct dma_desc_array {
- unsigned long start_addr;
- unsigned short cfg;
- unsigned short x_count;
- short x_modify;
-} __packed;
-
struct sdh_host {
struct mmc_host *mmc;
spinlock_t lock;
diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c
index 72dc3cd..af40d22 100644
--- a/drivers/mmc/host/dw_mmc.c
+++ b/drivers/mmc/host/dw_mmc.c
@@ -627,6 +627,7 @@
{
struct dw_mci *host = slot->host;
u32 div;
+ u32 clk_en_a;
if (slot->clock != host->current_speed) {
div = host->bus_hz / slot->clock;
@@ -659,9 +660,11 @@
mci_send_cmd(slot,
SDMMC_CMD_UPD_CLK | SDMMC_CMD_PRV_DAT_WAIT, 0);
- /* enable clock */
- mci_writel(host, CLKENA, ((SDMMC_CLKEN_ENABLE |
- SDMMC_CLKEN_LOW_PWR) << slot->id));
+ /* enable clock; only low power if no SDIO */
+ clk_en_a = SDMMC_CLKEN_ENABLE << slot->id;
+ if (!(mci_readl(host, INTMASK) & SDMMC_INT_SDIO(slot->id)))
+ clk_en_a |= SDMMC_CLKEN_LOW_PWR << slot->id;
+ mci_writel(host, CLKENA, clk_en_a);
/* inform CIU */
mci_send_cmd(slot,
@@ -862,6 +865,30 @@
return present;
}
+/*
+ * Disable lower power mode.
+ *
+ * Low power mode will stop the card clock when idle. According to the
+ * description of the CLKENA register we should disable low power mode
+ * for SDIO cards if we need SDIO interrupts to work.
+ *
+ * This function is fast if low power mode is already disabled.
+ */
+static void dw_mci_disable_low_power(struct dw_mci_slot *slot)
+{
+ struct dw_mci *host = slot->host;
+ u32 clk_en_a;
+ const u32 clken_low_pwr = SDMMC_CLKEN_LOW_PWR << slot->id;
+
+ clk_en_a = mci_readl(host, CLKENA);
+
+ if (clk_en_a & clken_low_pwr) {
+ mci_writel(host, CLKENA, clk_en_a & ~clken_low_pwr);
+ mci_send_cmd(slot, SDMMC_CMD_UPD_CLK |
+ SDMMC_CMD_PRV_DAT_WAIT, 0);
+ }
+}
+
static void dw_mci_enable_sdio_irq(struct mmc_host *mmc, int enb)
{
struct dw_mci_slot *slot = mmc_priv(mmc);
@@ -871,6 +898,14 @@
/* Enable/disable Slot Specific SDIO interrupt */
int_mask = mci_readl(host, INTMASK);
if (enb) {
+ /*
+ * Turn off low power mode if it was enabled. This is a bit of
+ * a heavy operation and we disable / enable IRQs a lot, so
+ * we'll leave low power mode disabled and it will get
+ * re-enabled again in dw_mci_setup_bus().
+ */
+ dw_mci_disable_low_power(slot);
+
mci_writel(host, INTMASK,
(int_mask | SDMMC_INT_SDIO(slot->id)));
} else {
@@ -1429,22 +1464,10 @@
nbytes += len;
remain -= len;
} while (remain);
- sg_miter->consumed = offset;
+ sg_miter->consumed = offset;
status = mci_readl(host, MINTSTS);
mci_writel(host, RINTSTS, SDMMC_INT_RXDR);
- if (status & DW_MCI_DATA_ERROR_FLAGS) {
- host->data_status = status;
- data->bytes_xfered += nbytes;
- sg_miter_stop(sg_miter);
- host->sg = NULL;
- smp_wmb();
-
- set_bit(EVENT_DATA_ERROR, &host->pending_events);
-
- tasklet_schedule(&host->tasklet);
- return;
- }
} while (status & SDMMC_INT_RXDR); /*if the RXDR is ready read again*/
data->bytes_xfered += nbytes;
@@ -1497,23 +1520,10 @@
nbytes += len;
remain -= len;
} while (remain);
- sg_miter->consumed = offset;
+ sg_miter->consumed = offset;
status = mci_readl(host, MINTSTS);
mci_writel(host, RINTSTS, SDMMC_INT_TXDR);
- if (status & DW_MCI_DATA_ERROR_FLAGS) {
- host->data_status = status;
- data->bytes_xfered += nbytes;
- sg_miter_stop(sg_miter);
- host->sg = NULL;
-
- smp_wmb();
-
- set_bit(EVENT_DATA_ERROR, &host->pending_events);
-
- tasklet_schedule(&host->tasklet);
- return;
- }
} while (status & SDMMC_INT_TXDR); /* if TXDR write again */
data->bytes_xfered += nbytes;
@@ -1547,12 +1557,11 @@
static irqreturn_t dw_mci_interrupt(int irq, void *dev_id)
{
struct dw_mci *host = dev_id;
- u32 status, pending;
+ u32 pending;
unsigned int pass_count = 0;
int i;
do {
- status = mci_readl(host, RINTSTS);
pending = mci_readl(host, MINTSTS); /* read-only mask reg */
/*
@@ -1570,7 +1579,7 @@
if (pending & DW_MCI_CMD_ERROR_FLAGS) {
mci_writel(host, RINTSTS, DW_MCI_CMD_ERROR_FLAGS);
- host->cmd_status = status;
+ host->cmd_status = pending;
smp_wmb();
set_bit(EVENT_CMD_COMPLETE, &host->pending_events);
}
@@ -1578,18 +1587,16 @@
if (pending & DW_MCI_DATA_ERROR_FLAGS) {
/* if there is an error report DATA_ERROR */
mci_writel(host, RINTSTS, DW_MCI_DATA_ERROR_FLAGS);
- host->data_status = status;
+ host->data_status = pending;
smp_wmb();
set_bit(EVENT_DATA_ERROR, &host->pending_events);
- if (!(pending & (SDMMC_INT_DTO | SDMMC_INT_DCRC |
- SDMMC_INT_SBE | SDMMC_INT_EBE)))
- tasklet_schedule(&host->tasklet);
+ tasklet_schedule(&host->tasklet);
}
if (pending & SDMMC_INT_DATA_OVER) {
mci_writel(host, RINTSTS, SDMMC_INT_DATA_OVER);
if (!host->data_status)
- host->data_status = status;
+ host->data_status = pending;
smp_wmb();
if (host->dir_status == DW_MCI_RECV_STATUS) {
if (host->sg != NULL)
@@ -1613,7 +1620,7 @@
if (pending & SDMMC_INT_CMD_DONE) {
mci_writel(host, RINTSTS, SDMMC_INT_CMD_DONE);
- dw_mci_cmd_interrupt(host, status);
+ dw_mci_cmd_interrupt(host, pending);
}
if (pending & SDMMC_INT_CD) {
diff --git a/drivers/mmc/host/mxs-mmc.c b/drivers/mmc/host/mxs-mmc.c
index a51f930..ad3fcea 100644
--- a/drivers/mmc/host/mxs-mmc.c
+++ b/drivers/mmc/host/mxs-mmc.c
@@ -285,11 +285,11 @@
writel(stat & MXS_MMC_IRQ_BITS,
host->base + HW_SSP_CTRL1(host) + STMP_OFFSET_REG_CLR);
+ spin_unlock(&host->lock);
+
if ((stat & BM_SSP_CTRL1_SDIO_IRQ) && (stat & BM_SSP_CTRL1_SDIO_IRQ_EN))
mmc_signal_sdio_irq(host->mmc);
- spin_unlock(&host->lock);
-
if (stat & BM_SSP_CTRL1_RESP_TIMEOUT_IRQ)
cmd->error = -ETIMEDOUT;
else if (stat & BM_SSP_CTRL1_RESP_ERR_IRQ)
@@ -644,11 +644,6 @@
host->base + HW_SSP_CTRL0 + STMP_OFFSET_REG_SET);
writel(BM_SSP_CTRL1_SDIO_IRQ_EN,
host->base + HW_SSP_CTRL1(host) + STMP_OFFSET_REG_SET);
-
- if (readl(host->base + HW_SSP_STATUS(host)) &
- BM_SSP_STATUS_SDIO_IRQ)
- mmc_signal_sdio_irq(host->mmc);
-
} else {
writel(BM_SSP_CTRL0_SDIO_IRQ_CHECK,
host->base + HW_SSP_CTRL0 + STMP_OFFSET_REG_CLR);
@@ -657,6 +652,11 @@
}
spin_unlock_irqrestore(&host->lock, flags);
+
+ if (enable && readl(host->base + HW_SSP_STATUS(host)) &
+ BM_SSP_STATUS_SDIO_IRQ)
+ mmc_signal_sdio_irq(host->mmc);
+
}
static const struct mmc_host_ops mxs_mmc_ops = {
diff --git a/drivers/mmc/host/omap.c b/drivers/mmc/host/omap.c
index 50e08f0..a5999a7 100644
--- a/drivers/mmc/host/omap.c
+++ b/drivers/mmc/host/omap.c
@@ -668,7 +668,7 @@
static void
mmc_omap_xfer_data(struct mmc_omap_host *host, int write)
{
- int n;
+ int n, nwords;
if (host->buffer_bytes_left == 0) {
host->sg_idx++;
@@ -678,15 +678,23 @@
n = 64;
if (n > host->buffer_bytes_left)
n = host->buffer_bytes_left;
+
+ nwords = n / 2;
+ nwords += n & 1; /* handle odd number of bytes to transfer */
+
host->buffer_bytes_left -= n;
host->total_bytes_left -= n;
host->data->bytes_xfered += n;
if (write) {
- __raw_writesw(host->virt_base + OMAP_MMC_REG(host, DATA), host->buffer, n);
+ __raw_writesw(host->virt_base + OMAP_MMC_REG(host, DATA),
+ host->buffer, nwords);
} else {
- __raw_readsw(host->virt_base + OMAP_MMC_REG(host, DATA), host->buffer, n);
+ __raw_readsw(host->virt_base + OMAP_MMC_REG(host, DATA),
+ host->buffer, nwords);
}
+
+ host->buffer += nwords;
}
static inline void mmc_omap_report_irq(u16 status)
diff --git a/drivers/mmc/host/omap_hsmmc.c b/drivers/mmc/host/omap_hsmmc.c
index 3a09f93..686e256 100644
--- a/drivers/mmc/host/omap_hsmmc.c
+++ b/drivers/mmc/host/omap_hsmmc.c
@@ -447,7 +447,7 @@
OMAP_HSMMC_WRITE(host->base, SYSCTL,
OMAP_HSMMC_READ(host->base, SYSCTL) & ~CEN);
if ((OMAP_HSMMC_READ(host->base, SYSCTL) & CEN) != 0x0)
- dev_dbg(mmc_dev(host->mmc), "MMC Clock is not stoped\n");
+ dev_dbg(mmc_dev(host->mmc), "MMC Clock is not stopped\n");
}
static void omap_hsmmc_enable_irq(struct omap_hsmmc_host *host,
diff --git a/drivers/mmc/host/sdhci-esdhc-imx.c b/drivers/mmc/host/sdhci-esdhc-imx.c
index e23f813..32f4a07 100644
--- a/drivers/mmc/host/sdhci-esdhc-imx.c
+++ b/drivers/mmc/host/sdhci-esdhc-imx.c
@@ -315,7 +315,7 @@
new_val = val & (SDHCI_CTRL_LED | \
SDHCI_CTRL_4BITBUS | \
SDHCI_CTRL_D3CD);
- /* ensure the endianess */
+ /* ensure the endianness */
new_val |= ESDHC_HOST_CONTROL_LE;
/* bits 8&9 are reserved on mx25 */
if (!is_imx25_esdhc(imx_data)) {
diff --git a/drivers/mmc/host/sdhci-esdhc.h b/drivers/mmc/host/sdhci-esdhc.h
index b97b2f5..d25f9ab 100644
--- a/drivers/mmc/host/sdhci-esdhc.h
+++ b/drivers/mmc/host/sdhci-esdhc.h
@@ -48,14 +48,14 @@
int div = 1;
u32 temp;
+ if (clock == 0)
+ goto out;
+
temp = sdhci_readl(host, ESDHC_SYSTEM_CONTROL);
temp &= ~(ESDHC_CLOCK_IPGEN | ESDHC_CLOCK_HCKEN | ESDHC_CLOCK_PEREN
| ESDHC_CLOCK_MASK);
sdhci_writel(host, temp, ESDHC_SYSTEM_CONTROL);
- if (clock == 0)
- goto out;
-
while (host->max_clk / pre_div / 16 > clock && pre_div < 256)
pre_div *= 2;
diff --git a/drivers/mmc/host/vub300.c b/drivers/mmc/host/vub300.c
index 3135a1a..58eab9a 100644
--- a/drivers/mmc/host/vub300.c
+++ b/drivers/mmc/host/vub300.c
@@ -806,7 +806,7 @@
* we suspect a buggy USB host controller
*/
} else if (!vub300->data) {
- /* this means that the command (typically CMD52) suceeded */
+ /* this means that the command (typically CMD52) succeeded */
} else if (vub300->resp.common.header_type != 0x02) {
/*
* this is an error response from the VUB300 chip
diff --git a/drivers/mtd/mtdchar.c b/drivers/mtd/mtdchar.c
index f2f482b..a6e7451 100644
--- a/drivers/mtd/mtdchar.c
+++ b/drivers/mtd/mtdchar.c
@@ -1123,6 +1123,33 @@
}
#endif
+static inline unsigned long get_vm_size(struct vm_area_struct *vma)
+{
+ return vma->vm_end - vma->vm_start;
+}
+
+static inline resource_size_t get_vm_offset(struct vm_area_struct *vma)
+{
+ return (resource_size_t) vma->vm_pgoff << PAGE_SHIFT;
+}
+
+/*
+ * Set a new vm offset.
+ *
+ * Verify that the incoming offset really works as a page offset,
+ * and that the offset and size fit in a resource_size_t.
+ */
+static inline int set_vm_offset(struct vm_area_struct *vma, resource_size_t off)
+{
+ pgoff_t pgoff = off >> PAGE_SHIFT;
+ if (off != (resource_size_t) pgoff << PAGE_SHIFT)
+ return -EINVAL;
+ if (off + get_vm_size(vma) - 1 < off)
+ return -EINVAL;
+ vma->vm_pgoff = pgoff;
+ return 0;
+}
+
/*
* set up a mapping for shared memory segments
*/
@@ -1132,20 +1159,29 @@
struct mtd_file_info *mfi = file->private_data;
struct mtd_info *mtd = mfi->mtd;
struct map_info *map = mtd->priv;
- unsigned long start;
- unsigned long off;
- u32 len;
+ resource_size_t start, off;
+ unsigned long len, vma_len;
if (mtd->type == MTD_RAM || mtd->type == MTD_ROM) {
- off = vma->vm_pgoff << PAGE_SHIFT;
+ off = get_vm_offset(vma);
start = map->phys;
len = PAGE_ALIGN((start & ~PAGE_MASK) + map->size);
start &= PAGE_MASK;
- if ((vma->vm_end - vma->vm_start + off) > len)
+ vma_len = get_vm_size(vma);
+
+ /* Overflow in off+len? */
+ if (vma_len + off < off)
+ return -EINVAL;
+ /* Does it fit in the mapping? */
+ if (vma_len + off > len)
return -EINVAL;
off += start;
- vma->vm_pgoff = off >> PAGE_SHIFT;
+ /* Did that overflow? */
+ if (off < start)
+ return -EINVAL;
+ if (set_vm_offset(vma, off) < 0)
+ return -EINVAL;
vma->vm_flags |= VM_IO | VM_RESERVED;
#ifdef pgprot_noncached
diff --git a/drivers/mtd/ubi/vtbl.c b/drivers/mtd/ubi/vtbl.c
index 437bc19..568307c 100644
--- a/drivers/mtd/ubi/vtbl.c
+++ b/drivers/mtd/ubi/vtbl.c
@@ -340,7 +340,7 @@
* of this LEB as it will be deleted and freed in 'ubi_add_to_av()'.
*/
err = ubi_add_to_av(ubi, ai, new_aeb->pnum, new_aeb->ec, vid_hdr, 0);
- kfree(new_aeb);
+ kmem_cache_free(ai->aeb_slab_cache, new_aeb);
ubi_free_vid_hdr(ubi, vid_hdr);
return err;
@@ -353,7 +353,7 @@
list_add(&new_aeb->u.list, &ai->erase);
goto retry;
}
- kfree(new_aeb);
+ kmem_cache_free(ai->aeb_slab_cache, new_aeb);
out_free:
ubi_free_vid_hdr(ubi, vid_hdr);
return err;
diff --git a/drivers/net/can/janz-ican3.c b/drivers/net/can/janz-ican3.c
index 98ee438..7edadee 100644
--- a/drivers/net/can/janz-ican3.c
+++ b/drivers/net/can/janz-ican3.c
@@ -1391,7 +1391,6 @@
*/
static int ican3_reset_module(struct ican3_dev *mod)
{
- u8 val = 1 << mod->num;
unsigned long start;
u8 runold, runnew;
@@ -1405,8 +1404,7 @@
runold = ioread8(mod->dpm + TARGET_RUNNING);
/* reset the module */
- iowrite8(val, &mod->ctrl->reset_assert);
- iowrite8(val, &mod->ctrl->reset_deassert);
+ iowrite8(0x00, &mod->dpmctrl->hwreset);
/* wait until the module has finished resetting and is running */
start = jiffies;
diff --git a/drivers/net/can/mcp251x.c b/drivers/net/can/mcp251x.c
index a580db2..26e7129 100644
--- a/drivers/net/can/mcp251x.c
+++ b/drivers/net/can/mcp251x.c
@@ -83,6 +83,11 @@
#define INSTRUCTION_LOAD_TXB(n) (0x40 + 2 * (n))
#define INSTRUCTION_READ_RXB(n) (((n) == 0) ? 0x90 : 0x94)
#define INSTRUCTION_RESET 0xC0
+#define RTS_TXB0 0x01
+#define RTS_TXB1 0x02
+#define RTS_TXB2 0x04
+#define INSTRUCTION_RTS(n) (0x80 | ((n) & 0x07))
+
/* MPC251x registers */
#define CANSTAT 0x0e
@@ -397,6 +402,7 @@
static void mcp251x_hw_tx(struct spi_device *spi, struct can_frame *frame,
int tx_buf_idx)
{
+ struct mcp251x_priv *priv = dev_get_drvdata(&spi->dev);
u32 sid, eid, exide, rtr;
u8 buf[SPI_TRANSFER_BUF_LEN];
@@ -418,7 +424,10 @@
buf[TXBDLC_OFF] = (rtr << DLC_RTR_SHIFT) | frame->can_dlc;
memcpy(buf + TXBDAT_OFF, frame->data, frame->can_dlc);
mcp251x_hw_tx_frame(spi, buf, frame->can_dlc, tx_buf_idx);
- mcp251x_write_reg(spi, TXBCTRL(tx_buf_idx), TXBCTRL_TXREQ);
+
+ /* use INSTRUCTION_RTS, to avoid "repeated frame problem" */
+ priv->spi_tx_buf[0] = INSTRUCTION_RTS(1 << tx_buf_idx);
+ mcp251x_spi_trans(priv->spi, 1);
}
static void mcp251x_hw_rx_frame(struct spi_device *spi, u8 *buf,
diff --git a/drivers/net/can/sja1000/sja1000_platform.c b/drivers/net/can/sja1000/sja1000_platform.c
index 4f50145..662c5f7 100644
--- a/drivers/net/can/sja1000/sja1000_platform.c
+++ b/drivers/net/can/sja1000/sja1000_platform.c
@@ -109,7 +109,9 @@
priv = netdev_priv(dev);
dev->irq = res_irq->start;
- priv->irq_flags = res_irq->flags & (IRQF_TRIGGER_MASK | IRQF_SHARED);
+ priv->irq_flags = res_irq->flags & IRQF_TRIGGER_MASK;
+ if (res_irq->flags & IORESOURCE_IRQ_SHAREABLE)
+ priv->irq_flags |= IRQF_SHARED;
priv->reg_base = addr;
/* The CAN clock frequency is half the oscillator clock frequency */
priv->can.clock.freq = pdata->osc_freq / 2;
diff --git a/drivers/net/can/softing/softing_fw.c b/drivers/net/can/softing/softing_fw.c
index 3105961..b595d34 100644
--- a/drivers/net/can/softing/softing_fw.c
+++ b/drivers/net/can/softing/softing_fw.c
@@ -150,7 +150,7 @@
const uint8_t *mem, *end, *dat;
uint16_t type, len;
uint32_t addr;
- uint8_t *buf = NULL;
+ uint8_t *buf = NULL, *new_buf;
int buflen = 0;
int8_t type_end = 0;
@@ -199,11 +199,12 @@
if (len > buflen) {
/* align buflen */
buflen = (len + (1024-1)) & ~(1024-1);
- buf = krealloc(buf, buflen, GFP_KERNEL);
- if (!buf) {
+ new_buf = krealloc(buf, buflen, GFP_KERNEL);
+ if (!new_buf) {
ret = -ENOMEM;
goto failed;
}
+ buf = new_buf;
}
/* verify record data */
memcpy_fromio(buf, &dpram[addr + offset], len);
diff --git a/drivers/net/can/ti_hecc.c b/drivers/net/can/ti_hecc.c
index 527dbcf..9ded21e 100644
--- a/drivers/net/can/ti_hecc.c
+++ b/drivers/net/can/ti_hecc.c
@@ -984,12 +984,12 @@
struct net_device *ndev = platform_get_drvdata(pdev);
struct ti_hecc_priv *priv = netdev_priv(ndev);
+ unregister_candev(ndev);
clk_disable(priv->clk);
clk_put(priv->clk);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
iounmap(priv->base);
release_mem_region(res->start, resource_size(res));
- unregister_candev(ndev);
free_candev(ndev);
platform_set_drvdata(pdev, NULL);
diff --git a/drivers/net/ethernet/3com/typhoon.c b/drivers/net/ethernet/3com/typhoon.c
index b153666..bb9670f 100644
--- a/drivers/net/ethernet/3com/typhoon.c
+++ b/drivers/net/ethernet/3com/typhoon.c
@@ -364,7 +364,7 @@
static inline void
typhoon_inc_tx_index(u32 *index, const int count)
{
- /* if we start using the Hi Tx ring, this needs updateing */
+ /* if we start using the Hi Tx ring, this needs updating */
typhoon_inc_index(index, count, TXLO_ENTRIES);
}
diff --git a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
index 1bf5bbf..55a2e37 100644
--- a/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
+++ b/drivers/net/ethernet/atheros/atl1c/atl1c_main.c
@@ -149,7 +149,7 @@
data &= ~(PCI_ERR_UNC_DLP | PCI_ERR_UNC_FCP);
pci_write_config_dword(pdev, pos + PCI_ERR_UNCOR_SEVER, data);
/* clear error status */
- pci_write_config_word(pdev, pci_pcie_cap(pdev) + PCI_EXP_DEVSTA,
+ pcie_capability_write_word(pdev, PCI_EXP_DEVSTA,
PCI_EXP_DEVSTA_NFED |
PCI_EXP_DEVSTA_FED |
PCI_EXP_DEVSTA_CED |
@@ -2685,7 +2685,7 @@
netif_device_attach(netdev);
}
-static struct pci_error_handlers atl1c_err_handler = {
+static const struct pci_error_handlers atl1c_err_handler = {
.error_detected = atl1c_io_error_detected,
.slot_reset = atl1c_io_slot_reset,
.resume = atl1c_io_resume,
diff --git a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c
index a98acc8..e213da2 100644
--- a/drivers/net/ethernet/atheros/atl1e/atl1e_main.c
+++ b/drivers/net/ethernet/atheros/atl1e/atl1e_main.c
@@ -2489,7 +2489,7 @@
netif_device_attach(netdev);
}
-static struct pci_error_handlers atl1e_err_handler = {
+static const struct pci_error_handlers atl1e_err_handler = {
.error_detected = atl1e_io_error_detected,
.slot_reset = atl1e_io_slot_reset,
.resume = atl1e_io_resume,
diff --git a/drivers/net/ethernet/broadcom/bnx2.c b/drivers/net/ethernet/broadcom/bnx2.c
index 79cebd8..d431070 100644
--- a/drivers/net/ethernet/broadcom/bnx2.c
+++ b/drivers/net/ethernet/broadcom/bnx2.c
@@ -8564,7 +8564,7 @@
return 0;
error:
- iounmap(bp->regview);
+ pci_iounmap(pdev, bp->regview);
pci_release_regions(pdev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
@@ -8742,7 +8742,7 @@
rtnl_unlock();
}
-static struct pci_error_handlers bnx2_err_handler = {
+static const struct pci_error_handlers bnx2_err_handler = {
.error_detected = bnx2_io_error_detected,
.slot_reset = bnx2_io_slot_reset,
.resume = bnx2_io_resume,
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
index 463b9ec..eac2523 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
@@ -1458,7 +1458,7 @@
int fw_stats_req_sz;
/*
- * FW statistics data shortcut (points at the begining of
+ * FW statistics data shortcut (points at the beginning of
* fw_stats buffer + fw_stats_req_sz).
*/
struct bnx2x_fw_stats_data *fw_stats_data;
@@ -1708,9 +1708,6 @@
continue; \
else
-#define for_each_napi_rx_queue(bp, var) \
- for ((var) = 0; (var) < bp->num_napi_queues; (var)++)
-
/* Skip OOO FP */
#define for_each_tx_queue(bp, var) \
for ((var) = 0; (var) < BNX2X_NUM_QUEUES(bp); (var)++) \
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
index e879e19..e8e97a7 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c
@@ -662,14 +662,16 @@
struct bnx2x_fastpath *fp,
struct bnx2x_eth_q_stats *qstats)
{
- /* Do nothing if no IP/L4 csum validation was done */
-
+ /* Do nothing if no L4 csum validation was done.
+ * We do not check whether IP csum was validated. For IPv4 we assume
+ * that if the card got as far as validating the L4 csum, it also
+ * validated the IP csum. IPv6 has no IP csum.
+ */
if (cqe->fast_path_cqe.status_flags &
- (ETH_FAST_PATH_RX_CQE_IP_XSUM_NO_VALIDATION_FLG |
- ETH_FAST_PATH_RX_CQE_L4_XSUM_NO_VALIDATION_FLG))
+ ETH_FAST_PATH_RX_CQE_L4_XSUM_NO_VALIDATION_FLG)
return;
- /* If both IP/L4 validation were done, check if an error was found. */
+ /* If L4 validation was done, check if an error was found. */
if (cqe->fast_path_cqe.type_error_flags &
(ETH_FAST_PATH_RX_CQE_IP_BAD_XSUM_FLG |
@@ -2046,6 +2048,8 @@
*/
bnx2x_setup_tc(bp->dev, bp->max_cos);
+ /* Add all NAPI objects */
+ bnx2x_add_all_napi(bp);
bnx2x_napi_enable(bp);
/* set pf load just before approaching the MCP */
@@ -2408,6 +2412,8 @@
/* Disable HW interrupts, NAPI */
bnx2x_netif_stop(bp, 1);
+ /* Delete all NAPI objects */
+ bnx2x_del_all_napi(bp);
/* Release IRQs */
bnx2x_free_irq(bp);
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
index dfa757e..dfd86a5 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h
@@ -710,17 +710,15 @@
prod = txdata->tx_bd_prod;
cons = txdata->tx_bd_cons;
- /* NUM_TX_RINGS = number of "next-page" entries
- It will be used as a threshold */
- used = SUB_S16(prod, cons) + (s16)NUM_TX_RINGS;
+ used = SUB_S16(prod, cons);
#ifdef BNX2X_STOP_ON_ERROR
WARN_ON(used < 0);
- WARN_ON(used > bp->tx_ring_size);
- WARN_ON((bp->tx_ring_size - used) > MAX_TX_AVAIL);
+ WARN_ON(used > txdata->tx_ring_size);
+ WARN_ON((txdata->tx_ring_size - used) > MAX_TX_AVAIL);
#endif
- return (s16)(bp->tx_ring_size) - used;
+ return (s16)(txdata->tx_ring_size) - used;
}
static inline int bnx2x_tx_queue_has_work(struct bnx2x_fp_txdata *txdata)
@@ -792,7 +790,7 @@
bp->num_napi_queues = bp->num_queues;
/* Add NAPI objects */
- for_each_napi_rx_queue(bp, i)
+ for_each_rx_queue(bp, i)
netif_napi_add(bp->dev, &bnx2x_fp(bp, i, napi),
bnx2x_poll, BNX2X_NAPI_WEIGHT);
}
@@ -801,7 +799,7 @@
{
int i;
- for_each_napi_rx_queue(bp, i)
+ for_each_rx_queue(bp, i)
netif_napi_del(&bnx2x_fp(bp, i, napi));
}
@@ -1088,6 +1086,7 @@
txdata->txq_index = txq_index;
txdata->tx_cons_sb = tx_cons_sb;
txdata->parent_fp = fp;
+ txdata->tx_ring_size = IS_FCOE_FP(fp) ? MAX_TX_AVAIL : bp->tx_ring_size;
DP(NETIF_MSG_IFUP, "created tx data cid %d, txq %d\n",
txdata->cid, txdata->txq_index);
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dump.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dump.h
index 3e4cff9..b926f58 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dump.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_dump.h
@@ -401,11 +401,11 @@
{ 0x70000, 8, RI_ALL_ONLINE },
{ 0x70020, 8184, RI_ALL_OFFLINE },
{ 0x78000, 8192, RI_E3E3B0_OFFLINE },
- { 0x85000, 3, RI_ALL_ONLINE },
- { 0x8501c, 7, RI_ALL_ONLINE },
- { 0x85048, 1, RI_ALL_ONLINE },
- { 0x85200, 32, RI_ALL_ONLINE },
- { 0xb0000, 16384, RI_E1H_ONLINE },
+ { 0x85000, 3, RI_ALL_OFFLINE },
+ { 0x8501c, 7, RI_ALL_OFFLINE },
+ { 0x85048, 1, RI_ALL_OFFLINE },
+ { 0x85200, 32, RI_ALL_OFFLINE },
+ { 0xb0000, 16384, RI_E1H_OFFLINE },
{ 0xc1000, 7, RI_ALL_ONLINE },
{ 0xc103c, 2, RI_E2E3E3B0_ONLINE },
{ 0xc1800, 2, RI_ALL_ONLINE },
@@ -581,17 +581,12 @@
{ 0x140188, 3, RI_E1E1HE2E3_ONLINE },
{ 0x140194, 13, RI_ALL_ONLINE },
{ 0x140200, 6, RI_E1E1HE2E3_ONLINE },
- { 0x140220, 4, RI_E2E3_ONLINE },
- { 0x140240, 4, RI_E2E3_ONLINE },
{ 0x140260, 4, RI_E2E3_ONLINE },
{ 0x140280, 4, RI_E2E3_ONLINE },
- { 0x1402a0, 4, RI_E2E3_ONLINE },
- { 0x1402c0, 4, RI_E2E3_ONLINE },
{ 0x1402e0, 2, RI_E2E3_ONLINE },
{ 0x1402e8, 2, RI_E2E3E3B0_ONLINE },
{ 0x1402f0, 9, RI_E2E3_ONLINE },
{ 0x140314, 44, RI_E3B0_ONLINE },
- { 0x1403d0, 70, RI_E3B0_ONLINE },
{ 0x144000, 4, RI_E1E1H_ONLINE },
{ 0x148000, 4, RI_E1E1H_ONLINE },
{ 0x14c000, 4, RI_E1E1H_ONLINE },
@@ -704,7 +699,6 @@
{ 0x180398, 1, RI_E2E3E3B0_ONLINE },
{ 0x1803a0, 5, RI_E2E3E3B0_ONLINE },
{ 0x1803b4, 2, RI_E3E3B0_ONLINE },
- { 0x180400, 1, RI_ALL_ONLINE },
{ 0x180404, 255, RI_E1E1H_OFFLINE },
{ 0x181000, 4, RI_ALL_ONLINE },
{ 0x181010, 1020, RI_ALL_OFFLINE },
@@ -800,9 +794,9 @@
{ 0x1b905c, 1, RI_E3E3B0_ONLINE },
{ 0x1b9064, 1, RI_E3B0_ONLINE },
{ 0x1b9080, 10, RI_E3B0_ONLINE },
- { 0x1b9400, 14, RI_E2E3E3B0_ONLINE },
- { 0x1b943c, 19, RI_E2E3E3B0_ONLINE },
- { 0x1b9490, 10, RI_E2E3E3B0_ONLINE },
+ { 0x1b9400, 14, RI_E2E3E3B0_OFFLINE },
+ { 0x1b943c, 19, RI_E2E3E3B0_OFFLINE },
+ { 0x1b9490, 10, RI_E2E3E3B0_OFFLINE },
{ 0x1c0000, 2, RI_ALL_ONLINE },
{ 0x200000, 65, RI_ALL_ONLINE },
{ 0x20014c, 2, RI_E1HE2E3E3B0_ONLINE },
@@ -814,7 +808,6 @@
{ 0x200398, 1, RI_E2E3E3B0_ONLINE },
{ 0x2003a0, 1, RI_E2E3E3B0_ONLINE },
{ 0x2003a8, 2, RI_E2E3E3B0_ONLINE },
- { 0x200400, 1, RI_ALL_ONLINE },
{ 0x200404, 255, RI_E1E1H_OFFLINE },
{ 0x202000, 4, RI_ALL_ONLINE },
{ 0x202010, 2044, RI_ALL_OFFLINE },
@@ -921,7 +914,6 @@
{ 0x280398, 1, RI_E2E3E3B0_ONLINE },
{ 0x2803a0, 1, RI_E2E3E3B0_ONLINE },
{ 0x2803a8, 2, RI_E2E3E3B0_ONLINE },
- { 0x280400, 1, RI_ALL_ONLINE },
{ 0x280404, 255, RI_E1E1H_OFFLINE },
{ 0x282000, 4, RI_ALL_ONLINE },
{ 0x282010, 2044, RI_ALL_OFFLINE },
@@ -1031,7 +1023,6 @@
{ 0x300398, 1, RI_E2E3E3B0_ONLINE },
{ 0x3003a0, 1, RI_E2E3E3B0_ONLINE },
{ 0x3003a8, 2, RI_E2E3E3B0_ONLINE },
- { 0x300400, 1, RI_ALL_ONLINE },
{ 0x300404, 255, RI_E1E1H_OFFLINE },
{ 0x302000, 4, RI_ALL_ONLINE },
{ 0x302010, 2044, RI_ALL_OFFLINE },
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
index fc4e0e3..ebf40cd 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_ethtool.c
@@ -775,7 +775,7 @@
struct bnx2x *bp = netdev_priv(dev);
struct dump_hdr dump_hdr = {0};
- regs->version = 0;
+ regs->version = 1;
memset(p, 0, regs->len);
if (!netif_running(bp->dev))
@@ -1587,6 +1587,12 @@
bp->link_params.req_flow_ctrl[cfg_idx] =
BNX2X_FLOW_CTRL_AUTO;
}
+ bp->link_params.req_fc_auto_adv = BNX2X_FLOW_CTRL_NONE;
+ if (epause->rx_pause)
+ bp->link_params.req_fc_auto_adv |= BNX2X_FLOW_CTRL_RX;
+
+ if (epause->tx_pause)
+ bp->link_params.req_fc_auto_adv |= BNX2X_FLOW_CTRL_TX;
}
DP(BNX2X_MSG_ETHTOOL,
@@ -2888,11 +2894,9 @@
*/
static void bnx2x_change_num_queues(struct bnx2x *bp, int num_rss)
{
- bnx2x_del_all_napi(bp);
bnx2x_disable_msi(bp);
BNX2X_NUM_QUEUES(bp) = num_rss + NON_ETH_CONTEXT_USE;
bnx2x_set_int_mode(bp);
- bnx2x_add_all_napi(bp);
}
/**
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_link.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_link.c
index f4beb46..b046beb 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_link.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_link.c
@@ -2667,9 +2667,11 @@
return bnx2x_status;
DP(NETIF_MSG_LINK, "About to update PFC in BMAC\n");
- if (CHIP_IS_E3(bp))
- bnx2x_update_pfc_xmac(params, vars, 0);
- else {
+
+ if (CHIP_IS_E3(bp)) {
+ if (vars->mac_type == MAC_TYPE_XMAC)
+ bnx2x_update_pfc_xmac(params, vars, 0);
+ } else {
val = REG_RD(bp, MISC_REG_RESET_REG_2);
if ((val &
(MISC_REGISTERS_RESET_REG_2_RST_BMAC0 << params->port))
@@ -5432,7 +5434,7 @@
switch (speed_mask) {
case GP_STATUS_10M:
vars->line_speed = SPEED_10;
- if (vars->duplex == DUPLEX_FULL)
+ if (is_duplex == DUPLEX_FULL)
vars->link_status |= LINK_10TFD;
else
vars->link_status |= LINK_10THD;
@@ -5440,7 +5442,7 @@
case GP_STATUS_100M:
vars->line_speed = SPEED_100;
- if (vars->duplex == DUPLEX_FULL)
+ if (is_duplex == DUPLEX_FULL)
vars->link_status |= LINK_100TXFD;
else
vars->link_status |= LINK_100TXHD;
@@ -5449,7 +5451,7 @@
case GP_STATUS_1G:
case GP_STATUS_1G_KX:
vars->line_speed = SPEED_1000;
- if (vars->duplex == DUPLEX_FULL)
+ if (is_duplex == DUPLEX_FULL)
vars->link_status |= LINK_1000TFD;
else
vars->link_status |= LINK_1000THD;
@@ -5457,7 +5459,7 @@
case GP_STATUS_2_5G:
vars->line_speed = SPEED_2500;
- if (vars->duplex == DUPLEX_FULL)
+ if (is_duplex == DUPLEX_FULL)
vars->link_status |= LINK_2500TFD;
else
vars->link_status |= LINK_2500THD;
@@ -5531,6 +5533,7 @@
if (gp_status & MDIO_GP_STATUS_TOP_AN_STATUS1_LINK_STATUS) {
if (SINGLE_MEDIA_DIRECT(params)) {
+ vars->duplex = duplex;
bnx2x_flow_ctrl_resolve(phy, params, vars, gp_status);
if (phy->req_line_speed == SPEED_AUTO_NEG)
bnx2x_xgxs_an_resolve(phy, params, vars,
@@ -5625,6 +5628,7 @@
LINK_STATUS_PARALLEL_DETECTION_USED;
}
bnx2x_ext_phy_resolve_fc(phy, params, vars);
+ vars->duplex = duplex;
}
}
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
index 02b5a34..e11485c 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
@@ -1162,14 +1162,9 @@
static u8 bnx2x_is_pcie_pending(struct pci_dev *dev)
{
- int pos;
u16 status;
- pos = pci_pcie_cap(dev);
- if (!pos)
- return false;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVSTA, &status);
+ pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &status);
return status & PCI_EXP_DEVSTA_TRPND;
}
@@ -6135,8 +6130,7 @@
u16 devctl;
int r_order, w_order;
- pci_read_config_word(bp->pdev,
- pci_pcie_cap(bp->pdev) + PCI_EXP_DEVCTL, &devctl);
+ pcie_capability_read_word(bp->pdev, PCI_EXP_DEVCTL, &devctl);
DP(NETIF_MSG_HW, "read 0x%x from devctl\n", devctl);
w_order = ((devctl & PCI_EXP_DEVCTL_PAYLOAD) >> 5);
if (bp->mrrs == -1)
@@ -7561,8 +7555,14 @@
}
rc = bnx2x_config_vlan_mac(bp, &ramrod_param);
- if (rc < 0)
+
+ if (rc == -EEXIST) {
+ DP(BNX2X_MSG_SP, "Failed to schedule ADD operations: %d\n", rc);
+ /* do not treat adding same MAC as error */
+ rc = 0;
+ } else if (rc < 0)
BNX2X_ERR("%s MAC failed\n", (set ? "Set" : "Del"));
+
return rc;
}
@@ -8427,6 +8427,8 @@
/* Disable HW interrupts, NAPI */
bnx2x_netif_stop(bp, 1);
+ /* Delete all NAPI objects */
+ bnx2x_del_all_napi(bp);
/* Release IRQs */
bnx2x_free_irq(bp);
@@ -9372,7 +9374,7 @@
static int __devinit bnx2x_do_flr(struct bnx2x *bp)
{
- int i, pos;
+ int i;
u16 status;
struct pci_dev *dev = bp->pdev;
@@ -9389,16 +9391,12 @@
return -EINVAL;
}
- pos = pci_pcie_cap(dev);
- if (!pos)
- return -ENOTTY;
-
/* Wait for Transaction Pending bit clean */
for (i = 0; i < 4; i++) {
if (i)
msleep((1 << (i - 1)) * 100);
- pci_read_config_word(dev, pos + PCI_EXP_DEVSTA, &status);
+ pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &status);
if (!(status & PCI_EXP_DEVSTA_TRPND))
goto clear;
}
@@ -9823,12 +9821,13 @@
}
#ifdef CONFIG_PCI_MSI
- /*
- * It's expected that number of CAM entries for this functions is equal
- * to the number evaluated based on the MSI-X table size. We want a
- * harsh warning if these values are different!
+ /* Due to new PF resource allocation by MFW T7.4 and above, it's
+ * optional that number of CAM entries will not be equal to the value
+ * advertised in PCI.
+ * Driver should use the minimal value of both as the actual status
+ * block count
*/
- WARN_ON(bp->igu_sb_cnt != igu_sb_cnt);
+ bp->igu_sb_cnt = min_t(int, bp->igu_sb_cnt, igu_sb_cnt);
#endif
if (igu_sb_cnt == 0)
@@ -10292,13 +10291,11 @@
dev_info.port_hw_config[port].
fcoe_wwn_node_name_lower);
} else if (!IS_MF_SD(bp)) {
- u32 cfg = MF_CFG_RD(bp, func_ext_config[func].func_cfg);
-
/*
* Read the WWN info only if the FCoE feature is enabled for
* this function.
*/
- if (cfg & MACP_FUNC_CFG_FLAGS_FCOE_OFFLOAD)
+ if (BNX2X_MF_EXT_PROTOCOL_FCOE(bp) && !CHIP_IS_E1x(bp))
bnx2x_get_ext_wwn_info(bp, func);
} else if (IS_MF_FCOE_SD(bp))
@@ -11071,7 +11068,14 @@
netdev_for_each_uc_addr(ha, dev) {
rc = bnx2x_set_mac_one(bp, bnx2x_uc_addr(ha), mac_obj, true,
BNX2X_UC_LIST_MAC, &ramrod_flags);
- if (rc < 0) {
+ if (rc == -EEXIST) {
+ DP(BNX2X_MSG_SP,
+ "Failed to schedule ADD operations: %d\n", rc);
+ /* do not treat adding same MAC as error */
+ rc = 0;
+
+ } else if (rc < 0) {
+
BNX2X_ERR("Failed to schedule ADD operations: %d\n",
rc);
return rc;
@@ -11229,10 +11233,12 @@
static void poll_bnx2x(struct net_device *dev)
{
struct bnx2x *bp = netdev_priv(dev);
+ int i;
- disable_irq(bp->pdev->irq);
- bnx2x_interrupt(bp->pdev->irq, dev);
- enable_irq(bp->pdev->irq);
+ for_each_eth_queue(bp, i) {
+ struct bnx2x_fastpath *fp = &bp->fp[i];
+ napi_schedule(&bnx2x_fp(bp, fp->index, napi));
+ }
}
#endif
@@ -11899,9 +11905,6 @@
*/
bnx2x_set_int_mode(bp);
- /* Add all NAPI objects */
- bnx2x_add_all_napi(bp);
-
rc = register_netdev(dev);
if (rc) {
dev_err(&pdev->dev, "Cannot register net device\n");
@@ -11976,9 +11979,6 @@
unregister_netdev(dev);
- /* Delete all NAPI objects */
- bnx2x_del_all_napi(bp);
-
/* Power on: we can't let PCI layer write to us while we are in D3 */
bnx2x_set_power_state(bp, PCI_D0);
@@ -12025,6 +12025,8 @@
bnx2x_tx_disable(bp);
bnx2x_netif_stop(bp, 0);
+ /* Delete all NAPI objects */
+ bnx2x_del_all_napi(bp);
del_timer_sync(&bp->timer);
@@ -12155,7 +12157,7 @@
rtnl_unlock();
}
-static struct pci_error_handlers bnx2x_err_handler = {
+static const struct pci_error_handlers bnx2x_err_handler = {
.error_detected = bnx2x_io_error_detected,
.slot_reset = bnx2x_io_slot_reset,
.resume = bnx2x_io_resume,
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.h b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.h
index f83e033..acf2fe4 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.h
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_sp.h
@@ -1321,7 +1321,7 @@
* the current command will be enqueued to the tail of the
* pending commands list.
*
- * Return: 0 is operation was sucessfull and there are no pending completions,
+ * Return: 0 is operation was successfull and there are no pending completions,
* negative if there were errors, positive if there are pending
* completions.
*/
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c
index 332db64..a1d0446 100644
--- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c
+++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c
@@ -101,6 +101,11 @@
if (CHIP_REV_IS_SLOW(bp))
return;
+ /* Update MCP's statistics if possible */
+ if (bp->func_stx)
+ memcpy(bnx2x_sp(bp, func_stats), &bp->func_stats,
+ sizeof(bp->func_stats));
+
/* loader */
if (bp->executer_idx) {
int loader_idx = PMF_DMAE_C(bp);
@@ -128,8 +133,6 @@
} else if (bp->func_stx) {
*stats_comp = 0;
- memcpy(bnx2x_sp(bp, func_stats), &bp->func_stats,
- sizeof(bp->func_stats));
bnx2x_post_dmae(bp, dmae, INIT_DMAE_C(bp));
}
}
@@ -1151,9 +1154,11 @@
if (bp->port.pmf)
bnx2x_hw_stats_update(bp);
- if (bnx2x_storm_stats_update(bp) && (bp->stats_pending++ == 3)) {
- BNX2X_ERR("storm stats were not updated for 3 times\n");
- bnx2x_panic();
+ if (bnx2x_storm_stats_update(bp)) {
+ if (bp->stats_pending++ == 3) {
+ BNX2X_ERR("storm stats were not updated for 3 times\n");
+ bnx2x_panic();
+ }
return;
}
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index bf906c5..388d322 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -3653,17 +3653,9 @@
tg3_enable_register_access(tp);
/* Restore the CLKREQ setting. */
- if (tg3_flag(tp, CLKREQ_BUG)) {
- u16 lnkctl;
-
- pci_read_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_LNKCTL,
- &lnkctl);
- lnkctl |= PCI_EXP_LNKCTL_CLKREQ_EN;
- pci_write_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_LNKCTL,
- lnkctl);
- }
+ if (tg3_flag(tp, CLKREQ_BUG))
+ pcie_capability_set_word(tp->pdev, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
misc_host_ctrl = tr32(TG3PCI_MISC_HOST_CTRL);
tw32(TG3PCI_MISC_HOST_CTRL,
@@ -4434,20 +4426,13 @@
/* Prevent send BD corruption. */
if (tg3_flag(tp, CLKREQ_BUG)) {
- u16 oldlnkctl, newlnkctl;
-
- pci_read_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_LNKCTL,
- &oldlnkctl);
if (tp->link_config.active_speed == SPEED_100 ||
tp->link_config.active_speed == SPEED_10)
- newlnkctl = oldlnkctl & ~PCI_EXP_LNKCTL_CLKREQ_EN;
+ pcie_capability_clear_word(tp->pdev, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
else
- newlnkctl = oldlnkctl | PCI_EXP_LNKCTL_CLKREQ_EN;
- if (newlnkctl != oldlnkctl)
- pci_write_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) +
- PCI_EXP_LNKCTL, newlnkctl);
+ pcie_capability_set_word(tp->pdev, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
}
if (current_link_up != netif_carrier_ok(tp->dev)) {
@@ -8054,7 +8039,7 @@
udelay(120);
- if (tg3_flag(tp, PCI_EXPRESS) && pci_pcie_cap(tp->pdev)) {
+ if (tg3_flag(tp, PCI_EXPRESS) && pci_is_pcie(tp->pdev)) {
u16 val16;
if (tp->pci_chip_rev_id == CHIPREV_ID_5750_A0) {
@@ -8071,24 +8056,17 @@
}
/* Clear the "no snoop" and "relaxed ordering" bits. */
- pci_read_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_DEVCTL,
- &val16);
- val16 &= ~(PCI_EXP_DEVCTL_RELAX_EN |
- PCI_EXP_DEVCTL_NOSNOOP_EN);
+ val16 = PCI_EXP_DEVCTL_RELAX_EN | PCI_EXP_DEVCTL_NOSNOOP_EN;
/*
* Older PCIe devices only support the 128 byte
* MPS setting. Enforce the restriction.
*/
if (!tg3_flag(tp, CPMU_PRESENT))
- val16 &= ~PCI_EXP_DEVCTL_PAYLOAD;
- pci_write_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_DEVCTL,
- val16);
+ val16 |= PCI_EXP_DEVCTL_PAYLOAD;
+ pcie_capability_clear_word(tp->pdev, PCI_EXP_DEVCTL, val16);
/* Clear error status */
- pci_write_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_DEVSTA,
+ pcie_capability_write_word(tp->pdev, PCI_EXP_DEVSTA,
PCI_EXP_DEVSTA_CED |
PCI_EXP_DEVSTA_NFED |
PCI_EXP_DEVSTA_FED |
@@ -14565,9 +14543,7 @@
tg3_flag_set(tp, PCI_EXPRESS);
- pci_read_config_word(tp->pdev,
- pci_pcie_cap(tp->pdev) + PCI_EXP_LNKCTL,
- &lnkctl);
+ pcie_capability_read_word(tp->pdev, PCI_EXP_LNKCTL, &lnkctl);
if (lnkctl & PCI_EXP_LNKCTL_CLKREQ_EN) {
if (GET_ASIC_REV(tp->pci_chip_rev_id) ==
ASIC_REV_5906) {
@@ -16397,7 +16373,7 @@
rtnl_unlock();
}
-static struct pci_error_handlers tg3_err_handler = {
+static const struct pci_error_handlers tg3_err_handler = {
.error_detected = tg3_io_error_detected,
.slot_reset = tg3_io_slot_reset,
.resume = tg3_io_resume
diff --git a/drivers/net/ethernet/cadence/at91_ether.c b/drivers/net/ethernet/cadence/at91_ether.c
index 7788419..4e980a78 100644
--- a/drivers/net/ethernet/cadence/at91_ether.c
+++ b/drivers/net/ethernet/cadence/at91_ether.c
@@ -1086,7 +1086,7 @@
/* Clock */
lp->ether_clk = clk_get(&pdev->dev, "ether_clk");
if (IS_ERR(lp->ether_clk)) {
- res = -ENODEV;
+ res = PTR_ERR(lp->ether_clk);
goto err_ioumap;
}
clk_enable(lp->ether_clk);
diff --git a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
index 6505070..875bbb9 100644
--- a/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c
@@ -3036,7 +3036,7 @@
t3_resume_ports(adapter);
}
-static struct pci_error_handlers t3_err_handler = {
+static const struct pci_error_handlers t3_err_handler = {
.error_detected = t3_io_error_detected,
.slot_reset = t3_io_slot_reset,
.resume = t3_io_resume,
diff --git a/drivers/net/ethernet/chelsio/cxgb3/t3_hw.c b/drivers/net/ethernet/chelsio/cxgb3/t3_hw.c
index bff8a3c..aef45d3 100644
--- a/drivers/net/ethernet/chelsio/cxgb3/t3_hw.c
+++ b/drivers/net/ethernet/chelsio/cxgb3/t3_hw.c
@@ -3289,22 +3289,18 @@
unsigned int log2_width, pldsize;
unsigned int fst_trn_rx, fst_trn_tx, acklat, rpllmt;
- pci_read_config_word(adap->pdev,
- adap->pdev->pcie_cap + PCI_EXP_DEVCTL,
- &val);
+ pcie_capability_read_word(adap->pdev, PCI_EXP_DEVCTL, &val);
pldsize = (val & PCI_EXP_DEVCTL_PAYLOAD) >> 5;
pci_read_config_word(adap->pdev, 0x2, &devid);
if (devid == 0x37) {
- pci_write_config_word(adap->pdev,
- adap->pdev->pcie_cap + PCI_EXP_DEVCTL,
- val & ~PCI_EXP_DEVCTL_READRQ &
- ~PCI_EXP_DEVCTL_PAYLOAD);
+ pcie_capability_write_word(adap->pdev, PCI_EXP_DEVCTL,
+ val & ~PCI_EXP_DEVCTL_READRQ &
+ ~PCI_EXP_DEVCTL_PAYLOAD);
pldsize = 0;
}
- pci_read_config_word(adap->pdev, adap->pdev->pcie_cap + PCI_EXP_LNKCTL,
- &val);
+ pcie_capability_read_word(adap->pdev, PCI_EXP_LNKCTL, &val);
fst_trn_tx = G_NUMFSTTRNSEQ(t3_read_reg(adap, A_PCIE_PEX_CTRL0));
fst_trn_rx = adap->params.rev == 0 ? fst_trn_tx :
@@ -3425,15 +3421,13 @@
static void get_pci_mode(struct adapter *adapter, struct pci_params *p)
{
static unsigned short speed_map[] = { 33, 66, 100, 133 };
- u32 pci_mode, pcie_cap;
+ u32 pci_mode;
- pcie_cap = pci_pcie_cap(adapter->pdev);
- if (pcie_cap) {
+ if (pci_is_pcie(adapter->pdev)) {
u16 val;
p->variant = PCI_VARIANT_PCIE;
- pci_read_config_word(adapter->pdev, pcie_cap + PCI_EXP_LNKSTA,
- &val);
+ pcie_capability_read_word(adapter->pdev, PCI_EXP_LNKSTA, &val);
p->width = (val >> 4) & 0x3f;
return;
}
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index 5ed49af..9339854 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -3453,7 +3453,7 @@
rtnl_unlock();
}
-static struct pci_error_handlers cxgb4_eeh = {
+static const struct pci_error_handlers cxgb4_eeh = {
.error_detected = eeh_err_detected,
.slot_reset = eeh_slot_reset,
.resume = eeh_resume,
@@ -3694,15 +3694,7 @@
static void __devinit enable_pcie_relaxed_ordering(struct pci_dev *dev)
{
- u16 v;
- int pos;
-
- pos = pci_pcie_cap(dev);
- if (pos > 0) {
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &v);
- v |= PCI_EXP_DEVCTL_RELAX_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, v);
- }
+ pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_RELAX_EN);
}
/*
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
index fa947df..af16013 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
@@ -2741,11 +2741,9 @@
struct pci_params *p)
{
u16 val;
- u32 pcie_cap = pci_pcie_cap(adapter->pdev);
- if (pcie_cap) {
- pci_read_config_word(adapter->pdev, pcie_cap + PCI_EXP_LNKSTA,
- &val);
+ if (pci_is_pcie(adapter->pdev)) {
+ pcie_capability_read_word(adapter->pdev, PCI_EXP_LNKSTA, &val);
p->speed = val & PCI_EXP_LNKSTA_CLS;
p->width = (val & PCI_EXP_LNKSTA_NLW) >> 4;
}
diff --git a/drivers/net/ethernet/cirrus/cs89x0.c b/drivers/net/ethernet/cirrus/cs89x0.c
index 845b202..1384469 100644
--- a/drivers/net/ethernet/cirrus/cs89x0.c
+++ b/drivers/net/ethernet/cirrus/cs89x0.c
@@ -1243,6 +1243,7 @@
{
struct net_local *lp = netdev_priv(dev);
unsigned long flags;
+ u16 cfg;
spin_lock_irqsave(&lp->lock, flags);
if (dev->flags & IFF_PROMISC)
@@ -1260,11 +1261,10 @@
/* in promiscuous mode, we accept errored packets,
* so we have to enable interrupts on them also
*/
- writereg(dev, PP_RxCFG,
- (lp->curr_rx_cfg |
- (lp->rx_mode == RX_ALL_ACCEPT)
- ? (RX_CRC_ERROR_ENBL | RX_RUNT_ENBL | RX_EXTRA_DATA_ENBL)
- : 0));
+ cfg = lp->curr_rx_cfg;
+ if (lp->rx_mode == RX_ALL_ACCEPT)
+ cfg |= RX_CRC_ERROR_ENBL | RX_RUNT_ENBL | RX_EXTRA_DATA_ENBL;
+ writereg(dev, PP_RxCFG, cfg);
spin_unlock_irqrestore(&lp->lock, flags);
}
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c b/drivers/net/ethernet/emulex/benet/be_cmds.c
index 7fac97b..8c63d06 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.c
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.c
@@ -259,7 +259,7 @@
int num = 0, status = 0;
struct be_mcc_obj *mcc_obj = &adapter->mcc_obj;
- spin_lock_bh(&adapter->mcc_cq_lock);
+ spin_lock(&adapter->mcc_cq_lock);
while ((compl = be_mcc_compl_get(adapter))) {
if (compl->flags & CQE_FLAGS_ASYNC_MASK) {
/* Interpret flags as an async trailer */
@@ -280,7 +280,7 @@
if (num)
be_cq_notify(adapter, mcc_obj->cq.id, mcc_obj->rearm_cq, num);
- spin_unlock_bh(&adapter->mcc_cq_lock);
+ spin_unlock(&adapter->mcc_cq_lock);
return status;
}
@@ -295,7 +295,9 @@
if (be_error(adapter))
return -EIO;
+ local_bh_disable();
status = be_process_mcc(adapter);
+ local_bh_enable();
if (atomic_read(&mcc_obj->q.used) == 0)
break;
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index 90a903d8..95d1047 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -3763,7 +3763,9 @@
/* when interrupts are not yet enabled, just reap any pending
* mcc completions */
if (!netif_running(adapter->netdev)) {
+ local_bh_disable();
be_process_mcc(adapter);
+ local_bh_enable();
goto reschedule;
}
@@ -4104,7 +4106,7 @@
dev_err(&adapter->pdev->dev, "EEH resume failed\n");
}
-static struct pci_error_handlers be_eeh_handlers = {
+static const struct pci_error_handlers be_eeh_handlers = {
.error_detected = be_eeh_err_detected,
.slot_reset = be_eeh_reset,
.resume = be_eeh_resume,
diff --git a/drivers/net/ethernet/freescale/gianfar.c b/drivers/net/ethernet/freescale/gianfar.c
index 4605f72..d3233f5 100644
--- a/drivers/net/ethernet/freescale/gianfar.c
+++ b/drivers/net/ethernet/freescale/gianfar.c
@@ -1041,7 +1041,7 @@
if (priv->device_flags & FSL_GIANFAR_DEV_HAS_VLAN) {
dev->hw_features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
- dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
+ dev->features |= NETIF_F_HW_VLAN_RX;
}
if (priv->device_flags & FSL_GIANFAR_DEV_HAS_EXTENDED_HASH) {
diff --git a/drivers/net/ethernet/freescale/gianfar_ethtool.c b/drivers/net/ethernet/freescale/gianfar_ethtool.c
index 8971921..ab6762c 100644
--- a/drivers/net/ethernet/freescale/gianfar_ethtool.c
+++ b/drivers/net/ethernet/freescale/gianfar_ethtool.c
@@ -1773,6 +1773,7 @@
}
int gfar_phc_index = -1;
+EXPORT_SYMBOL(gfar_phc_index);
static int gfar_get_ts_info(struct net_device *dev,
struct ethtool_ts_info *info)
diff --git a/drivers/net/ethernet/freescale/gianfar_ptp.c b/drivers/net/ethernet/freescale/gianfar_ptp.c
index c08e5d4..0daa66b 100644
--- a/drivers/net/ethernet/freescale/gianfar_ptp.c
+++ b/drivers/net/ethernet/freescale/gianfar_ptp.c
@@ -515,7 +515,7 @@
err = PTR_ERR(etsects->clock);
goto no_clock;
}
- gfar_phc_clock = ptp_clock_index(etsects->clock);
+ gfar_phc_index = ptp_clock_index(etsects->clock);
dev_set_drvdata(&dev->dev, etsects);
@@ -539,7 +539,7 @@
gfar_write(&etsects->regs->tmr_temask, 0);
gfar_write(&etsects->regs->tmr_ctrl, 0);
- gfar_phc_clock = -1;
+ gfar_phc_index = -1;
ptp_clock_unregister(etsects->clock);
iounmap(etsects->regs);
release_resource(etsects->rsrc);
diff --git a/drivers/net/ethernet/i825xx/znet.c b/drivers/net/ethernet/i825xx/znet.c
index bd1f1ef..ba4e0ce 100644
--- a/drivers/net/ethernet/i825xx/znet.c
+++ b/drivers/net/ethernet/i825xx/znet.c
@@ -139,8 +139,11 @@
/* Only one can be built-in;-> */
static struct net_device *znet_dev;
+#define NETIDBLK_MAGIC "NETIDBLK"
+#define NETIDBLK_MAGIC_SIZE 8
+
struct netidblk {
- char magic[8]; /* The magic number (string) "NETIDBLK" */
+ char magic[NETIDBLK_MAGIC_SIZE]; /* The magic number (string) "NETIDBLK" */
unsigned char netid[8]; /* The physical station address */
char nettype, globalopt;
char vendor[8]; /* The machine vendor and product name. */
@@ -373,14 +376,16 @@
struct znet_private *znet;
struct net_device *dev;
char *p;
+ char *plast = phys_to_virt(0x100000 - NETIDBLK_MAGIC_SIZE);
int err = -ENOMEM;
/* This code scans the region 0xf0000 to 0xfffff for a "NETIDBLK". */
- for(p = (char *)phys_to_virt(0xf0000); p < (char *)phys_to_virt(0x100000); p++)
- if (*p == 'N' && strncmp(p, "NETIDBLK", 8) == 0)
+ for(p = (char *)phys_to_virt(0xf0000); p <= plast; p++)
+ if (*p == 'N' &&
+ strncmp(p, NETIDBLK_MAGIC, NETIDBLK_MAGIC_SIZE) == 0)
break;
- if (p >= (char *)phys_to_virt(0x100000)) {
+ if (p > plast) {
if (znet_debug > 1)
printk(KERN_INFO "No Z-Note ethernet adaptor found.\n");
return -ENODEV;
diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c
index 9010cea..b68d28a 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -472,14 +472,9 @@
}
if (adapter->rx_queue.queue_addr != NULL) {
- if (!dma_mapping_error(dev, adapter->rx_queue.queue_dma)) {
- dma_unmap_single(dev,
- adapter->rx_queue.queue_dma,
- adapter->rx_queue.queue_len,
- DMA_BIDIRECTIONAL);
- adapter->rx_queue.queue_dma = DMA_ERROR_CODE;
- }
- kfree(adapter->rx_queue.queue_addr);
+ dma_free_coherent(dev, adapter->rx_queue.queue_len,
+ adapter->rx_queue.queue_addr,
+ adapter->rx_queue.queue_dma);
adapter->rx_queue.queue_addr = NULL;
}
@@ -556,10 +551,13 @@
goto err_out;
}
+ dev = &adapter->vdev->dev;
+
adapter->rx_queue.queue_len = sizeof(struct ibmveth_rx_q_entry) *
rxq_entries;
- adapter->rx_queue.queue_addr = kmalloc(adapter->rx_queue.queue_len,
- GFP_KERNEL);
+ adapter->rx_queue.queue_addr =
+ dma_alloc_coherent(dev, adapter->rx_queue.queue_len,
+ &adapter->rx_queue.queue_dma, GFP_KERNEL);
if (!adapter->rx_queue.queue_addr) {
netdev_err(netdev, "unable to allocate rx queue pages\n");
@@ -567,19 +565,13 @@
goto err_out;
}
- dev = &adapter->vdev->dev;
-
adapter->buffer_list_dma = dma_map_single(dev,
adapter->buffer_list_addr, 4096, DMA_BIDIRECTIONAL);
adapter->filter_list_dma = dma_map_single(dev,
adapter->filter_list_addr, 4096, DMA_BIDIRECTIONAL);
- adapter->rx_queue.queue_dma = dma_map_single(dev,
- adapter->rx_queue.queue_addr,
- adapter->rx_queue.queue_len, DMA_BIDIRECTIONAL);
if ((dma_mapping_error(dev, adapter->buffer_list_dma)) ||
- (dma_mapping_error(dev, adapter->filter_list_dma)) ||
- (dma_mapping_error(dev, adapter->rx_queue.queue_dma))) {
+ (dma_mapping_error(dev, adapter->filter_list_dma))) {
netdev_err(netdev, "unable to map filter or buffer list "
"pages\n");
rc = -ENOMEM;
diff --git a/drivers/net/ethernet/intel/e100.c b/drivers/net/ethernet/intel/e100.c
index 535f94f..29ce9bd 100644
--- a/drivers/net/ethernet/intel/e100.c
+++ b/drivers/net/ethernet/intel/e100.c
@@ -3157,7 +3157,7 @@
}
}
-static struct pci_error_handlers e100_err_handler = {
+static const struct pci_error_handlers e100_err_handler = {
.error_detected = e100_io_error_detected,
.slot_reset = e100_io_slot_reset,
.resume = e100_io_resume,
diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c
index 3bfbb8d..f3f9aeb 100644
--- a/drivers/net/ethernet/intel/e1000/e1000_main.c
+++ b/drivers/net/ethernet/intel/e1000/e1000_main.c
@@ -192,7 +192,7 @@
static pci_ers_result_t e1000_io_slot_reset(struct pci_dev *pdev);
static void e1000_io_resume(struct pci_dev *pdev);
-static struct pci_error_handlers e1000_err_handler = {
+static const struct pci_error_handlers e1000_err_handler = {
.error_detected = e1000_io_error_detected,
.slot_reset = e1000_io_slot_reset,
.resume = e1000_io_resume,
@@ -3149,6 +3149,17 @@
return NETDEV_TX_OK;
}
+ /* On PCI/PCI-X HW, if packet size is less than ETH_ZLEN,
+ * packets may get corrupted during padding by HW.
+ * To WA this issue, pad all small packets manually.
+ */
+ if (skb->len < ETH_ZLEN) {
+ if (skb_pad(skb, ETH_ZLEN - skb->len))
+ return NETDEV_TX_OK;
+ skb->len = ETH_ZLEN;
+ skb_set_tail_pointer(skb, ETH_ZLEN);
+ }
+
mss = skb_shinfo(skb)->gso_size;
/* The controller does a simple calculation to
* make sure there is enough room in the FIFO before
diff --git a/drivers/net/ethernet/intel/e1000e/e1000.h b/drivers/net/ethernet/intel/e1000e/e1000.h
index cd15332..cb3356c 100644
--- a/drivers/net/ethernet/intel/e1000e/e1000.h
+++ b/drivers/net/ethernet/intel/e1000e/e1000.h
@@ -310,6 +310,7 @@
*/
struct e1000_ring *tx_ring /* One per active queue */
____cacheline_aligned_in_smp;
+ u32 tx_fifo_limit;
struct napi_struct napi;
diff --git a/drivers/net/ethernet/intel/e1000e/netdev.c b/drivers/net/ethernet/intel/e1000e/netdev.c
index 46c3b1f..3f0223a 100644
--- a/drivers/net/ethernet/intel/e1000e/netdev.c
+++ b/drivers/net/ethernet/intel/e1000e/netdev.c
@@ -3517,6 +3517,15 @@
}
/*
+ * Alignment of Tx data is on an arbitrary byte boundary with the
+ * maximum size per Tx descriptor limited only to the transmit
+ * allocation of the packet buffer minus 96 bytes with an upper
+ * limit of 24KB due to receive synchronization limitations.
+ */
+ adapter->tx_fifo_limit = min_t(u32, ((er32(PBA) >> 16) << 10) - 96,
+ 24 << 10);
+
+ /*
* Disable Adaptive Interrupt Moderation if 2 full packets cannot
* fit in receive buffer.
*/
@@ -4785,12 +4794,9 @@
return 1;
}
-#define E1000_MAX_PER_TXD 8192
-#define E1000_MAX_TXD_PWR 12
-
static int e1000_tx_map(struct e1000_ring *tx_ring, struct sk_buff *skb,
unsigned int first, unsigned int max_per_txd,
- unsigned int nr_frags, unsigned int mss)
+ unsigned int nr_frags)
{
struct e1000_adapter *adapter = tx_ring->adapter;
struct pci_dev *pdev = adapter->pdev;
@@ -5023,20 +5029,19 @@
static int e1000_maybe_stop_tx(struct e1000_ring *tx_ring, int size)
{
+ BUG_ON(size > tx_ring->count);
+
if (e1000_desc_unused(tx_ring) >= size)
return 0;
return __e1000_maybe_stop_tx(tx_ring, size);
}
-#define TXD_USE_COUNT(S, X) (((S) >> (X)) + 1)
static netdev_tx_t e1000_xmit_frame(struct sk_buff *skb,
struct net_device *netdev)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
struct e1000_ring *tx_ring = adapter->tx_ring;
unsigned int first;
- unsigned int max_per_txd = E1000_MAX_PER_TXD;
- unsigned int max_txd_pwr = E1000_MAX_TXD_PWR;
unsigned int tx_flags = 0;
unsigned int len = skb_headlen(skb);
unsigned int nr_frags;
@@ -5056,18 +5061,8 @@
}
mss = skb_shinfo(skb)->gso_size;
- /*
- * The controller does a simple calculation to
- * make sure there is enough room in the FIFO before
- * initiating the DMA for each buffer. The calc is:
- * 4 = ceil(buffer len/mss). To make sure we don't
- * overrun the FIFO, adjust the max buffer len if mss
- * drops.
- */
if (mss) {
u8 hdr_len;
- max_per_txd = min(mss << 2, max_per_txd);
- max_txd_pwr = fls(max_per_txd) - 1;
/*
* TSO Workaround for 82571/2/3 Controllers -- if skb->data
@@ -5097,12 +5092,12 @@
count++;
count++;
- count += TXD_USE_COUNT(len, max_txd_pwr);
+ count += DIV_ROUND_UP(len, adapter->tx_fifo_limit);
nr_frags = skb_shinfo(skb)->nr_frags;
for (f = 0; f < nr_frags; f++)
- count += TXD_USE_COUNT(skb_frag_size(&skb_shinfo(skb)->frags[f]),
- max_txd_pwr);
+ count += DIV_ROUND_UP(skb_frag_size(&skb_shinfo(skb)->frags[f]),
+ adapter->tx_fifo_limit);
if (adapter->hw.mac.tx_pkt_filtering)
e1000_transfer_dhcp_info(adapter, skb);
@@ -5144,15 +5139,18 @@
tx_flags |= E1000_TX_FLAGS_NO_FCS;
/* if count is 0 then mapping error has occurred */
- count = e1000_tx_map(tx_ring, skb, first, max_per_txd, nr_frags, mss);
+ count = e1000_tx_map(tx_ring, skb, first, adapter->tx_fifo_limit,
+ nr_frags);
if (count) {
skb_tx_timestamp(skb);
netdev_sent_queue(netdev, skb->len);
e1000_tx_queue(tx_ring, tx_flags, count);
/* Make sure there is space in the ring for the next send. */
- e1000_maybe_stop_tx(tx_ring, MAX_SKB_FRAGS + 2);
-
+ e1000_maybe_stop_tx(tx_ring,
+ (MAX_SKB_FRAGS *
+ DIV_ROUND_UP(PAGE_SIZE,
+ adapter->tx_fifo_limit) + 2));
} else {
dev_kfree_skb_any(skb);
tx_ring->buffer_info[first].time_stamp = 0;
@@ -5586,16 +5584,15 @@
*/
if (adapter->flags & FLAG_IS_QUAD_PORT) {
struct pci_dev *us_dev = pdev->bus->self;
- int pos = pci_pcie_cap(us_dev);
u16 devctl;
- pci_read_config_word(us_dev, pos + PCI_EXP_DEVCTL, &devctl);
- pci_write_config_word(us_dev, pos + PCI_EXP_DEVCTL,
- (devctl & ~PCI_EXP_DEVCTL_CERE));
+ pcie_capability_read_word(us_dev, PCI_EXP_DEVCTL, &devctl);
+ pcie_capability_write_word(us_dev, PCI_EXP_DEVCTL,
+ (devctl & ~PCI_EXP_DEVCTL_CERE));
e1000_power_off(pdev, sleep, wake);
- pci_write_config_word(us_dev, pos + PCI_EXP_DEVCTL, devctl);
+ pcie_capability_write_word(us_dev, PCI_EXP_DEVCTL, devctl);
} else {
e1000_power_off(pdev, sleep, wake);
}
@@ -5609,25 +5606,15 @@
#else
static void __e1000e_disable_aspm(struct pci_dev *pdev, u16 state)
{
- int pos;
- u16 reg16;
-
/*
* Both device and parent should have the same ASPM setting.
* Disable ASPM in downstream component first and then upstream.
*/
- pos = pci_pcie_cap(pdev);
- pci_read_config_word(pdev, pos + PCI_EXP_LNKCTL, ®16);
- reg16 &= ~state;
- pci_write_config_word(pdev, pos + PCI_EXP_LNKCTL, reg16);
+ pcie_capability_clear_word(pdev, PCI_EXP_LNKCTL, state);
- if (!pdev->bus->self)
- return;
-
- pos = pci_pcie_cap(pdev->bus->self);
- pci_read_config_word(pdev->bus->self, pos + PCI_EXP_LNKCTL, ®16);
- reg16 &= ~state;
- pci_write_config_word(pdev->bus->self, pos + PCI_EXP_LNKCTL, reg16);
+ if (pdev->bus->self)
+ pcie_capability_clear_word(pdev->bus->self, PCI_EXP_LNKCTL,
+ state);
}
#endif
static void e1000e_disable_aspm(struct pci_dev *pdev, u16 state)
@@ -6327,8 +6314,8 @@
adapter->hw.phy.autoneg_advertised = 0x2f;
/* ring size defaults */
- adapter->rx_ring->count = 256;
- adapter->tx_ring->count = 256;
+ adapter->rx_ring->count = E1000_DEFAULT_RXD;
+ adapter->tx_ring->count = E1000_DEFAULT_TXD;
/*
* Initial Wake on LAN setting - If APM wake is enabled in
@@ -6488,7 +6475,7 @@
}
/* PCI Error Recovery (ERS) */
-static struct pci_error_handlers e1000_err_handler = {
+static const struct pci_error_handlers e1000_err_handler = {
.error_detected = e1000_io_error_detected,
.slot_reset = e1000_io_slot_reset,
.resume = e1000_io_resume,
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 48cc4fb..f88c822 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -217,7 +217,7 @@
static pci_ers_result_t igb_io_slot_reset(struct pci_dev *);
static void igb_io_resume(struct pci_dev *);
-static struct pci_error_handlers igb_err_handler = {
+static const struct pci_error_handlers igb_err_handler = {
.error_detected = igb_io_error_detected,
.slot_reset = igb_io_slot_reset,
.resume = igb_io_resume,
@@ -6538,28 +6538,20 @@
s32 igb_read_pcie_cap_reg(struct e1000_hw *hw, u32 reg, u16 *value)
{
struct igb_adapter *adapter = hw->back;
- u16 cap_offset;
- cap_offset = adapter->pdev->pcie_cap;
- if (!cap_offset)
+ if (pcie_capability_read_word(adapter->pdev, reg, value))
return -E1000_ERR_CONFIG;
- pci_read_config_word(adapter->pdev, cap_offset + reg, value);
-
return 0;
}
s32 igb_write_pcie_cap_reg(struct e1000_hw *hw, u32 reg, u16 *value)
{
struct igb_adapter *adapter = hw->back;
- u16 cap_offset;
- cap_offset = adapter->pdev->pcie_cap;
- if (!cap_offset)
+ if (pcie_capability_write_word(adapter->pdev, reg, *value))
return -E1000_ERR_CONFIG;
- pci_write_config_word(adapter->pdev, cap_offset + reg, *value);
-
return 0;
}
diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c
index 0696abf..0ac11f5 100644
--- a/drivers/net/ethernet/intel/igbvf/netdev.c
+++ b/drivers/net/ethernet/intel/igbvf/netdev.c
@@ -2833,7 +2833,7 @@
}
/* PCI Error Recovery (ERS) */
-static struct pci_error_handlers igbvf_err_handler = {
+static const struct pci_error_handlers igbvf_err_handler = {
.error_detected = igbvf_io_error_detected,
.slot_reset = igbvf_io_slot_reset,
.resume = igbvf_io_resume,
diff --git a/drivers/net/ethernet/intel/ixgb/ixgb_main.c b/drivers/net/ethernet/intel/ixgb/ixgb_main.c
index d05fc95..d99a2d5 100644
--- a/drivers/net/ethernet/intel/ixgb/ixgb_main.c
+++ b/drivers/net/ethernet/intel/ixgb/ixgb_main.c
@@ -115,7 +115,7 @@
static pci_ers_result_t ixgb_io_slot_reset (struct pci_dev *pdev);
static void ixgb_io_resume (struct pci_dev *pdev);
-static struct pci_error_handlers ixgb_err_handler = {
+static const struct pci_error_handlers ixgb_err_handler = {
.error_detected = ixgb_io_error_detected,
.slot_reset = ixgb_io_slot_reset,
.resume = ixgb_io_resume,
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 4326f74..ee61819 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7527,7 +7527,7 @@
goto skip_bad_vf_detection;
bdev = pdev->bus->self;
- while (bdev && (bdev->pcie_type != PCI_EXP_TYPE_ROOT_PORT))
+ while (bdev && (pci_pcie_type(bdev) != PCI_EXP_TYPE_ROOT_PORT))
bdev = bdev->bus->self;
if (!bdev)
@@ -7677,7 +7677,7 @@
netif_device_attach(netdev);
}
-static struct pci_error_handlers ixgbe_err_handler = {
+static const struct pci_error_handlers ixgbe_err_handler = {
.error_detected = ixgbe_io_error_detected,
.slot_reset = ixgbe_io_slot_reset,
.resume = ixgbe_io_resume,
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 60ef645..6647383 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -3256,7 +3256,7 @@
}
/* PCI Error Recovery (ERS) */
-static struct pci_error_handlers ixgbevf_err_handler = {
+static const struct pci_error_handlers ixgbevf_err_handler = {
.error_detected = ixgbevf_io_error_detected,
.slot_reset = ixgbevf_io_slot_reset,
.resume = ixgbevf_io_resume,
diff --git a/drivers/net/ethernet/mellanox/mlx4/icm.c b/drivers/net/ethernet/mellanox/mlx4/icm.c
index daf4179..31d0264 100644
--- a/drivers/net/ethernet/mellanox/mlx4/icm.c
+++ b/drivers/net/ethernet/mellanox/mlx4/icm.c
@@ -227,9 +227,10 @@
MLX4_CMD_TIME_CLASS_B, MLX4_CMD_NATIVE);
}
-int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj)
+int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj)
{
- int i = (obj & (table->num_obj - 1)) / (MLX4_TABLE_CHUNK_SIZE / table->obj_size);
+ u32 i = (obj & (table->num_obj - 1)) /
+ (MLX4_TABLE_CHUNK_SIZE / table->obj_size);
int ret = 0;
mutex_lock(&table->mutex);
@@ -262,16 +263,18 @@
return ret;
}
-void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj)
+void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj)
{
- int i;
+ u32 i;
+ u64 offset;
i = (obj & (table->num_obj - 1)) / (MLX4_TABLE_CHUNK_SIZE / table->obj_size);
mutex_lock(&table->mutex);
if (--table->icm[i]->refcount == 0) {
- mlx4_UNMAP_ICM(dev, table->virt + i * MLX4_TABLE_CHUNK_SIZE,
+ offset = (u64) i * MLX4_TABLE_CHUNK_SIZE;
+ mlx4_UNMAP_ICM(dev, table->virt + offset,
MLX4_TABLE_CHUNK_SIZE / MLX4_ICM_PAGE_SIZE);
mlx4_free_icm(dev, table->icm[i], table->coherent);
table->icm[i] = NULL;
@@ -280,9 +283,11 @@
mutex_unlock(&table->mutex);
}
-void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_handle)
+void *mlx4_table_find(struct mlx4_icm_table *table, u32 obj,
+ dma_addr_t *dma_handle)
{
- int idx, offset, dma_offset, i;
+ int offset, dma_offset, i;
+ u64 idx;
struct mlx4_icm_chunk *chunk;
struct mlx4_icm *icm;
struct page *page = NULL;
@@ -292,7 +297,7 @@
mutex_lock(&table->mutex);
- idx = (obj & (table->num_obj - 1)) * table->obj_size;
+ idx = (u64) (obj & (table->num_obj - 1)) * table->obj_size;
icm = table->icm[idx / MLX4_TABLE_CHUNK_SIZE];
dma_offset = offset = idx % MLX4_TABLE_CHUNK_SIZE;
@@ -326,10 +331,11 @@
}
int mlx4_table_get_range(struct mlx4_dev *dev, struct mlx4_icm_table *table,
- int start, int end)
+ u32 start, u32 end)
{
int inc = MLX4_TABLE_CHUNK_SIZE / table->obj_size;
- int i, err;
+ int err;
+ u32 i;
for (i = start; i <= end; i += inc) {
err = mlx4_table_get(dev, table, i);
@@ -349,9 +355,9 @@
}
void mlx4_table_put_range(struct mlx4_dev *dev, struct mlx4_icm_table *table,
- int start, int end)
+ u32 start, u32 end)
{
- int i;
+ u32 i;
for (i = start; i <= end; i += MLX4_TABLE_CHUNK_SIZE / table->obj_size)
mlx4_table_put(dev, table, i);
diff --git a/drivers/net/ethernet/mellanox/mlx4/icm.h b/drivers/net/ethernet/mellanox/mlx4/icm.h
index a67744f..dee67fa 100644
--- a/drivers/net/ethernet/mellanox/mlx4/icm.h
+++ b/drivers/net/ethernet/mellanox/mlx4/icm.h
@@ -71,17 +71,17 @@
gfp_t gfp_mask, int coherent);
void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent);
-int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj);
-void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, int obj);
+int mlx4_table_get(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj);
+void mlx4_table_put(struct mlx4_dev *dev, struct mlx4_icm_table *table, u32 obj);
int mlx4_table_get_range(struct mlx4_dev *dev, struct mlx4_icm_table *table,
- int start, int end);
+ u32 start, u32 end);
void mlx4_table_put_range(struct mlx4_dev *dev, struct mlx4_icm_table *table,
- int start, int end);
+ u32 start, u32 end);
int mlx4_init_icm_table(struct mlx4_dev *dev, struct mlx4_icm_table *table,
u64 virt, int obj_size, u32 nobj, int reserved,
int use_lowmem, int use_coherent);
void mlx4_cleanup_icm_table(struct mlx4_dev *dev, struct mlx4_icm_table *table);
-void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_handle);
+void *mlx4_table_find(struct mlx4_icm_table *table, u32 obj, dma_addr_t *dma_handle);
static inline void mlx4_icm_first(struct mlx4_icm *icm,
struct mlx4_icm_iter *iter)
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index 827b72d..dd6ea94 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -1234,13 +1234,13 @@
mlx4_info(dev, "non-primary physical function, skipping.\n");
else
mlx4_err(dev, "QUERY_FW command failed, aborting.\n");
- goto unmap_bf;
+ return err;
}
err = mlx4_load_fw(dev);
if (err) {
mlx4_err(dev, "Failed to start FW, aborting.\n");
- goto unmap_bf;
+ return err;
}
mlx4_cfg.log_pg_sz_m = 1;
@@ -1304,7 +1304,7 @@
err = mlx4_init_slave(dev);
if (err) {
mlx4_err(dev, "Failed to initialize slave\n");
- goto unmap_bf;
+ return err;
}
err = mlx4_slave_cap(dev);
@@ -1324,7 +1324,7 @@
err = mlx4_QUERY_ADAPTER(dev, &adapter);
if (err) {
mlx4_err(dev, "QUERY_ADAPTER command failed, aborting.\n");
- goto err_close;
+ goto unmap_bf;
}
priv->eq_table.inta_pin = adapter.inta_pin;
@@ -1332,6 +1332,9 @@
return 0;
+unmap_bf:
+ unmap_bf_area(dev);
+
err_close:
mlx4_close_hca(dev);
@@ -1344,8 +1347,6 @@
mlx4_UNMAP_FA(dev);
mlx4_free_icm(dev, priv->fw.fw_icm, 0);
}
-unmap_bf:
- unmap_bf_area(dev);
return err;
}
@@ -1996,7 +1997,8 @@
}
slave_start:
- if (mlx4_cmd_init(dev)) {
+ err = mlx4_cmd_init(dev);
+ if (err) {
mlx4_err(dev, "Failed to init command interface, aborting.\n");
goto err_sriov;
}
@@ -2298,7 +2300,7 @@
return ret ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED;
}
-static struct pci_error_handlers mlx4_err_handler = {
+static const struct pci_error_handlers mlx4_err_handler = {
.error_detected = mlx4_pci_err_detected,
.slot_reset = mlx4_pci_slot_reset,
};
diff --git a/drivers/net/ethernet/mellanox/mlx4/mcg.c b/drivers/net/ethernet/mellanox/mlx4/mcg.c
index a018ea2..e151c21 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mcg.c
+++ b/drivers/net/ethernet/mellanox/mlx4/mcg.c
@@ -137,11 +137,11 @@
return err;
}
-static struct mlx4_promisc_qp *get_promisc_qp(struct mlx4_dev *dev, u8 pf_num,
+static struct mlx4_promisc_qp *get_promisc_qp(struct mlx4_dev *dev, u8 port,
enum mlx4_steer_type steer,
u32 qpn)
{
- struct mlx4_steer *s_steer = &mlx4_priv(dev)->steer[pf_num];
+ struct mlx4_steer *s_steer = &mlx4_priv(dev)->steer[port - 1];
struct mlx4_promisc_qp *pqp;
list_for_each_entry(pqp, &s_steer->promisc_qps[steer], list) {
@@ -182,7 +182,7 @@
/* If the given qpn is also a promisc qp,
* it should be inserted to duplicates list
*/
- pqp = get_promisc_qp(dev, 0, steer, qpn);
+ pqp = get_promisc_qp(dev, port, steer, qpn);
if (pqp) {
dqp = kmalloc(sizeof *dqp, GFP_KERNEL);
if (!dqp) {
@@ -256,7 +256,7 @@
s_steer = &mlx4_priv(dev)->steer[port - 1];
- pqp = get_promisc_qp(dev, 0, steer, qpn);
+ pqp = get_promisc_qp(dev, port, steer, qpn);
if (!pqp)
return 0; /* nothing to do */
@@ -302,7 +302,7 @@
s_steer = &mlx4_priv(dev)->steer[port - 1];
/* if qp is not promisc, it cannot be duplicated */
- if (!get_promisc_qp(dev, 0, steer, qpn))
+ if (!get_promisc_qp(dev, port, steer, qpn))
return false;
/* The qp is promisc qp so it is a duplicate on this index
@@ -352,7 +352,7 @@
members_count = be32_to_cpu(mgm->members_count) & 0xffffff;
for (i = 0; i < members_count; i++) {
qpn = be32_to_cpu(mgm->qp[i]) & MGM_QPN_MASK;
- if (!get_promisc_qp(dev, 0, steer, qpn) && qpn != tqpn) {
+ if (!get_promisc_qp(dev, port, steer, qpn) && qpn != tqpn) {
/* the qp is not promisc, the entry can't be removed */
goto out;
}
@@ -398,7 +398,7 @@
mutex_lock(&priv->mcg_table.mutex);
- if (get_promisc_qp(dev, 0, steer, qpn)) {
+ if (get_promisc_qp(dev, port, steer, qpn)) {
err = 0; /* Noting to do, already exists */
goto out_mutex;
}
@@ -503,7 +503,7 @@
s_steer = &mlx4_priv(dev)->steer[port - 1];
mutex_lock(&priv->mcg_table.mutex);
- pqp = get_promisc_qp(dev, 0, steer, qpn);
+ pqp = get_promisc_qp(dev, port, steer, qpn);
if (unlikely(!pqp)) {
mlx4_warn(dev, "QP %x is not promiscuous QP\n", qpn);
/* nothing to do */
@@ -650,13 +650,6 @@
return err;
}
-struct mlx4_net_trans_rule_hw_ctrl {
- __be32 ctrl;
- __be32 vf_vep_port;
- __be32 qpn;
- __be32 reserved;
-};
-
static void trans_rule_ctrl_to_hw(struct mlx4_net_trans_rule *ctrl,
struct mlx4_net_trans_rule_hw_ctrl *hw)
{
@@ -680,87 +673,18 @@
hw->qpn = cpu_to_be32(ctrl->qpn);
}
-struct mlx4_net_trans_rule_hw_ib {
- u8 size;
- u8 rsvd1;
- __be16 id;
- u32 rsvd2;
- __be32 qpn;
- __be32 qpn_mask;
- u8 dst_gid[16];
- u8 dst_gid_msk[16];
-} __packed;
-
-struct mlx4_net_trans_rule_hw_eth {
- u8 size;
- u8 rsvd;
- __be16 id;
- u8 rsvd1[6];
- u8 dst_mac[6];
- u16 rsvd2;
- u8 dst_mac_msk[6];
- u16 rsvd3;
- u8 src_mac[6];
- u16 rsvd4;
- u8 src_mac_msk[6];
- u8 rsvd5;
- u8 ether_type_enable;
- __be16 ether_type;
- __be16 vlan_id_msk;
- __be16 vlan_id;
-} __packed;
-
-struct mlx4_net_trans_rule_hw_tcp_udp {
- u8 size;
- u8 rsvd;
- __be16 id;
- __be16 rsvd1[3];
- __be16 dst_port;
- __be16 rsvd2;
- __be16 dst_port_msk;
- __be16 rsvd3;
- __be16 src_port;
- __be16 rsvd4;
- __be16 src_port_msk;
-} __packed;
-
-struct mlx4_net_trans_rule_hw_ipv4 {
- u8 size;
- u8 rsvd;
- __be16 id;
- __be32 rsvd1;
- __be32 dst_ip;
- __be32 dst_ip_msk;
- __be32 src_ip;
- __be32 src_ip_msk;
-} __packed;
-
-struct _rule_hw {
- union {
- struct {
- u8 size;
- u8 rsvd;
- __be16 id;
- };
- struct mlx4_net_trans_rule_hw_eth eth;
- struct mlx4_net_trans_rule_hw_ib ib;
- struct mlx4_net_trans_rule_hw_ipv4 ipv4;
- struct mlx4_net_trans_rule_hw_tcp_udp tcp_udp;
- };
+const u16 __sw_id_hw[] = {
+ [MLX4_NET_TRANS_RULE_ID_ETH] = 0xE001,
+ [MLX4_NET_TRANS_RULE_ID_IB] = 0xE005,
+ [MLX4_NET_TRANS_RULE_ID_IPV6] = 0xE003,
+ [MLX4_NET_TRANS_RULE_ID_IPV4] = 0xE002,
+ [MLX4_NET_TRANS_RULE_ID_TCP] = 0xE004,
+ [MLX4_NET_TRANS_RULE_ID_UDP] = 0xE006
};
static int parse_trans_rule(struct mlx4_dev *dev, struct mlx4_spec_list *spec,
struct _rule_hw *rule_hw)
{
- static const u16 __sw_id_hw[] = {
- [MLX4_NET_TRANS_RULE_ID_ETH] = 0xE001,
- [MLX4_NET_TRANS_RULE_ID_IB] = 0xE005,
- [MLX4_NET_TRANS_RULE_ID_IPV6] = 0xE003,
- [MLX4_NET_TRANS_RULE_ID_IPV4] = 0xE002,
- [MLX4_NET_TRANS_RULE_ID_TCP] = 0xE004,
- [MLX4_NET_TRANS_RULE_ID_UDP] = 0xE006
- };
-
static const size_t __rule_hw_sz[] = {
[MLX4_NET_TRANS_RULE_ID_ETH] =
sizeof(struct mlx4_net_trans_rule_hw_eth),
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4.h b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
index 4d9df8f..dba69d9 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
@@ -690,6 +690,82 @@
struct list_head steer_entries[MLX4_NUM_STEERS];
};
+struct mlx4_net_trans_rule_hw_ctrl {
+ __be32 ctrl;
+ __be32 vf_vep_port;
+ __be32 qpn;
+ __be32 reserved;
+};
+
+struct mlx4_net_trans_rule_hw_ib {
+ u8 size;
+ u8 rsvd1;
+ __be16 id;
+ u32 rsvd2;
+ __be32 qpn;
+ __be32 qpn_mask;
+ u8 dst_gid[16];
+ u8 dst_gid_msk[16];
+} __packed;
+
+struct mlx4_net_trans_rule_hw_eth {
+ u8 size;
+ u8 rsvd;
+ __be16 id;
+ u8 rsvd1[6];
+ u8 dst_mac[6];
+ u16 rsvd2;
+ u8 dst_mac_msk[6];
+ u16 rsvd3;
+ u8 src_mac[6];
+ u16 rsvd4;
+ u8 src_mac_msk[6];
+ u8 rsvd5;
+ u8 ether_type_enable;
+ __be16 ether_type;
+ __be16 vlan_id_msk;
+ __be16 vlan_id;
+} __packed;
+
+struct mlx4_net_trans_rule_hw_tcp_udp {
+ u8 size;
+ u8 rsvd;
+ __be16 id;
+ __be16 rsvd1[3];
+ __be16 dst_port;
+ __be16 rsvd2;
+ __be16 dst_port_msk;
+ __be16 rsvd3;
+ __be16 src_port;
+ __be16 rsvd4;
+ __be16 src_port_msk;
+} __packed;
+
+struct mlx4_net_trans_rule_hw_ipv4 {
+ u8 size;
+ u8 rsvd;
+ __be16 id;
+ __be32 rsvd1;
+ __be32 dst_ip;
+ __be32 dst_ip_msk;
+ __be32 src_ip;
+ __be32 src_ip_msk;
+} __packed;
+
+struct _rule_hw {
+ union {
+ struct {
+ u8 size;
+ u8 rsvd;
+ __be16 id;
+ };
+ struct mlx4_net_trans_rule_hw_eth eth;
+ struct mlx4_net_trans_rule_hw_ib ib;
+ struct mlx4_net_trans_rule_hw_ipv4 ipv4;
+ struct mlx4_net_trans_rule_hw_tcp_udp tcp_udp;
+ };
+};
+
struct mlx4_priv {
struct mlx4_dev dev;
diff --git a/drivers/net/ethernet/mellanox/mlx4/reset.c b/drivers/net/ethernet/mellanox/mlx4/reset.c
index 11e7c1c..dd1b509 100644
--- a/drivers/net/ethernet/mellanox/mlx4/reset.c
+++ b/drivers/net/ethernet/mellanox/mlx4/reset.c
@@ -141,16 +141,16 @@
/* Now restore the PCI headers */
if (pcie_cap) {
devctl = hca_header[(pcie_cap + PCI_EXP_DEVCTL) / 4];
- if (pci_write_config_word(dev->pdev, pcie_cap + PCI_EXP_DEVCTL,
- devctl)) {
+ if (pcie_capability_write_word(dev->pdev, PCI_EXP_DEVCTL,
+ devctl)) {
err = -ENODEV;
mlx4_err(dev, "Couldn't restore HCA PCI Express "
"Device Control register, aborting.\n");
goto out;
}
linkctl = hca_header[(pcie_cap + PCI_EXP_LNKCTL) / 4];
- if (pci_write_config_word(dev->pdev, pcie_cap + PCI_EXP_LNKCTL,
- linkctl)) {
+ if (pcie_capability_write_word(dev->pdev, PCI_EXP_LNKCTL,
+ linkctl)) {
err = -ENODEV;
mlx4_err(dev, "Couldn't restore HCA PCI Express "
"Link control register, aborting.\n");
diff --git a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
index 94ceddd..293c9e8 100644
--- a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
+++ b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
@@ -42,6 +42,7 @@
#include <linux/mlx4/cmd.h>
#include <linux/mlx4/qp.h>
#include <linux/if_ether.h>
+#include <linux/etherdevice.h>
#include "mlx4.h"
#include "fw.h"
@@ -2776,18 +2777,133 @@
return err;
}
+/*
+ * MAC validation for Flow Steering rules.
+ * VF can attach rules only with a mac address which is assigned to it.
+ */
+static int validate_eth_header_mac(int slave, struct _rule_hw *eth_header,
+ struct list_head *rlist)
+{
+ struct mac_res *res, *tmp;
+ __be64 be_mac;
+
+ /* make sure it isn't multicast or broadcast mac*/
+ if (!is_multicast_ether_addr(eth_header->eth.dst_mac) &&
+ !is_broadcast_ether_addr(eth_header->eth.dst_mac)) {
+ list_for_each_entry_safe(res, tmp, rlist, list) {
+ be_mac = cpu_to_be64(res->mac << 16);
+ if (!memcmp(&be_mac, eth_header->eth.dst_mac, ETH_ALEN))
+ return 0;
+ }
+ pr_err("MAC %pM doesn't belong to VF %d, Steering rule rejected\n",
+ eth_header->eth.dst_mac, slave);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/*
+ * In case of missing eth header, append eth header with a MAC address
+ * assigned to the VF.
+ */
+static int add_eth_header(struct mlx4_dev *dev, int slave,
+ struct mlx4_cmd_mailbox *inbox,
+ struct list_head *rlist, int header_id)
+{
+ struct mac_res *res, *tmp;
+ u8 port;
+ struct mlx4_net_trans_rule_hw_ctrl *ctrl;
+ struct mlx4_net_trans_rule_hw_eth *eth_header;
+ struct mlx4_net_trans_rule_hw_ipv4 *ip_header;
+ struct mlx4_net_trans_rule_hw_tcp_udp *l4_header;
+ __be64 be_mac = 0;
+ __be64 mac_msk = cpu_to_be64(MLX4_MAC_MASK << 16);
+
+ ctrl = (struct mlx4_net_trans_rule_hw_ctrl *)inbox->buf;
+ port = be32_to_cpu(ctrl->vf_vep_port) & 0xff;
+ eth_header = (struct mlx4_net_trans_rule_hw_eth *)(ctrl + 1);
+
+ /* Clear a space in the inbox for eth header */
+ switch (header_id) {
+ case MLX4_NET_TRANS_RULE_ID_IPV4:
+ ip_header =
+ (struct mlx4_net_trans_rule_hw_ipv4 *)(eth_header + 1);
+ memmove(ip_header, eth_header,
+ sizeof(*ip_header) + sizeof(*l4_header));
+ break;
+ case MLX4_NET_TRANS_RULE_ID_TCP:
+ case MLX4_NET_TRANS_RULE_ID_UDP:
+ l4_header = (struct mlx4_net_trans_rule_hw_tcp_udp *)
+ (eth_header + 1);
+ memmove(l4_header, eth_header, sizeof(*l4_header));
+ break;
+ default:
+ return -EINVAL;
+ }
+ list_for_each_entry_safe(res, tmp, rlist, list) {
+ if (port == res->port) {
+ be_mac = cpu_to_be64(res->mac << 16);
+ break;
+ }
+ }
+ if (!be_mac) {
+ pr_err("Failed adding eth header to FS rule, Can't find matching MAC for port %d .\n",
+ port);
+ return -EINVAL;
+ }
+
+ memset(eth_header, 0, sizeof(*eth_header));
+ eth_header->size = sizeof(*eth_header) >> 2;
+ eth_header->id = cpu_to_be16(__sw_id_hw[MLX4_NET_TRANS_RULE_ID_ETH]);
+ memcpy(eth_header->dst_mac, &be_mac, ETH_ALEN);
+ memcpy(eth_header->dst_mac_msk, &mac_msk, ETH_ALEN);
+
+ return 0;
+
+}
+
int mlx4_QP_FLOW_STEERING_ATTACH_wrapper(struct mlx4_dev *dev, int slave,
struct mlx4_vhcr *vhcr,
struct mlx4_cmd_mailbox *inbox,
struct mlx4_cmd_mailbox *outbox,
struct mlx4_cmd_info *cmd)
{
+
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_resource_tracker *tracker = &priv->mfunc.master.res_tracker;
+ struct list_head *rlist = &tracker->slave_list[slave].res_list[RES_MAC];
int err;
+ struct mlx4_net_trans_rule_hw_ctrl *ctrl;
+ struct _rule_hw *rule_header;
+ int header_id;
if (dev->caps.steering_mode !=
MLX4_STEERING_MODE_DEVICE_MANAGED)
return -EOPNOTSUPP;
+ ctrl = (struct mlx4_net_trans_rule_hw_ctrl *)inbox->buf;
+ rule_header = (struct _rule_hw *)(ctrl + 1);
+ header_id = map_hw_to_sw_id(be16_to_cpu(rule_header->id));
+
+ switch (header_id) {
+ case MLX4_NET_TRANS_RULE_ID_ETH:
+ if (validate_eth_header_mac(slave, rule_header, rlist))
+ return -EINVAL;
+ break;
+ case MLX4_NET_TRANS_RULE_ID_IPV4:
+ case MLX4_NET_TRANS_RULE_ID_TCP:
+ case MLX4_NET_TRANS_RULE_ID_UDP:
+ pr_warn("Can't attach FS rule without L2 headers, adding L2 header.\n");
+ if (add_eth_header(dev, slave, inbox, rlist, header_id))
+ return -EINVAL;
+ vhcr->in_modifier +=
+ sizeof(struct mlx4_net_trans_rule_hw_eth) >> 2;
+ break;
+ default:
+ pr_err("Corrupted mailbox.\n");
+ return -EINVAL;
+ }
+
err = mlx4_cmd_imm(dev, inbox->dma, &vhcr->out_param,
vhcr->in_modifier, 0,
MLX4_QP_FLOW_STEERING_ATTACH, MLX4_CMD_TIME_CLASS_A,
diff --git a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
index fa85cf1..83516e3 100644
--- a/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
+++ b/drivers/net/ethernet/myricom/myri10ge/myri10ge.c
@@ -1078,22 +1078,16 @@
#ifdef CONFIG_MYRI10GE_DCA
static int myri10ge_toggle_relaxed(struct pci_dev *pdev, int on)
{
- int ret, cap, err;
+ int ret;
u16 ctl;
- cap = pci_pcie_cap(pdev);
- if (!cap)
- return 0;
-
- err = pci_read_config_word(pdev, cap + PCI_EXP_DEVCTL, &ctl);
- if (err)
- return 0;
+ pcie_capability_read_word(pdev, PCI_EXP_DEVCTL, &ctl);
ret = (ctl & PCI_EXP_DEVCTL_RELAX_EN) >> 4;
if (ret != on) {
ctl &= ~PCI_EXP_DEVCTL_RELAX_EN;
ctl |= (on << 4);
- pci_write_config_word(pdev, cap + PCI_EXP_DEVCTL, ctl);
+ pcie_capability_write_word(pdev, PCI_EXP_DEVCTL, ctl);
}
return ret;
}
@@ -3192,18 +3186,13 @@
struct device *dev = &mgp->pdev->dev;
int cap;
unsigned err_cap;
- u16 val;
- u8 ext_type;
int ret;
if (!myri10ge_ecrc_enable || !bridge)
return;
/* check that the bridge is a root port */
- cap = pci_pcie_cap(bridge);
- pci_read_config_word(bridge, cap + PCI_CAP_FLAGS, &val);
- ext_type = (val & PCI_EXP_FLAGS_TYPE) >> 4;
- if (ext_type != PCI_EXP_TYPE_ROOT_PORT) {
+ if (pci_pcie_type(bridge) != PCI_EXP_TYPE_ROOT_PORT) {
if (myri10ge_ecrc_enable > 1) {
struct pci_dev *prev_bridge, *old_bridge = bridge;
@@ -3218,11 +3207,8 @@
" to force ECRC\n");
return;
}
- cap = pci_pcie_cap(bridge);
- pci_read_config_word(bridge,
- cap + PCI_CAP_FLAGS, &val);
- ext_type = (val & PCI_EXP_FLAGS_TYPE) >> 4;
- } while (ext_type != PCI_EXP_TYPE_ROOT_PORT);
+ } while (pci_pcie_type(bridge) !=
+ PCI_EXP_TYPE_ROOT_PORT);
dev_info(dev,
"Forcing ECRC on non-root port %s"
@@ -3335,11 +3321,10 @@
int overridden = 0;
if (myri10ge_force_firmware == 0) {
- int link_width, exp_cap;
+ int link_width;
u16 lnk;
- exp_cap = pci_pcie_cap(mgp->pdev);
- pci_read_config_word(mgp->pdev, exp_cap + PCI_EXP_LNKSTA, &lnk);
+ pcie_capability_read_word(mgp->pdev, PCI_EXP_LNKSTA, &lnk);
link_width = (lnk >> 4) & 0x3f;
/* Check to see if Link is less than 8 or if the
diff --git a/drivers/net/ethernet/neterion/s2io.c b/drivers/net/ethernet/neterion/s2io.c
index d958c22..de50547 100644
--- a/drivers/net/ethernet/neterion/s2io.c
+++ b/drivers/net/ethernet/neterion/s2io.c
@@ -484,7 +484,7 @@
MODULE_DEVICE_TABLE(pci, s2io_tbl);
-static struct pci_error_handlers s2io_err_handler = {
+static const struct pci_error_handlers s2io_err_handler = {
.error_detected = s2io_io_error_detected,
.slot_reset = s2io_io_slot_reset,
.resume = s2io_io_resume,
diff --git a/drivers/net/ethernet/neterion/vxge/vxge-config.c b/drivers/net/ethernet/neterion/vxge/vxge-config.c
index 32d0682..c2e420a 100644
--- a/drivers/net/ethernet/neterion/vxge/vxge-config.c
+++ b/drivers/net/ethernet/neterion/vxge/vxge-config.c
@@ -757,7 +757,7 @@
u16 lnk;
/* Get the negotiated link width and speed from PCI config space */
- pci_read_config_word(dev, dev->pcie_cap + PCI_EXP_LNKSTA, &lnk);
+ pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnk);
if ((lnk & PCI_EXP_LNKSTA_CLS) != 1)
return VXGE_HW_ERR_INVALID_PCI_INFO;
@@ -1982,7 +1982,7 @@
struct pci_dev *dev = hldev->pdev;
u16 lnk;
- pci_read_config_word(dev, dev->pcie_cap + PCI_EXP_LNKSTA, &lnk);
+ pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnk);
return (lnk & VXGE_HW_PCI_EXP_LNKCAP_LNK_WIDTH) >> 4;
}
diff --git a/drivers/net/ethernet/neterion/vxge/vxge-main.c b/drivers/net/ethernet/neterion/vxge/vxge-main.c
index de21904..cfa71a3 100644
--- a/drivers/net/ethernet/neterion/vxge/vxge-main.c
+++ b/drivers/net/ethernet/neterion/vxge/vxge-main.c
@@ -4799,7 +4799,7 @@
__LINE__);
}
-static struct pci_error_handlers vxge_err_handler = {
+static const struct pci_error_handlers vxge_err_handler = {
.error_detected = vxge_io_error_detected,
.slot_reset = vxge_io_slot_reset,
.resume = vxge_io_resume,
diff --git a/drivers/net/ethernet/octeon/octeon_mgmt.c b/drivers/net/ethernet/octeon/octeon_mgmt.c
index c42bbb1..a688a2d 100644
--- a/drivers/net/ethernet/octeon/octeon_mgmt.c
+++ b/drivers/net/ethernet/octeon/octeon_mgmt.c
@@ -722,10 +722,8 @@
octeon_mgmt_adjust_link, 0,
PHY_INTERFACE_MODE_MII);
- if (IS_ERR(p->phydev)) {
- p->phydev = NULL;
+ if (!p->phydev)
return -1;
- }
phy_start_aneg(p->phydev);
diff --git a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c
index feb85d5..b2a94d0 100644
--- a/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c
+++ b/drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c
@@ -2795,7 +2795,7 @@
};
#endif
-static struct pci_error_handlers pch_gbe_err_handler = {
+static const struct pci_error_handlers pch_gbe_err_handler = {
.error_detected = pch_gbe_io_error_detected,
.slot_reset = pch_gbe_io_slot_reset,
.resume = pch_gbe_io_resume
diff --git a/drivers/net/ethernet/pasemi/pasemi_mac.c b/drivers/net/ethernet/pasemi/pasemi_mac.c
index e559dfa..6fa74d5 100644
--- a/drivers/net/ethernet/pasemi/pasemi_mac.c
+++ b/drivers/net/ethernet/pasemi/pasemi_mac.c
@@ -1101,9 +1101,9 @@
phydev = of_phy_connect(dev, phy_dn, &pasemi_adjust_link, 0,
PHY_INTERFACE_MODE_SGMII);
- if (IS_ERR(phydev)) {
+ if (!phydev) {
printk(KERN_ERR "%s: Could not attach to phy\n", dev->name);
- return PTR_ERR(phydev);
+ return -ENODEV;
}
mac->phydev = phydev;
diff --git a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c
index 342b3a7..df45061 100644
--- a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c
+++ b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c
@@ -1378,11 +1378,15 @@
struct pci_dev *root = pdev->bus->self;
u32 aer_pos;
+ /* root bus? */
+ if (!root)
+ return;
+
if (adapter->ahw.board_type != NETXEN_BRDTYPE_P3_4_GB_MM &&
adapter->ahw.board_type != NETXEN_BRDTYPE_P3_10G_TP)
return;
- if (root->pcie_type != PCI_EXP_TYPE_ROOT_PORT)
+ if (pci_pcie_type(root) != PCI_EXP_TYPE_ROOT_PORT)
return;
aer_pos = pci_find_ext_capability(root, PCI_EXT_CAP_ID_ERR);
@@ -3336,7 +3340,7 @@
{ }
#endif
-static struct pci_error_handlers netxen_err_handler = {
+static const struct pci_error_handlers netxen_err_handler = {
.error_detected = netxen_io_error_detected,
.slot_reset = netxen_io_slot_reset,
.resume = netxen_io_resume,
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ctx.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ctx.c
index b8ead69..2a179d0 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ctx.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_ctx.c
@@ -15,7 +15,7 @@
do {
/* give atleast 1ms for firmware to respond */
- msleep(1);
+ mdelay(1);
if (++timeout > QLCNIC_OS_CRB_RETRY_COUNT)
return QLCNIC_CDRP_RSP_TIMEOUT;
@@ -601,7 +601,7 @@
qlcnic_fw_cmd_destroy_tx_ctx(adapter);
/* Allow dma queues to drain after context reset */
- msleep(20);
+ mdelay(20);
}
}
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
index 212c121..473ce13 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
@@ -4522,7 +4522,7 @@
qlcnic_restore_indev_addr(struct net_device *dev, unsigned long event)
{ }
#endif
-static struct pci_error_handlers qlcnic_err_handler = {
+static const struct pci_error_handlers qlcnic_err_handler = {
.error_detected = qlcnic_io_error_detected,
.slot_reset = qlcnic_io_slot_reset,
.resume = qlcnic_io_resume,
diff --git a/drivers/net/ethernet/qlogic/qlge/qlge_main.c b/drivers/net/ethernet/qlogic/qlge/qlge_main.c
index b53a3b6..b262d61 100644
--- a/drivers/net/ethernet/qlogic/qlge/qlge_main.c
+++ b/drivers/net/ethernet/qlogic/qlge/qlge_main.c
@@ -4847,7 +4847,7 @@
netif_device_attach(ndev);
}
-static struct pci_error_handlers qlge_err_handler = {
+static const struct pci_error_handlers qlge_err_handler = {
.error_detected = qlge_io_error_detected,
.slot_reset = qlge_io_slot_reset,
.resume = qlge_io_resume,
diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
index b47d5b3..a7cc560 100644
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -833,15 +833,8 @@
static void rtl_tx_performance_tweak(struct pci_dev *pdev, u16 force)
{
- int cap = pci_pcie_cap(pdev);
-
- if (cap) {
- u16 ctl;
-
- pci_read_config_word(pdev, cap + PCI_EXP_DEVCTL, &ctl);
- ctl = (ctl & ~PCI_EXP_DEVCTL_READRQ) | force;
- pci_write_config_word(pdev, cap + PCI_EXP_DEVCTL, ctl);
- }
+ pcie_capability_clear_and_set_word(pdev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_READRQ, force);
}
struct rtl_cond {
@@ -4739,28 +4732,14 @@
static void rtl_disable_clock_request(struct pci_dev *pdev)
{
- int cap = pci_pcie_cap(pdev);
-
- if (cap) {
- u16 ctl;
-
- pci_read_config_word(pdev, cap + PCI_EXP_LNKCTL, &ctl);
- ctl &= ~PCI_EXP_LNKCTL_CLKREQ_EN;
- pci_write_config_word(pdev, cap + PCI_EXP_LNKCTL, ctl);
- }
+ pcie_capability_clear_word(pdev, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
}
static void rtl_enable_clock_request(struct pci_dev *pdev)
{
- int cap = pci_pcie_cap(pdev);
-
- if (cap) {
- u16 ctl;
-
- pci_read_config_word(pdev, cap + PCI_EXP_LNKCTL, &ctl);
- ctl |= PCI_EXP_LNKCTL_CLKREQ_EN;
- pci_write_config_word(pdev, cap + PCI_EXP_LNKCTL, ctl);
- }
+ pcie_capability_set_word(pdev, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
}
#define R8168_CPCMD_QUIRK_MASK (\
@@ -5405,14 +5384,9 @@
tp->event_slow &= ~RxFIFOOver;
if (tp->mac_version == RTL_GIGA_MAC_VER_13 ||
- tp->mac_version == RTL_GIGA_MAC_VER_16) {
- int cap = pci_pcie_cap(pdev);
-
- if (cap) {
- pci_write_config_word(pdev, cap + PCI_EXP_DEVCTL,
- PCI_EXP_DEVCTL_NOSNOOP_EN);
- }
- }
+ tp->mac_version == RTL_GIGA_MAC_VER_16)
+ pcie_capability_set_word(pdev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_NOSNOOP_EN);
RTL_W8(Cfg9346, Cfg9346_Unlock);
diff --git a/drivers/net/ethernet/seeq/sgiseeq.c b/drivers/net/ethernet/seeq/sgiseeq.c
index bb8c822..4d15bf4 100644
--- a/drivers/net/ethernet/seeq/sgiseeq.c
+++ b/drivers/net/ethernet/seeq/sgiseeq.c
@@ -751,6 +751,7 @@
sp->srings = sr;
sp->rx_desc = sp->srings->rxvector;
sp->tx_desc = sp->srings->txvector;
+ spin_lock_init(&sp->tx_lock);
/* A couple calculations now, saves many cycles later. */
setup_rx_ring(dev, sp->rx_desc, SEEQ_RX_BUFFERS);
diff --git a/drivers/net/ethernet/sfc/ethtool.c b/drivers/net/ethernet/sfc/ethtool.c
index 8cba2df..5faedd8 100644
--- a/drivers/net/ethernet/sfc/ethtool.c
+++ b/drivers/net/ethernet/sfc/ethtool.c
@@ -863,8 +863,8 @@
&ip_entry->ip4dst, &ip_entry->pdst);
if (rc != 0) {
rc = efx_filter_get_ipv4_full(
- &spec, &proto, &ip_entry->ip4src, &ip_entry->psrc,
- &ip_entry->ip4dst, &ip_entry->pdst);
+ &spec, &proto, &ip_entry->ip4dst, &ip_entry->pdst,
+ &ip_entry->ip4src, &ip_entry->psrc);
EFX_WARN_ON_PARANOID(rc);
ip_mask->ip4src = ~0;
ip_mask->psrc = ~0;
diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h
index e2d0832..719be39 100644
--- a/drivers/net/ethernet/stmicro/stmmac/common.h
+++ b/drivers/net/ethernet/stmicro/stmmac/common.h
@@ -22,6 +22,9 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __COMMON_H__
+#define __COMMON_H__
+
#include <linux/etherdevice.h>
#include <linux/netdevice.h>
#include <linux/phy.h>
@@ -366,3 +369,5 @@
extern void dwmac_dma_flush_tx_fifo(void __iomem *ioaddr);
extern const struct stmmac_ring_mode_ops ring_mode_ops;
+
+#endif /* __COMMON_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/descs.h b/drivers/net/ethernet/stmicro/stmmac/descs.h
index 9820ec8..223adf9 100644
--- a/drivers/net/ethernet/stmicro/stmmac/descs.h
+++ b/drivers/net/ethernet/stmicro/stmmac/descs.h
@@ -20,6 +20,10 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+
+#ifndef __DESCS_H__
+#define __DESCS_H__
+
struct dma_desc {
/* Receive descriptor */
union {
@@ -166,3 +170,5 @@
* is not calculated */
cic_full = 3, /* IP header and pseudoheader */
};
+
+#endif /* __DESCS_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/descs_com.h b/drivers/net/ethernet/stmicro/stmmac/descs_com.h
index dd8d6e1..7ee9499 100644
--- a/drivers/net/ethernet/stmicro/stmmac/descs_com.h
+++ b/drivers/net/ethernet/stmicro/stmmac/descs_com.h
@@ -27,6 +27,9 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __DESC_COM_H__
+#define __DESC_COM_H__
+
#if defined(CONFIG_STMMAC_RING)
static inline void ehn_desc_rx_set_on_ring_chain(struct dma_desc *p, int end)
{
@@ -124,3 +127,5 @@
p->des01.tx.buffer1_size = len;
}
#endif
+
+#endif /* __DESC_COM_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac100.h b/drivers/net/ethernet/stmicro/stmmac/dwmac100.h
index 7c6d857..2ec6aea 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac100.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac100.h
@@ -22,6 +22,9 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __DWMAC100_H__
+#define __DWMAC100_H__
+
#include <linux/phy.h>
#include "common.h"
@@ -119,3 +122,5 @@
#define DMA_MISSED_FRAME_M_CNTR 0x0000ffff /* Missed Frame Couinter */
extern const struct stmmac_dma_ops dwmac100_dma_ops;
+
+#endif /* __DWMAC100_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h b/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h
index f90fcb5..0e4cace 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000.h
@@ -19,6 +19,8 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __DWMAC1000_H__
+#define __DWMAC1000_H__
#include <linux/phy.h>
#include "common.h"
@@ -229,6 +231,7 @@
#define GMAC_MMC_RX_CSUM_OFFLOAD 0x208
/* Synopsys Core versions */
-#define DWMAC_CORE_3_40 34
+#define DWMAC_CORE_3_40 0x34
extern const struct stmmac_dma_ops dwmac1000_dma_ops;
+#endif /* __DWMAC1000_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h b/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h
index e678ce3..e49c9a0 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h
@@ -22,6 +22,9 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __DWMAC_DMA_H__
+#define __DWMAC_DMA_H__
+
/* DMA CRS Control and Status Register Mapping */
#define DMA_BUS_MODE 0x00001000 /* Bus Mode */
#define DMA_XMT_POLL_DEMAND 0x00001004 /* Transmit Poll Demand */
@@ -109,3 +112,5 @@
extern void dwmac_dma_stop_rx(void __iomem *ioaddr);
extern int dwmac_dma_interrupt(void __iomem *ioaddr,
struct stmmac_extra_stats *x);
+
+#endif /* __DWMAC_DMA_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/mmc.h b/drivers/net/ethernet/stmicro/stmmac/mmc.h
index a3835202..67995ef 100644
--- a/drivers/net/ethernet/stmicro/stmmac/mmc.h
+++ b/drivers/net/ethernet/stmicro/stmmac/mmc.h
@@ -22,6 +22,9 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __MMC_H__
+#define __MMC_H__
+
/* MMC control register */
/* When set, all counter are reset */
#define MMC_CNTRL_COUNTER_RESET 0x1
@@ -129,3 +132,5 @@
extern void dwmac_mmc_ctrl(void __iomem *ioaddr, unsigned int mode);
extern void dwmac_mmc_intr_all_mask(void __iomem *ioaddr);
extern void dwmac_mmc_read(void __iomem *ioaddr, struct stmmac_counters *mmc);
+
+#endif /* __MMC_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/mmc_core.c b/drivers/net/ethernet/stmicro/stmmac/mmc_core.c
index c07cfe9..0c74a70 100644
--- a/drivers/net/ethernet/stmicro/stmmac/mmc_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/mmc_core.c
@@ -33,7 +33,7 @@
#define MMC_TX_INTR 0x00000108 /* MMC TX Interrupt */
#define MMC_RX_INTR_MASK 0x0000010c /* MMC Interrupt Mask */
#define MMC_TX_INTR_MASK 0x00000110 /* MMC Interrupt Mask */
-#define MMC_DEFAUL_MASK 0xffffffff
+#define MMC_DEFAULT_MASK 0xffffffff
/* MMC TX counter registers */
@@ -147,8 +147,8 @@
/* To mask all all interrupts.*/
void dwmac_mmc_intr_all_mask(void __iomem *ioaddr)
{
- writel(MMC_DEFAUL_MASK, ioaddr + MMC_RX_INTR_MASK);
- writel(MMC_DEFAUL_MASK, ioaddr + MMC_TX_INTR_MASK);
+ writel(MMC_DEFAULT_MASK, ioaddr + MMC_RX_INTR_MASK);
+ writel(MMC_DEFAULT_MASK, ioaddr + MMC_TX_INTR_MASK);
}
/* This reads the MAC core counters (if actaully supported).
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
index f2d3665..e872e1d 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
@@ -20,6 +20,9 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __STMMAC_H__
+#define __STMMAC_H__
+
#define STMMAC_RESOURCE_NAME "stmmaceth"
#define DRV_MODULE_VERSION "March_2012"
@@ -166,3 +169,5 @@
{
}
#endif /* CONFIG_STMMAC_PCI */
+
+#endif /* __STMMAC_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index c136162..3be8833 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -1066,7 +1066,7 @@
} else
priv->tm->enable = 1;
#endif
- clk_enable(priv->stmmac_clk);
+ clk_prepare_enable(priv->stmmac_clk);
stmmac_check_ether_addr(priv);
@@ -1188,7 +1188,7 @@
if (priv->phydev)
phy_disconnect(priv->phydev);
- clk_disable(priv->stmmac_clk);
+ clk_disable_unprepare(priv->stmmac_clk);
return ret;
}
@@ -1246,7 +1246,7 @@
#ifdef CONFIG_STMMAC_DEBUG_FS
stmmac_exit_fs();
#endif
- clk_disable(priv->stmmac_clk);
+ clk_disable_unprepare(priv->stmmac_clk);
return 0;
}
@@ -2178,7 +2178,7 @@
else {
stmmac_set_mac(priv->ioaddr, false);
/* Disable clock in case of PWM is off */
- clk_disable(priv->stmmac_clk);
+ clk_disable_unprepare(priv->stmmac_clk);
}
spin_unlock_irqrestore(&priv->lock, flags);
return 0;
@@ -2203,7 +2203,7 @@
priv->hw->mac->pmt(priv->ioaddr, 0);
else
/* enable the clk prevously disabled */
- clk_enable(priv->stmmac_clk);
+ clk_prepare_enable(priv->stmmac_clk);
netif_device_attach(ndev);
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.c
index 2a0e1ab..4ccd4e2 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.c
@@ -97,19 +97,19 @@
static void stmmac_tmu_start(unsigned int new_freq)
{
clk_set_rate(timer_clock, new_freq);
- clk_enable(timer_clock);
+ clk_prepare_enable(timer_clock);
}
static void stmmac_tmu_stop(void)
{
- clk_disable(timer_clock);
+ clk_disable_unprepare(timer_clock);
}
int stmmac_open_ext_timer(struct net_device *dev, struct stmmac_timer *tm)
{
timer_clock = clk_get(NULL, TMU_CHANNEL);
- if (timer_clock == NULL)
+ if (IS_ERR(timer_clock))
return -1;
if (tmu2_register_user(stmmac_timer_handler, (void *)dev) < 0) {
@@ -126,7 +126,7 @@
int stmmac_close_ext_timer(void)
{
- clk_disable(timer_clock);
+ clk_disable_unprepare(timer_clock);
tmu2_unregister_user();
clk_put(timer_clock);
return 0;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.h b/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.h
index 6863590..aea9b14 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.h
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_timer.h
@@ -21,6 +21,8 @@
Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
*******************************************************************************/
+#ifndef __STMMAC_TIMER_H__
+#define __STMMAC_TIMER_H__
struct stmmac_timer {
void (*timer_start) (unsigned int new_freq);
@@ -40,3 +42,5 @@
extern int tmu2_register_user(void *fnt, void *data);
extern void tmu2_unregister_user(void);
#endif
+
+#endif /* __STMMAC_TIMER_H__ */
diff --git a/drivers/net/ethernet/sun/niu.c b/drivers/net/ethernet/sun/niu.c
index c2a0fe3..3208dca 100644
--- a/drivers/net/ethernet/sun/niu.c
+++ b/drivers/net/ethernet/sun/niu.c
@@ -9762,9 +9762,8 @@
union niu_parent_id parent_id;
struct net_device *dev;
struct niu *np;
- int err, pos;
+ int err;
u64 dma_mask;
- u16 val16;
niu_driver_version();
@@ -9787,8 +9786,7 @@
goto err_out_disable_pdev;
}
- pos = pci_pcie_cap(pdev);
- if (pos <= 0) {
+ if (!pci_is_pcie(pdev)) {
dev_err(&pdev->dev, "Cannot find PCI Express capability, aborting\n");
goto err_out_free_res;
}
@@ -9813,14 +9811,11 @@
goto err_out_free_dev;
}
- pci_read_config_word(pdev, pos + PCI_EXP_DEVCTL, &val16);
- val16 &= ~PCI_EXP_DEVCTL_NOSNOOP_EN;
- val16 |= (PCI_EXP_DEVCTL_CERE |
- PCI_EXP_DEVCTL_NFERE |
- PCI_EXP_DEVCTL_FERE |
- PCI_EXP_DEVCTL_URRE |
- PCI_EXP_DEVCTL_RELAX_EN);
- pci_write_config_word(pdev, pos + PCI_EXP_DEVCTL, val16);
+ pcie_capability_clear_and_set_word(pdev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_NOSNOOP_EN,
+ PCI_EXP_DEVCTL_CERE | PCI_EXP_DEVCTL_NFERE |
+ PCI_EXP_DEVCTL_FERE | PCI_EXP_DEVCTL_URRE |
+ PCI_EXP_DEVCTL_RELAX_EN);
dma_mask = DMA_BIT_MASK(44);
err = pci_set_dma_mask(pdev, dma_mask);
diff --git a/drivers/net/ethernet/ti/davinci_mdio.c b/drivers/net/ethernet/ti/davinci_mdio.c
index cd7ee20..a9ca4a0 100644
--- a/drivers/net/ethernet/ti/davinci_mdio.c
+++ b/drivers/net/ethernet/ti/davinci_mdio.c
@@ -394,8 +394,10 @@
struct device *dev = &pdev->dev;
struct davinci_mdio_data *data = dev_get_drvdata(dev);
- if (data->bus)
+ if (data->bus) {
+ mdiobus_unregister(data->bus);
mdiobus_free(data->bus);
+ }
if (data->clk)
clk_put(data->clk);
diff --git a/drivers/net/fddi/skfp/pmf.c b/drivers/net/fddi/skfp/pmf.c
index 24d8566..441b4dc 100644
--- a/drivers/net/fddi/skfp/pmf.c
+++ b/drivers/net/fddi/skfp/pmf.c
@@ -673,7 +673,7 @@
sm_pm_get_ls(smc,port_to_mib(smc,port))) ;
break ;
case SMT_P_REASON :
- * (u_long *) to = 0 ;
+ *(u32 *)to = 0 ;
sp_len = 4 ;
goto sp_done ;
case SMT_P1033 : /* time stamp */
diff --git a/drivers/net/irda/sh_sir.c b/drivers/net/irda/sh_sir.c
index 256eddf..7951094 100644
--- a/drivers/net/irda/sh_sir.c
+++ b/drivers/net/irda/sh_sir.c
@@ -280,7 +280,7 @@
}
clk = clk_get(NULL, "irda_clk");
- if (!clk) {
+ if (IS_ERR(clk)) {
dev_err(dev, "can not get irda_clk\n");
return -EIO;
}
diff --git a/drivers/net/phy/bcm87xx.c b/drivers/net/phy/bcm87xx.c
index 2346b38..7997895 100644
--- a/drivers/net/phy/bcm87xx.c
+++ b/drivers/net/phy/bcm87xx.c
@@ -229,3 +229,5 @@
ARRAY_SIZE(bcm87xx_driver));
}
module_exit(bcm87xx_exit);
+
+MODULE_LICENSE("GPL");
diff --git a/drivers/net/phy/micrel.c b/drivers/net/phy/micrel.c
index cf287e0..2165d5f 100644
--- a/drivers/net/phy/micrel.c
+++ b/drivers/net/phy/micrel.c
@@ -21,6 +21,12 @@
#include <linux/phy.h>
#include <linux/micrel_phy.h>
+/* Operation Mode Strap Override */
+#define MII_KSZPHY_OMSO 0x16
+#define KSZPHY_OMSO_B_CAST_OFF (1 << 9)
+#define KSZPHY_OMSO_RMII_OVERRIDE (1 << 1)
+#define KSZPHY_OMSO_MII_OVERRIDE (1 << 0)
+
/* general Interrupt control/status reg in vendor specific block. */
#define MII_KSZPHY_INTCS 0x1B
#define KSZPHY_INTCS_JABBER (1 << 15)
@@ -101,6 +107,13 @@
return 0;
}
+static int ksz8021_config_init(struct phy_device *phydev)
+{
+ const u16 val = KSZPHY_OMSO_B_CAST_OFF | KSZPHY_OMSO_RMII_OVERRIDE;
+ phy_write(phydev, MII_KSZPHY_OMSO, val);
+ return 0;
+}
+
static int ks8051_config_init(struct phy_device *phydev)
{
int regval;
@@ -128,9 +141,22 @@
.config_intr = ks8737_config_intr,
.driver = { .owner = THIS_MODULE,},
}, {
- .phy_id = PHY_ID_KS8041,
+ .phy_id = PHY_ID_KSZ8021,
+ .phy_id_mask = 0x00ffffff,
+ .name = "Micrel KSZ8021",
+ .features = (PHY_BASIC_FEATURES | SUPPORTED_Pause |
+ SUPPORTED_Asym_Pause),
+ .flags = PHY_HAS_MAGICANEG | PHY_HAS_INTERRUPT,
+ .config_init = ksz8021_config_init,
+ .config_aneg = genphy_config_aneg,
+ .read_status = genphy_read_status,
+ .ack_interrupt = kszphy_ack_interrupt,
+ .config_intr = kszphy_config_intr,
+ .driver = { .owner = THIS_MODULE,},
+}, {
+ .phy_id = PHY_ID_KSZ8041,
.phy_id_mask = 0x00fffff0,
- .name = "Micrel KS8041",
+ .name = "Micrel KSZ8041",
.features = (PHY_BASIC_FEATURES | SUPPORTED_Pause
| SUPPORTED_Asym_Pause),
.flags = PHY_HAS_MAGICANEG | PHY_HAS_INTERRUPT,
@@ -141,9 +167,9 @@
.config_intr = kszphy_config_intr,
.driver = { .owner = THIS_MODULE,},
}, {
- .phy_id = PHY_ID_KS8051,
+ .phy_id = PHY_ID_KSZ8051,
.phy_id_mask = 0x00fffff0,
- .name = "Micrel KS8051",
+ .name = "Micrel KSZ8051",
.features = (PHY_BASIC_FEATURES | SUPPORTED_Pause
| SUPPORTED_Asym_Pause),
.flags = PHY_HAS_MAGICANEG | PHY_HAS_INTERRUPT,
@@ -154,8 +180,8 @@
.config_intr = kszphy_config_intr,
.driver = { .owner = THIS_MODULE,},
}, {
- .phy_id = PHY_ID_KS8001,
- .name = "Micrel KS8001 or KS8721",
+ .phy_id = PHY_ID_KSZ8001,
+ .name = "Micrel KSZ8001 or KS8721",
.phy_id_mask = 0x00ffffff,
.features = (PHY_BASIC_FEATURES | SUPPORTED_Pause),
.flags = PHY_HAS_MAGICANEG | PHY_HAS_INTERRUPT,
@@ -201,10 +227,11 @@
static struct mdio_device_id __maybe_unused micrel_tbl[] = {
{ PHY_ID_KSZ9021, 0x000ffffe },
- { PHY_ID_KS8001, 0x00ffffff },
+ { PHY_ID_KSZ8001, 0x00ffffff },
{ PHY_ID_KS8737, 0x00fffff0 },
- { PHY_ID_KS8041, 0x00fffff0 },
- { PHY_ID_KS8051, 0x00fffff0 },
+ { PHY_ID_KSZ8021, 0x00ffffff },
+ { PHY_ID_KSZ8041, 0x00fffff0 },
+ { PHY_ID_KSZ8051, 0x00fffff0 },
{ }
};
diff --git a/drivers/net/phy/smsc.c b/drivers/net/phy/smsc.c
index 6d61923..88e3991 100644
--- a/drivers/net/phy/smsc.c
+++ b/drivers/net/phy/smsc.c
@@ -56,6 +56,32 @@
return smsc_phy_ack_interrupt (phydev);
}
+static int lan87xx_config_init(struct phy_device *phydev)
+{
+ /*
+ * Make sure the EDPWRDOWN bit is NOT set. Setting this bit on
+ * LAN8710/LAN8720 PHY causes the PHY to misbehave, likely due
+ * to a bug on the chip.
+ *
+ * When the system is powered on with the network cable being
+ * disconnected all the way until after ifconfig ethX up is
+ * issued for the LAN port with this PHY, connecting the cable
+ * afterwards does not cause LINK change detection, while the
+ * expected behavior is the Link UP being detected.
+ */
+ int rc = phy_read(phydev, MII_LAN83C185_CTRL_STATUS);
+ if (rc < 0)
+ return rc;
+
+ rc &= ~MII_LAN83C185_EDPWRDOWN;
+
+ rc = phy_write(phydev, MII_LAN83C185_CTRL_STATUS, rc);
+ if (rc < 0)
+ return rc;
+
+ return smsc_phy_ack_interrupt(phydev);
+}
+
static int lan911x_config_init(struct phy_device *phydev)
{
return smsc_phy_ack_interrupt(phydev);
@@ -162,7 +188,7 @@
/* basic functions */
.config_aneg = genphy_config_aneg,
.read_status = genphy_read_status,
- .config_init = smsc_phy_config_init,
+ .config_init = lan87xx_config_init,
/* IRQ related */
.ack_interrupt = smsc_phy_ack_interrupt,
diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c
index cbf7047..20f31d0 100644
--- a/drivers/net/ppp/pppoe.c
+++ b/drivers/net/ppp/pppoe.c
@@ -570,7 +570,7 @@
po = pppox_sk(sk);
- if (sk->sk_state & (PPPOX_CONNECTED | PPPOX_BOUND)) {
+ if (sk->sk_state & (PPPOX_CONNECTED | PPPOX_BOUND | PPPOX_ZOMBIE)) {
dev_put(po->pppoe_dev);
po->pppoe_dev = NULL;
}
diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c
index 341b65d..f8cd61f 100644
--- a/drivers/net/team/team.c
+++ b/drivers/net/team/team.c
@@ -848,7 +848,7 @@
}
#endif
-static void __team_port_change_check(struct team_port *port, bool linkup);
+static void __team_port_change_port_added(struct team_port *port, bool linkup);
static int team_port_add(struct team *team, struct net_device *port_dev)
{
@@ -948,7 +948,7 @@
team_port_enable(team, port);
list_add_tail_rcu(&port->list, &team->port_list);
__team_compute_features(team);
- __team_port_change_check(port, !!netif_carrier_ok(port_dev));
+ __team_port_change_port_added(port, !!netif_carrier_ok(port_dev));
__team_options_change_check(team);
netdev_info(dev, "Port device %s added\n", portname);
@@ -983,6 +983,8 @@
return err;
}
+static void __team_port_change_port_removed(struct team_port *port);
+
static int team_port_del(struct team *team, struct net_device *port_dev)
{
struct net_device *dev = team->dev;
@@ -999,8 +1001,7 @@
__team_option_inst_mark_removed_port(team, port);
__team_options_change_check(team);
__team_option_inst_del_port(team, port);
- port->removed = true;
- __team_port_change_check(port, false);
+ __team_port_change_port_removed(port);
team_port_disable(team, port);
list_del_rcu(&port->list);
netdev_rx_handler_unregister(port_dev);
@@ -1652,8 +1653,8 @@
hdr = genlmsg_put(msg, info->snd_pid, info->snd_seq,
&team_nl_family, 0, TEAM_CMD_NOOP);
- if (IS_ERR(hdr)) {
- err = PTR_ERR(hdr);
+ if (!hdr) {
+ err = -EMSGSIZE;
goto err_msg_put;
}
@@ -1847,8 +1848,8 @@
hdr = genlmsg_put(skb, pid, seq, &team_nl_family, flags | NLM_F_MULTI,
TEAM_CMD_OPTIONS_GET);
- if (IS_ERR(hdr))
- return PTR_ERR(hdr);
+ if (!hdr)
+ return -EMSGSIZE;
if (nla_put_u32(skb, TEAM_ATTR_TEAM_IFINDEX, team->dev->ifindex))
goto nla_put_failure;
@@ -2067,8 +2068,8 @@
hdr = genlmsg_put(skb, pid, seq, &team_nl_family, flags,
TEAM_CMD_PORT_LIST_GET);
- if (IS_ERR(hdr))
- return PTR_ERR(hdr);
+ if (!hdr)
+ return -EMSGSIZE;
if (nla_put_u32(skb, TEAM_ATTR_TEAM_IFINDEX, team->dev->ifindex))
goto nla_put_failure;
@@ -2251,13 +2252,11 @@
}
/* rtnl lock is held */
-static void __team_port_change_check(struct team_port *port, bool linkup)
+
+static void __team_port_change_send(struct team_port *port, bool linkup)
{
int err;
- if (!port->removed && port->state.linkup == linkup)
- return;
-
port->changed = true;
port->state.linkup = linkup;
team_refresh_port_linkup(port);
@@ -2282,6 +2281,23 @@
}
+static void __team_port_change_check(struct team_port *port, bool linkup)
+{
+ if (port->state.linkup != linkup)
+ __team_port_change_send(port, linkup);
+}
+
+static void __team_port_change_port_added(struct team_port *port, bool linkup)
+{
+ __team_port_change_send(port, linkup);
+}
+
+static void __team_port_change_port_removed(struct team_port *port)
+{
+ port->removed = true;
+ __team_port_change_send(port, false);
+}
+
static void team_port_change_check(struct team_port *port, bool linkup)
{
struct team *team = port->team;
diff --git a/drivers/net/usb/asix_devices.c b/drivers/net/usb/asix_devices.c
index 4fd48df..32e31c5 100644
--- a/drivers/net/usb/asix_devices.c
+++ b/drivers/net/usb/asix_devices.c
@@ -962,6 +962,10 @@
USB_DEVICE (0x2001, 0x3c05),
.driver_info = (unsigned long) &ax88772_info,
}, {
+ // DLink DUB-E100 H/W Ver C1
+ USB_DEVICE (0x2001, 0x1a02),
+ .driver_info = (unsigned long) &ax88772_info,
+}, {
// Linksys USB1000
USB_DEVICE (0x1737, 0x0039),
.driver_info = (unsigned long) &ax88178_info,
diff --git a/drivers/net/usb/qmi_wwan.c b/drivers/net/usb/qmi_wwan.c
index 328397c..3543c9e 100644
--- a/drivers/net/usb/qmi_wwan.c
+++ b/drivers/net/usb/qmi_wwan.c
@@ -297,7 +297,7 @@
if (ret < 0)
goto err;
- if (info->subdriver && info->subdriver->suspend)
+ if (intf == info->control && info->subdriver && info->subdriver->suspend)
ret = info->subdriver->suspend(intf, message);
if (ret < 0)
usbnet_resume(intf);
@@ -310,13 +310,14 @@
struct usbnet *dev = usb_get_intfdata(intf);
struct qmi_wwan_state *info = (void *)&dev->data;
int ret = 0;
+ bool callsub = (intf == info->control && info->subdriver && info->subdriver->resume);
- if (info->subdriver && info->subdriver->resume)
+ if (callsub)
ret = info->subdriver->resume(intf);
if (ret < 0)
goto err;
ret = usbnet_resume(intf);
- if (ret < 0 && info->subdriver && info->subdriver->resume && info->subdriver->suspend)
+ if (ret < 0 && callsub && info->subdriver->suspend)
info->subdriver->suspend(intf, PMSG_SUSPEND);
err:
return ret;
@@ -365,16 +366,20 @@
},
/* 2. Combined interface devices matching on class+protocol */
+ { /* Huawei E367 and possibly others in "Windows mode" */
+ USB_VENDOR_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, USB_CLASS_VENDOR_SPEC, 1, 7),
+ .driver_info = (unsigned long)&qmi_wwan_info,
+ },
{ /* Huawei E392, E398 and possibly others in "Windows mode" */
USB_VENDOR_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, USB_CLASS_VENDOR_SPEC, 1, 17),
.driver_info = (unsigned long)&qmi_wwan_shared,
},
- { /* Pantech UML290 */
- USB_DEVICE_AND_INTERFACE_INFO(0x106c, 0x3718, USB_CLASS_VENDOR_SPEC, 0xf0, 0xff),
+ { /* Pantech UML290, P4200 and more */
+ USB_VENDOR_AND_INTERFACE_INFO(0x106c, USB_CLASS_VENDOR_SPEC, 0xf0, 0xff),
.driver_info = (unsigned long)&qmi_wwan_shared,
},
{ /* Pantech UML290 - newer firmware */
- USB_DEVICE_AND_INTERFACE_INFO(0x106c, 0x3718, USB_CLASS_VENDOR_SPEC, 0xf1, 0xff),
+ USB_VENDOR_AND_INTERFACE_INFO(0x106c, USB_CLASS_VENDOR_SPEC, 0xf1, 0xff),
.driver_info = (unsigned long)&qmi_wwan_shared,
},
@@ -382,6 +387,7 @@
{QMI_FIXED_INTF(0x19d2, 0x0055, 1)}, /* ZTE (Vodafone) K3520-Z */
{QMI_FIXED_INTF(0x19d2, 0x0063, 4)}, /* ZTE (Vodafone) K3565-Z */
{QMI_FIXED_INTF(0x19d2, 0x0104, 4)}, /* ZTE (Vodafone) K4505-Z */
+ {QMI_FIXED_INTF(0x19d2, 0x0157, 5)}, /* ZTE MF683 */
{QMI_FIXED_INTF(0x19d2, 0x0167, 4)}, /* ZTE MF820D */
{QMI_FIXED_INTF(0x19d2, 0x0326, 4)}, /* ZTE MF821D */
{QMI_FIXED_INTF(0x19d2, 0x1008, 4)}, /* ZTE (Vodafone) K3570-Z */
@@ -398,7 +404,6 @@
/* 4. Gobi 1000 devices */
{QMI_GOBI1K_DEVICE(0x05c6, 0x9212)}, /* Acer Gobi Modem Device */
{QMI_GOBI1K_DEVICE(0x03f0, 0x1f1d)}, /* HP un2400 Gobi Modem Device */
- {QMI_GOBI1K_DEVICE(0x03f0, 0x371d)}, /* HP un2430 Mobile Broadband Module */
{QMI_GOBI1K_DEVICE(0x04da, 0x250d)}, /* Panasonic Gobi Modem device */
{QMI_GOBI1K_DEVICE(0x413c, 0x8172)}, /* Dell Gobi Modem device */
{QMI_GOBI1K_DEVICE(0x1410, 0xa001)}, /* Novatel Gobi Modem device */
@@ -413,7 +418,9 @@
/* 5. Gobi 2000 and 3000 devices */
{QMI_GOBI_DEVICE(0x413c, 0x8186)}, /* Dell Gobi 2000 Modem device (N0218, VU936) */
+ {QMI_GOBI_DEVICE(0x413c, 0x8194)}, /* Dell Gobi 3000 Composite */
{QMI_GOBI_DEVICE(0x05c6, 0x920b)}, /* Generic Gobi 2000 Modem device */
+ {QMI_GOBI_DEVICE(0x05c6, 0x920d)}, /* Gobi 3000 Composite */
{QMI_GOBI_DEVICE(0x05c6, 0x9225)}, /* Sony Gobi 2000 Modem device (N0279, VU730) */
{QMI_GOBI_DEVICE(0x05c6, 0x9245)}, /* Samsung Gobi 2000 Modem device (VL176) */
{QMI_GOBI_DEVICE(0x03f0, 0x251d)}, /* HP Gobi 2000 Modem device (VP412) */
@@ -438,9 +445,12 @@
{QMI_GOBI_DEVICE(0x16d8, 0x8002)}, /* CMDTech Gobi 2000 Modem device (VU922) */
{QMI_GOBI_DEVICE(0x05c6, 0x9205)}, /* Gobi 2000 Modem device */
{QMI_GOBI_DEVICE(0x1199, 0x9013)}, /* Sierra Wireless Gobi 3000 Modem device (MC8355) */
+ {QMI_GOBI_DEVICE(0x03f0, 0x371d)}, /* HP un2430 Mobile Broadband Module */
{QMI_GOBI_DEVICE(0x1199, 0x9015)}, /* Sierra Wireless Gobi 3000 Modem device */
{QMI_GOBI_DEVICE(0x1199, 0x9019)}, /* Sierra Wireless Gobi 3000 Modem device */
{QMI_GOBI_DEVICE(0x1199, 0x901b)}, /* Sierra Wireless MC7770 */
+ {QMI_GOBI_DEVICE(0x12d1, 0x14f1)}, /* Sony Gobi 3000 Composite */
+ {QMI_GOBI_DEVICE(0x1410, 0xa021)}, /* Foxconn Gobi 3000 Modem device (Novatel E396) */
{ } /* END */
};
diff --git a/drivers/net/usb/sierra_net.c b/drivers/net/usb/sierra_net.c
index 7be49ea..8e22417 100644
--- a/drivers/net/usb/sierra_net.c
+++ b/drivers/net/usb/sierra_net.c
@@ -656,7 +656,7 @@
return -EIO;
}
- *datap = *attrdata;
+ *datap = le16_to_cpu(*attrdata);
kfree(attrdata);
return result;
diff --git a/drivers/net/usb/smsc75xx.c b/drivers/net/usb/smsc75xx.c
index f5ab6e6..376143e 100644
--- a/drivers/net/usb/smsc75xx.c
+++ b/drivers/net/usb/smsc75xx.c
@@ -1253,6 +1253,7 @@
.probe = usbnet_probe,
.suspend = usbnet_suspend,
.resume = usbnet_resume,
+ .reset_resume = usbnet_resume,
.disconnect = usbnet_disconnect,
.disable_hub_initiated_lpm = 1,
};
diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c
index 8531c1c..fc9f578 100644
--- a/drivers/net/usb/usbnet.c
+++ b/drivers/net/usb/usbnet.c
@@ -1201,19 +1201,26 @@
}
EXPORT_SYMBOL_GPL(usbnet_start_xmit);
-static void rx_alloc_submit(struct usbnet *dev, gfp_t flags)
+static int rx_alloc_submit(struct usbnet *dev, gfp_t flags)
{
struct urb *urb;
int i;
+ int ret = 0;
/* don't refill the queue all at once */
for (i = 0; i < 10 && dev->rxq.qlen < RX_QLEN(dev); i++) {
urb = usb_alloc_urb(0, flags);
if (urb != NULL) {
- if (rx_submit(dev, urb, flags) == -ENOLINK)
- return;
+ ret = rx_submit(dev, urb, flags);
+ if (ret)
+ goto err;
+ } else {
+ ret = -ENOMEM;
+ goto err;
}
}
+err:
+ return ret;
}
/*-------------------------------------------------------------------------*/
@@ -1257,7 +1264,8 @@
int temp = dev->rxq.qlen;
if (temp < RX_QLEN(dev)) {
- rx_alloc_submit(dev, GFP_ATOMIC);
+ if (rx_alloc_submit(dev, GFP_ATOMIC) == -ENOLINK)
+ return;
if (temp != dev->rxq.qlen)
netif_dbg(dev, link, dev->net,
"rxqlen %d --> %d\n",
@@ -1573,7 +1581,7 @@
netif_device_present(dev->net) &&
!timer_pending(&dev->delay) &&
!test_bit(EVENT_RX_HALT, &dev->flags))
- rx_alloc_submit(dev, GFP_KERNEL);
+ rx_alloc_submit(dev, GFP_NOIO);
if (!(dev->txq.qlen >= TX_QLEN(dev)))
netif_tx_wake_all_queues(dev->net);
diff --git a/drivers/net/wan/ixp4xx_hss.c b/drivers/net/wan/ixp4xx_hss.c
index aaaca9a..3f575af 100644
--- a/drivers/net/wan/ixp4xx_hss.c
+++ b/drivers/net/wan/ixp4xx_hss.c
@@ -10,6 +10,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/module.h>
#include <linux/bitops.h>
#include <linux/cdev.h>
#include <linux/dma-mapping.h>
diff --git a/drivers/net/wireless/ath/ath5k/eeprom.c b/drivers/net/wireless/ath/ath5k/eeprom.c
index 4026c90..b7e0258 100644
--- a/drivers/net/wireless/ath/ath5k/eeprom.c
+++ b/drivers/net/wireless/ath/ath5k/eeprom.c
@@ -1482,7 +1482,7 @@
case AR5K_EEPROM_MODE_11A:
offset += AR5K_EEPROM_TARGET_PWR_OFF_11A(ee->ee_version);
rate_pcal_info = ee->ee_rate_tpwr_a;
- ee->ee_rate_target_pwr_num[mode] = AR5K_EEPROM_N_5GHZ_CHAN;
+ ee->ee_rate_target_pwr_num[mode] = AR5K_EEPROM_N_5GHZ_RATE_CHAN;
break;
case AR5K_EEPROM_MODE_11B:
offset += AR5K_EEPROM_TARGET_PWR_OFF_11B(ee->ee_version);
diff --git a/drivers/net/wireless/ath/ath5k/eeprom.h b/drivers/net/wireless/ath/ath5k/eeprom.h
index dc2bcfe..94a9bbe 100644
--- a/drivers/net/wireless/ath/ath5k/eeprom.h
+++ b/drivers/net/wireless/ath/ath5k/eeprom.h
@@ -182,6 +182,7 @@
#define AR5K_EEPROM_EEP_DELTA 10
#define AR5K_EEPROM_N_MODES 3
#define AR5K_EEPROM_N_5GHZ_CHAN 10
+#define AR5K_EEPROM_N_5GHZ_RATE_CHAN 8
#define AR5K_EEPROM_N_2GHZ_CHAN 3
#define AR5K_EEPROM_N_2GHZ_CHAN_2413 4
#define AR5K_EEPROM_N_2GHZ_CHAN_MAX 4
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c b/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c
index 2588848..d066f25 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c
+++ b/drivers/net/wireless/ath/ath9k/ar9003_eeprom.c
@@ -2982,6 +2982,10 @@
case EEP_RX_MASK:
return pBase->txrxMask & 0xf;
case EEP_PAPRD:
+ if (AR_SREV_9462(ah))
+ return false;
+ if (!ah->config.enable_paprd);
+ return false;
return !!(pBase->featureEnable & BIT(5));
case EEP_CHAIN_MASK_REDUCE:
return (pBase->miscConfiguration >> 0x3) & 0x1;
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_paprd.c b/drivers/net/wireless/ath/ath9k/ar9003_paprd.c
index 2c9f7d7..0ed3846 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_paprd.c
+++ b/drivers/net/wireless/ath/ath9k/ar9003_paprd.c
@@ -142,6 +142,7 @@
};
int training_power;
int i, val;
+ u32 am2pm_mask = ah->paprd_ratemask;
if (IS_CHAN_2GHZ(ah->curchan))
training_power = ar9003_get_training_power_2g(ah);
@@ -158,10 +159,13 @@
}
ah->paprd_training_power = training_power;
+ if (AR_SREV_9330(ah))
+ am2pm_mask = 0;
+
REG_RMW_FIELD(ah, AR_PHY_PAPRD_AM2AM, AR_PHY_PAPRD_AM2AM_MASK,
ah->paprd_ratemask);
REG_RMW_FIELD(ah, AR_PHY_PAPRD_AM2PM, AR_PHY_PAPRD_AM2PM_MASK,
- ah->paprd_ratemask);
+ am2pm_mask);
REG_RMW_FIELD(ah, AR_PHY_PAPRD_HT40, AR_PHY_PAPRD_HT40_MASK,
ah->paprd_ratemask_ht40);
@@ -782,6 +786,102 @@
}
EXPORT_SYMBOL(ar9003_paprd_setup_gain_table);
+static bool ar9003_paprd_retrain_pa_in(struct ath_hw *ah,
+ struct ath9k_hw_cal_data *caldata,
+ int chain)
+{
+ u32 *pa_in = caldata->pa_table[chain];
+ int capdiv_offset, quick_drop_offset;
+ int capdiv2g, quick_drop;
+ int count = 0;
+ int i;
+
+ if (!AR_SREV_9485(ah) && !AR_SREV_9330(ah))
+ return false;
+
+ capdiv2g = REG_READ_FIELD(ah, AR_PHY_65NM_CH0_TXRF3,
+ AR_PHY_65NM_CH0_TXRF3_CAPDIV2G);
+
+ quick_drop = REG_READ_FIELD(ah, AR_PHY_PAPRD_TRAINER_CNTL3,
+ AR_PHY_PAPRD_TRAINER_CNTL3_CF_PAPRD_QUICK_DROP);
+
+ if (quick_drop)
+ quick_drop -= 0x40;
+
+ for (i = 0; i < NUM_BIN + 1; i++) {
+ if (pa_in[i] == 1400)
+ count++;
+ }
+
+ if (AR_SREV_9485(ah)) {
+ if (pa_in[23] < 800) {
+ capdiv_offset = (int)((1000 - pa_in[23] + 75) / 150);
+ capdiv2g += capdiv_offset;
+ if (capdiv2g > 7) {
+ capdiv2g = 7;
+ if (pa_in[23] < 600) {
+ quick_drop++;
+ if (quick_drop > 0)
+ quick_drop = 0;
+ }
+ }
+ } else if (pa_in[23] == 1400) {
+ quick_drop_offset = min_t(int, count / 3, 2);
+ quick_drop += quick_drop_offset;
+ capdiv2g += quick_drop_offset / 2;
+
+ if (capdiv2g > 7)
+ capdiv2g = 7;
+
+ if (quick_drop > 0) {
+ quick_drop = 0;
+ capdiv2g -= quick_drop_offset;
+ if (capdiv2g < 0)
+ capdiv2g = 0;
+ }
+ } else {
+ return false;
+ }
+ } else if (AR_SREV_9330(ah)) {
+ if (pa_in[23] < 1000) {
+ capdiv_offset = (1000 - pa_in[23]) / 100;
+ capdiv2g += capdiv_offset;
+ if (capdiv_offset > 3) {
+ capdiv_offset = 1;
+ quick_drop--;
+ }
+
+ capdiv2g += capdiv_offset;
+ if (capdiv2g > 6)
+ capdiv2g = 6;
+ if (quick_drop < -4)
+ quick_drop = -4;
+ } else if (pa_in[23] == 1400) {
+ if (count > 3) {
+ quick_drop++;
+ capdiv2g -= count / 4;
+ if (quick_drop > -2)
+ quick_drop = -2;
+ } else {
+ capdiv2g--;
+ }
+
+ if (capdiv2g < 0)
+ capdiv2g = 0;
+ } else {
+ return false;
+ }
+ }
+
+ REG_RMW_FIELD(ah, AR_PHY_65NM_CH0_TXRF3,
+ AR_PHY_65NM_CH0_TXRF3_CAPDIV2G, capdiv2g);
+ REG_RMW_FIELD(ah, AR_PHY_PAPRD_TRAINER_CNTL3,
+ AR_PHY_PAPRD_TRAINER_CNTL3_CF_PAPRD_QUICK_DROP,
+ quick_drop);
+
+ return true;
+}
+
int ar9003_paprd_create_curve(struct ath_hw *ah,
struct ath9k_hw_cal_data *caldata, int chain)
{
@@ -817,6 +917,9 @@
if (!create_pa_curve(data_L, data_U, pa_table, small_signal_gain))
status = -2;
+ if (ar9003_paprd_retrain_pa_in(ah, caldata, chain))
+ status = -EINPROGRESS;
+
REG_CLR_BIT(ah, AR_PHY_PAPRD_TRAINER_STAT1,
AR_PHY_PAPRD_TRAINER_STAT1_PAPRD_TRAIN_DONE);
diff --git a/drivers/net/wireless/ath/ath9k/ar9003_phy.h b/drivers/net/wireless/ath/ath9k/ar9003_phy.h
index 7bfbaf0..84d3d49 100644
--- a/drivers/net/wireless/ath/ath9k/ar9003_phy.h
+++ b/drivers/net/wireless/ath/ath9k/ar9003_phy.h
@@ -625,6 +625,10 @@
#define AR_PHY_AIC_CTRL_4_B0 (AR_SM_BASE + 0x4c0)
#define AR_PHY_AIC_STAT_2_B0 (AR_SM_BASE + 0x4cc)
+#define AR_PHY_65NM_CH0_TXRF3 0x16048
+#define AR_PHY_65NM_CH0_TXRF3_CAPDIV2G 0x0000001e
+#define AR_PHY_65NM_CH0_TXRF3_CAPDIV2G_S 1
+
#define AR_PHY_65NM_CH0_SYNTH4 0x1608c
#define AR_PHY_SYNTH4_LONG_SHIFT_SELECT (AR_SREV_9462(ah) ? 0x00000001 : 0x00000002)
#define AR_PHY_SYNTH4_LONG_SHIFT_SELECT_S (AR_SREV_9462(ah) ? 0 : 1)
diff --git a/drivers/net/wireless/ath/ath9k/debug.c b/drivers/net/wireless/ath/ath9k/debug.c
index 68b643c..c8ef301 100644
--- a/drivers/net/wireless/ath/ath9k/debug.c
+++ b/drivers/net/wireless/ath/ath9k/debug.c
@@ -1577,6 +1577,8 @@
sc->debug.debugfs_phy, sc, &fops_tx_chainmask);
debugfs_create_file("disable_ani", S_IRUSR | S_IWUSR,
sc->debug.debugfs_phy, sc, &fops_disable_ani);
+ debugfs_create_bool("paprd", S_IRUSR | S_IWUSR, sc->debug.debugfs_phy,
+ &sc->sc_ah->config.enable_paprd);
debugfs_create_file("regidx", S_IRUSR | S_IWUSR, sc->debug.debugfs_phy,
sc, &fops_regidx);
debugfs_create_file("regval", S_IRUSR | S_IWUSR, sc->debug.debugfs_phy,
diff --git a/drivers/net/wireless/ath/ath9k/gpio.c b/drivers/net/wireless/ath/ath9k/gpio.c
index bacdb8f..9f83f71 100644
--- a/drivers/net/wireless/ath/ath9k/gpio.c
+++ b/drivers/net/wireless/ath/ath9k/gpio.c
@@ -341,7 +341,8 @@
{
struct ath_btcoex *btcoex = &sc->btcoex;
- ath9k_gen_timer_stop(sc->sc_ah, btcoex->no_stomp_timer);
+ if (btcoex->hw_timer_enabled)
+ ath9k_gen_timer_stop(sc->sc_ah, btcoex->no_stomp_timer);
}
u16 ath9k_btcoex_aggr_limit(struct ath_softc *sc, u32 max_4ms_framelen)
diff --git a/drivers/net/wireless/ath/ath9k/hw.c b/drivers/net/wireless/ath/ath9k/hw.c
index 60b6a9d..4faf0a3 100644
--- a/drivers/net/wireless/ath/ath9k/hw.c
+++ b/drivers/net/wireless/ath/ath9k/hw.c
@@ -463,9 +463,6 @@
ah->config.spurchans[i][1] = AR_NO_SPUR;
}
- /* PAPRD needs some more work to be enabled */
- ah->config.paprd_disable = 1;
-
ah->config.rx_intr_mitigation = true;
ah->config.pcieSerDesWrite = true;
@@ -978,9 +975,6 @@
else
imr_reg |= AR_IMR_TXOK;
- if (opmode == NL80211_IFTYPE_AP)
- imr_reg |= AR_IMR_MIB;
-
ENABLE_REGWRITE_BUFFER(ah);
REG_WRITE(ah, AR_IMR, imr_reg);
@@ -1778,6 +1772,8 @@
/* Operating channel changed, reset channel calibration data */
memset(caldata, 0, sizeof(*caldata));
ath9k_init_nfcal_hist_buffer(ah, chan);
+ } else if (caldata) {
+ caldata->paprd_packet_sent = false;
}
ah->noise = ath9k_hw_getchan_noise(ah, chan);
@@ -2501,9 +2497,6 @@
pCap->rx_status_len = sizeof(struct ar9003_rxs);
pCap->tx_desc_len = sizeof(struct ar9003_txc);
pCap->txs_len = sizeof(struct ar9003_txs);
- if (!ah->config.paprd_disable &&
- ah->eep_ops->get_eeprom(ah, EEP_PAPRD))
- pCap->hw_caps |= ATH9K_HW_CAP_PAPRD;
} else {
pCap->tx_desc_len = sizeof(struct ath_desc);
if (AR_SREV_9280_20(ah))
diff --git a/drivers/net/wireless/ath/ath9k/hw.h b/drivers/net/wireless/ath/ath9k/hw.h
index ce7332c..de6968f 100644
--- a/drivers/net/wireless/ath/ath9k/hw.h
+++ b/drivers/net/wireless/ath/ath9k/hw.h
@@ -236,7 +236,6 @@
ATH9K_HW_CAP_LDPC = BIT(6),
ATH9K_HW_CAP_FASTCLOCK = BIT(7),
ATH9K_HW_CAP_SGI_20 = BIT(8),
- ATH9K_HW_CAP_PAPRD = BIT(9),
ATH9K_HW_CAP_ANT_DIV_COMB = BIT(10),
ATH9K_HW_CAP_2GHZ = BIT(11),
ATH9K_HW_CAP_5GHZ = BIT(12),
@@ -287,12 +286,12 @@
u8 pcie_clock_req;
u32 pcie_waen;
u8 analog_shiftreg;
- u8 paprd_disable;
u32 ofdm_trig_low;
u32 ofdm_trig_high;
u32 cck_trig_high;
u32 cck_trig_low;
u32 enable_ani;
+ u32 enable_paprd;
int serialize_regmode;
bool rx_intr_mitigation;
bool tx_intr_mitigation;
@@ -405,6 +404,7 @@
int8_t iCoff;
int8_t qCoff;
bool rtt_done;
+ bool paprd_packet_sent;
bool paprd_done;
bool nfcal_pending;
bool nfcal_interference;
diff --git a/drivers/net/wireless/ath/ath9k/link.c b/drivers/net/wireless/ath/ath9k/link.c
index d4549e9..7b88b9c 100644
--- a/drivers/net/wireless/ath/ath9k/link.c
+++ b/drivers/net/wireless/ath/ath9k/link.c
@@ -254,8 +254,9 @@
int chain_ok = 0;
int chain;
int len = 1800;
+ int ret;
- if (!caldata)
+ if (!caldata || !caldata->paprd_packet_sent || caldata->paprd_done)
return;
ath9k_ps_wakeup(sc);
@@ -282,13 +283,6 @@
continue;
chain_ok = 0;
-
- ath_dbg(common, CALIBRATE,
- "Sending PAPRD frame for thermal measurement on chain %d\n",
- chain);
- if (!ath_paprd_send_frame(sc, skb, chain))
- goto fail_paprd;
-
ar9003_paprd_setup_gain_table(ah, chain);
ath_dbg(common, CALIBRATE,
@@ -302,7 +296,13 @@
break;
}
- if (ar9003_paprd_create_curve(ah, caldata, chain)) {
+ ret = ar9003_paprd_create_curve(ah, caldata, chain);
+ if (ret == -EINPROGRESS) {
+ ath_dbg(common, CALIBRATE,
+ "PAPRD curve on chain %d needs to be re-trained\n",
+ chain);
+ break;
+ } else if (ret) {
ath_dbg(common, CALIBRATE,
"PAPRD create curve failed on chain %d\n",
chain);
@@ -423,7 +423,7 @@
cal_interval = min(cal_interval, (u32)short_cal_interval);
mod_timer(&common->ani.timer, jiffies + msecs_to_jiffies(cal_interval));
- if ((sc->sc_ah->caps.hw_caps & ATH9K_HW_CAP_PAPRD) && ah->caldata) {
+ if (ah->eep_ops->get_eeprom(ah, EEP_PAPRD) && ah->caldata) {
if (!ah->caldata->paprd_done)
ieee80211_queue_work(sc->hw, &sc->paprd_work);
else if (!ah->paprd_table_write_done)
diff --git a/drivers/net/wireless/ath/ath9k/pci.c b/drivers/net/wireless/ath/ath9k/pci.c
index a978984..ef11dc6 100644
--- a/drivers/net/wireless/ath/ath9k/pci.c
+++ b/drivers/net/wireless/ath/ath9k/pci.c
@@ -113,41 +113,32 @@
struct ath_hw *ah = sc->sc_ah;
struct pci_dev *pdev = to_pci_dev(sc->dev);
struct pci_dev *parent;
- int pos;
- u8 aspm;
+ u16 aspm;
if (!ah->is_pciexpress)
return;
- pos = pci_pcie_cap(pdev);
- if (!pos)
- return;
-
parent = pdev->bus->self;
if (!parent)
return;
if (ath9k_hw_get_btcoex_scheme(ah) != ATH_BTCOEX_CFG_NONE) {
/* Bluetooth coexistance requires disabling ASPM. */
- pci_read_config_byte(pdev, pos + PCI_EXP_LNKCTL, &aspm);
- aspm &= ~(PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1);
- pci_write_config_byte(pdev, pos + PCI_EXP_LNKCTL, aspm);
+ pcie_capability_clear_word(pdev, PCI_EXP_LNKCTL,
+ PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1);
/*
* Both upstream and downstream PCIe components should
* have the same ASPM settings.
*/
- pos = pci_pcie_cap(parent);
- pci_read_config_byte(parent, pos + PCI_EXP_LNKCTL, &aspm);
- aspm &= ~(PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1);
- pci_write_config_byte(parent, pos + PCI_EXP_LNKCTL, aspm);
+ pcie_capability_clear_word(parent, PCI_EXP_LNKCTL,
+ PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1);
ath_info(common, "Disabling ASPM since BTCOEX is enabled\n");
return;
}
- pos = pci_pcie_cap(parent);
- pci_read_config_byte(parent, pos + PCI_EXP_LNKCTL, &aspm);
+ pcie_capability_read_word(parent, PCI_EXP_LNKCTL, &aspm);
if (aspm & (PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1)) {
ah->aspm_enabled = true;
/* Initialize PCIe PM and SERDES registers. */
diff --git a/drivers/net/wireless/ath/ath9k/xmit.c b/drivers/net/wireless/ath/ath9k/xmit.c
index 2c9da6b..0d4155a 100644
--- a/drivers/net/wireless/ath/ath9k/xmit.c
+++ b/drivers/net/wireless/ath/ath9k/xmit.c
@@ -2018,6 +2018,9 @@
ath_dbg(common, XMIT, "TX complete: skb: %p\n", skb);
+ if (sc->sc_ah->caldata)
+ sc->sc_ah->caldata->paprd_packet_sent = true;
+
if (!(tx_flags & ATH_TX_ERROR))
/* Frame was ACKed */
tx_info->flags |= IEEE80211_TX_STAT_ACK;
diff --git a/drivers/net/wireless/b43/Kconfig b/drivers/net/wireless/b43/Kconfig
index 3876c7e..7a28d21 100644
--- a/drivers/net/wireless/b43/Kconfig
+++ b/drivers/net/wireless/b43/Kconfig
@@ -34,8 +34,8 @@
config B43_BCMA_EXTRA
bool "Hardware support that overlaps with the brcmsmac driver"
depends on B43_BCMA
- default n if BRCMSMAC || BRCMSMAC_MODULE
- default y
+ default n if BRCMSMAC
+ default y
config B43_SSB
bool
diff --git a/drivers/net/wireless/brcm80211/brcmfmac/bcmsdh_sdmmc.c b/drivers/net/wireless/brcm80211/brcmfmac/bcmsdh_sdmmc.c
index 49765d3..7c4ee72 100644
--- a/drivers/net/wireless/brcm80211/brcmfmac/bcmsdh_sdmmc.c
+++ b/drivers/net/wireless/brcm80211/brcmfmac/bcmsdh_sdmmc.c
@@ -638,6 +638,8 @@
oobirq_entry = kzalloc(sizeof(struct brcmf_sdio_oobirq),
GFP_KERNEL);
+ if (!oobirq_entry)
+ return -ENOMEM;
oobirq_entry->irq = res->start;
oobirq_entry->flags = res->flags & IRQF_TRIGGER_MASK;
list_add_tail(&oobirq_entry->list, &oobirq_lh);
diff --git a/drivers/net/wireless/brcm80211/brcmfmac/dhd_common.c b/drivers/net/wireless/brcm80211/brcmfmac/dhd_common.c
index 2621dd3..6f70953 100644
--- a/drivers/net/wireless/brcm80211/brcmfmac/dhd_common.c
+++ b/drivers/net/wireless/brcm80211/brcmfmac/dhd_common.c
@@ -764,8 +764,11 @@
{
char iovbuf[32];
int retcode;
+ __le32 arp_mode_le;
- brcmf_c_mkiovar("arp_ol", (char *)&arp_mode, 4, iovbuf, sizeof(iovbuf));
+ arp_mode_le = cpu_to_le32(arp_mode);
+ brcmf_c_mkiovar("arp_ol", (char *)&arp_mode_le, 4, iovbuf,
+ sizeof(iovbuf));
retcode = brcmf_proto_cdc_set_dcmd(drvr, 0, BRCMF_C_SET_VAR,
iovbuf, sizeof(iovbuf));
retcode = retcode >= 0 ? 0 : retcode;
@@ -781,8 +784,11 @@
{
char iovbuf[32];
int retcode;
+ __le32 arp_enable_le;
- brcmf_c_mkiovar("arpoe", (char *)&arp_enable, 4,
+ arp_enable_le = cpu_to_le32(arp_enable);
+
+ brcmf_c_mkiovar("arpoe", (char *)&arp_enable_le, 4,
iovbuf, sizeof(iovbuf));
retcode = brcmf_proto_cdc_set_dcmd(drvr, 0, BRCMF_C_SET_VAR,
iovbuf, sizeof(iovbuf));
@@ -800,10 +806,10 @@
char iovbuf[BRCMF_EVENTING_MASK_LEN + 12]; /* Room for
"event_msgs" + '\0' + bitvec */
char buf[128], *ptr;
- u32 roaming = 1;
- uint bcn_timeout = 3;
- int scan_assoc_time = 40;
- int scan_unassoc_time = 40;
+ __le32 roaming_le = cpu_to_le32(1);
+ __le32 bcn_timeout_le = cpu_to_le32(3);
+ __le32 scan_assoc_time_le = cpu_to_le32(40);
+ __le32 scan_unassoc_time_le = cpu_to_le32(40);
int i;
struct brcmf_bus_dcmd *cmdlst;
struct list_head *cur, *q;
@@ -829,14 +835,14 @@
/* Setup timeout if Beacons are lost and roam is off to report
link down */
- brcmf_c_mkiovar("bcn_timeout", (char *)&bcn_timeout, 4, iovbuf,
+ brcmf_c_mkiovar("bcn_timeout", (char *)&bcn_timeout_le, 4, iovbuf,
sizeof(iovbuf));
brcmf_proto_cdc_set_dcmd(drvr, 0, BRCMF_C_SET_VAR, iovbuf,
sizeof(iovbuf));
/* Enable/Disable build-in roaming to allowed ext supplicant to take
of romaing */
- brcmf_c_mkiovar("roam_off", (char *)&roaming, 4,
+ brcmf_c_mkiovar("roam_off", (char *)&roaming_le, 4,
iovbuf, sizeof(iovbuf));
brcmf_proto_cdc_set_dcmd(drvr, 0, BRCMF_C_SET_VAR, iovbuf,
sizeof(iovbuf));
@@ -848,9 +854,9 @@
sizeof(iovbuf));
brcmf_proto_cdc_set_dcmd(drvr, 0, BRCMF_C_SET_SCAN_CHANNEL_TIME,
- (char *)&scan_assoc_time, sizeof(scan_assoc_time));
+ (char *)&scan_assoc_time_le, sizeof(scan_assoc_time_le));
brcmf_proto_cdc_set_dcmd(drvr, 0, BRCMF_C_SET_SCAN_UNASSOC_TIME,
- (char *)&scan_unassoc_time, sizeof(scan_unassoc_time));
+ (char *)&scan_unassoc_time_le, sizeof(scan_unassoc_time_le));
/* Set and enable ARP offload feature */
brcmf_c_arp_offload_set(drvr, BRCMF_ARPOL_MODE);
diff --git a/drivers/net/wireless/brcm80211/brcmfmac/usb.c b/drivers/net/wireless/brcm80211/brcmfmac/usb.c
index a299d42..58f89fa 100644
--- a/drivers/net/wireless/brcm80211/brcmfmac/usb.c
+++ b/drivers/net/wireless/brcm80211/brcmfmac/usb.c
@@ -519,7 +519,7 @@
else
devinfo->bus_pub.bus->dstats.tx_errors++;
- dev_kfree_skb(req->skb);
+ brcmu_pkt_buf_free_skb(req->skb);
req->skb = NULL;
brcmf_usb_enq(devinfo, &devinfo->tx_freeq, req);
@@ -540,7 +540,7 @@
devinfo->bus_pub.bus->dstats.rx_packets++;
} else {
devinfo->bus_pub.bus->dstats.rx_errors++;
- dev_kfree_skb(skb);
+ brcmu_pkt_buf_free_skb(skb);
brcmf_usb_enq(devinfo, &devinfo->rx_freeq, req);
return;
}
@@ -550,13 +550,15 @@
if (brcmf_proto_hdrpull(devinfo->dev, &ifidx, skb) != 0) {
brcmf_dbg(ERROR, "rx protocol error\n");
brcmu_pkt_buf_free_skb(skb);
+ brcmf_usb_enq(devinfo, &devinfo->rx_freeq, req);
devinfo->bus_pub.bus->dstats.rx_errors++;
} else {
brcmf_rx_packet(devinfo->dev, ifidx, skb);
brcmf_usb_rx_refill(devinfo, req);
}
} else {
- dev_kfree_skb(skb);
+ brcmu_pkt_buf_free_skb(skb);
+ brcmf_usb_enq(devinfo, &devinfo->rx_freeq, req);
}
return;
@@ -581,14 +583,13 @@
usb_fill_bulk_urb(req->urb, devinfo->usbdev, devinfo->rx_pipe,
skb->data, skb_tailroom(skb), brcmf_usb_rx_complete,
req);
- req->urb->transfer_flags |= URB_ZERO_PACKET;
req->devinfo = devinfo;
+ brcmf_usb_enq(devinfo, &devinfo->rx_postq, req);
ret = usb_submit_urb(req->urb, GFP_ATOMIC);
- if (ret == 0) {
- brcmf_usb_enq(devinfo, &devinfo->rx_postq, req);
- } else {
- dev_kfree_skb(req->skb);
+ if (ret) {
+ brcmf_usb_del_fromq(devinfo, req);
+ brcmu_pkt_buf_free_skb(req->skb);
req->skb = NULL;
brcmf_usb_enq(devinfo, &devinfo->rx_freeq, req);
}
@@ -683,23 +684,22 @@
req = brcmf_usb_deq(devinfo, &devinfo->tx_freeq);
if (!req) {
+ brcmu_pkt_buf_free_skb(skb);
brcmf_dbg(ERROR, "no req to send\n");
return -ENOMEM;
}
- if (!req->urb) {
- brcmf_dbg(ERROR, "no urb for req %p\n", req);
- return -ENOBUFS;
- }
req->skb = skb;
req->devinfo = devinfo;
usb_fill_bulk_urb(req->urb, devinfo->usbdev, devinfo->tx_pipe,
skb->data, skb->len, brcmf_usb_tx_complete, req);
req->urb->transfer_flags |= URB_ZERO_PACKET;
+ brcmf_usb_enq(devinfo, &devinfo->tx_postq, req);
ret = usb_submit_urb(req->urb, GFP_ATOMIC);
- if (!ret) {
- brcmf_usb_enq(devinfo, &devinfo->tx_postq, req);
- } else {
+ if (ret) {
+ brcmf_dbg(ERROR, "brcmf_usb_tx usb_submit_urb FAILED\n");
+ brcmf_usb_del_fromq(devinfo, req);
+ brcmu_pkt_buf_free_skb(req->skb);
req->skb = NULL;
brcmf_usb_enq(devinfo, &devinfo->tx_freeq, req);
}
diff --git a/drivers/net/wireless/brcm80211/brcmfmac/wl_cfg80211.c b/drivers/net/wireless/brcm80211/brcmfmac/wl_cfg80211.c
index 28c5fbb..50b5553 100644
--- a/drivers/net/wireless/brcm80211/brcmfmac/wl_cfg80211.c
+++ b/drivers/net/wireless/brcm80211/brcmfmac/wl_cfg80211.c
@@ -500,8 +500,10 @@
params_le->active_time = cpu_to_le32(-1);
params_le->passive_time = cpu_to_le32(-1);
params_le->home_time = cpu_to_le32(-1);
- if (ssid && ssid->SSID_len)
- memcpy(¶ms_le->ssid_le, ssid, sizeof(struct brcmf_ssid));
+ if (ssid && ssid->SSID_len) {
+ params_le->ssid_le.SSID_len = cpu_to_le32(ssid->SSID_len);
+ memcpy(¶ms_le->ssid_le.SSID, ssid->SSID, ssid->SSID_len);
+ }
}
static s32
@@ -1876,16 +1878,17 @@
}
if (test_bit(WL_STATUS_CONNECTED, &cfg_priv->status)) {
- scb_val.val = cpu_to_le32(0);
+ memset(&scb_val, 0, sizeof(scb_val));
err = brcmf_exec_dcmd(ndev, BRCMF_C_GET_RSSI, &scb_val,
sizeof(struct brcmf_scb_val_le));
- if (err)
+ if (err) {
WL_ERR("Could not get rssi (%d)\n", err);
-
- rssi = le32_to_cpu(scb_val.val);
- sinfo->filled |= STATION_INFO_SIGNAL;
- sinfo->signal = rssi;
- WL_CONN("RSSI %d dBm\n", rssi);
+ } else {
+ rssi = le32_to_cpu(scb_val.val);
+ sinfo->filled |= STATION_INFO_SIGNAL;
+ sinfo->signal = rssi;
+ WL_CONN("RSSI %d dBm\n", rssi);
+ }
}
done:
diff --git a/drivers/net/wireless/brcm80211/brcmsmac/channel.c b/drivers/net/wireless/brcm80211/brcmsmac/channel.c
index 7ed7d75..64a48f0 100644
--- a/drivers/net/wireless/brcm80211/brcmsmac/channel.c
+++ b/drivers/net/wireless/brcm80211/brcmsmac/channel.c
@@ -77,7 +77,7 @@
NL80211_RRF_NO_IBSS)
static const struct ieee80211_regdomain brcms_regdom_x2 = {
- .n_reg_rules = 7,
+ .n_reg_rules = 6,
.alpha2 = "X2",
.reg_rules = {
BRCM_2GHZ_2412_2462,
diff --git a/drivers/net/wireless/brcm80211/brcmsmac/mac80211_if.c b/drivers/net/wireless/brcm80211/brcmsmac/mac80211_if.c
index 192ad5c..a5edebe 100644
--- a/drivers/net/wireless/brcm80211/brcmsmac/mac80211_if.c
+++ b/drivers/net/wireless/brcm80211/brcmsmac/mac80211_if.c
@@ -1233,6 +1233,9 @@
/* dpc will not be rescheduled */
wl->resched = false;
+ /* inform publicly that interface is down */
+ wl->pub->up = false;
+
return 0;
}
diff --git a/drivers/net/wireless/ipw2x00/ipw2100.c b/drivers/net/wireless/ipw2x00/ipw2100.c
index 95aa8e1..83324b3 100644
--- a/drivers/net/wireless/ipw2x00/ipw2100.c
+++ b/drivers/net/wireless/ipw2x00/ipw2100.c
@@ -2042,7 +2042,8 @@
return;
}
len = ETH_ALEN;
- ipw2100_get_ordinal(priv, IPW_ORD_STAT_ASSN_AP_BSSID, &bssid, &len);
+ ret = ipw2100_get_ordinal(priv, IPW_ORD_STAT_ASSN_AP_BSSID, bssid,
+ &len);
if (ret) {
IPW_DEBUG_INFO("failed querying ordinals at line %d\n",
__LINE__);
diff --git a/drivers/net/wireless/iwlegacy/common.h b/drivers/net/wireless/iwlegacy/common.h
index 5f50177..7246826 100644
--- a/drivers/net/wireless/iwlegacy/common.h
+++ b/drivers/net/wireless/iwlegacy/common.h
@@ -1832,10 +1832,8 @@
static inline u16
il_pcie_link_ctl(struct il_priv *il)
{
- int pos;
u16 pci_lnk_ctl;
- pos = pci_pcie_cap(il->pci_dev);
- pci_read_config_word(il->pci_dev, pos + PCI_EXP_LNKCTL, &pci_lnk_ctl);
+ pcie_capability_read_word(il->pci_dev, PCI_EXP_LNKCTL, &pci_lnk_ctl);
return pci_lnk_ctl;
}
diff --git a/drivers/net/wireless/iwlwifi/dvm/debugfs.c b/drivers/net/wireless/iwlwifi/dvm/debugfs.c
index 46782f1..a47b306 100644
--- a/drivers/net/wireless/iwlwifi/dvm/debugfs.c
+++ b/drivers/net/wireless/iwlwifi/dvm/debugfs.c
@@ -124,6 +124,9 @@
const struct fw_img *img;
size_t bufsz;
+ if (!iwl_is_ready_rf(priv))
+ return -EAGAIN;
+
/* default is to dump the entire data segment */
if (!priv->dbgfs_sram_offset && !priv->dbgfs_sram_len) {
priv->dbgfs_sram_offset = 0x800000;
diff --git a/drivers/net/wireless/iwlwifi/pcie/internal.h b/drivers/net/wireless/iwlwifi/pcie/internal.h
index d9694c5..4ffc18d 100644
--- a/drivers/net/wireless/iwlwifi/pcie/internal.h
+++ b/drivers/net/wireless/iwlwifi/pcie/internal.h
@@ -350,7 +350,7 @@
/*****************************************************
* Error handling
******************************************************/
-int iwl_dump_fh(struct iwl_trans *trans, char **buf, bool display);
+int iwl_dump_fh(struct iwl_trans *trans, char **buf);
void iwl_dump_csr(struct iwl_trans *trans);
/*****************************************************
diff --git a/drivers/net/wireless/iwlwifi/pcie/rx.c b/drivers/net/wireless/iwlwifi/pcie/rx.c
index 39a6ca1..d1a61ba 100644
--- a/drivers/net/wireless/iwlwifi/pcie/rx.c
+++ b/drivers/net/wireless/iwlwifi/pcie/rx.c
@@ -555,7 +555,7 @@
}
iwl_dump_csr(trans);
- iwl_dump_fh(trans, NULL, false);
+ iwl_dump_fh(trans, NULL);
iwl_op_mode_nic_error(trans->op_mode);
}
diff --git a/drivers/net/wireless/iwlwifi/pcie/trans.c b/drivers/net/wireless/iwlwifi/pcie/trans.c
index 939c2f7..063ecaf 100644
--- a/drivers/net/wireless/iwlwifi/pcie/trans.c
+++ b/drivers/net/wireless/iwlwifi/pcie/trans.c
@@ -675,13 +675,10 @@
static u16 iwl_pciexp_link_ctrl(struct iwl_trans *trans)
{
struct iwl_trans_pcie *trans_pcie = IWL_TRANS_GET_PCIE_TRANS(trans);
- int pos;
u16 pci_lnk_ctl;
- struct pci_dev *pci_dev = trans_pcie->pci_dev;
-
- pos = pci_pcie_cap(pci_dev);
- pci_read_config_word(pci_dev, pos + PCI_EXP_LNKCTL, &pci_lnk_ctl);
+ pcie_capability_read_word(trans_pcie->pci_dev, PCI_EXP_LNKCTL,
+ &pci_lnk_ctl);
return pci_lnk_ctl;
}
@@ -1442,6 +1439,7 @@
return err;
err_free_irq:
+ trans_pcie->irq_requested = false;
free_irq(trans_pcie->irq, trans);
error:
iwl_free_isr_ict(trans);
@@ -1649,13 +1647,9 @@
#undef IWL_CMD
}
-int iwl_dump_fh(struct iwl_trans *trans, char **buf, bool display)
+int iwl_dump_fh(struct iwl_trans *trans, char **buf)
{
int i;
-#ifdef CONFIG_IWLWIFI_DEBUG
- int pos = 0;
- size_t bufsz = 0;
-#endif
static const u32 fh_tbl[] = {
FH_RSCSR_CHNL0_STTS_WPTR_REG,
FH_RSCSR_CHNL0_RBDCB_BASE_REG,
@@ -1667,29 +1661,35 @@
FH_TSSR_TX_STATUS_REG,
FH_TSSR_TX_ERROR_REG
};
-#ifdef CONFIG_IWLWIFI_DEBUG
- if (display) {
- bufsz = ARRAY_SIZE(fh_tbl) * 48 + 40;
+
+#ifdef CONFIG_IWLWIFI_DEBUGFS
+ if (buf) {
+ int pos = 0;
+ size_t bufsz = ARRAY_SIZE(fh_tbl) * 48 + 40;
+
*buf = kmalloc(bufsz, GFP_KERNEL);
if (!*buf)
return -ENOMEM;
+
pos += scnprintf(*buf + pos, bufsz - pos,
"FH register values:\n");
- for (i = 0; i < ARRAY_SIZE(fh_tbl); i++) {
+
+ for (i = 0; i < ARRAY_SIZE(fh_tbl); i++)
pos += scnprintf(*buf + pos, bufsz - pos,
" %34s: 0X%08x\n",
get_fh_string(fh_tbl[i]),
iwl_read_direct32(trans, fh_tbl[i]));
- }
+
return pos;
}
#endif
+
IWL_ERR(trans, "FH register values:\n");
- for (i = 0; i < ARRAY_SIZE(fh_tbl); i++) {
+ for (i = 0; i < ARRAY_SIZE(fh_tbl); i++)
IWL_ERR(trans, " %34s: 0X%08x\n",
get_fh_string(fh_tbl[i]),
iwl_read_direct32(trans, fh_tbl[i]));
- }
+
return 0;
}
@@ -1982,11 +1982,11 @@
size_t count, loff_t *ppos)
{
struct iwl_trans *trans = file->private_data;
- char *buf;
+ char *buf = NULL;
int pos = 0;
ssize_t ret = -EFAULT;
- ret = pos = iwl_dump_fh(trans, &buf, true);
+ ret = pos = iwl_dump_fh(trans, &buf);
if (buf) {
ret = simple_read_from_buffer(user_buf,
count, ppos, buf, pos);
diff --git a/drivers/net/wireless/libertas/if_sdio.c b/drivers/net/wireless/libertas/if_sdio.c
index e970897..4cb2343 100644
--- a/drivers/net/wireless/libertas/if_sdio.c
+++ b/drivers/net/wireless/libertas/if_sdio.c
@@ -1326,6 +1326,11 @@
mmc_pm_flag_t flags = sdio_get_host_pm_caps(func);
+ /* If we're powered off anyway, just let the mmc layer remove the
+ * card. */
+ if (!lbs_iface_active(card->priv))
+ return -ENOSYS;
+
dev_info(dev, "%s: suspend: PM flags = 0x%x\n",
sdio_func_id(func), flags);
diff --git a/drivers/net/wireless/mwifiex/cmdevt.c b/drivers/net/wireless/mwifiex/cmdevt.c
index c68adec..565527a 100644
--- a/drivers/net/wireless/mwifiex/cmdevt.c
+++ b/drivers/net/wireless/mwifiex/cmdevt.c
@@ -170,7 +170,20 @@
cmd_code = le16_to_cpu(host_cmd->command);
cmd_size = le16_to_cpu(host_cmd->size);
- skb_trim(cmd_node->cmd_skb, cmd_size);
+ /* Adjust skb length */
+ if (cmd_node->cmd_skb->len > cmd_size)
+ /*
+ * cmd_size is less than sizeof(struct host_cmd_ds_command).
+ * Trim off the unused portion.
+ */
+ skb_trim(cmd_node->cmd_skb, cmd_size);
+ else if (cmd_node->cmd_skb->len < cmd_size)
+ /*
+ * cmd_size is larger than sizeof(struct host_cmd_ds_command)
+ * because we have appended custom IE TLV. Increase skb length
+ * accordingly.
+ */
+ skb_put(cmd_node->cmd_skb, cmd_size - cmd_node->cmd_skb->len);
do_gettimeofday(&tstamp);
dev_dbg(adapter->dev, "cmd: DNLD_CMD: (%lu.%lu): %#x, act %#x, len %d,"
diff --git a/drivers/net/wireless/rt2x00/rt2400pci.c b/drivers/net/wireless/rt2x00/rt2400pci.c
index 8b9dbd7..64328af 100644
--- a/drivers/net/wireless/rt2x00/rt2400pci.c
+++ b/drivers/net/wireless/rt2x00/rt2400pci.c
@@ -1611,6 +1611,7 @@
static int rt2400pci_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u32 reg;
/*
* Allocate eeprom data.
@@ -1624,6 +1625,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2x00pci_register_read(rt2x00dev, GPIOCSR, ®);
+ rt2x00_set_field32(®, GPIOCSR_BIT8, 1);
+ rt2x00pci_register_write(rt2x00dev, GPIOCSR, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt2400pci_probe_hw_mode(rt2x00dev);
diff --git a/drivers/net/wireless/rt2x00/rt2400pci.h b/drivers/net/wireless/rt2x00/rt2400pci.h
index d3a4a68..7564ae9 100644
--- a/drivers/net/wireless/rt2x00/rt2400pci.h
+++ b/drivers/net/wireless/rt2x00/rt2400pci.h
@@ -670,6 +670,7 @@
#define GPIOCSR_BIT5 FIELD32(0x00000020)
#define GPIOCSR_BIT6 FIELD32(0x00000040)
#define GPIOCSR_BIT7 FIELD32(0x00000080)
+#define GPIOCSR_BIT8 FIELD32(0x00000100)
/*
* BBPPCSR: BBP Pin control register.
diff --git a/drivers/net/wireless/rt2x00/rt2500pci.c b/drivers/net/wireless/rt2x00/rt2500pci.c
index d2cf8a4..3de0406 100644
--- a/drivers/net/wireless/rt2x00/rt2500pci.c
+++ b/drivers/net/wireless/rt2x00/rt2500pci.c
@@ -1929,6 +1929,7 @@
static int rt2500pci_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u32 reg;
/*
* Allocate eeprom data.
@@ -1942,6 +1943,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2x00pci_register_read(rt2x00dev, GPIOCSR, ®);
+ rt2x00_set_field32(®, GPIOCSR_DIR0, 1);
+ rt2x00pci_register_write(rt2x00dev, GPIOCSR, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt2500pci_probe_hw_mode(rt2x00dev);
diff --git a/drivers/net/wireless/rt2x00/rt2500usb.c b/drivers/net/wireless/rt2x00/rt2500usb.c
index 3aae36b..89fee31 100644
--- a/drivers/net/wireless/rt2x00/rt2500usb.c
+++ b/drivers/net/wireless/rt2x00/rt2500usb.c
@@ -283,7 +283,7 @@
u16 reg;
rt2500usb_register_read(rt2x00dev, MAC_CSR19, ®);
- return rt2x00_get_field32(reg, MAC_CSR19_BIT7);
+ return rt2x00_get_field16(reg, MAC_CSR19_BIT7);
}
#ifdef CONFIG_RT2X00_LIB_LEDS
@@ -1768,6 +1768,7 @@
static int rt2500usb_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u16 reg;
/*
* Allocate eeprom data.
@@ -1781,6 +1782,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2500usb_register_read(rt2x00dev, MAC_CSR19, ®);
+ rt2x00_set_field16(®, MAC_CSR19_BIT8, 0);
+ rt2500usb_register_write(rt2x00dev, MAC_CSR19, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt2500usb_probe_hw_mode(rt2x00dev);
diff --git a/drivers/net/wireless/rt2x00/rt2500usb.h b/drivers/net/wireless/rt2x00/rt2500usb.h
index b493306..196bd510 100644
--- a/drivers/net/wireless/rt2x00/rt2500usb.h
+++ b/drivers/net/wireless/rt2x00/rt2500usb.h
@@ -189,14 +189,15 @@
* MAC_CSR19: GPIO control register.
*/
#define MAC_CSR19 0x0426
-#define MAC_CSR19_BIT0 FIELD32(0x0001)
-#define MAC_CSR19_BIT1 FIELD32(0x0002)
-#define MAC_CSR19_BIT2 FIELD32(0x0004)
-#define MAC_CSR19_BIT3 FIELD32(0x0008)
-#define MAC_CSR19_BIT4 FIELD32(0x0010)
-#define MAC_CSR19_BIT5 FIELD32(0x0020)
-#define MAC_CSR19_BIT6 FIELD32(0x0040)
-#define MAC_CSR19_BIT7 FIELD32(0x0080)
+#define MAC_CSR19_BIT0 FIELD16(0x0001)
+#define MAC_CSR19_BIT1 FIELD16(0x0002)
+#define MAC_CSR19_BIT2 FIELD16(0x0004)
+#define MAC_CSR19_BIT3 FIELD16(0x0008)
+#define MAC_CSR19_BIT4 FIELD16(0x0010)
+#define MAC_CSR19_BIT5 FIELD16(0x0020)
+#define MAC_CSR19_BIT6 FIELD16(0x0040)
+#define MAC_CSR19_BIT7 FIELD16(0x0080)
+#define MAC_CSR19_BIT8 FIELD16(0x0100)
/*
* MAC_CSR20: LED control register.
diff --git a/drivers/net/wireless/rt2x00/rt2800lib.c b/drivers/net/wireless/rt2x00/rt2800lib.c
index cb8c2ac..b93516d 100644
--- a/drivers/net/wireless/rt2x00/rt2800lib.c
+++ b/drivers/net/wireless/rt2x00/rt2800lib.c
@@ -4089,6 +4089,7 @@
rt2800_register_write(rt2x00dev, LDO_CFG0, reg);
msleep(1);
rt2800_register_read(rt2x00dev, LDO_CFG0, ®);
+ rt2x00_set_field32(®, LDO_CFG0_LDO_CORE_VLEVEL, 0);
rt2x00_set_field32(®, LDO_CFG0_BGSEL, 1);
rt2800_register_write(rt2x00dev, LDO_CFG0, reg);
}
diff --git a/drivers/net/wireless/rt2x00/rt2800pci.c b/drivers/net/wireless/rt2x00/rt2800pci.c
index 98aa426..4765bbd 100644
--- a/drivers/net/wireless/rt2x00/rt2800pci.c
+++ b/drivers/net/wireless/rt2x00/rt2800pci.c
@@ -983,6 +983,7 @@
static int rt2800pci_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u32 reg;
/*
* Allocate eeprom data.
@@ -996,6 +997,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2x00pci_register_read(rt2x00dev, GPIO_CTRL_CFG, ®);
+ rt2x00_set_field32(®, GPIO_CTRL_CFG_GPIOD_BIT2, 1);
+ rt2x00pci_register_write(rt2x00dev, GPIO_CTRL_CFG, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt2800_probe_hw_mode(rt2x00dev);
diff --git a/drivers/net/wireless/rt2x00/rt2800usb.c b/drivers/net/wireless/rt2x00/rt2800usb.c
index 6cf3365..6b4226b 100644
--- a/drivers/net/wireless/rt2x00/rt2800usb.c
+++ b/drivers/net/wireless/rt2x00/rt2800usb.c
@@ -667,8 +667,16 @@
skb_pull(entry->skb, RXINFO_DESC_SIZE);
/*
- * FIXME: we need to check for rx_pkt_len validity
+ * Check for rx_pkt_len validity. Return if invalid, leaving
+ * rxdesc->size zeroed out by the upper level.
*/
+ if (unlikely(rx_pkt_len == 0 ||
+ rx_pkt_len > entry->queue->data_size)) {
+ ERROR(entry->queue->rt2x00dev,
+ "Bad frame size %d, forcing to 0\n", rx_pkt_len);
+ return;
+ }
+
rxd = (__le32 *)(entry->skb->data + rx_pkt_len);
/*
@@ -736,6 +744,7 @@
static int rt2800usb_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u32 reg;
/*
* Allocate eeprom data.
@@ -749,6 +758,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2x00usb_register_read(rt2x00dev, GPIO_CTRL_CFG, ®);
+ rt2x00_set_field32(®, GPIO_CTRL_CFG_GPIOD_BIT2, 1);
+ rt2x00usb_register_write(rt2x00dev, GPIO_CTRL_CFG, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt2800_probe_hw_mode(rt2x00dev);
@@ -1157,6 +1174,8 @@
{ USB_DEVICE(0x1690, 0x0744) },
{ USB_DEVICE(0x1690, 0x0761) },
{ USB_DEVICE(0x1690, 0x0764) },
+ /* ASUS */
+ { USB_DEVICE(0x0b05, 0x179d) },
/* Cisco */
{ USB_DEVICE(0x167b, 0x4001) },
/* EnGenius */
@@ -1222,7 +1241,6 @@
{ USB_DEVICE(0x0b05, 0x1760) },
{ USB_DEVICE(0x0b05, 0x1761) },
{ USB_DEVICE(0x0b05, 0x1790) },
- { USB_DEVICE(0x0b05, 0x179d) },
/* AzureWave */
{ USB_DEVICE(0x13d3, 0x3262) },
{ USB_DEVICE(0x13d3, 0x3284) },
diff --git a/drivers/net/wireless/rt2x00/rt2x00dev.c b/drivers/net/wireless/rt2x00/rt2x00dev.c
index a6b88bd..3f07e36 100644
--- a/drivers/net/wireless/rt2x00/rt2x00dev.c
+++ b/drivers/net/wireless/rt2x00/rt2x00dev.c
@@ -629,7 +629,7 @@
*/
if (unlikely(rxdesc.size == 0 ||
rxdesc.size > entry->queue->data_size)) {
- WARNING(rt2x00dev, "Wrong frame size %d max %d.\n",
+ ERROR(rt2x00dev, "Wrong frame size %d max %d.\n",
rxdesc.size, entry->queue->data_size);
dev_kfree_skb(entry->skb);
goto renew_skb;
diff --git a/drivers/net/wireless/rt2x00/rt61pci.c b/drivers/net/wireless/rt2x00/rt61pci.c
index 3f7bc5c..b8ec961 100644
--- a/drivers/net/wireless/rt2x00/rt61pci.c
+++ b/drivers/net/wireless/rt2x00/rt61pci.c
@@ -2832,6 +2832,7 @@
static int rt61pci_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u32 reg;
/*
* Disable power saving.
@@ -2850,6 +2851,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2x00pci_register_read(rt2x00dev, MAC_CSR13, ®);
+ rt2x00_set_field32(®, MAC_CSR13_BIT13, 1);
+ rt2x00pci_register_write(rt2x00dev, MAC_CSR13, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt61pci_probe_hw_mode(rt2x00dev);
diff --git a/drivers/net/wireless/rt2x00/rt61pci.h b/drivers/net/wireless/rt2x00/rt61pci.h
index e3cd6db..8f3da5a 100644
--- a/drivers/net/wireless/rt2x00/rt61pci.h
+++ b/drivers/net/wireless/rt2x00/rt61pci.h
@@ -372,6 +372,7 @@
#define MAC_CSR13_BIT10 FIELD32(0x00000400)
#define MAC_CSR13_BIT11 FIELD32(0x00000800)
#define MAC_CSR13_BIT12 FIELD32(0x00001000)
+#define MAC_CSR13_BIT13 FIELD32(0x00002000)
/*
* MAC_CSR14: LED control register.
diff --git a/drivers/net/wireless/rt2x00/rt73usb.c b/drivers/net/wireless/rt2x00/rt73usb.c
index ba6e434..248436c 100644
--- a/drivers/net/wireless/rt2x00/rt73usb.c
+++ b/drivers/net/wireless/rt2x00/rt73usb.c
@@ -2177,6 +2177,7 @@
static int rt73usb_probe_hw(struct rt2x00_dev *rt2x00dev)
{
int retval;
+ u32 reg;
/*
* Allocate eeprom data.
@@ -2190,6 +2191,14 @@
return retval;
/*
+ * Enable rfkill polling by setting GPIO direction of the
+ * rfkill switch GPIO pin correctly.
+ */
+ rt2x00usb_register_read(rt2x00dev, MAC_CSR13, ®);
+ rt2x00_set_field32(®, MAC_CSR13_BIT15, 0);
+ rt2x00usb_register_write(rt2x00dev, MAC_CSR13, reg);
+
+ /*
* Initialize hw specifications.
*/
retval = rt73usb_probe_hw_mode(rt2x00dev);
diff --git a/drivers/net/wireless/rt2x00/rt73usb.h b/drivers/net/wireless/rt2x00/rt73usb.h
index 9f6b470..df1cc11 100644
--- a/drivers/net/wireless/rt2x00/rt73usb.h
+++ b/drivers/net/wireless/rt2x00/rt73usb.h
@@ -282,6 +282,9 @@
#define MAC_CSR13_BIT10 FIELD32(0x00000400)
#define MAC_CSR13_BIT11 FIELD32(0x00000800)
#define MAC_CSR13_BIT12 FIELD32(0x00001000)
+#define MAC_CSR13_BIT13 FIELD32(0x00002000)
+#define MAC_CSR13_BIT14 FIELD32(0x00004000)
+#define MAC_CSR13_BIT15 FIELD32(0x00008000)
/*
* MAC_CSR14: LED control register.
diff --git a/drivers/net/wireless/rtlwifi/pci.c b/drivers/net/wireless/rtlwifi/pci.c
index 80f75d3..5983631 100644
--- a/drivers/net/wireless/rtlwifi/pci.c
+++ b/drivers/net/wireless/rtlwifi/pci.c
@@ -372,13 +372,11 @@
struct rtl_pci_priv *pcipriv = rtl_pcipriv(hw);
u8 tmp;
- int pos;
- u8 linkctrl_reg;
+ u16 linkctrl_reg;
/*Link Control Register */
- pos = pci_pcie_cap(pdev);
- pci_read_config_byte(pdev, pos + PCI_EXP_LNKCTL, &linkctrl_reg);
- pcipriv->ndis_adapter.linkctrl_reg = linkctrl_reg;
+ pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &linkctrl_reg);
+ pcipriv->ndis_adapter.linkctrl_reg = (u8)linkctrl_reg;
RT_TRACE(rtlpriv, COMP_INIT, DBG_TRACE, "Link Control Register =%x\n",
pcipriv->ndis_adapter.linkctrl_reg);
diff --git a/drivers/net/wireless/rtlwifi/rtl8192c/fw_common.c b/drivers/net/wireless/rtlwifi/rtl8192c/fw_common.c
index 44febfd..8a7b864 100644
--- a/drivers/net/wireless/rtlwifi/rtl8192c/fw_common.c
+++ b/drivers/net/wireless/rtlwifi/rtl8192c/fw_common.c
@@ -315,7 +315,7 @@
u16 box_reg = 0, box_extreg = 0;
u8 u1b_tmp;
bool isfw_read = false;
- bool bwrite_sucess = false;
+ bool bwrite_success = false;
u8 wait_h2c_limmit = 100;
u8 wait_writeh2c_limmit = 100;
u8 boxcontent[4], boxextcontent[2];
@@ -354,7 +354,7 @@
}
}
- while (!bwrite_sucess) {
+ while (!bwrite_success) {
wait_writeh2c_limmit--;
if (wait_writeh2c_limmit == 0) {
RT_TRACE(rtlpriv, COMP_ERR, DBG_EMERG,
@@ -491,7 +491,7 @@
break;
}
- bwrite_sucess = true;
+ bwrite_success = true;
rtlhal->last_hmeboxnum = boxnum + 1;
if (rtlhal->last_hmeboxnum == 4)
diff --git a/drivers/net/wireless/rtlwifi/rtl8192ce/def.h b/drivers/net/wireless/rtlwifi/rtl8192ce/def.h
index 04c3aef..2925094 100644
--- a/drivers/net/wireless/rtlwifi/rtl8192ce/def.h
+++ b/drivers/net/wireless/rtlwifi/rtl8192ce/def.h
@@ -117,6 +117,7 @@
#define CHIP_VER_B BIT(4)
#define CHIP_92C_BITMASK BIT(0)
+#define CHIP_UNKNOWN BIT(7)
#define CHIP_92C_1T2R 0x03
#define CHIP_92C 0x01
#define CHIP_88C 0x00
diff --git a/drivers/net/wireless/rtlwifi/rtl8192ce/hw.c b/drivers/net/wireless/rtlwifi/rtl8192ce/hw.c
index bd0da7e..dd4bb09 100644
--- a/drivers/net/wireless/rtlwifi/rtl8192ce/hw.c
+++ b/drivers/net/wireless/rtlwifi/rtl8192ce/hw.c
@@ -994,8 +994,16 @@
version = (value32 & TYPE_ID) ? VERSION_A_CHIP_92C :
VERSION_A_CHIP_88C;
} else {
- version = (value32 & TYPE_ID) ? VERSION_B_CHIP_92C :
- VERSION_B_CHIP_88C;
+ version = (enum version_8192c) (CHIP_VER_B |
+ ((value32 & TYPE_ID) ? CHIP_92C_BITMASK : 0) |
+ ((value32 & VENDOR_ID) ? CHIP_VENDOR_UMC : 0));
+ if ((!IS_CHIP_VENDOR_UMC(version)) && (value32 &
+ CHIP_VER_RTL_MASK)) {
+ version = (enum version_8192c)(version |
+ ((((value32 & CHIP_VER_RTL_MASK) == BIT(12))
+ ? CHIP_VENDOR_UMC_B_CUT : CHIP_UNKNOWN) |
+ CHIP_VENDOR_UMC));
+ }
}
switch (version) {
diff --git a/drivers/net/wireless/rtlwifi/rtl8192ce/sw.c b/drivers/net/wireless/rtlwifi/rtl8192ce/sw.c
index 3aa927f..7d8f964 100644
--- a/drivers/net/wireless/rtlwifi/rtl8192ce/sw.c
+++ b/drivers/net/wireless/rtlwifi/rtl8192ce/sw.c
@@ -162,10 +162,12 @@
/* request fw */
if (IS_VENDOR_UMC_A_CUT(rtlhal->version) &&
- !IS_92C_SERIAL(rtlhal->version))
+ !IS_92C_SERIAL(rtlhal->version)) {
rtlpriv->cfg->fw_name = "rtlwifi/rtl8192cfwU.bin";
- else if (IS_81xxC_VENDOR_UMC_B_CUT(rtlhal->version))
+ } else if (IS_81xxC_VENDOR_UMC_B_CUT(rtlhal->version)) {
rtlpriv->cfg->fw_name = "rtlwifi/rtl8192cfwU_B.bin";
+ pr_info("****** This B_CUT device may not work with kernels 3.6 and earlier\n");
+ }
rtlpriv->max_fw_size = 0x4000;
pr_info("Using firmware %s\n", rtlpriv->cfg->fw_name);
diff --git a/drivers/net/wireless/rtlwifi/rtl8192de/fw.c b/drivers/net/wireless/rtlwifi/rtl8192de/fw.c
index 895ae6c..eb22dcc 100644
--- a/drivers/net/wireless/rtlwifi/rtl8192de/fw.c
+++ b/drivers/net/wireless/rtlwifi/rtl8192de/fw.c
@@ -365,7 +365,7 @@
u8 u1b_tmp;
bool isfw_read = false;
u8 buf_index = 0;
- bool bwrite_sucess = false;
+ bool bwrite_success = false;
u8 wait_h2c_limmit = 100;
u8 wait_writeh2c_limmit = 100;
u8 boxcontent[4], boxextcontent[2];
@@ -408,7 +408,7 @@
break;
}
}
- while (!bwrite_sucess) {
+ while (!bwrite_success) {
wait_writeh2c_limmit--;
if (wait_writeh2c_limmit == 0) {
RT_TRACE(rtlpriv, COMP_ERR, DBG_EMERG,
@@ -515,7 +515,7 @@
"switch case not processed\n");
break;
}
- bwrite_sucess = true;
+ bwrite_success = true;
rtlhal->last_hmeboxnum = boxnum + 1;
if (rtlhal->last_hmeboxnum == 4)
rtlhal->last_hmeboxnum = 0;
diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index 3089990..650f79a 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -57,8 +57,7 @@
static const struct ethtool_ops xennet_ethtool_ops;
struct netfront_cb {
- struct page *page;
- unsigned offset;
+ int pull_to;
};
#define NETFRONT_SKB_CB(skb) ((struct netfront_cb *)((skb)->cb))
@@ -867,15 +866,9 @@
struct sk_buff *skb;
while ((skb = __skb_dequeue(rxq)) != NULL) {
- struct page *page = NETFRONT_SKB_CB(skb)->page;
- void *vaddr = page_address(page);
- unsigned offset = NETFRONT_SKB_CB(skb)->offset;
+ int pull_to = NETFRONT_SKB_CB(skb)->pull_to;
- memcpy(skb->data, vaddr + offset,
- skb_headlen(skb));
-
- if (page != skb_frag_page(&skb_shinfo(skb)->frags[0]))
- __free_page(page);
+ __pskb_pull_tail(skb, pull_to - skb_headlen(skb));
/* Ethernet work: Delayed to here as it peeks the header. */
skb->protocol = eth_type_trans(skb, dev);
@@ -913,7 +906,6 @@
struct sk_buff_head errq;
struct sk_buff_head tmpq;
unsigned long flags;
- unsigned int len;
int err;
spin_lock(&np->rx_lock);
@@ -955,24 +947,13 @@
}
}
- NETFRONT_SKB_CB(skb)->page =
- skb_frag_page(&skb_shinfo(skb)->frags[0]);
- NETFRONT_SKB_CB(skb)->offset = rx->offset;
+ NETFRONT_SKB_CB(skb)->pull_to = rx->status;
+ if (NETFRONT_SKB_CB(skb)->pull_to > RX_COPY_THRESHOLD)
+ NETFRONT_SKB_CB(skb)->pull_to = RX_COPY_THRESHOLD;
- len = rx->status;
- if (len > RX_COPY_THRESHOLD)
- len = RX_COPY_THRESHOLD;
- skb_put(skb, len);
-
- if (rx->status > len) {
- skb_shinfo(skb)->frags[0].page_offset =
- rx->offset + len;
- skb_frag_size_set(&skb_shinfo(skb)->frags[0], rx->status - len);
- skb->data_len = rx->status - len;
- } else {
- __skb_fill_page_desc(skb, 0, NULL, 0, 0);
- skb_shinfo(skb)->nr_frags = 0;
- }
+ skb_shinfo(skb)->frags[0].page_offset = rx->offset;
+ skb_frag_size_set(&skb_shinfo(skb)->frags[0], rx->status);
+ skb->data_len = rx->status;
i = xennet_fill_frags(np, skb, &tmpq);
@@ -999,7 +980,7 @@
* receive throughout using the standard receive
* buffer size was cut by 25%(!!!).
*/
- skb->truesize += skb->data_len - (RX_COPY_THRESHOLD - len);
+ skb->truesize += skb->data_len - RX_COPY_THRESHOLD;
skb->len += skb->data_len;
if (rx->flags & XEN_NETRXF_csum_blank)
diff --git a/drivers/oprofile/cpu_buffer.c b/drivers/oprofile/cpu_buffer.c
index b8ef8dd..8aa73fa 100644
--- a/drivers/oprofile/cpu_buffer.c
+++ b/drivers/oprofile/cpu_buffer.c
@@ -451,14 +451,9 @@
{
struct oprofile_cpu_buffer *b =
container_of(work, struct oprofile_cpu_buffer, work.work);
- if (b->cpu != smp_processor_id()) {
- printk(KERN_DEBUG "WQ on CPU%d, prefer CPU%d\n",
- smp_processor_id(), b->cpu);
-
- if (!cpu_online(b->cpu)) {
- cancel_delayed_work(&b->work);
- return;
- }
+ if (b->cpu != smp_processor_id() && !cpu_online(b->cpu)) {
+ cancel_delayed_work(&b->work);
+ return;
}
sync_buffer(b->cpu);
diff --git a/drivers/parisc/dino.c b/drivers/parisc/dino.c
index ffddc4f..fb6a1fe 100644
--- a/drivers/parisc/dino.c
+++ b/drivers/parisc/dino.c
@@ -477,14 +477,12 @@
if (ccio_allocate_resource(dino_dev->hba.dev, res, _8MB,
F_EXTEND(0xf0000000UL) | _8MB,
F_EXTEND(0xffffffffUL) &~ _8MB, _8MB) < 0) {
- struct list_head *ln, *tmp_ln;
+ struct pci_dev *dev, *tmp;
printk(KERN_ERR "Dino: cannot attach bus %s\n",
dev_name(bus->bridge));
/* kill the bus, we can't do anything with it */
- list_for_each_safe(ln, tmp_ln, &bus->devices) {
- struct pci_dev *dev = pci_dev_b(ln);
-
+ list_for_each_entry_safe(dev, tmp, &bus->devices, bus_list) {
list_del(&dev->bus_list);
}
@@ -549,7 +547,6 @@
static void __init
dino_fixup_bus(struct pci_bus *bus)
{
- struct list_head *ln;
struct pci_dev *dev;
struct dino_device *dino_dev = DINO_DEV(parisc_walk_tree(bus->bridge));
@@ -596,8 +593,7 @@
}
- list_for_each(ln, &bus->devices) {
- dev = pci_dev_b(ln);
+ list_for_each_entry(dev, &bus->devices, bus_list) {
if (is_card_dino(&dino_dev->hba.dev->id))
dino_card_fixup(dev);
diff --git a/drivers/parisc/lba_pci.c b/drivers/parisc/lba_pci.c
index 4f9cf24..fdd63a6 100644
--- a/drivers/parisc/lba_pci.c
+++ b/drivers/parisc/lba_pci.c
@@ -629,7 +629,7 @@
static void
lba_fixup_bus(struct pci_bus *bus)
{
- struct list_head *ln;
+ struct pci_dev *dev;
#ifdef FBB_SUPPORT
u16 status;
#endif
@@ -710,9 +710,8 @@
}
- list_for_each(ln, &bus->devices) {
+ list_for_each_entry(dev, &bus->devices, bus_list) {
int i;
- struct pci_dev *dev = pci_dev_b(ln);
DBG("lba_fixup_bus() %s\n", pci_name(dev));
@@ -770,7 +769,7 @@
}
/* Lastly enable FBB/PERR/SERR on all devices too */
- list_for_each(ln, &bus->devices) {
+ list_for_each_entry(dev, &bus->devices, bus_list) {
(void) pci_read_config_word(dev, PCI_COMMAND, &status);
status |= PCI_COMMAND_PARITY | PCI_COMMAND_SERR | fbb_enable;
(void) pci_write_config_word(dev, PCI_COMMAND, status);
diff --git a/drivers/pci/.gitignore b/drivers/pci/.gitignore
deleted file mode 100644
index f297ca8..0000000
--- a/drivers/pci/.gitignore
+++ /dev/null
@@ -1,4 +0,0 @@
-classlist.h
-devlist.h
-gen-devlist
-
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 848bfb8..6d51aa6 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -3,7 +3,6 @@
#
config ARCH_SUPPORTS_MSI
bool
- default n
config PCI_MSI
bool "Message Signaled Interrupts (MSI and MSI-X)"
diff --git a/drivers/pci/access.c b/drivers/pci/access.c
index ba91a7e..3af0478 100644
--- a/drivers/pci/access.c
+++ b/drivers/pci/access.c
@@ -469,3 +469,205 @@
raw_spin_unlock_irqrestore(&pci_lock, flags);
}
EXPORT_SYMBOL_GPL(pci_cfg_access_unlock);
+
+static inline int pcie_cap_version(const struct pci_dev *dev)
+{
+ return dev->pcie_flags_reg & PCI_EXP_FLAGS_VERS;
+}
+
+static inline bool pcie_cap_has_devctl(const struct pci_dev *dev)
+{
+ return true;
+}
+
+static inline bool pcie_cap_has_lnkctl(const struct pci_dev *dev)
+{
+ int type = pci_pcie_type(dev);
+
+ return pcie_cap_version(dev) > 1 ||
+ type == PCI_EXP_TYPE_ROOT_PORT ||
+ type == PCI_EXP_TYPE_ENDPOINT ||
+ type == PCI_EXP_TYPE_LEG_END;
+}
+
+static inline bool pcie_cap_has_sltctl(const struct pci_dev *dev)
+{
+ int type = pci_pcie_type(dev);
+
+ return pcie_cap_version(dev) > 1 ||
+ type == PCI_EXP_TYPE_ROOT_PORT ||
+ (type == PCI_EXP_TYPE_DOWNSTREAM &&
+ dev->pcie_flags_reg & PCI_EXP_FLAGS_SLOT);
+}
+
+static inline bool pcie_cap_has_rtctl(const struct pci_dev *dev)
+{
+ int type = pci_pcie_type(dev);
+
+ return pcie_cap_version(dev) > 1 ||
+ type == PCI_EXP_TYPE_ROOT_PORT ||
+ type == PCI_EXP_TYPE_RC_EC;
+}
+
+static bool pcie_capability_reg_implemented(struct pci_dev *dev, int pos)
+{
+ if (!pci_is_pcie(dev))
+ return false;
+
+ switch (pos) {
+ case PCI_EXP_FLAGS_TYPE:
+ return true;
+ case PCI_EXP_DEVCAP:
+ case PCI_EXP_DEVCTL:
+ case PCI_EXP_DEVSTA:
+ return pcie_cap_has_devctl(dev);
+ case PCI_EXP_LNKCAP:
+ case PCI_EXP_LNKCTL:
+ case PCI_EXP_LNKSTA:
+ return pcie_cap_has_lnkctl(dev);
+ case PCI_EXP_SLTCAP:
+ case PCI_EXP_SLTCTL:
+ case PCI_EXP_SLTSTA:
+ return pcie_cap_has_sltctl(dev);
+ case PCI_EXP_RTCTL:
+ case PCI_EXP_RTCAP:
+ case PCI_EXP_RTSTA:
+ return pcie_cap_has_rtctl(dev);
+ case PCI_EXP_DEVCAP2:
+ case PCI_EXP_DEVCTL2:
+ case PCI_EXP_LNKCAP2:
+ case PCI_EXP_LNKCTL2:
+ case PCI_EXP_LNKSTA2:
+ return pcie_cap_version(dev) > 1;
+ default:
+ return false;
+ }
+}
+
+/*
+ * Note that these accessor functions are only for the "PCI Express
+ * Capability" (see PCIe spec r3.0, sec 7.8). They do not apply to the
+ * other "PCI Express Extended Capabilities" (AER, VC, ACS, MFVC, etc.)
+ */
+int pcie_capability_read_word(struct pci_dev *dev, int pos, u16 *val)
+{
+ int ret;
+
+ *val = 0;
+ if (pos & 1)
+ return -EINVAL;
+
+ if (pcie_capability_reg_implemented(dev, pos)) {
+ ret = pci_read_config_word(dev, pci_pcie_cap(dev) + pos, val);
+ /*
+ * Reset *val to 0 if pci_read_config_word() fails, it may
+ * have been written as 0xFFFF if hardware error happens
+ * during pci_read_config_word().
+ */
+ if (ret)
+ *val = 0;
+ return ret;
+ }
+
+ /*
+ * For Functions that do not implement the Slot Capabilities,
+ * Slot Status, and Slot Control registers, these spaces must
+ * be hardwired to 0b, with the exception of the Presence Detect
+ * State bit in the Slot Status register of Downstream Ports,
+ * which must be hardwired to 1b. (PCIe Base Spec 3.0, sec 7.8)
+ */
+ if (pci_is_pcie(dev) && pos == PCI_EXP_SLTSTA &&
+ pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) {
+ *val = PCI_EXP_SLTSTA_PDS;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(pcie_capability_read_word);
+
+int pcie_capability_read_dword(struct pci_dev *dev, int pos, u32 *val)
+{
+ int ret;
+
+ *val = 0;
+ if (pos & 3)
+ return -EINVAL;
+
+ if (pcie_capability_reg_implemented(dev, pos)) {
+ ret = pci_read_config_dword(dev, pci_pcie_cap(dev) + pos, val);
+ /*
+ * Reset *val to 0 if pci_read_config_dword() fails, it may
+ * have been written as 0xFFFFFFFF if hardware error happens
+ * during pci_read_config_dword().
+ */
+ if (ret)
+ *val = 0;
+ return ret;
+ }
+
+ if (pci_is_pcie(dev) && pos == PCI_EXP_SLTCTL &&
+ pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) {
+ *val = PCI_EXP_SLTSTA_PDS;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(pcie_capability_read_dword);
+
+int pcie_capability_write_word(struct pci_dev *dev, int pos, u16 val)
+{
+ if (pos & 1)
+ return -EINVAL;
+
+ if (!pcie_capability_reg_implemented(dev, pos))
+ return 0;
+
+ return pci_write_config_word(dev, pci_pcie_cap(dev) + pos, val);
+}
+EXPORT_SYMBOL(pcie_capability_write_word);
+
+int pcie_capability_write_dword(struct pci_dev *dev, int pos, u32 val)
+{
+ if (pos & 3)
+ return -EINVAL;
+
+ if (!pcie_capability_reg_implemented(dev, pos))
+ return 0;
+
+ return pci_write_config_dword(dev, pci_pcie_cap(dev) + pos, val);
+}
+EXPORT_SYMBOL(pcie_capability_write_dword);
+
+int pcie_capability_clear_and_set_word(struct pci_dev *dev, int pos,
+ u16 clear, u16 set)
+{
+ int ret;
+ u16 val;
+
+ ret = pcie_capability_read_word(dev, pos, &val);
+ if (!ret) {
+ val &= ~clear;
+ val |= set;
+ ret = pcie_capability_write_word(dev, pos, val);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(pcie_capability_clear_and_set_word);
+
+int pcie_capability_clear_and_set_dword(struct pci_dev *dev, int pos,
+ u32 clear, u32 set)
+{
+ int ret;
+ u32 val;
+
+ ret = pcie_capability_read_dword(dev, pos, &val);
+ if (!ret) {
+ val &= ~clear;
+ val |= set;
+ ret = pcie_capability_write_dword(dev, pos, val);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(pcie_capability_clear_and_set_dword);
diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index 4b0970b..6241fd0 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -87,11 +87,15 @@
void pci_bus_remove_resources(struct pci_bus *bus)
{
int i;
+ struct pci_bus_resource *bus_res, *tmp;
for (i = 0; i < PCI_BRIDGE_RESOURCE_NUM; i++)
bus->resource[i] = NULL;
- pci_free_resource_list(&bus->resources);
+ list_for_each_entry_safe(bus_res, tmp, &bus->resources, list) {
+ list_del(&bus_res->list);
+ kfree(bus_res);
+ }
}
/**
diff --git a/drivers/pci/hotplug/Kconfig b/drivers/pci/hotplug/Kconfig
index 66f29bc..b0e46de 100644
--- a/drivers/pci/hotplug/Kconfig
+++ b/drivers/pci/hotplug/Kconfig
@@ -17,28 +17,6 @@
if HOTPLUG_PCI
-config HOTPLUG_PCI_FAKE
- tristate "Fake PCI Hotplug driver"
- help
- Say Y here if you want to use the fake PCI hotplug driver. It can
- be used to simulate PCI hotplug events if even if your system is
- not PCI hotplug capable.
-
- This driver will "emulate" removing PCI devices from the system.
- If the "power" file is written to with "0" then the specified PCI
- device will be completely removed from the kernel.
-
- WARNING, this does NOT turn off the power to the PCI device.
- This is a "logical" removal, not a physical or electrical
- removal.
-
- Use this module at your own risk. You have been warned!
-
- To compile this driver as a module, choose M here: the
- module will be called fakephp.
-
- When in doubt, say N.
-
config HOTPLUG_PCI_COMPAQ
tristate "Compaq PCI Hotplug driver"
depends on X86 && PCI_BIOS
@@ -143,7 +121,7 @@
config HOTPLUG_PCI_RPA
tristate "RPA PCI Hotplug driver"
- depends on PPC_PSERIES && EEH && !HOTPLUG_PCI_FAKE
+ depends on PPC_PSERIES && EEH
help
Say Y here if you have a RPA system that supports PCI Hotplug.
diff --git a/drivers/pci/hotplug/Makefile b/drivers/pci/hotplug/Makefile
index 6cd9f3c..c459cd4 100644
--- a/drivers/pci/hotplug/Makefile
+++ b/drivers/pci/hotplug/Makefile
@@ -23,9 +23,6 @@
obj-$(CONFIG_HOTPLUG_PCI_ACPI_IBM) += acpiphp_ibm.o
-# Link this last so it doesn't claim devices that have a real hotplug driver
-obj-$(CONFIG_HOTPLUG_PCI_FAKE) += fakephp.o
-
pci_hotplug-objs := pci_hotplug_core.o pcihp_slot.o
ifdef CONFIG_HOTPLUG_PCI_CPCI
diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
index ad6fd66..3d6d4fd 100644
--- a/drivers/pci/hotplug/acpiphp_glue.c
+++ b/drivers/pci/hotplug/acpiphp_glue.c
@@ -115,6 +115,35 @@
.handler = handle_hotplug_event_func,
};
+/* Check whether the PCI device is managed by native PCIe hotplug driver */
+static bool device_is_managed_by_native_pciehp(struct pci_dev *pdev)
+{
+ u32 reg32;
+ acpi_handle tmp;
+ struct acpi_pci_root *root;
+
+ /* Check whether the PCIe port supports native PCIe hotplug */
+ if (pcie_capability_read_dword(pdev, PCI_EXP_SLTCAP, ®32))
+ return false;
+ if (!(reg32 & PCI_EXP_SLTCAP_HPC))
+ return false;
+
+ /*
+ * Check whether native PCIe hotplug has been enabled for
+ * this PCIe hierarchy.
+ */
+ tmp = acpi_find_root_bridge_handle(pdev);
+ if (!tmp)
+ return false;
+ root = acpi_pci_find_root(tmp);
+ if (!root)
+ return false;
+ if (!(root->osc_control_set & OSC_PCI_EXPRESS_NATIVE_HP_CONTROL))
+ return false;
+
+ return true;
+}
+
/* callback routine to register each ACPI PCI slot object */
static acpi_status
register_slot(acpi_handle handle, u32 lvl, void *context, void **rv)
@@ -142,16 +171,8 @@
function = adr & 0xffff;
pdev = pbus->self;
- if (pdev && pci_is_pcie(pdev)) {
- tmp = acpi_find_root_bridge_handle(pdev);
- if (tmp) {
- struct acpi_pci_root *root = acpi_pci_find_root(tmp);
-
- if (root && (root->osc_control_set &
- OSC_PCI_EXPRESS_NATIVE_HP_CONTROL))
- return AE_OK;
- }
- }
+ if (pdev && device_is_managed_by_native_pciehp(pdev))
+ return AE_OK;
newfunc = kzalloc(sizeof(struct acpiphp_func), GFP_KERNEL);
if (!newfunc)
@@ -382,10 +403,10 @@
/* allocate and initialize host bridge data structure */
-static void add_host_bridge(acpi_handle *handle)
+static void add_host_bridge(struct acpi_pci_root *root)
{
struct acpiphp_bridge *bridge;
- struct acpi_pci_root *root = acpi_pci_find_root(handle);
+ acpi_handle handle = root->device->handle;
bridge = kzalloc(sizeof(struct acpiphp_bridge), GFP_KERNEL);
if (bridge == NULL)
@@ -468,11 +489,12 @@
/* find hot-pluggable slots, and then find P2P bridge */
-static int add_bridge(acpi_handle handle)
+static int add_bridge(struct acpi_pci_root *root)
{
acpi_status status;
unsigned long long tmp;
acpi_handle dummy_handle;
+ acpi_handle handle = root->device->handle;
/* if the bridge doesn't have _STA, we assume it is always there */
status = acpi_get_handle(handle, "_STA", &dummy_handle);
@@ -490,7 +512,7 @@
/* check if this bridge has ejectable slots */
if (detect_ejectable_slots(handle) > 0) {
dbg("found PCI host-bus bridge with hot-pluggable slots\n");
- add_host_bridge(handle);
+ add_host_bridge(root);
}
/* search P2P bridges under this host bridge */
@@ -588,9 +610,10 @@
return AE_OK;
}
-static void remove_bridge(acpi_handle handle)
+static void remove_bridge(struct acpi_pci_root *root)
{
struct acpiphp_bridge *bridge;
+ acpi_handle handle = root->device->handle;
/* cleanup p2p bridges under this host bridge
in a depth-first manner */
@@ -869,17 +892,6 @@
return retval;
}
-static void disable_bridges(struct pci_bus *bus)
-{
- struct pci_dev *dev;
- list_for_each_entry(dev, &bus->devices, bus_list) {
- if (dev->subordinate) {
- disable_bridges(dev->subordinate);
- pci_disable_device(dev);
- }
- }
-}
-
/* return first device in slot, acquiring a reference on it */
static struct pci_dev *dev_in_slot(struct acpiphp_slot *slot)
{
@@ -931,12 +943,7 @@
* here.
*/
while ((pdev = dev_in_slot(slot))) {
- pci_stop_bus_device(pdev);
- if (pdev->subordinate) {
- disable_bridges(pdev->subordinate);
- pci_disable_device(pdev);
- }
- __pci_remove_bus_device(pdev);
+ pci_stop_and_remove_bus_device(pdev);
pci_dev_put(pdev);
}
@@ -1477,34 +1484,6 @@
}
-#if 0
-/**
- * acpiphp_for_each_slot - call function for each slot
- * @fn: callback function
- * @data: context to be passed to callback function
- */
-static int acpiphp_for_each_slot(acpiphp_callback fn, void *data)
-{
- struct list_head *node;
- struct acpiphp_bridge *bridge;
- struct acpiphp_slot *slot;
- int retval = 0;
-
- list_for_each (node, &bridge_list) {
- bridge = (struct acpiphp_bridge *)node;
- for (slot = bridge->slots; slot; slot = slot->next) {
- retval = fn(slot, data);
- if (!retval)
- goto err_exit;
- }
- }
-
- err_exit:
- return retval;
-}
-#endif
-
-
/**
* acpiphp_enable_slot - power on slot
* @slot: ACPI PHP slot
diff --git a/drivers/pci/hotplug/cpcihp_generic.c b/drivers/pci/hotplug/cpcihp_generic.c
index 81af764..a6a71c4 100644
--- a/drivers/pci/hotplug/cpcihp_generic.c
+++ b/drivers/pci/hotplug/cpcihp_generic.c
@@ -154,12 +154,8 @@
if(!r)
return -EBUSY;
- bus = pci_find_bus(0, bridge_busnr);
- if (!bus) {
- err("Invalid bus number %d", bridge_busnr);
- return -EINVAL;
- }
- dev = pci_get_slot(bus, PCI_DEVFN(bridge_slot, 0));
+ dev = pci_get_domain_bus_and_slot(0, bridge_busnr,
+ PCI_DEVFN(bridge_slot, 0));
if(!dev || dev->hdr_type != PCI_HEADER_TYPE_BRIDGE) {
err("Invalid bridge device %s", bridge);
pci_dev_put(dev);
diff --git a/drivers/pci/hotplug/cpqphp_ctrl.c b/drivers/pci/hotplug/cpqphp_ctrl.c
index e43908d..36112fe 100644
--- a/drivers/pci/hotplug/cpqphp_ctrl.c
+++ b/drivers/pci/hotplug/cpqphp_ctrl.c
@@ -2890,27 +2890,8 @@
func->mem_head = mem_node;
} else
return -ENOMEM;
- } else if ((temp_register & 0x0BL) == 0x04) {
- /* Map memory */
- base = temp_register & 0xFFFFFFF0;
- base = ~base + 1;
-
- dbg("CND: length = 0x%x\n", base);
- mem_node = get_resource(&(resources->mem_head), base);
-
- /* allocate the resource to the board */
- if (mem_node) {
- base = mem_node->base;
-
- mem_node->next = func->mem_head;
- func->mem_head = mem_node;
- } else
- return -ENOMEM;
- } else if ((temp_register & 0x0BL) == 0x06) {
- /* Those bits are reserved, we can't handle this */
- return 1;
} else {
- /* Requesting space below 1M */
+ /* Reserved bits or requesting space below 1M */
return NOT_ENOUGH_RESOURCES;
}
diff --git a/drivers/pci/hotplug/fakephp.c b/drivers/pci/hotplug/fakephp.c
deleted file mode 100644
index a019c9a..0000000
--- a/drivers/pci/hotplug/fakephp.c
+++ /dev/null
@@ -1,164 +0,0 @@
-/* Works like the fakephp driver used to, except a little better.
- *
- * - It's possible to remove devices with subordinate busses.
- * - New PCI devices that appear via any method, not just a fakephp triggered
- * rescan, will be noticed.
- * - Devices that are removed via any method, not just a fakephp triggered
- * removal, will also be noticed.
- *
- * Uses nothing from the pci-hotplug subsystem.
- *
- */
-
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/list.h>
-#include <linux/kobject.h>
-#include <linux/sysfs.h>
-#include <linux/init.h>
-#include <linux/pci.h>
-#include <linux/device.h>
-#include <linux/slab.h>
-#include "../pci.h"
-
-struct legacy_slot {
- struct kobject kobj;
- struct pci_dev *dev;
- struct list_head list;
-};
-
-static LIST_HEAD(legacy_list);
-
-static ssize_t legacy_show(struct kobject *kobj, struct attribute *attr,
- char *buf)
-{
- struct legacy_slot *slot = container_of(kobj, typeof(*slot), kobj);
- strcpy(buf, "1\n");
- return 2;
-}
-
-static void remove_callback(void *data)
-{
- pci_stop_and_remove_bus_device((struct pci_dev *)data);
-}
-
-static ssize_t legacy_store(struct kobject *kobj, struct attribute *attr,
- const char *buf, size_t len)
-{
- struct legacy_slot *slot = container_of(kobj, typeof(*slot), kobj);
- unsigned long val;
-
- if (strict_strtoul(buf, 0, &val) < 0)
- return -EINVAL;
-
- if (val)
- pci_rescan_bus(slot->dev->bus);
- else
- sysfs_schedule_callback(&slot->dev->dev.kobj, remove_callback,
- slot->dev, THIS_MODULE);
- return len;
-}
-
-static struct attribute *legacy_attrs[] = {
- &(struct attribute){ .name = "power", .mode = 0644 },
- NULL,
-};
-
-static void legacy_release(struct kobject *kobj)
-{
- struct legacy_slot *slot = container_of(kobj, typeof(*slot), kobj);
-
- pci_dev_put(slot->dev);
- kfree(slot);
-}
-
-static struct kobj_type legacy_ktype = {
- .sysfs_ops = &(const struct sysfs_ops){
- .store = legacy_store, .show = legacy_show
- },
- .release = &legacy_release,
- .default_attrs = legacy_attrs,
-};
-
-static int legacy_add_slot(struct pci_dev *pdev)
-{
- struct legacy_slot *slot = kzalloc(sizeof(*slot), GFP_KERNEL);
-
- if (!slot)
- return -ENOMEM;
-
- if (kobject_init_and_add(&slot->kobj, &legacy_ktype,
- &pci_slots_kset->kobj, "%s",
- dev_name(&pdev->dev))) {
- dev_warn(&pdev->dev, "Failed to created legacy fake slot\n");
- return -EINVAL;
- }
- slot->dev = pci_dev_get(pdev);
-
- list_add(&slot->list, &legacy_list);
-
- return 0;
-}
-
-static int legacy_notify(struct notifier_block *nb,
- unsigned long action, void *data)
-{
- struct pci_dev *pdev = to_pci_dev(data);
-
- if (action == BUS_NOTIFY_ADD_DEVICE) {
- legacy_add_slot(pdev);
- } else if (action == BUS_NOTIFY_DEL_DEVICE) {
- struct legacy_slot *slot;
-
- list_for_each_entry(slot, &legacy_list, list)
- if (slot->dev == pdev)
- goto found;
-
- dev_warn(&pdev->dev, "Missing legacy fake slot?");
- return -ENODEV;
-found:
- kobject_del(&slot->kobj);
- list_del(&slot->list);
- kobject_put(&slot->kobj);
- }
-
- return 0;
-}
-
-static struct notifier_block legacy_notifier = {
- .notifier_call = legacy_notify
-};
-
-static int __init init_legacy(void)
-{
- struct pci_dev *pdev = NULL;
-
- /* Add existing devices */
- for_each_pci_dev(pdev)
- legacy_add_slot(pdev);
-
- /* Be alerted of any new ones */
- bus_register_notifier(&pci_bus_type, &legacy_notifier);
- return 0;
-}
-module_init(init_legacy);
-
-static void __exit remove_legacy(void)
-{
- struct legacy_slot *slot, *tmp;
-
- bus_unregister_notifier(&pci_bus_type, &legacy_notifier);
-
- list_for_each_entry_safe(slot, tmp, &legacy_list, list) {
- list_del(&slot->list);
- kobject_del(&slot->kobj);
- kobject_put(&slot->kobj);
- }
-}
-module_exit(remove_legacy);
-
-
-MODULE_AUTHOR("Trent Piepho <xyzzy@speakeasy.org>");
-MODULE_DESCRIPTION("Legacy version of the fakephp interface");
-MODULE_LICENSE("GPL");
diff --git a/drivers/pci/hotplug/pciehp_acpi.c b/drivers/pci/hotplug/pciehp_acpi.c
index 376d70d..24d709b 100644
--- a/drivers/pci/hotplug/pciehp_acpi.c
+++ b/drivers/pci/hotplug/pciehp_acpi.c
@@ -81,16 +81,12 @@
/* Dummy driver for dumplicate name detection */
static int __init dummy_probe(struct pcie_device *dev)
{
- int pos;
u32 slot_cap;
acpi_handle handle;
struct dummy_slot *slot, *tmp;
struct pci_dev *pdev = dev->port;
- pos = pci_pcie_cap(pdev);
- if (!pos)
- return -ENODEV;
- pci_read_config_dword(pdev, pos + PCI_EXP_SLTCAP, &slot_cap);
+ pcie_capability_read_dword(pdev, PCI_EXP_SLTCAP, &slot_cap);
slot = kzalloc(sizeof(*slot), GFP_KERNEL);
if (!slot)
return -ENOMEM;
diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c
index 365c6b9..916bf4f 100644
--- a/drivers/pci/hotplug/pciehp_core.c
+++ b/drivers/pci/hotplug/pciehp_core.c
@@ -300,24 +300,24 @@
static int pciehp_resume (struct pcie_device *dev)
{
+ struct controller *ctrl;
+ struct slot *slot;
+ u8 status;
+
dev_info(&dev->device, "%s ENTRY\n", __func__);
- if (pciehp_force) {
- struct controller *ctrl = get_service_data(dev);
- struct slot *slot;
- u8 status;
+ ctrl = get_service_data(dev);
- /* reinitialize the chipset's event detection logic */
- pcie_enable_notification(ctrl);
+ /* reinitialize the chipset's event detection logic */
+ pcie_enable_notification(ctrl);
- slot = ctrl->slot;
+ slot = ctrl->slot;
- /* Check if slot is occupied */
- pciehp_get_adapter_status(slot, &status);
- if (status)
- pciehp_enable_slot(slot);
- else
- pciehp_disable_slot(slot);
- }
+ /* Check if slot is occupied */
+ pciehp_get_adapter_status(slot, &status);
+ if (status)
+ pciehp_enable_slot(slot);
+ else
+ pciehp_disable_slot(slot);
return 0;
}
#endif /* PM */
diff --git a/drivers/pci/hotplug/pciehp_hpc.c b/drivers/pci/hotplug/pciehp_hpc.c
index 302451e..13b2eaf 100644
--- a/drivers/pci/hotplug/pciehp_hpc.c
+++ b/drivers/pci/hotplug/pciehp_hpc.c
@@ -44,25 +44,25 @@
static inline int pciehp_readw(struct controller *ctrl, int reg, u16 *value)
{
struct pci_dev *dev = ctrl->pcie->port;
- return pci_read_config_word(dev, pci_pcie_cap(dev) + reg, value);
+ return pcie_capability_read_word(dev, reg, value);
}
static inline int pciehp_readl(struct controller *ctrl, int reg, u32 *value)
{
struct pci_dev *dev = ctrl->pcie->port;
- return pci_read_config_dword(dev, pci_pcie_cap(dev) + reg, value);
+ return pcie_capability_read_dword(dev, reg, value);
}
static inline int pciehp_writew(struct controller *ctrl, int reg, u16 value)
{
struct pci_dev *dev = ctrl->pcie->port;
- return pci_write_config_word(dev, pci_pcie_cap(dev) + reg, value);
+ return pcie_capability_write_word(dev, reg, value);
}
static inline int pciehp_writel(struct controller *ctrl, int reg, u32 value)
{
struct pci_dev *dev = ctrl->pcie->port;
- return pci_write_config_dword(dev, pci_pcie_cap(dev) + reg, value);
+ return pcie_capability_write_dword(dev, reg, value);
}
/* Power Control Command */
@@ -855,10 +855,6 @@
goto abort;
}
ctrl->pcie = dev;
- if (!pci_pcie_cap(pdev)) {
- ctrl_err(ctrl, "Cannot find PCI Express capability\n");
- goto abort_ctrl;
- }
if (pciehp_readl(ctrl, PCI_EXP_SLTCAP, &slot_cap)) {
ctrl_err(ctrl, "Cannot read SLOTCAP register\n");
goto abort_ctrl;
diff --git a/drivers/pci/hotplug/pcihp_slot.c b/drivers/pci/hotplug/pcihp_slot.c
index 8c05a18..fec2d5b 100644
--- a/drivers/pci/hotplug/pcihp_slot.c
+++ b/drivers/pci/hotplug/pcihp_slot.c
@@ -96,17 +96,11 @@
static void program_hpp_type2(struct pci_dev *dev, struct hpp_type2 *hpp)
{
int pos;
- u16 reg16;
u32 reg32;
if (!hpp)
return;
- /* Find PCI Express capability */
- pos = pci_pcie_cap(dev);
- if (!pos)
- return;
-
if (hpp->revision > 1) {
dev_warn(&dev->dev, "PCIe settings rev %d not supported\n",
hpp->revision);
@@ -114,17 +108,13 @@
}
/* Initialize Device Control Register */
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, ®16);
- reg16 = (reg16 & hpp->pci_exp_devctl_and) | hpp->pci_exp_devctl_or;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, reg16);
+ pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL,
+ ~hpp->pci_exp_devctl_and, hpp->pci_exp_devctl_or);
/* Initialize Link Control Register */
- if (dev->subordinate) {
- pci_read_config_word(dev, pos + PCI_EXP_LNKCTL, ®16);
- reg16 = (reg16 & hpp->pci_exp_lnkctl_and)
- | hpp->pci_exp_lnkctl_or;
- pci_write_config_word(dev, pos + PCI_EXP_LNKCTL, reg16);
- }
+ if (dev->subordinate)
+ pcie_capability_clear_and_set_word(dev, PCI_EXP_LNKCTL,
+ ~hpp->pci_exp_lnkctl_and, hpp->pci_exp_lnkctl_or);
/* Find Advanced Error Reporting Enhanced Capability */
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index 74bbaf8..aeccc91 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -433,8 +433,8 @@
struct resource *res;
struct pci_dev *pdev;
- if (dev->pcie_type != PCI_EXP_TYPE_RC_END &&
- dev->pcie_type != PCI_EXP_TYPE_ENDPOINT)
+ if (pci_pcie_type(dev) != PCI_EXP_TYPE_RC_END &&
+ pci_pcie_type(dev) != PCI_EXP_TYPE_ENDPOINT)
return -ENODEV;
pci_read_config_word(dev, pos + PCI_SRIOV_CTRL, &ctrl);
@@ -503,7 +503,7 @@
iov->self = dev;
pci_read_config_dword(dev, pos + PCI_SRIOV_CAP, &iov->cap);
pci_read_config_byte(dev, pos + PCI_SRIOV_FUNC_LINK, &iov->link);
- if (dev->pcie_type == PCI_EXP_TYPE_RC_END)
+ if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END)
iov->link = PCI_DEVFN(PCI_SLOT(dev->devfn), iov->link);
if (pdev)
diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index 5270f1a..9e1d295 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -139,7 +139,6 @@
return retval;
return count;
}
-static DRIVER_ATTR(new_id, S_IWUSR, NULL, store_new_id);
/**
* store_remove_id - remove a PCI device ID from this driver
@@ -185,38 +184,16 @@
return retval;
return count;
}
-static DRIVER_ATTR(remove_id, S_IWUSR, NULL, store_remove_id);
-static int
-pci_create_newid_files(struct pci_driver *drv)
-{
- int error = 0;
+static struct driver_attribute pci_drv_attrs[] = {
+ __ATTR(new_id, S_IWUSR, NULL, store_new_id),
+ __ATTR(remove_id, S_IWUSR, NULL, store_remove_id),
+ __ATTR_NULL,
+};
- if (drv->probe != NULL) {
- error = driver_create_file(&drv->driver, &driver_attr_new_id);
- if (error == 0) {
- error = driver_create_file(&drv->driver,
- &driver_attr_remove_id);
- if (error)
- driver_remove_file(&drv->driver,
- &driver_attr_new_id);
- }
- }
- return error;
-}
-
-static void pci_remove_newid_files(struct pci_driver *drv)
-{
- driver_remove_file(&drv->driver, &driver_attr_remove_id);
- driver_remove_file(&drv->driver, &driver_attr_new_id);
-}
-#else /* !CONFIG_HOTPLUG */
-static inline int pci_create_newid_files(struct pci_driver *drv)
-{
- return 0;
-}
-static inline void pci_remove_newid_files(struct pci_driver *drv) {}
-#endif
+#else
+#define pci_drv_attrs NULL
+#endif /* CONFIG_HOTPLUG */
/**
* pci_match_id - See if a pci device matches a given pci_id table
@@ -280,8 +257,12 @@
{
struct drv_dev_and_id *ddi = _ddi;
struct device *dev = &ddi->dev->dev;
+ struct device *parent = dev->parent;
int rc;
+ /* The parent bridge must be in active state when probing */
+ if (parent)
+ pm_runtime_get_sync(parent);
/* Unbound PCI devices are always set to disabled and suspended.
* During probe, the device is set to enabled and active and the
* usage count is incremented. If the driver supports runtime PM,
@@ -298,6 +279,8 @@
pm_runtime_set_suspended(dev);
pm_runtime_put_noidle(dev);
}
+ if (parent)
+ pm_runtime_put(parent);
return rc;
}
@@ -1156,8 +1139,6 @@
int __pci_register_driver(struct pci_driver *drv, struct module *owner,
const char *mod_name)
{
- int error;
-
/* initialize common driver fields */
drv->driver.name = drv->name;
drv->driver.bus = &pci_bus_type;
@@ -1168,19 +1149,7 @@
INIT_LIST_HEAD(&drv->dynids.list);
/* register with core */
- error = driver_register(&drv->driver);
- if (error)
- goto out;
-
- error = pci_create_newid_files(drv);
- if (error)
- goto out_newid;
-out:
- return error;
-
-out_newid:
- driver_unregister(&drv->driver);
- goto out;
+ return driver_register(&drv->driver);
}
/**
@@ -1196,7 +1165,6 @@
void
pci_unregister_driver(struct pci_driver *drv)
{
- pci_remove_newid_files(drv);
driver_unregister(&drv->driver);
pci_free_dynids(drv);
}
@@ -1296,6 +1264,7 @@
.shutdown = pci_device_shutdown,
.dev_attrs = pci_dev_attrs,
.bus_attrs = pci_bus_attrs,
+ .drv_attrs = pci_drv_attrs,
.pm = PCI_PM_OPS_PTR,
};
diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c
index 6869009..02d107b 100644
--- a/drivers/pci/pci-sysfs.c
+++ b/drivers/pci/pci-sysfs.c
@@ -458,6 +458,40 @@
}
struct device_attribute vga_attr = __ATTR_RO(boot_vga);
+static void
+pci_config_pm_runtime_get(struct pci_dev *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device *parent = dev->parent;
+
+ if (parent)
+ pm_runtime_get_sync(parent);
+ pm_runtime_get_noresume(dev);
+ /*
+ * pdev->current_state is set to PCI_D3cold during suspending,
+ * so wait until suspending completes
+ */
+ pm_runtime_barrier(dev);
+ /*
+ * Only need to resume devices in D3cold, because config
+ * registers are still accessible for devices suspended but
+ * not in D3cold.
+ */
+ if (pdev->current_state == PCI_D3cold)
+ pm_runtime_resume(dev);
+}
+
+static void
+pci_config_pm_runtime_put(struct pci_dev *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device *parent = dev->parent;
+
+ pm_runtime_put(dev);
+ if (parent)
+ pm_runtime_put_sync(parent);
+}
+
static ssize_t
pci_read_config(struct file *filp, struct kobject *kobj,
struct bin_attribute *bin_attr,
@@ -484,6 +518,8 @@
size = count;
}
+ pci_config_pm_runtime_get(dev);
+
if ((off & 1) && size) {
u8 val;
pci_user_read_config_byte(dev, off, &val);
@@ -529,6 +565,8 @@
--size;
}
+ pci_config_pm_runtime_put(dev);
+
return count;
}
@@ -549,6 +587,8 @@
count = size;
}
+ pci_config_pm_runtime_get(dev);
+
if ((off & 1) && size) {
pci_user_write_config_byte(dev, off, data[off - init_off]);
off++;
@@ -587,6 +627,8 @@
--size;
}
+ pci_config_pm_runtime_put(dev);
+
return count;
}
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index f3ea977..5485883 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -254,36 +254,56 @@
}
/**
- * pci_pcie_cap2 - query for devices' PCI_CAP_ID_EXP v2 capability structure
- * @dev: PCI device to check
+ * pci_find_next_ext_capability - Find an extended capability
+ * @dev: PCI device to query
+ * @start: address at which to start looking (0 to start at beginning of list)
+ * @cap: capability code
*
- * Like pci_pcie_cap() but also checks that the PCIe capability version is
- * >= 2. Note that v1 capability structures could be sparse in that not
- * all register fields were required. v2 requires the entire structure to
- * be present size wise, while still allowing for non-implemented registers
- * to exist but they must be hardwired to 0.
- *
- * Due to the differences in the versions of capability structures, one
- * must be careful not to try and access non-existant registers that may
- * exist in early versions - v1 - of Express devices.
- *
- * Returns the offset of the PCIe capability structure as long as the
- * capability version is >= 2; otherwise 0 is returned.
+ * Returns the address of the next matching extended capability structure
+ * within the device's PCI configuration space or 0 if the device does
+ * not support it. Some capabilities can occur several times, e.g., the
+ * vendor-specific capability, and this provides a way to find them all.
*/
-static int pci_pcie_cap2(struct pci_dev *dev)
+int pci_find_next_ext_capability(struct pci_dev *dev, int start, int cap)
{
- u16 flags;
- int pos;
+ u32 header;
+ int ttl;
+ int pos = PCI_CFG_SPACE_SIZE;
- pos = pci_pcie_cap(dev);
- if (pos) {
- pci_read_config_word(dev, pos + PCI_EXP_FLAGS, &flags);
- if ((flags & PCI_EXP_FLAGS_VERS) < 2)
- pos = 0;
+ /* minimum 8 bytes per capability */
+ ttl = (PCI_CFG_SPACE_EXP_SIZE - PCI_CFG_SPACE_SIZE) / 8;
+
+ if (dev->cfg_size <= PCI_CFG_SPACE_SIZE)
+ return 0;
+
+ if (start)
+ pos = start;
+
+ if (pci_read_config_dword(dev, pos, &header) != PCIBIOS_SUCCESSFUL)
+ return 0;
+
+ /*
+ * If we have no capabilities, this is indicated by cap ID,
+ * cap version and next pointer all being 0.
+ */
+ if (header == 0)
+ return 0;
+
+ while (ttl-- > 0) {
+ if (PCI_EXT_CAP_ID(header) == cap && pos != start)
+ return pos;
+
+ pos = PCI_EXT_CAP_NEXT(header);
+ if (pos < PCI_CFG_SPACE_SIZE)
+ break;
+
+ if (pci_read_config_dword(dev, pos, &header) != PCIBIOS_SUCCESSFUL)
+ break;
}
- return pos;
+ return 0;
}
+EXPORT_SYMBOL_GPL(pci_find_next_ext_capability);
/**
* pci_find_ext_capability - Find an extended capability
@@ -301,39 +321,7 @@
*/
int pci_find_ext_capability(struct pci_dev *dev, int cap)
{
- u32 header;
- int ttl;
- int pos = PCI_CFG_SPACE_SIZE;
-
- /* minimum 8 bytes per capability */
- ttl = (PCI_CFG_SPACE_EXP_SIZE - PCI_CFG_SPACE_SIZE) / 8;
-
- if (dev->cfg_size <= PCI_CFG_SPACE_SIZE)
- return 0;
-
- if (pci_read_config_dword(dev, pos, &header) != PCIBIOS_SUCCESSFUL)
- return 0;
-
- /*
- * If we have no capabilities, this is indicated by cap ID,
- * cap version and next pointer all being 0.
- */
- if (header == 0)
- return 0;
-
- while (ttl-- > 0) {
- if (PCI_EXT_CAP_ID(header) == cap)
- return pos;
-
- pos = PCI_EXT_CAP_NEXT(header);
- if (pos < PCI_CFG_SPACE_SIZE)
- break;
-
- if (pci_read_config_dword(dev, pos, &header) != PCIBIOS_SUCCESSFUL)
- break;
- }
-
- return 0;
+ return pci_find_next_ext_capability(dev, 0, cap);
}
EXPORT_SYMBOL_GPL(pci_find_ext_capability);
@@ -854,21 +842,6 @@
#define PCI_EXP_SAVE_REGS 7
-#define pcie_cap_has_devctl(type, flags) 1
-#define pcie_cap_has_lnkctl(type, flags) \
- ((flags & PCI_EXP_FLAGS_VERS) > 1 || \
- (type == PCI_EXP_TYPE_ROOT_PORT || \
- type == PCI_EXP_TYPE_ENDPOINT || \
- type == PCI_EXP_TYPE_LEG_END))
-#define pcie_cap_has_sltctl(type, flags) \
- ((flags & PCI_EXP_FLAGS_VERS) > 1 || \
- ((type == PCI_EXP_TYPE_ROOT_PORT) || \
- (type == PCI_EXP_TYPE_DOWNSTREAM && \
- (flags & PCI_EXP_FLAGS_SLOT))))
-#define pcie_cap_has_rtctl(type, flags) \
- ((flags & PCI_EXP_FLAGS_VERS) > 1 || \
- (type == PCI_EXP_TYPE_ROOT_PORT || \
- type == PCI_EXP_TYPE_RC_EC))
static struct pci_cap_saved_state *pci_find_saved_cap(
struct pci_dev *pci_dev, char cap)
@@ -885,13 +858,11 @@
static int pci_save_pcie_state(struct pci_dev *dev)
{
- int pos, i = 0;
+ int i = 0;
struct pci_cap_saved_state *save_state;
u16 *cap;
- u16 flags;
- pos = pci_pcie_cap(dev);
- if (!pos)
+ if (!pci_is_pcie(dev))
return 0;
save_state = pci_find_saved_cap(dev, PCI_CAP_ID_EXP);
@@ -899,60 +870,37 @@
dev_err(&dev->dev, "buffer not found in %s\n", __func__);
return -ENOMEM;
}
+
cap = (u16 *)&save_state->cap.data[0];
+ pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &cap[i++]);
+ pcie_capability_read_word(dev, PCI_EXP_LNKCTL, &cap[i++]);
+ pcie_capability_read_word(dev, PCI_EXP_SLTCTL, &cap[i++]);
+ pcie_capability_read_word(dev, PCI_EXP_RTCTL, &cap[i++]);
+ pcie_capability_read_word(dev, PCI_EXP_DEVCTL2, &cap[i++]);
+ pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &cap[i++]);
+ pcie_capability_read_word(dev, PCI_EXP_SLTCTL2, &cap[i++]);
- pci_read_config_word(dev, pos + PCI_EXP_FLAGS, &flags);
-
- if (pcie_cap_has_devctl(dev->pcie_type, flags))
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &cap[i++]);
- if (pcie_cap_has_lnkctl(dev->pcie_type, flags))
- pci_read_config_word(dev, pos + PCI_EXP_LNKCTL, &cap[i++]);
- if (pcie_cap_has_sltctl(dev->pcie_type, flags))
- pci_read_config_word(dev, pos + PCI_EXP_SLTCTL, &cap[i++]);
- if (pcie_cap_has_rtctl(dev->pcie_type, flags))
- pci_read_config_word(dev, pos + PCI_EXP_RTCTL, &cap[i++]);
-
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return 0;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &cap[i++]);
- pci_read_config_word(dev, pos + PCI_EXP_LNKCTL2, &cap[i++]);
- pci_read_config_word(dev, pos + PCI_EXP_SLTCTL2, &cap[i++]);
return 0;
}
static void pci_restore_pcie_state(struct pci_dev *dev)
{
- int i = 0, pos;
+ int i = 0;
struct pci_cap_saved_state *save_state;
u16 *cap;
- u16 flags;
save_state = pci_find_saved_cap(dev, PCI_CAP_ID_EXP);
- pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
- if (!save_state || pos <= 0)
+ if (!save_state)
return;
+
cap = (u16 *)&save_state->cap.data[0];
-
- pci_read_config_word(dev, pos + PCI_EXP_FLAGS, &flags);
-
- if (pcie_cap_has_devctl(dev->pcie_type, flags))
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, cap[i++]);
- if (pcie_cap_has_lnkctl(dev->pcie_type, flags))
- pci_write_config_word(dev, pos + PCI_EXP_LNKCTL, cap[i++]);
- if (pcie_cap_has_sltctl(dev->pcie_type, flags))
- pci_write_config_word(dev, pos + PCI_EXP_SLTCTL, cap[i++]);
- if (pcie_cap_has_rtctl(dev->pcie_type, flags))
- pci_write_config_word(dev, pos + PCI_EXP_RTCTL, cap[i++]);
-
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return;
-
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, cap[i++]);
- pci_write_config_word(dev, pos + PCI_EXP_LNKCTL2, cap[i++]);
- pci_write_config_word(dev, pos + PCI_EXP_SLTCTL2, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_DEVCTL, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_LNKCTL, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_SLTCTL, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_RTCTL, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_DEVCTL2, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_LNKCTL2, cap[i++]);
+ pcie_capability_write_word(dev, PCI_EXP_SLTCTL2, cap[i++]);
}
@@ -1543,7 +1491,7 @@
/**
* pci_wakeup - Wake up a PCI device
- * @dev: Device to handle.
+ * @pci_dev: Device to handle.
* @ign: ignored parameter
*/
static int pci_wakeup(struct pci_dev *pci_dev, void *ign)
@@ -1941,6 +1889,7 @@
dev->pm_cap = pm;
dev->d3_delay = PCI_PM_D3_WAIT;
dev->d3cold_delay = PCI_PM_D3COLD_WAIT;
+ dev->d3cold_allowed = true;
dev->d1_support = false;
dev->d2_support = false;
@@ -2066,35 +2015,24 @@
*/
void pci_enable_ari(struct pci_dev *dev)
{
- int pos;
u32 cap;
- u16 ctrl;
struct pci_dev *bridge;
if (pcie_ari_disabled || !pci_is_pcie(dev) || dev->devfn)
return;
- pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ARI);
- if (!pos)
+ if (!pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ARI))
return;
bridge = dev->bus->self;
if (!bridge)
return;
- /* ARI is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(bridge);
- if (!pos)
- return;
-
- pci_read_config_dword(bridge, pos + PCI_EXP_DEVCAP2, &cap);
+ pcie_capability_read_dword(bridge, PCI_EXP_DEVCAP2, &cap);
if (!(cap & PCI_EXP_DEVCAP2_ARI))
return;
- pci_read_config_word(bridge, pos + PCI_EXP_DEVCTL2, &ctrl);
- ctrl |= PCI_EXP_DEVCTL2_ARI;
- pci_write_config_word(bridge, pos + PCI_EXP_DEVCTL2, ctrl);
-
+ pcie_capability_set_word(bridge, PCI_EXP_DEVCTL2, PCI_EXP_DEVCTL2_ARI);
bridge->ari_enabled = 1;
}
@@ -2109,20 +2047,14 @@
*/
void pci_enable_ido(struct pci_dev *dev, unsigned long type)
{
- int pos;
- u16 ctrl;
+ u16 ctrl = 0;
- /* ID-based Ordering is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &ctrl);
if (type & PCI_EXP_IDO_REQUEST)
ctrl |= PCI_EXP_IDO_REQ_EN;
if (type & PCI_EXP_IDO_COMPLETION)
ctrl |= PCI_EXP_IDO_CMP_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, ctrl);
+ if (ctrl)
+ pcie_capability_set_word(dev, PCI_EXP_DEVCTL2, ctrl);
}
EXPORT_SYMBOL(pci_enable_ido);
@@ -2133,20 +2065,14 @@
*/
void pci_disable_ido(struct pci_dev *dev, unsigned long type)
{
- int pos;
- u16 ctrl;
+ u16 ctrl = 0;
- /* ID-based Ordering is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &ctrl);
if (type & PCI_EXP_IDO_REQUEST)
- ctrl &= ~PCI_EXP_IDO_REQ_EN;
+ ctrl |= PCI_EXP_IDO_REQ_EN;
if (type & PCI_EXP_IDO_COMPLETION)
- ctrl &= ~PCI_EXP_IDO_CMP_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, ctrl);
+ ctrl |= PCI_EXP_IDO_CMP_EN;
+ if (ctrl)
+ pcie_capability_clear_word(dev, PCI_EXP_DEVCTL2, ctrl);
}
EXPORT_SYMBOL(pci_disable_ido);
@@ -2171,17 +2097,11 @@
*/
int pci_enable_obff(struct pci_dev *dev, enum pci_obff_signal_type type)
{
- int pos;
u32 cap;
u16 ctrl;
int ret;
- /* OBFF is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return -ENOTSUPP;
-
- pci_read_config_dword(dev, pos + PCI_EXP_DEVCAP2, &cap);
+ pcie_capability_read_dword(dev, PCI_EXP_DEVCAP2, &cap);
if (!(cap & PCI_EXP_OBFF_MASK))
return -ENOTSUPP; /* no OBFF support at all */
@@ -2192,7 +2112,7 @@
return ret;
}
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &ctrl);
+ pcie_capability_read_word(dev, PCI_EXP_DEVCTL2, &ctrl);
if (cap & PCI_EXP_OBFF_WAKE)
ctrl |= PCI_EXP_OBFF_WAKE_EN;
else {
@@ -2210,7 +2130,7 @@
return -ENOTSUPP;
}
}
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, ctrl);
+ pcie_capability_write_word(dev, PCI_EXP_DEVCTL2, ctrl);
return 0;
}
@@ -2224,17 +2144,7 @@
*/
void pci_disable_obff(struct pci_dev *dev)
{
- int pos;
- u16 ctrl;
-
- /* OBFF is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &ctrl);
- ctrl &= ~PCI_EXP_OBFF_WAKE_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, ctrl);
+ pcie_capability_clear_word(dev, PCI_EXP_DEVCTL2, PCI_EXP_OBFF_WAKE_EN);
}
EXPORT_SYMBOL(pci_disable_obff);
@@ -2247,15 +2157,9 @@
*/
static bool pci_ltr_supported(struct pci_dev *dev)
{
- int pos;
u32 cap;
- /* LTR is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return false;
-
- pci_read_config_dword(dev, pos + PCI_EXP_DEVCAP2, &cap);
+ pcie_capability_read_dword(dev, PCI_EXP_DEVCAP2, &cap);
return cap & PCI_EXP_DEVCAP2_LTR;
}
@@ -2272,22 +2176,15 @@
*/
int pci_enable_ltr(struct pci_dev *dev)
{
- int pos;
- u16 ctrl;
int ret;
- if (!pci_ltr_supported(dev))
- return -ENOTSUPP;
-
- /* LTR is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return -ENOTSUPP;
-
/* Only primary function can enable/disable LTR */
if (PCI_FUNC(dev->devfn) != 0)
return -EINVAL;
+ if (!pci_ltr_supported(dev))
+ return -ENOTSUPP;
+
/* Enable upstream ports first */
if (dev->bus->self) {
ret = pci_enable_ltr(dev->bus->self);
@@ -2295,11 +2192,7 @@
return ret;
}
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &ctrl);
- ctrl |= PCI_EXP_LTR_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, ctrl);
-
- return 0;
+ return pcie_capability_set_word(dev, PCI_EXP_DEVCTL2, PCI_EXP_LTR_EN);
}
EXPORT_SYMBOL(pci_enable_ltr);
@@ -2309,24 +2202,14 @@
*/
void pci_disable_ltr(struct pci_dev *dev)
{
- int pos;
- u16 ctrl;
-
- if (!pci_ltr_supported(dev))
- return;
-
- /* LTR is a PCIe cap v2 feature */
- pos = pci_pcie_cap2(dev);
- if (!pos)
- return;
-
/* Only primary function can enable/disable LTR */
if (PCI_FUNC(dev->devfn) != 0)
return;
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL2, &ctrl);
- ctrl &= ~PCI_EXP_LTR_EN;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL2, ctrl);
+ if (!pci_ltr_supported(dev))
+ return;
+
+ pcie_capability_clear_word(dev, PCI_EXP_DEVCTL2, PCI_EXP_LTR_EN);
}
EXPORT_SYMBOL(pci_disable_ltr);
@@ -2409,9 +2292,6 @@
if (!pci_acs_enable)
return;
- if (!pci_is_pcie(dev))
- return;
-
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS);
if (!pos)
return;
@@ -2459,8 +2339,8 @@
acs_flags &= (PCI_ACS_RR | PCI_ACS_CR |
PCI_ACS_EC | PCI_ACS_DT);
- if (pdev->pcie_type == PCI_EXP_TYPE_DOWNSTREAM ||
- pdev->pcie_type == PCI_EXP_TYPE_ROOT_PORT ||
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_DOWNSTREAM ||
+ pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT ||
pdev->multifunction) {
pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_ACS);
if (!pos)
@@ -3176,15 +3056,10 @@
static int pcie_flr(struct pci_dev *dev, int probe)
{
int i;
- int pos;
u32 cap;
- u16 status, control;
+ u16 status;
- pos = pci_pcie_cap(dev);
- if (!pos)
- return -ENOTTY;
-
- pci_read_config_dword(dev, pos + PCI_EXP_DEVCAP, &cap);
+ pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &cap);
if (!(cap & PCI_EXP_DEVCAP_FLR))
return -ENOTTY;
@@ -3196,7 +3071,7 @@
if (i)
msleep((1 << (i - 1)) * 100);
- pci_read_config_word(dev, pos + PCI_EXP_DEVSTA, &status);
+ pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &status);
if (!(status & PCI_EXP_DEVSTA_TRPND))
goto clear;
}
@@ -3205,9 +3080,7 @@
"proceeding with reset anyway\n");
clear:
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &control);
- control |= PCI_EXP_DEVCTL_BCR_FLR;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, control);
+ pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR);
msleep(100);
@@ -3575,18 +3448,11 @@
*/
int pcie_get_readrq(struct pci_dev *dev)
{
- int ret, cap;
u16 ctl;
- cap = pci_pcie_cap(dev);
- if (!cap)
- return -EINVAL;
+ pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &ctl);
- ret = pci_read_config_word(dev, cap + PCI_EXP_DEVCTL, &ctl);
- if (!ret)
- ret = 128 << ((ctl & PCI_EXP_DEVCTL_READRQ) >> 12);
-
- return ret;
+ return 128 << ((ctl & PCI_EXP_DEVCTL_READRQ) >> 12);
}
EXPORT_SYMBOL(pcie_get_readrq);
@@ -3600,19 +3466,11 @@
*/
int pcie_set_readrq(struct pci_dev *dev, int rq)
{
- int cap, err = -EINVAL;
- u16 ctl, v;
+ u16 v;
if (rq < 128 || rq > 4096 || !is_power_of_2(rq))
- goto out;
+ return -EINVAL;
- cap = pci_pcie_cap(dev);
- if (!cap)
- goto out;
-
- err = pci_read_config_word(dev, cap + PCI_EXP_DEVCTL, &ctl);
- if (err)
- goto out;
/*
* If using the "performance" PCIe config, we clamp the
* read rq size to the max packet size to prevent the
@@ -3630,14 +3488,8 @@
v = (ffs(rq) - 8) << 12;
- if ((ctl & PCI_EXP_DEVCTL_READRQ) != v) {
- ctl &= ~PCI_EXP_DEVCTL_READRQ;
- ctl |= v;
- err = pci_write_config_word(dev, cap + PCI_EXP_DEVCTL, ctl);
- }
-
-out:
- return err;
+ return pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_READRQ, v);
}
EXPORT_SYMBOL(pcie_set_readrq);
@@ -3650,18 +3502,11 @@
*/
int pcie_get_mps(struct pci_dev *dev)
{
- int ret, cap;
u16 ctl;
- cap = pci_pcie_cap(dev);
- if (!cap)
- return -EINVAL;
+ pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &ctl);
- ret = pci_read_config_word(dev, cap + PCI_EXP_DEVCTL, &ctl);
- if (!ret)
- ret = 128 << ((ctl & PCI_EXP_DEVCTL_PAYLOAD) >> 5);
-
- return ret;
+ return 128 << ((ctl & PCI_EXP_DEVCTL_PAYLOAD) >> 5);
}
/**
@@ -3674,32 +3519,18 @@
*/
int pcie_set_mps(struct pci_dev *dev, int mps)
{
- int cap, err = -EINVAL;
- u16 ctl, v;
+ u16 v;
if (mps < 128 || mps > 4096 || !is_power_of_2(mps))
- goto out;
+ return -EINVAL;
v = ffs(mps) - 8;
if (v > dev->pcie_mpss)
- goto out;
+ return -EINVAL;
v <<= 5;
- cap = pci_pcie_cap(dev);
- if (!cap)
- goto out;
-
- err = pci_read_config_word(dev, cap + PCI_EXP_DEVCTL, &ctl);
- if (err)
- goto out;
-
- if ((ctl & PCI_EXP_DEVCTL_PAYLOAD) != v) {
- ctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
- ctl |= v;
- err = pci_write_config_word(dev, cap + PCI_EXP_DEVCTL, ctl);
- }
-out:
- return err;
+ return pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_PAYLOAD, v);
}
/**
diff --git a/drivers/pci/pcie/aer/aer_inject.c b/drivers/pci/pcie/aer/aer_inject.c
index 5222986..4e24cb8 100644
--- a/drivers/pci/pcie/aer/aer_inject.c
+++ b/drivers/pci/pcie/aer/aer_inject.c
@@ -288,7 +288,7 @@
while (1) {
if (!pci_is_pcie(dev))
break;
- if (dev->pcie_type == PCI_EXP_TYPE_ROOT_PORT)
+ if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT)
return dev;
if (!dev->bus->self)
break;
diff --git a/drivers/pci/pcie/aer/aerdrv.c b/drivers/pci/pcie/aer/aerdrv.c
index 58ad791..030cf12 100644
--- a/drivers/pci/pcie/aer/aerdrv.c
+++ b/drivers/pci/pcie/aer/aerdrv.c
@@ -48,7 +48,7 @@
static void aer_error_resume(struct pci_dev *dev);
static pci_ers_result_t aer_root_reset(struct pci_dev *dev);
-static struct pci_error_handlers aer_error_handlers = {
+static const struct pci_error_handlers aer_error_handlers = {
.error_detected = aer_error_detected,
.resume = aer_error_resume,
};
@@ -81,10 +81,11 @@
static int set_device_error_reporting(struct pci_dev *dev, void *data)
{
bool enable = *((bool *)data);
+ int type = pci_pcie_type(dev);
- if ((dev->pcie_type == PCI_EXP_TYPE_ROOT_PORT) ||
- (dev->pcie_type == PCI_EXP_TYPE_UPSTREAM) ||
- (dev->pcie_type == PCI_EXP_TYPE_DOWNSTREAM)) {
+ if ((type == PCI_EXP_TYPE_ROOT_PORT) ||
+ (type == PCI_EXP_TYPE_UPSTREAM) ||
+ (type == PCI_EXP_TYPE_DOWNSTREAM)) {
if (enable)
pci_enable_pcie_error_reporting(dev);
else
@@ -121,19 +122,17 @@
static void aer_enable_rootport(struct aer_rpc *rpc)
{
struct pci_dev *pdev = rpc->rpd->port;
- int pos, aer_pos;
+ int aer_pos;
u16 reg16;
u32 reg32;
- pos = pci_pcie_cap(pdev);
/* Clear PCIe Capability's Device Status */
- pci_read_config_word(pdev, pos+PCI_EXP_DEVSTA, ®16);
- pci_write_config_word(pdev, pos+PCI_EXP_DEVSTA, reg16);
+ pcie_capability_read_word(pdev, PCI_EXP_DEVSTA, ®16);
+ pcie_capability_write_word(pdev, PCI_EXP_DEVSTA, reg16);
/* Disable system error generation in response to error messages */
- pci_read_config_word(pdev, pos + PCI_EXP_RTCTL, ®16);
- reg16 &= ~(SYSTEM_ERROR_INTR_ON_MESG_MASK);
- pci_write_config_word(pdev, pos + PCI_EXP_RTCTL, reg16);
+ pcie_capability_clear_word(pdev, PCI_EXP_RTCTL,
+ SYSTEM_ERROR_INTR_ON_MESG_MASK);
aer_pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_ERR);
/* Clear error status */
@@ -395,9 +394,8 @@
u16 reg16;
/* Clean up Root device status */
- pos = pci_pcie_cap(dev);
- pci_read_config_word(dev, pos + PCI_EXP_DEVSTA, ®16);
- pci_write_config_word(dev, pos + PCI_EXP_DEVSTA, reg16);
+ pcie_capability_read_word(dev, PCI_EXP_DEVSTA, ®16);
+ pcie_capability_write_word(dev, PCI_EXP_DEVSTA, reg16);
/* Clean AER Root Error Status */
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);
diff --git a/drivers/pci/pcie/aer/aerdrv_acpi.c b/drivers/pci/pcie/aer/aerdrv_acpi.c
index 124f20f..5194a7d 100644
--- a/drivers/pci/pcie/aer/aerdrv_acpi.c
+++ b/drivers/pci/pcie/aer/aerdrv_acpi.c
@@ -60,7 +60,7 @@
p = (struct acpi_hest_aer_common *)(hest_hdr + 1);
if (p->flags & ACPI_HEST_GLOBAL) {
if ((pci_is_pcie(info->pci_dev) &&
- info->pci_dev->pcie_type == pcie_type) || bridge)
+ pci_pcie_type(info->pci_dev) == pcie_type) || bridge)
ff = !!(p->flags & ACPI_HEST_FIRMWARE_FIRST);
} else
if (hest_match_pci(p, info->pci_dev))
diff --git a/drivers/pci/pcie/aer/aerdrv_core.c b/drivers/pci/pcie/aer/aerdrv_core.c
index 0ca0535..06bad96 100644
--- a/drivers/pci/pcie/aer/aerdrv_core.c
+++ b/drivers/pci/pcie/aer/aerdrv_core.c
@@ -32,53 +32,28 @@
module_param(forceload, bool, 0);
module_param(nosourceid, bool, 0);
+#define PCI_EXP_AER_FLAGS (PCI_EXP_DEVCTL_CERE | PCI_EXP_DEVCTL_NFERE | \
+ PCI_EXP_DEVCTL_FERE | PCI_EXP_DEVCTL_URRE)
+
int pci_enable_pcie_error_reporting(struct pci_dev *dev)
{
- u16 reg16 = 0;
- int pos;
-
if (pcie_aer_get_firmware_first(dev))
return -EIO;
- pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);
- if (!pos)
+ if (!pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR))
return -EIO;
- pos = pci_pcie_cap(dev);
- if (!pos)
- return -EIO;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, ®16);
- reg16 |= (PCI_EXP_DEVCTL_CERE |
- PCI_EXP_DEVCTL_NFERE |
- PCI_EXP_DEVCTL_FERE |
- PCI_EXP_DEVCTL_URRE);
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, reg16);
-
- return 0;
+ return pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_AER_FLAGS);
}
EXPORT_SYMBOL_GPL(pci_enable_pcie_error_reporting);
int pci_disable_pcie_error_reporting(struct pci_dev *dev)
{
- u16 reg16 = 0;
- int pos;
-
if (pcie_aer_get_firmware_first(dev))
return -EIO;
- pos = pci_pcie_cap(dev);
- if (!pos)
- return -EIO;
-
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, ®16);
- reg16 &= ~(PCI_EXP_DEVCTL_CERE |
- PCI_EXP_DEVCTL_NFERE |
- PCI_EXP_DEVCTL_FERE |
- PCI_EXP_DEVCTL_URRE);
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, reg16);
-
- return 0;
+ return pcie_capability_clear_word(dev, PCI_EXP_DEVCTL,
+ PCI_EXP_AER_FLAGS);
}
EXPORT_SYMBOL_GPL(pci_disable_pcie_error_reporting);
@@ -151,18 +126,12 @@
*/
if (atomic_read(&dev->enable_cnt) == 0)
return false;
- pos = pci_pcie_cap(dev);
- if (!pos)
- return false;
/* Check if AER is enabled */
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, ®16);
- if (!(reg16 & (
- PCI_EXP_DEVCTL_CERE |
- PCI_EXP_DEVCTL_NFERE |
- PCI_EXP_DEVCTL_FERE |
- PCI_EXP_DEVCTL_URRE)))
+ pcie_capability_read_word(dev, PCI_EXP_DEVCTL, ®16);
+ if (!(reg16 & PCI_EXP_AER_FLAGS))
return false;
+
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR);
if (!pos)
return false;
@@ -240,7 +209,7 @@
static int report_error_detected(struct pci_dev *dev, void *data)
{
pci_ers_result_t vote;
- struct pci_error_handlers *err_handler;
+ const struct pci_error_handlers *err_handler;
struct aer_broadcast_data *result_data;
result_data = (struct aer_broadcast_data *) data;
@@ -274,7 +243,7 @@
static int report_mmio_enabled(struct pci_dev *dev, void *data)
{
pci_ers_result_t vote;
- struct pci_error_handlers *err_handler;
+ const struct pci_error_handlers *err_handler;
struct aer_broadcast_data *result_data;
result_data = (struct aer_broadcast_data *) data;
@@ -292,7 +261,7 @@
static int report_slot_reset(struct pci_dev *dev, void *data)
{
pci_ers_result_t vote;
- struct pci_error_handlers *err_handler;
+ const struct pci_error_handlers *err_handler;
struct aer_broadcast_data *result_data;
result_data = (struct aer_broadcast_data *) data;
@@ -309,7 +278,7 @@
static int report_resume(struct pci_dev *dev, void *data)
{
- struct pci_error_handlers *err_handler;
+ const struct pci_error_handlers *err_handler;
dev->error_state = pci_channel_io_normal;
@@ -465,7 +434,7 @@
if (driver && driver->reset_link) {
status = driver->reset_link(udev);
- } else if (udev->pcie_type == PCI_EXP_TYPE_DOWNSTREAM) {
+ } else if (pci_pcie_type(udev) == PCI_EXP_TYPE_DOWNSTREAM) {
status = default_downstream_reset_link(udev);
} else {
dev_printk(KERN_DEBUG, &dev->dev,
@@ -540,14 +509,12 @@
"resume",
report_resume);
- dev_printk(KERN_DEBUG, &dev->dev,
- "AER driver successfully recovered\n");
+ dev_info(&dev->dev, "AER: Device recovery successful\n");
return;
failed:
/* TODO: Should kernel panic here? */
- dev_printk(KERN_DEBUG, &dev->dev,
- "AER driver didn't recover\n");
+ dev_info(&dev->dev, "AER: Device recovery failed\n");
}
/**
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index b500840..213753b 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -125,21 +125,16 @@
static void pcie_set_clkpm_nocheck(struct pcie_link_state *link, int enable)
{
- int pos;
- u16 reg16;
struct pci_dev *child;
struct pci_bus *linkbus = link->pdev->subordinate;
list_for_each_entry(child, &linkbus->devices, bus_list) {
- pos = pci_pcie_cap(child);
- if (!pos)
- return;
- pci_read_config_word(child, pos + PCI_EXP_LNKCTL, ®16);
if (enable)
- reg16 |= PCI_EXP_LNKCTL_CLKREQ_EN;
+ pcie_capability_set_word(child, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
else
- reg16 &= ~PCI_EXP_LNKCTL_CLKREQ_EN;
- pci_write_config_word(child, pos + PCI_EXP_LNKCTL, reg16);
+ pcie_capability_clear_word(child, PCI_EXP_LNKCTL,
+ PCI_EXP_LNKCTL_CLKREQ_EN);
}
link->clkpm_enabled = !!enable;
}
@@ -157,7 +152,7 @@
static void pcie_clkpm_cap_init(struct pcie_link_state *link, int blacklist)
{
- int pos, capable = 1, enabled = 1;
+ int capable = 1, enabled = 1;
u32 reg32;
u16 reg16;
struct pci_dev *child;
@@ -165,16 +160,13 @@
/* All functions should have the same cap and state, take the worst */
list_for_each_entry(child, &linkbus->devices, bus_list) {
- pos = pci_pcie_cap(child);
- if (!pos)
- return;
- pci_read_config_dword(child, pos + PCI_EXP_LNKCAP, ®32);
+ pcie_capability_read_dword(child, PCI_EXP_LNKCAP, ®32);
if (!(reg32 & PCI_EXP_LNKCAP_CLKPM)) {
capable = 0;
enabled = 0;
break;
}
- pci_read_config_word(child, pos + PCI_EXP_LNKCTL, ®16);
+ pcie_capability_read_word(child, PCI_EXP_LNKCTL, ®16);
if (!(reg16 & PCI_EXP_LNKCTL_CLKREQ_EN))
enabled = 0;
}
@@ -190,7 +182,7 @@
*/
static void pcie_aspm_configure_common_clock(struct pcie_link_state *link)
{
- int ppos, cpos, same_clock = 1;
+ int same_clock = 1;
u16 reg16, parent_reg, child_reg[8];
unsigned long start_jiffies;
struct pci_dev *child, *parent = link->pdev;
@@ -203,46 +195,43 @@
BUG_ON(!pci_is_pcie(child));
/* Check downstream component if bit Slot Clock Configuration is 1 */
- cpos = pci_pcie_cap(child);
- pci_read_config_word(child, cpos + PCI_EXP_LNKSTA, ®16);
+ pcie_capability_read_word(child, PCI_EXP_LNKSTA, ®16);
if (!(reg16 & PCI_EXP_LNKSTA_SLC))
same_clock = 0;
/* Check upstream component if bit Slot Clock Configuration is 1 */
- ppos = pci_pcie_cap(parent);
- pci_read_config_word(parent, ppos + PCI_EXP_LNKSTA, ®16);
+ pcie_capability_read_word(parent, PCI_EXP_LNKSTA, ®16);
if (!(reg16 & PCI_EXP_LNKSTA_SLC))
same_clock = 0;
/* Configure downstream component, all functions */
list_for_each_entry(child, &linkbus->devices, bus_list) {
- cpos = pci_pcie_cap(child);
- pci_read_config_word(child, cpos + PCI_EXP_LNKCTL, ®16);
+ pcie_capability_read_word(child, PCI_EXP_LNKCTL, ®16);
child_reg[PCI_FUNC(child->devfn)] = reg16;
if (same_clock)
reg16 |= PCI_EXP_LNKCTL_CCC;
else
reg16 &= ~PCI_EXP_LNKCTL_CCC;
- pci_write_config_word(child, cpos + PCI_EXP_LNKCTL, reg16);
+ pcie_capability_write_word(child, PCI_EXP_LNKCTL, reg16);
}
/* Configure upstream component */
- pci_read_config_word(parent, ppos + PCI_EXP_LNKCTL, ®16);
+ pcie_capability_read_word(parent, PCI_EXP_LNKCTL, ®16);
parent_reg = reg16;
if (same_clock)
reg16 |= PCI_EXP_LNKCTL_CCC;
else
reg16 &= ~PCI_EXP_LNKCTL_CCC;
- pci_write_config_word(parent, ppos + PCI_EXP_LNKCTL, reg16);
+ pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16);
/* Retrain link */
reg16 |= PCI_EXP_LNKCTL_RL;
- pci_write_config_word(parent, ppos + PCI_EXP_LNKCTL, reg16);
+ pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16);
/* Wait for link training end. Break out after waiting for timeout */
start_jiffies = jiffies;
for (;;) {
- pci_read_config_word(parent, ppos + PCI_EXP_LNKSTA, ®16);
+ pcie_capability_read_word(parent, PCI_EXP_LNKSTA, ®16);
if (!(reg16 & PCI_EXP_LNKSTA_LT))
break;
if (time_after(jiffies, start_jiffies + LINK_RETRAIN_TIMEOUT))
@@ -255,12 +244,10 @@
/* Training failed. Restore common clock configurations */
dev_printk(KERN_ERR, &parent->dev,
"ASPM: Could not configure common clock\n");
- list_for_each_entry(child, &linkbus->devices, bus_list) {
- cpos = pci_pcie_cap(child);
- pci_write_config_word(child, cpos + PCI_EXP_LNKCTL,
- child_reg[PCI_FUNC(child->devfn)]);
- }
- pci_write_config_word(parent, ppos + PCI_EXP_LNKCTL, parent_reg);
+ list_for_each_entry(child, &linkbus->devices, bus_list)
+ pcie_capability_write_word(child, PCI_EXP_LNKCTL,
+ child_reg[PCI_FUNC(child->devfn)]);
+ pcie_capability_write_word(parent, PCI_EXP_LNKCTL, parent_reg);
}
/* Convert L0s latency encoding to ns */
@@ -305,16 +292,14 @@
static void pcie_get_aspm_reg(struct pci_dev *pdev,
struct aspm_register_info *info)
{
- int pos;
u16 reg16;
u32 reg32;
- pos = pci_pcie_cap(pdev);
- pci_read_config_dword(pdev, pos + PCI_EXP_LNKCAP, ®32);
+ pcie_capability_read_dword(pdev, PCI_EXP_LNKCAP, ®32);
info->support = (reg32 & PCI_EXP_LNKCAP_ASPMS) >> 10;
info->latency_encoding_l0s = (reg32 & PCI_EXP_LNKCAP_L0SEL) >> 12;
info->latency_encoding_l1 = (reg32 & PCI_EXP_LNKCAP_L1EL) >> 15;
- pci_read_config_word(pdev, pos + PCI_EXP_LNKCTL, ®16);
+ pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, ®16);
info->enabled = reg16 & PCI_EXP_LNKCTL_ASPMC;
}
@@ -412,7 +397,7 @@
* do ASPM for now.
*/
list_for_each_entry(child, &linkbus->devices, bus_list) {
- if (child->pcie_type == PCI_EXP_TYPE_PCI_BRIDGE) {
+ if (pci_pcie_type(child) == PCI_EXP_TYPE_PCI_BRIDGE) {
link->aspm_disable = ASPM_STATE_ALL;
break;
}
@@ -420,17 +405,15 @@
/* Get and check endpoint acceptable latencies */
list_for_each_entry(child, &linkbus->devices, bus_list) {
- int pos;
u32 reg32, encoding;
struct aspm_latency *acceptable =
&link->acceptable[PCI_FUNC(child->devfn)];
- if (child->pcie_type != PCI_EXP_TYPE_ENDPOINT &&
- child->pcie_type != PCI_EXP_TYPE_LEG_END)
+ if (pci_pcie_type(child) != PCI_EXP_TYPE_ENDPOINT &&
+ pci_pcie_type(child) != PCI_EXP_TYPE_LEG_END)
continue;
- pos = pci_pcie_cap(child);
- pci_read_config_dword(child, pos + PCI_EXP_DEVCAP, ®32);
+ pcie_capability_read_dword(child, PCI_EXP_DEVCAP, ®32);
/* Calculate endpoint L0s acceptable latency */
encoding = (reg32 & PCI_EXP_DEVCAP_L0S) >> 6;
acceptable->l0s = calc_l0s_acceptable(encoding);
@@ -444,13 +427,7 @@
static void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val)
{
- u16 reg16;
- int pos = pci_pcie_cap(pdev);
-
- pci_read_config_word(pdev, pos + PCI_EXP_LNKCTL, ®16);
- reg16 &= ~0x3;
- reg16 |= val;
- pci_write_config_word(pdev, pos + PCI_EXP_LNKCTL, reg16);
+ pcie_capability_clear_and_set_word(pdev, PCI_EXP_LNKCTL, 0x3, val);
}
static void pcie_config_aspm_link(struct pcie_link_state *link, u32 state)
@@ -505,7 +482,6 @@
static int pcie_aspm_sanity_check(struct pci_dev *pdev)
{
struct pci_dev *child;
- int pos;
u32 reg32;
/*
@@ -513,8 +489,7 @@
* very strange. Disable ASPM for the whole slot
*/
list_for_each_entry(child, &pdev->subordinate->devices, bus_list) {
- pos = pci_pcie_cap(child);
- if (!pos)
+ if (!pci_is_pcie(child))
return -EINVAL;
/*
@@ -530,7 +505,7 @@
* Disable ASPM for pre-1.1 PCIe device, we follow MS to use
* RBER bit to determine if a function is 1.1 version device
*/
- pci_read_config_dword(child, pos + PCI_EXP_DEVCAP, ®32);
+ pcie_capability_read_dword(child, PCI_EXP_DEVCAP, ®32);
if (!(reg32 & PCI_EXP_DEVCAP_RBER) && !aspm_force) {
dev_printk(KERN_INFO, &child->dev, "disabling ASPM"
" on pre-1.1 PCIe device. You can enable it"
@@ -552,7 +527,7 @@
INIT_LIST_HEAD(&link->children);
INIT_LIST_HEAD(&link->link);
link->pdev = pdev;
- if (pdev->pcie_type == PCI_EXP_TYPE_DOWNSTREAM) {
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_DOWNSTREAM) {
struct pcie_link_state *parent;
parent = pdev->bus->parent->self->link_state;
if (!parent) {
@@ -585,12 +560,12 @@
if (!pci_is_pcie(pdev) || pdev->link_state)
return;
- if (pdev->pcie_type != PCI_EXP_TYPE_ROOT_PORT &&
- pdev->pcie_type != PCI_EXP_TYPE_DOWNSTREAM)
+ if (pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT &&
+ pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM)
return;
/* VIA has a strange chipset, root port is under a bridge */
- if (pdev->pcie_type == PCI_EXP_TYPE_ROOT_PORT &&
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT &&
pdev->bus->self)
return;
@@ -647,8 +622,8 @@
if (link->root != root)
continue;
list_for_each_entry(child, &linkbus->devices, bus_list) {
- if ((child->pcie_type != PCI_EXP_TYPE_ENDPOINT) &&
- (child->pcie_type != PCI_EXP_TYPE_LEG_END))
+ if ((pci_pcie_type(child) != PCI_EXP_TYPE_ENDPOINT) &&
+ (pci_pcie_type(child) != PCI_EXP_TYPE_LEG_END))
continue;
pcie_aspm_check_latency(child);
}
@@ -663,8 +638,8 @@
if (!pci_is_pcie(pdev) || !parent || !parent->link_state)
return;
- if ((parent->pcie_type != PCI_EXP_TYPE_ROOT_PORT) &&
- (parent->pcie_type != PCI_EXP_TYPE_DOWNSTREAM))
+ if ((pci_pcie_type(parent) != PCI_EXP_TYPE_ROOT_PORT) &&
+ (pci_pcie_type(parent) != PCI_EXP_TYPE_DOWNSTREAM))
return;
down_read(&pci_bus_sem);
@@ -704,8 +679,8 @@
if (aspm_disabled || !pci_is_pcie(pdev) || !link)
return;
- if ((pdev->pcie_type != PCI_EXP_TYPE_ROOT_PORT) &&
- (pdev->pcie_type != PCI_EXP_TYPE_DOWNSTREAM))
+ if ((pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT) &&
+ (pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM))
return;
/*
* Devices changed PM state, we should recheck if latency
@@ -729,8 +704,8 @@
if (aspm_policy != POLICY_POWERSAVE)
return;
- if ((pdev->pcie_type != PCI_EXP_TYPE_ROOT_PORT) &&
- (pdev->pcie_type != PCI_EXP_TYPE_DOWNSTREAM))
+ if ((pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT) &&
+ (pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM))
return;
down_read(&pci_bus_sem);
@@ -757,8 +732,8 @@
if (!pci_is_pcie(pdev))
return;
- if (pdev->pcie_type == PCI_EXP_TYPE_ROOT_PORT ||
- pdev->pcie_type == PCI_EXP_TYPE_DOWNSTREAM)
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT ||
+ pci_pcie_type(pdev) == PCI_EXP_TYPE_DOWNSTREAM)
parent = pdev;
if (!parent || !parent->link_state)
return;
@@ -933,8 +908,8 @@
struct pcie_link_state *link_state = pdev->link_state;
if (!pci_is_pcie(pdev) ||
- (pdev->pcie_type != PCI_EXP_TYPE_ROOT_PORT &&
- pdev->pcie_type != PCI_EXP_TYPE_DOWNSTREAM) || !link_state)
+ (pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT &&
+ pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM) || !link_state)
return;
if (link_state->aspm_support)
@@ -950,8 +925,8 @@
struct pcie_link_state *link_state = pdev->link_state;
if (!pci_is_pcie(pdev) ||
- (pdev->pcie_type != PCI_EXP_TYPE_ROOT_PORT &&
- pdev->pcie_type != PCI_EXP_TYPE_DOWNSTREAM) || !link_state)
+ (pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT &&
+ pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM) || !link_state)
return;
if (link_state->aspm_support)
diff --git a/drivers/pci/pcie/pme.c b/drivers/pci/pcie/pme.c
index 001f1b7..9ca0dc9 100644
--- a/drivers/pci/pcie/pme.c
+++ b/drivers/pci/pcie/pme.c
@@ -57,17 +57,12 @@
*/
void pcie_pme_interrupt_enable(struct pci_dev *dev, bool enable)
{
- int rtctl_pos;
- u16 rtctl;
-
- rtctl_pos = pci_pcie_cap(dev) + PCI_EXP_RTCTL;
-
- pci_read_config_word(dev, rtctl_pos, &rtctl);
if (enable)
- rtctl |= PCI_EXP_RTCTL_PMEIE;
+ pcie_capability_set_word(dev, PCI_EXP_RTCTL,
+ PCI_EXP_RTCTL_PMEIE);
else
- rtctl &= ~PCI_EXP_RTCTL_PMEIE;
- pci_write_config_word(dev, rtctl_pos, rtctl);
+ pcie_capability_clear_word(dev, PCI_EXP_RTCTL,
+ PCI_EXP_RTCTL_PMEIE);
}
/**
@@ -120,7 +115,7 @@
if (!dev)
return false;
- if (pci_is_pcie(dev) && dev->pcie_type == PCI_EXP_TYPE_PCI_BRIDGE) {
+ if (pci_is_pcie(dev) && pci_pcie_type(dev) == PCI_EXP_TYPE_PCI_BRIDGE) {
down_read(&pci_bus_sem);
if (pcie_pme_walk_bus(bus))
found = true;
@@ -226,18 +221,15 @@
struct pcie_pme_service_data *data =
container_of(work, struct pcie_pme_service_data, work);
struct pci_dev *port = data->srv->port;
- int rtsta_pos;
u32 rtsta;
- rtsta_pos = pci_pcie_cap(port) + PCI_EXP_RTSTA;
-
spin_lock_irq(&data->lock);
for (;;) {
if (data->noirq)
break;
- pci_read_config_dword(port, rtsta_pos, &rtsta);
+ pcie_capability_read_dword(port, PCI_EXP_RTSTA, &rtsta);
if (rtsta & PCI_EXP_RTSTA_PME) {
/*
* Clear PME status of the port. If there are other
@@ -276,17 +268,14 @@
{
struct pci_dev *port;
struct pcie_pme_service_data *data;
- int rtsta_pos;
u32 rtsta;
unsigned long flags;
port = ((struct pcie_device *)context)->port;
data = get_service_data((struct pcie_device *)context);
- rtsta_pos = pci_pcie_cap(port) + PCI_EXP_RTSTA;
-
spin_lock_irqsave(&data->lock, flags);
- pci_read_config_dword(port, rtsta_pos, &rtsta);
+ pcie_capability_read_dword(port, PCI_EXP_RTSTA, &rtsta);
if (!(rtsta & PCI_EXP_RTSTA_PME)) {
spin_unlock_irqrestore(&data->lock, flags);
@@ -335,13 +324,13 @@
struct pci_dev *dev;
/* Check if this is a root port event collector. */
- if (port->pcie_type != PCI_EXP_TYPE_RC_EC || !bus)
+ if (pci_pcie_type(port) != PCI_EXP_TYPE_RC_EC || !bus)
return;
down_read(&pci_bus_sem);
list_for_each_entry(dev, &bus->devices, bus_list)
if (pci_is_pcie(dev)
- && dev->pcie_type == PCI_EXP_TYPE_RC_END)
+ && pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END)
pcie_pme_set_native(dev, NULL);
up_read(&pci_bus_sem);
}
diff --git a/drivers/pci/pcie/portdrv_bus.c b/drivers/pci/pcie/portdrv_bus.c
index 18bf90f..67be55a 100644
--- a/drivers/pci/pcie/portdrv_bus.c
+++ b/drivers/pci/pcie/portdrv_bus.c
@@ -38,7 +38,7 @@
return 0;
if ((driver->port_type != PCIE_ANY_PORT) &&
- (driver->port_type != pciedev->port->pcie_type))
+ (driver->port_type != pci_pcie_type(pciedev->port)))
return 0;
return 1;
diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c
index 75915b3..d03a7a3 100644
--- a/drivers/pci/pcie/portdrv_core.c
+++ b/drivers/pci/pcie/portdrv_core.c
@@ -200,10 +200,13 @@
{
int i, irq = -1;
- /* We have to use INTx if MSI cannot be used for PCIe PME or pciehp. */
+ /*
+ * If MSI cannot be used for PCIe PME or hotplug, we have to use
+ * INTx or other interrupts, e.g. system shared interrupt.
+ */
if (((mask & PCIE_PORT_SERVICE_PME) && pcie_pme_no_msi()) ||
((mask & PCIE_PORT_SERVICE_HP) && pciehp_no_msi())) {
- if (dev->pin)
+ if (dev->irq)
irq = dev->irq;
goto no_msi;
}
@@ -212,8 +215,12 @@
if (!pcie_port_enable_msix(dev, irqs, mask))
return 0;
- /* We're not going to use MSI-X, so try MSI and fall back to INTx */
- if (!pci_enable_msi(dev) || dev->pin)
+ /*
+ * We're not going to use MSI-X, so try MSI and fall back to INTx.
+ * If neither MSI/MSI-X nor INTx available, try other interrupt. On
+ * some platforms, root port doesn't support MSI/MSI-X/INTx in RC mode.
+ */
+ if (!pci_enable_msi(dev) || dev->irq)
irq = dev->irq;
no_msi:
@@ -246,8 +253,7 @@
*/
static int get_port_device_capability(struct pci_dev *dev)
{
- int services = 0, pos;
- u16 reg16;
+ int services = 0;
u32 reg32;
int cap_mask = 0;
int err;
@@ -265,11 +271,9 @@
return 0;
}
- pos = pci_pcie_cap(dev);
- pci_read_config_word(dev, pos + PCI_EXP_FLAGS, ®16);
/* Hot-Plug Capable */
- if ((cap_mask & PCIE_PORT_SERVICE_HP) && (reg16 & PCI_EXP_FLAGS_SLOT)) {
- pci_read_config_dword(dev, pos + PCI_EXP_SLTCAP, ®32);
+ if (cap_mask & PCIE_PORT_SERVICE_HP) {
+ pcie_capability_read_dword(dev, PCI_EXP_SLTCAP, ®32);
if (reg32 & PCI_EXP_SLTCAP_HPC) {
services |= PCIE_PORT_SERVICE_HP;
/*
@@ -277,10 +281,8 @@
* enabled by the BIOS and the hot-plug service driver
* is not loaded.
*/
- pos += PCI_EXP_SLTCTL;
- pci_read_config_word(dev, pos, ®16);
- reg16 &= ~(PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE);
- pci_write_config_word(dev, pos, reg16);
+ pcie_capability_clear_word(dev, PCI_EXP_SLTCTL,
+ PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE);
}
}
/* AER capable */
@@ -298,7 +300,7 @@
services |= PCIE_PORT_SERVICE_VC;
/* Root ports are capable of generating PME too */
if ((cap_mask & PCIE_PORT_SERVICE_PME)
- && dev->pcie_type == PCI_EXP_TYPE_ROOT_PORT) {
+ && pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) {
services |= PCIE_PORT_SERVICE_PME;
/*
* Disable PME interrupt on this port in case it's been enabled
@@ -336,7 +338,7 @@
device->release = release_pcie_device; /* callback to free pcie dev */
dev_set_name(device, "%s:pcie%02x",
pci_name(pdev),
- get_descriptor_id(pdev->pcie_type, service));
+ get_descriptor_id(pci_pcie_type(pdev), service));
device->parent = &pdev->dev;
device_enable_async_suspend(device);
diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c
index 3a7eefc..0761d90 100644
--- a/drivers/pci/pcie/portdrv_pci.c
+++ b/drivers/pci/pcie/portdrv_pci.c
@@ -64,14 +64,7 @@
*/
void pcie_clear_root_pme_status(struct pci_dev *dev)
{
- int rtsta_pos;
- u32 rtsta;
-
- rtsta_pos = pci_pcie_cap(dev) + PCI_EXP_RTSTA;
-
- pci_read_config_dword(dev, rtsta_pos, &rtsta);
- rtsta |= PCI_EXP_RTSTA_PME;
- pci_write_config_dword(dev, rtsta_pos, rtsta);
+ pcie_capability_set_dword(dev, PCI_EXP_RTSTA, PCI_EXP_RTSTA_PME);
}
static int pcie_portdrv_restore_config(struct pci_dev *dev)
@@ -95,7 +88,7 @@
* which breaks ACPI-based runtime wakeup on PCI Express, so clear those
* bits now just in case (shouldn't hurt).
*/
- if(pdev->pcie_type == PCI_EXP_TYPE_ROOT_PORT)
+ if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT)
pcie_clear_root_pme_status(pdev);
return 0;
}
@@ -140,9 +133,17 @@
{
return 0;
}
+
+static int pcie_port_runtime_idle(struct device *dev)
+{
+ /* Delay for a short while to prevent too frequent suspend/resume */
+ pm_schedule_suspend(dev, 10);
+ return -EBUSY;
+}
#else
#define pcie_port_runtime_suspend NULL
#define pcie_port_runtime_resume NULL
+#define pcie_port_runtime_idle NULL
#endif
static const struct dev_pm_ops pcie_portdrv_pm_ops = {
@@ -155,6 +156,7 @@
.resume_noirq = pcie_port_resume_noirq,
.runtime_suspend = pcie_port_runtime_suspend,
.runtime_resume = pcie_port_runtime_resume,
+ .runtime_idle = pcie_port_runtime_idle,
};
#define PCIE_PORTDRV_PM_OPS (&pcie_portdrv_pm_ops)
@@ -186,9 +188,9 @@
int status;
if (!pci_is_pcie(dev) ||
- ((dev->pcie_type != PCI_EXP_TYPE_ROOT_PORT) &&
- (dev->pcie_type != PCI_EXP_TYPE_UPSTREAM) &&
- (dev->pcie_type != PCI_EXP_TYPE_DOWNSTREAM)))
+ ((pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT) &&
+ (pci_pcie_type(dev) != PCI_EXP_TYPE_UPSTREAM) &&
+ (pci_pcie_type(dev) != PCI_EXP_TYPE_DOWNSTREAM)))
return -ENODEV;
if (!dev->irq && dev->pin) {
@@ -200,6 +202,11 @@
return status;
pci_save_state(dev);
+ /*
+ * D3cold may not work properly on some PCIe port, so disable
+ * it by default.
+ */
+ dev->d3cold_allowed = false;
if (!pci_match_id(port_runtime_pm_black_list, dev))
pm_runtime_put_noidle(&dev->dev);
@@ -371,11 +378,11 @@
};
MODULE_DEVICE_TABLE(pci, port_pci_ids);
-static struct pci_error_handlers pcie_portdrv_err_handler = {
- .error_detected = pcie_portdrv_error_detected,
- .mmio_enabled = pcie_portdrv_mmio_enabled,
- .slot_reset = pcie_portdrv_slot_reset,
- .resume = pcie_portdrv_err_resume,
+static const struct pci_error_handlers pcie_portdrv_err_handler = {
+ .error_detected = pcie_portdrv_error_detected,
+ .mmio_enabled = pcie_portdrv_mmio_enabled,
+ .slot_reset = pcie_portdrv_slot_reset,
+ .resume = pcie_portdrv_err_resume,
};
static struct pci_driver pcie_portdriver = {
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 6c143b4..ec909af 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -144,15 +144,13 @@
case PCI_BASE_ADDRESS_MEM_TYPE_32:
break;
case PCI_BASE_ADDRESS_MEM_TYPE_1M:
- dev_info(&dev->dev, "1M mem BAR treated as 32-bit BAR\n");
+ /* 1M mem BAR treated as 32-bit BAR */
break;
case PCI_BASE_ADDRESS_MEM_TYPE_64:
flags |= IORESOURCE_MEM_64;
break;
default:
- dev_warn(&dev->dev,
- "mem unknown type %x treated as 32-bit BAR\n",
- mem_type);
+ /* mem unknown type treated as 32-bit BAR */
break;
}
return flags;
@@ -173,9 +171,11 @@
u32 l, sz, mask;
u16 orig_cmd;
struct pci_bus_region region;
+ bool bar_too_big = false, bar_disabled = false;
mask = type ? PCI_ROM_ADDRESS_MASK : ~0;
+ /* No printks while decoding is disabled! */
if (!dev->mmio_always_on) {
pci_read_config_word(dev, PCI_COMMAND, &orig_cmd);
pci_write_config_word(dev, PCI_COMMAND,
@@ -240,8 +240,7 @@
goto fail;
if ((sizeof(resource_size_t) < 8) && (sz64 > 0x100000000ULL)) {
- dev_err(&dev->dev, "reg %x: can't handle 64-bit BAR\n",
- pos);
+ bar_too_big = true;
goto fail;
}
@@ -252,12 +251,11 @@
region.start = 0;
region.end = sz64;
pcibios_bus_to_resource(dev, res, ®ion);
+ bar_disabled = true;
} else {
region.start = l64;
region.end = l64 + sz64;
pcibios_bus_to_resource(dev, res, ®ion);
- dev_printk(KERN_DEBUG, &dev->dev, "reg %x: %pR\n",
- pos, res);
}
} else {
sz = pci_size(l, sz, mask);
@@ -268,18 +266,23 @@
region.start = l;
region.end = l + sz;
pcibios_bus_to_resource(dev, res, ®ion);
-
- dev_printk(KERN_DEBUG, &dev->dev, "reg %x: %pR\n", pos, res);
}
- out:
+ goto out;
+
+
+fail:
+ res->flags = 0;
+out:
if (!dev->mmio_always_on)
pci_write_config_word(dev, PCI_COMMAND, orig_cmd);
+ if (bar_too_big)
+ dev_err(&dev->dev, "reg %x: can't handle 64-bit BAR\n", pos);
+ if (res->flags && !bar_disabled)
+ dev_printk(KERN_DEBUG, &dev->dev, "reg %x: %pR\n", pos, res);
+
return (res->flags & IORESOURCE_MEM_64) ? 1 : 0;
- fail:
- res->flags = 0;
- goto out;
}
static void pci_read_bases(struct pci_dev *dev, unsigned int howmany, int rom)
@@ -603,10 +606,10 @@
u32 linkcap;
u16 linksta;
- pci_read_config_dword(bridge, pos + PCI_EXP_LNKCAP, &linkcap);
+ pcie_capability_read_dword(bridge, PCI_EXP_LNKCAP, &linkcap);
bus->max_bus_speed = pcie_link_speed[linkcap & 0xf];
- pci_read_config_word(bridge, pos + PCI_EXP_LNKSTA, &linksta);
+ pcie_capability_read_word(bridge, PCI_EXP_LNKSTA, &linksta);
pcie_update_link_speed(bus, linksta);
}
}
@@ -726,8 +729,10 @@
/* Check if setup is sensible at all */
if (!pass &&
- (primary != bus->number || secondary <= bus->number)) {
- dev_dbg(&dev->dev, "bus configuration invalid, reconfiguring\n");
+ (primary != bus->number || secondary <= bus->number ||
+ secondary > subordinate)) {
+ dev_info(&dev->dev, "bridge configuration invalid ([bus %02x-%02x]), reconfiguring\n",
+ secondary, subordinate);
broken = 1;
}
@@ -929,24 +934,16 @@
pdev->is_pcie = 1;
pdev->pcie_cap = pos;
pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16);
- pdev->pcie_type = (reg16 & PCI_EXP_FLAGS_TYPE) >> 4;
+ pdev->pcie_flags_reg = reg16;
pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16);
pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
}
void set_pcie_hotplug_bridge(struct pci_dev *pdev)
{
- int pos;
- u16 reg16;
u32 reg32;
- pos = pci_pcie_cap(pdev);
- if (!pos)
- return;
- pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16);
- if (!(reg16 & PCI_EXP_FLAGS_SLOT))
- return;
- pci_read_config_dword(pdev, pos + PCI_EXP_SLTCAP, ®32);
+ pcie_capability_read_dword(pdev, PCI_EXP_SLTCAP, ®32);
if (reg32 & PCI_EXP_SLTCAP_HPC)
pdev->is_hotplug_bridge = 1;
}
@@ -1160,8 +1157,7 @@
if (class == PCI_CLASS_BRIDGE_HOST)
return pci_cfg_space_size_ext(dev);
- pos = pci_pcie_cap(dev);
- if (!pos) {
+ if (!pci_is_pcie(dev)) {
pos = pci_find_capability(dev, PCI_CAP_ID_PCIX);
if (!pos)
goto fail;
@@ -1383,9 +1379,9 @@
if (!parent || !pci_is_pcie(parent))
return 0;
- if (parent->pcie_type == PCI_EXP_TYPE_ROOT_PORT)
+ if (pci_pcie_type(parent) == PCI_EXP_TYPE_ROOT_PORT)
return 1;
- if (parent->pcie_type == PCI_EXP_TYPE_DOWNSTREAM &&
+ if (pci_pcie_type(parent) == PCI_EXP_TYPE_DOWNSTREAM &&
!pci_has_flag(PCI_SCAN_ALL_PCIE_DEVS))
return 1;
return 0;
@@ -1462,7 +1458,7 @@
*/
if (dev->is_hotplug_bridge && (!list_is_singular(&dev->bus->devices) ||
(dev->bus->self &&
- dev->bus->self->pcie_type != PCI_EXP_TYPE_ROOT_PORT)))
+ pci_pcie_type(dev->bus->self) != PCI_EXP_TYPE_ROOT_PORT)))
*smpss = 0;
if (*smpss > dev->pcie_mpss)
@@ -1478,7 +1474,8 @@
if (pcie_bus_config == PCIE_BUS_PERFORMANCE) {
mps = 128 << dev->pcie_mpss;
- if (dev->pcie_type != PCI_EXP_TYPE_ROOT_PORT && dev->bus->self)
+ if (pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT &&
+ dev->bus->self)
/* For "Performance", the assumption is made that
* downstream communication will never be larger than
* the MRRS. So, the MPS only needs to be configured
@@ -1753,11 +1750,6 @@
"busn_res: can not insert %pR under %s%pR (conflicts with %s %pR)\n",
res, pci_is_root_bus(b) ? "domain " : "",
parent_res, conflict->name, conflict);
- else
- dev_printk(KERN_DEBUG, &b->dev,
- "busn_res: %pR is inserted under %s%pR\n",
- res, pci_is_root_bus(b) ? "domain " : "",
- parent_res);
return conflict == NULL;
}
diff --git a/drivers/pci/proc.c b/drivers/pci/proc.c
index 27911b5..eb907a8f 100644
--- a/drivers/pci/proc.c
+++ b/drivers/pci/proc.c
@@ -434,25 +434,6 @@
return 0;
}
-#if 0
-int pci_proc_attach_bus(struct pci_bus* bus)
-{
- struct proc_dir_entry *de = bus->procdir;
-
- if (!proc_initialized)
- return -EACCES;
-
- if (!de) {
- char name[16];
- sprintf(name, "%02x", bus->number);
- de = bus->procdir = proc_mkdir(name, proc_bus_pci_dir);
- if (!de)
- return -ENOMEM;
- }
- return 0;
-}
-#endif /* 0 */
-
int pci_proc_detach_bus(struct pci_bus* bus)
{
struct proc_dir_entry *de = bus->procdir;
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 5155317..7a451ff 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3081,17 +3081,36 @@
static int reset_intel_82599_sfp_virtfn(struct pci_dev *dev, int probe)
{
- int pos;
+ int i;
+ u16 status;
- pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
- if (!pos)
- return -ENOTTY;
+ /*
+ * http://www.intel.com/content/dam/doc/datasheet/82599-10-gbe-controller-datasheet.pdf
+ *
+ * The 82599 supports FLR on VFs, but FLR support is reported only
+ * in the PF DEVCAP (sec 9.3.10.4), not in the VF DEVCAP (sec 9.5).
+ * Therefore, we can't use pcie_flr(), which checks the VF DEVCAP.
+ */
if (probe)
return 0;
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL,
- PCI_EXP_DEVCTL_BCR_FLR);
+ /* Wait for Transaction Pending bit clean */
+ for (i = 0; i < 4; i++) {
+ if (i)
+ msleep((1 << (i - 1)) * 100);
+
+ pcie_capability_read_word(dev, PCI_EXP_DEVSTA, &status);
+ if (!(status & PCI_EXP_DEVSTA_TRPND))
+ goto clear;
+ }
+
+ dev_err(&dev->dev, "transaction is not cleared; "
+ "proceeding with reset anyway\n");
+
+clear:
+ pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_BCR_FLR);
+
msleep(100);
return 0;
diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c
index 04a4861..513972f 100644
--- a/drivers/pci/remove.c
+++ b/drivers/pci/remove.c
@@ -32,53 +32,67 @@
static void pci_destroy_dev(struct pci_dev *dev)
{
- /* Remove the device from the device lists, and prevent any further
- * list accesses from this device */
down_write(&pci_bus_sem);
list_del(&dev->bus_list);
- dev->bus_list.next = dev->bus_list.prev = NULL;
up_write(&pci_bus_sem);
pci_free_resources(dev);
pci_dev_put(dev);
}
-/**
- * pci_remove_device_safe - remove an unused hotplug device
- * @dev: the device to remove
- *
- * Delete the device structure from the device lists and
- * notify userspace (/sbin/hotplug), but only if the device
- * in question is not being used by a driver.
- * Returns 0 on success.
- */
-#if 0
-int pci_remove_device_safe(struct pci_dev *dev)
+void pci_remove_bus(struct pci_bus *bus)
{
- if (pci_dev_driver(dev))
- return -EBUSY;
- pci_destroy_dev(dev);
- return 0;
-}
-#endif /* 0 */
-
-void pci_remove_bus(struct pci_bus *pci_bus)
-{
- pci_proc_detach_bus(pci_bus);
+ pci_proc_detach_bus(bus);
down_write(&pci_bus_sem);
- list_del(&pci_bus->node);
- pci_bus_release_busn_res(pci_bus);
+ list_del(&bus->node);
+ pci_bus_release_busn_res(bus);
up_write(&pci_bus_sem);
- if (!pci_bus->is_added)
+ if (!bus->is_added)
return;
- pci_remove_legacy_files(pci_bus);
- device_unregister(&pci_bus->dev);
+ pci_remove_legacy_files(bus);
+ device_unregister(&bus->dev);
}
EXPORT_SYMBOL(pci_remove_bus);
-static void __pci_remove_behind_bridge(struct pci_dev *dev);
+static void pci_stop_bus_device(struct pci_dev *dev)
+{
+ struct pci_bus *bus = dev->subordinate;
+ struct pci_dev *child, *tmp;
+
+ /*
+ * Stopping an SR-IOV PF device removes all the associated VFs,
+ * which will update the bus->devices list and confuse the
+ * iterator. Therefore, iterate in reverse so we remove the VFs
+ * first, then the PF.
+ */
+ if (bus) {
+ list_for_each_entry_safe_reverse(child, tmp,
+ &bus->devices, bus_list)
+ pci_stop_bus_device(child);
+ }
+
+ pci_stop_dev(dev);
+}
+
+static void pci_remove_bus_device(struct pci_dev *dev)
+{
+ struct pci_bus *bus = dev->subordinate;
+ struct pci_dev *child, *tmp;
+
+ if (bus) {
+ list_for_each_entry_safe(child, tmp,
+ &bus->devices, bus_list)
+ pci_remove_bus_device(child);
+
+ pci_remove_bus(bus);
+ dev->subordinate = NULL;
+ }
+
+ pci_destroy_dev(dev);
+}
+
/**
* pci_stop_and_remove_bus_device - remove a PCI device and any children
* @dev: the device to remove
@@ -91,93 +105,9 @@
* device lists, remove the /proc entry, and notify userspace
* (/sbin/hotplug).
*/
-void __pci_remove_bus_device(struct pci_dev *dev)
-{
- if (dev->subordinate) {
- struct pci_bus *b = dev->subordinate;
-
- __pci_remove_behind_bridge(dev);
- pci_remove_bus(b);
- dev->subordinate = NULL;
- }
-
- pci_destroy_dev(dev);
-}
-EXPORT_SYMBOL(__pci_remove_bus_device);
-
void pci_stop_and_remove_bus_device(struct pci_dev *dev)
{
pci_stop_bus_device(dev);
- __pci_remove_bus_device(dev);
+ pci_remove_bus_device(dev);
}
-
-static void __pci_remove_behind_bridge(struct pci_dev *dev)
-{
- struct list_head *l, *n;
-
- if (dev->subordinate)
- list_for_each_safe(l, n, &dev->subordinate->devices)
- __pci_remove_bus_device(pci_dev_b(l));
-}
-
-static void pci_stop_behind_bridge(struct pci_dev *dev)
-{
- struct list_head *l, *n;
-
- if (dev->subordinate)
- list_for_each_safe(l, n, &dev->subordinate->devices)
- pci_stop_bus_device(pci_dev_b(l));
-}
-
-/**
- * pci_stop_and_remove_behind_bridge - stop and remove all devices behind
- * a PCI bridge
- * @dev: PCI bridge device
- *
- * Remove all devices on the bus, except for the parent bridge.
- * This also removes any child buses, and any devices they may
- * contain in a depth-first manner.
- */
-void pci_stop_and_remove_behind_bridge(struct pci_dev *dev)
-{
- pci_stop_behind_bridge(dev);
- __pci_remove_behind_bridge(dev);
-}
-
-static void pci_stop_bus_devices(struct pci_bus *bus)
-{
- struct list_head *l, *n;
-
- /*
- * VFs could be removed by pci_stop_and_remove_bus_device() in the
- * pci_stop_bus_devices() code path for PF.
- * aka, bus->devices get updated in the process.
- * but VFs are inserted after PFs when SRIOV is enabled for PF,
- * We can iterate the list backwards to get prev valid PF instead
- * of removed VF.
- */
- list_for_each_prev_safe(l, n, &bus->devices) {
- struct pci_dev *dev = pci_dev_b(l);
- pci_stop_bus_device(dev);
- }
-}
-
-/**
- * pci_stop_bus_device - stop a PCI device and any children
- * @dev: the device to stop
- *
- * Stop a PCI device (detach the driver, remove from the global list
- * and so on). This also stop any subordinate buses and children in a
- * depth-first manner.
- */
-void pci_stop_bus_device(struct pci_dev *dev)
-{
- if (dev->subordinate)
- pci_stop_bus_devices(dev->subordinate);
-
- pci_stop_dev(dev);
-}
-
EXPORT_SYMBOL(pci_stop_and_remove_bus_device);
-EXPORT_SYMBOL(pci_stop_and_remove_behind_bridge);
-EXPORT_SYMBOL_GPL(pci_stop_bus_device);
diff --git a/drivers/pci/rom.c b/drivers/pci/rom.c
index 48ebdb2..0b3037a 100644
--- a/drivers/pci/rom.c
+++ b/drivers/pci/rom.c
@@ -167,44 +167,6 @@
return rom;
}
-#if 0
-/**
- * pci_map_rom_copy - map a PCI ROM to kernel space, create a copy
- * @pdev: pointer to pci device struct
- * @size: pointer to receive size of pci window over ROM
- *
- * Return: kernel virtual pointer to image of ROM
- *
- * Map a PCI ROM into kernel space. If ROM is boot video ROM,
- * the shadow BIOS copy will be returned instead of the
- * actual ROM.
- */
-void __iomem *pci_map_rom_copy(struct pci_dev *pdev, size_t *size)
-{
- struct resource *res = &pdev->resource[PCI_ROM_RESOURCE];
- void __iomem *rom;
-
- rom = pci_map_rom(pdev, size);
- if (!rom)
- return NULL;
-
- if (res->flags & (IORESOURCE_ROM_COPY | IORESOURCE_ROM_SHADOW |
- IORESOURCE_ROM_BIOS_COPY))
- return rom;
-
- res->start = (unsigned long)kmalloc(*size, GFP_KERNEL);
- if (!res->start)
- return rom;
-
- res->end = res->start + *size;
- memcpy_fromio((void*)(unsigned long)res->start, rom, *size);
- pci_unmap_rom(pdev, rom);
- res->flags |= IORESOURCE_ROM_COPY;
-
- return (void __iomem *)(unsigned long)res->start;
-}
-#endif /* 0 */
-
/**
* pci_unmap_rom - unmap the ROM from kernel space
* @pdev: pointer to pci device struct
@@ -226,27 +188,6 @@
pci_disable_rom(pdev);
}
-#if 0
-/**
- * pci_remove_rom - disable the ROM and remove its sysfs attribute
- * @pdev: pointer to pci device struct
- *
- * Remove the rom file in sysfs and disable ROM decoding.
- */
-void pci_remove_rom(struct pci_dev *pdev)
-{
- struct resource *res = &pdev->resource[PCI_ROM_RESOURCE];
-
- if (pci_resource_len(pdev, PCI_ROM_RESOURCE))
- sysfs_remove_bin_file(&pdev->dev.kobj, pdev->rom_attr);
- if (!(res->flags & (IORESOURCE_ROM_ENABLE |
- IORESOURCE_ROM_SHADOW |
- IORESOURCE_ROM_BIOS_COPY |
- IORESOURCE_ROM_COPY)))
- pci_disable_rom(pdev);
-}
-#endif /* 0 */
-
/**
* pci_cleanup_rom - free the ROM copy created by pci_map_rom_copy
* @pdev: pointer to pci device struct
diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 993d4a0..bf969ba 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -41,7 +41,7 @@
continue;
}
/* PCI device should connect to a PCIe bridge */
- if (pdev->pcie_type != PCI_EXP_TYPE_PCI_BRIDGE) {
+ if (pci_pcie_type(pdev) != PCI_EXP_TYPE_PCI_BRIDGE) {
/* Busted hardware? */
WARN_ON_ONCE(1);
return NULL;
@@ -130,16 +130,14 @@
* decrement the reference count by calling pci_dev_put().
* If no device is found, %NULL is returned.
*/
-struct pci_dev * pci_get_slot(struct pci_bus *bus, unsigned int devfn)
+struct pci_dev *pci_get_slot(struct pci_bus *bus, unsigned int devfn)
{
- struct list_head *tmp;
struct pci_dev *dev;
WARN_ON(in_interrupt());
down_read(&pci_bus_sem);
- list_for_each(tmp, &bus->devices) {
- dev = pci_dev_b(tmp);
+ list_for_each_entry(dev, &bus->devices, bus_list) {
if (dev->devfn == devfn)
goto out;
}
@@ -245,30 +243,14 @@
unsigned int ss_vendor, unsigned int ss_device,
struct pci_dev *from)
{
- struct pci_dev *pdev;
- struct pci_device_id *id;
+ struct pci_device_id id = {
+ .vendor = vendor,
+ .device = device,
+ .subvendor = ss_vendor,
+ .subdevice = ss_device,
+ };
- /*
- * pci_find_subsys() can be called on the ide_setup() path,
- * super-early in boot. But the down_read() will enable local
- * interrupts, which can cause some machines to crash. So here we
- * detect and flag that situation and bail out early.
- */
- if (unlikely(no_pci_devices()))
- return NULL;
-
- id = kzalloc(sizeof(*id), GFP_KERNEL);
- if (!id)
- return NULL;
- id->vendor = vendor;
- id->device = device;
- id->subvendor = ss_vendor;
- id->subdevice = ss_device;
-
- pdev = pci_get_dev_by_id(id, from);
- kfree(id);
-
- return pdev;
+ return pci_get_dev_by_id(&id, from);
}
/**
@@ -307,19 +289,16 @@
*/
struct pci_dev *pci_get_class(unsigned int class, struct pci_dev *from)
{
- struct pci_dev *dev;
- struct pci_device_id *id;
+ struct pci_device_id id = {
+ .vendor = PCI_ANY_ID,
+ .device = PCI_ANY_ID,
+ .subvendor = PCI_ANY_ID,
+ .subdevice = PCI_ANY_ID,
+ .class_mask = PCI_ANY_ID,
+ .class = class,
+ };
- id = kzalloc(sizeof(*id), GFP_KERNEL);
- if (!id)
- return NULL;
- id->vendor = id->device = id->subvendor = id->subdevice = PCI_ANY_ID;
- id->class_mask = PCI_ANY_ID;
- id->class = class;
-
- dev = pci_get_dev_by_id(id, from);
- kfree(id);
- return dev;
+ return pci_get_dev_by_id(&id, from);
}
/**
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index fb50613..1e808ca 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -697,6 +697,38 @@
return size;
}
+resource_size_t __weak pcibios_window_alignment(struct pci_bus *bus,
+ unsigned long type)
+{
+ return 1;
+}
+
+#define PCI_P2P_DEFAULT_MEM_ALIGN 0x100000 /* 1MiB */
+#define PCI_P2P_DEFAULT_IO_ALIGN 0x1000 /* 4KiB */
+#define PCI_P2P_DEFAULT_IO_ALIGN_1K 0x400 /* 1KiB */
+
+static resource_size_t window_alignment(struct pci_bus *bus,
+ unsigned long type)
+{
+ resource_size_t align = 1, arch_align;
+
+ if (type & IORESOURCE_MEM)
+ align = PCI_P2P_DEFAULT_MEM_ALIGN;
+ else if (type & IORESOURCE_IO) {
+ /*
+ * Per spec, I/O windows are 4K-aligned, but some
+ * bridges have an extension to support 1K alignment.
+ */
+ if (bus->self->io_window_1k)
+ align = PCI_P2P_DEFAULT_IO_ALIGN_1K;
+ else
+ align = PCI_P2P_DEFAULT_IO_ALIGN;
+ }
+
+ arch_align = pcibios_window_alignment(bus, type);
+ return max(align, arch_align);
+}
+
/**
* pbus_size_io() - size the io window of a given bus
*
@@ -717,17 +749,12 @@
struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO);
unsigned long size = 0, size0 = 0, size1 = 0;
resource_size_t children_add_size = 0;
- resource_size_t min_align = 4096, align;
+ resource_size_t min_align, io_align, align;
if (!b_res)
return;
- /*
- * Per spec, I/O windows are 4K-aligned, but some bridges have an
- * extension to support 1K alignment.
- */
- if (bus->self->io_window_1k)
- min_align = 1024;
+ io_align = min_align = window_alignment(bus, IORESOURCE_IO);
list_for_each_entry(dev, &bus->devices, bus_list) {
int i;
@@ -754,8 +781,8 @@
}
}
- if (min_align > 4096)
- min_align = 4096;
+ if (min_align > io_align)
+ min_align = io_align;
size0 = calculate_iosize(size, min_size, size1,
resource_size(b_res), min_align);
@@ -785,6 +812,28 @@
}
}
+static inline resource_size_t calculate_mem_align(resource_size_t *aligns,
+ int max_order)
+{
+ resource_size_t align = 0;
+ resource_size_t min_align = 0;
+ int order;
+
+ for (order = 0; order <= max_order; order++) {
+ resource_size_t align1 = 1;
+
+ align1 <<= (order + 20);
+
+ if (!align)
+ min_align = align1;
+ else if (ALIGN(align + min_align, min_align) < align1)
+ min_align = align1 >> 1;
+ align += aligns[order];
+ }
+
+ return min_align;
+}
+
/**
* pbus_size_mem() - size the memory window of a given bus
*
@@ -864,19 +913,9 @@
children_add_size += get_res_add_size(realloc_head, r);
}
}
- align = 0;
- min_align = 0;
- for (order = 0; order <= max_order; order++) {
- resource_size_t align1 = 1;
- align1 <<= (order + 20);
-
- if (!align)
- min_align = align1;
- else if (ALIGN(align + min_align, min_align) < align1)
- min_align = align1 >> 1;
- align += aligns[order];
- }
+ min_align = calculate_mem_align(aligns, max_order);
+ min_align = max(min_align, window_alignment(bus, b_res->flags & mask));
size0 = calculate_memsize(size, min_size, 0, resource_size(b_res), min_align);
if (children_add_size > add_size)
add_size = children_add_size;
diff --git a/drivers/pci/setup-irq.c b/drivers/pci/setup-irq.c
index eb219a1..9bd6864 100644
--- a/drivers/pci/setup-irq.c
+++ b/drivers/pci/setup-irq.c
@@ -17,8 +17,13 @@
#include <linux/ioport.h>
#include <linux/cache.h>
+void __weak pcibios_update_irq(struct pci_dev *dev, int irq)
+{
+ dev_dbg(&dev->dev, "assigning IRQ %02d\n", irq);
+ pci_write_config_byte(dev, PCI_INTERRUPT_LINE, irq);
+}
-static void __init
+static void
pdev_fixup_irq(struct pci_dev *dev,
u8 (*swizzle)(struct pci_dev *, u8 *),
int (*map_irq)(const struct pci_dev *, u8, u8))
@@ -54,7 +59,7 @@
pcibios_update_irq(dev, irq);
}
-void __init
+void
pci_fixup_irqs(u8 (*swizzle)(struct pci_dev *, u8 *),
int (*map_irq)(const struct pci_dev *, u8, u8))
{
diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index d6cc62c..def8d0b 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -982,7 +982,6 @@
int err = 0;
int i, num_devs;
unsigned int domain, bus, slot, func;
- struct pci_bus *pci_bus;
struct pci_dev *pci_dev;
char str[64];
@@ -1032,13 +1031,8 @@
goto out;
}
- pci_bus = pci_find_bus(domain, bus);
- if (!pci_bus) {
- dev_dbg(&pdev->xdev->dev, "Cannot get bus %04x:%02x\n",
- domain, bus);
- continue;
- }
- pci_dev = pci_get_slot(pci_bus, PCI_DEVFN(slot, func));
+ pci_dev = pci_get_domain_bus_and_slot(domain, bus,
+ PCI_DEVFN(slot, func));
if (!pci_dev) {
dev_dbg(&pdev->xdev->dev,
"Cannot get PCI device %04x:%02x:%02x.%d\n",
diff --git a/drivers/pcmcia/cardbus.c b/drivers/pcmcia/cardbus.c
index 24caeaf..9d3ac99 100644
--- a/drivers/pcmcia/cardbus.c
+++ b/drivers/pcmcia/cardbus.c
@@ -105,8 +105,17 @@
*/
void cb_free(struct pcmcia_socket *s)
{
- struct pci_dev *bridge = s->cb_dev;
+ struct pci_dev *bridge, *dev, *tmp;
+ struct pci_bus *bus;
- if (bridge)
- pci_stop_and_remove_behind_bridge(bridge);
+ bridge = s->cb_dev;
+ if (!bridge)
+ return;
+
+ bus = bridge->subordinate;
+ if (!bus)
+ return;
+
+ list_for_each_entry_safe(dev, tmp, &bus->devices, bus_list)
+ pci_stop_and_remove_bus_device(dev);
}
diff --git a/drivers/platform/x86/acer-wmi.c b/drivers/platform/x86/acer-wmi.c
index 3782e1c..934d861 100644
--- a/drivers/platform/x86/acer-wmi.c
+++ b/drivers/platform/x86/acer-wmi.c
@@ -2196,10 +2196,8 @@
interface->capability &= ~ACER_CAP_BRIGHTNESS;
pr_info("Brightness must be controlled by acpi video driver\n");
} else {
-#ifdef CONFIG_ACPI_VIDEO
pr_info("Disabling ACPI video driver\n");
acpi_video_unregister();
-#endif
}
if (wmi_has_guid(WMID_GUID3)) {
diff --git a/drivers/platform/x86/apple-gmux.c b/drivers/platform/x86/apple-gmux.c
index dfb1a92..db8f638 100644
--- a/drivers/platform/x86/apple-gmux.c
+++ b/drivers/platform/x86/apple-gmux.c
@@ -101,7 +101,7 @@
for (i = 0; i < 4; i++) {
tmpval = (val >> (i * 8)) & 0xff;
- outb(tmpval, port + i);
+ outb(tmpval, gmux_data->iostart + port + i);
}
}
@@ -142,8 +142,9 @@
u8 val;
mutex_lock(&gmux_data->index_lock);
- outb((port & 0xff), gmux_data->iostart + GMUX_PORT_READ);
gmux_index_wait_ready(gmux_data);
+ outb((port & 0xff), gmux_data->iostart + GMUX_PORT_READ);
+ gmux_index_wait_complete(gmux_data);
val = inb(gmux_data->iostart + GMUX_PORT_VALUE);
mutex_unlock(&gmux_data->index_lock);
@@ -166,8 +167,9 @@
u32 val;
mutex_lock(&gmux_data->index_lock);
- outb((port & 0xff), gmux_data->iostart + GMUX_PORT_READ);
gmux_index_wait_ready(gmux_data);
+ outb((port & 0xff), gmux_data->iostart + GMUX_PORT_READ);
+ gmux_index_wait_complete(gmux_data);
val = inl(gmux_data->iostart + GMUX_PORT_VALUE);
mutex_unlock(&gmux_data->index_lock);
@@ -461,18 +463,22 @@
ver_release = gmux_read8(gmux_data, GMUX_PORT_VERSION_RELEASE);
if (ver_major == 0xff && ver_minor == 0xff && ver_release == 0xff) {
if (gmux_is_indexed(gmux_data)) {
+ u32 version;
mutex_init(&gmux_data->index_lock);
gmux_data->indexed = true;
+ version = gmux_read32(gmux_data,
+ GMUX_PORT_VERSION_MAJOR);
+ ver_major = (version >> 24) & 0xff;
+ ver_minor = (version >> 16) & 0xff;
+ ver_release = (version >> 8) & 0xff;
} else {
pr_info("gmux device not present\n");
ret = -ENODEV;
goto err_release;
}
- pr_info("Found indexed gmux\n");
- } else {
- pr_info("Found gmux version %d.%d.%d\n", ver_major, ver_minor,
- ver_release);
}
+ pr_info("Found gmux version %d.%d.%d [%s]\n", ver_major, ver_minor,
+ ver_release, (gmux_data->indexed ? "indexed" : "classic"));
memset(&props, 0, sizeof(props));
props.type = BACKLIGHT_PLATFORM;
@@ -505,9 +511,7 @@
* Disable the other backlight choices.
*/
acpi_video_dmi_promote_vendor();
-#if defined (CONFIG_ACPI_VIDEO) || defined (CONFIG_ACPI_VIDEO_MODULE)
acpi_video_unregister();
-#endif
apple_bl_unregister();
gmux_data->power_state = VGA_SWITCHEROO_ON;
@@ -593,9 +597,7 @@
kfree(gmux_data);
acpi_video_dmi_demote_vendor();
-#if defined (CONFIG_ACPI_VIDEO) || defined (CONFIG_ACPI_VIDEO_MODULE)
acpi_video_register();
-#endif
apple_bl_register();
}
diff --git a/drivers/platform/x86/asus-laptop.c b/drivers/platform/x86/asus-laptop.c
index e38f91b..4b568df 100644
--- a/drivers/platform/x86/asus-laptop.c
+++ b/drivers/platform/x86/asus-laptop.c
@@ -85,7 +85,7 @@
static char *bled_type = "unknown";
module_param(wled_type, charp, 0444);
-MODULE_PARM_DESC(wlan_status, "Set the wled type on boot "
+MODULE_PARM_DESC(wled_type, "Set the wled type on boot "
"(unknown, led or rfkill). "
"default is unknown");
@@ -863,9 +863,9 @@
* The significance of others is yet to be found.
* If we don't find the method, we assume the device are present.
*/
- rv = acpi_evaluate_integer(asus->handle, "HRWS", NULL, &temp);
+ rv = acpi_evaluate_integer(asus->handle, "HWRS", NULL, &temp);
if (!ACPI_FAILURE(rv))
- len += sprintf(page + len, "HRWS value : %#x\n",
+ len += sprintf(page + len, "HWRS value : %#x\n",
(uint) temp);
/*
* Another value for userspace: the ASYM method returns 0x02 for
@@ -1751,9 +1751,9 @@
* The significance of others is yet to be found.
*/
status =
- acpi_evaluate_integer(asus->handle, "HRWS", NULL, &hwrs_result);
+ acpi_evaluate_integer(asus->handle, "HWRS", NULL, &hwrs_result);
if (!ACPI_FAILURE(status))
- pr_notice(" HRWS returned %x", (int)hwrs_result);
+ pr_notice(" HWRS returned %x", (int)hwrs_result);
if (!acpi_check_handle(asus->handle, METHOD_WL_STATUS, NULL))
asus->have_rsts = true;
diff --git a/drivers/platform/x86/asus-wmi.c b/drivers/platform/x86/asus-wmi.c
index 2eb9fe8..c0e9ff4 100644
--- a/drivers/platform/x86/asus-wmi.c
+++ b/drivers/platform/x86/asus-wmi.c
@@ -47,9 +47,7 @@
#include <linux/thermal.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
-#ifdef CONFIG_ACPI_VIDEO
#include <acpi/video.h>
-#endif
#include "asus-wmi.h"
@@ -1704,10 +1702,8 @@
if (asus->driver->quirks->wmi_backlight_power)
acpi_video_dmi_promote_vendor();
if (!acpi_video_backlight_support()) {
-#ifdef CONFIG_ACPI_VIDEO
pr_info("Disabling ACPI video driver\n");
acpi_video_unregister();
-#endif
err = asus_wmi_backlight_init(asus);
if (err && err != -ENODEV)
goto fail_backlight;
diff --git a/drivers/platform/x86/eeepc-laptop.c b/drivers/platform/x86/eeepc-laptop.c
index dab91b4..5ca2641 100644
--- a/drivers/platform/x86/eeepc-laptop.c
+++ b/drivers/platform/x86/eeepc-laptop.c
@@ -610,12 +610,12 @@
if (!bus) {
pr_warn("Unable to find PCI bus 1?\n");
- goto out_unlock;
+ goto out_put_dev;
}
if (pci_bus_read_config_dword(bus, 0, PCI_VENDOR_ID, &l)) {
pr_err("Unable to read PCI config space?\n");
- goto out_unlock;
+ goto out_put_dev;
}
absent = (l == 0xffffffff);
@@ -627,7 +627,7 @@
absent ? "absent" : "present");
pr_warn("skipped wireless hotplug as probably "
"inappropriate for this model\n");
- goto out_unlock;
+ goto out_put_dev;
}
if (!blocked) {
@@ -635,7 +635,7 @@
if (dev) {
/* Device already present */
pci_dev_put(dev);
- goto out_unlock;
+ goto out_put_dev;
}
dev = pci_scan_single_device(bus, 0);
if (dev) {
@@ -650,6 +650,8 @@
pci_dev_put(dev);
}
}
+out_put_dev:
+ pci_dev_put(port);
}
out_unlock:
diff --git a/drivers/platform/x86/samsung-laptop.c b/drivers/platform/x86/samsung-laptop.c
index c1ca7bc..dd90d15 100644
--- a/drivers/platform/x86/samsung-laptop.c
+++ b/drivers/platform/x86/samsung-laptop.c
@@ -26,9 +26,7 @@
#include <linux/seq_file.h>
#include <linux/debugfs.h>
#include <linux/ctype.h>
-#ifdef CONFIG_ACPI_VIDEO
#include <acpi/video.h>
-#endif
/*
* This driver is needed because a number of Samsung laptops do not hook
@@ -1558,9 +1556,7 @@
samsung->handle_backlight = false;
} else if (samsung->quirks->broken_acpi_video) {
pr_info("Disabling ACPI video driver\n");
-#ifdef CONFIG_ACPI_VIDEO
acpi_video_unregister();
-#endif
}
#endif
diff --git a/drivers/platform/x86/thinkpad_acpi.c b/drivers/platform/x86/thinkpad_acpi.c
index 80e3779..52daaa8 100644
--- a/drivers/platform/x86/thinkpad_acpi.c
+++ b/drivers/platform/x86/thinkpad_acpi.c
@@ -545,7 +545,7 @@
*/
static int acpi_evalf(acpi_handle handle,
- void *res, char *method, char *fmt, ...)
+ int *res, char *method, char *fmt, ...)
{
char *fmt0 = fmt;
struct acpi_object_list params;
@@ -606,7 +606,7 @@
success = (status == AE_OK &&
out_obj.type == ACPI_TYPE_INTEGER);
if (success && res)
- *(int *)res = out_obj.integer.value;
+ *res = out_obj.integer.value;
break;
case 'v': /* void */
success = status == AE_OK;
@@ -7386,17 +7386,18 @@
* Add TPACPI_FAN_RD_ACPI_FANS ? */
switch (fan_status_access_mode) {
- case TPACPI_FAN_RD_ACPI_GFAN:
+ case TPACPI_FAN_RD_ACPI_GFAN: {
/* 570, 600e/x, 770e, 770x */
+ int res;
- if (unlikely(!acpi_evalf(gfan_handle, &s, NULL, "d")))
+ if (unlikely(!acpi_evalf(gfan_handle, &res, NULL, "d")))
return -EIO;
if (likely(status))
- *status = s & 0x07;
+ *status = res & 0x07;
break;
-
+ }
case TPACPI_FAN_RD_TPEC:
/* all except 570, 600e/x, 770e, 770x */
if (unlikely(!acpi_ec_read(fan_status_offset, &s)))
diff --git a/drivers/pwm/pwm-tiecap.c b/drivers/pwm/pwm-tiecap.c
index 0b66d0f..4b66889 100644
--- a/drivers/pwm/pwm-tiecap.c
+++ b/drivers/pwm/pwm-tiecap.c
@@ -100,6 +100,13 @@
writel(period_cycles, pc->mmio_base + CAP3);
}
+ if (!test_bit(PWMF_ENABLED, &pwm->flags)) {
+ reg_val = readw(pc->mmio_base + ECCTL2);
+ /* Disable APWM mode to put APWM output Low */
+ reg_val &= ~ECCTL2_APWM_MODE;
+ writew(reg_val, pc->mmio_base + ECCTL2);
+ }
+
pm_runtime_put_sync(pc->chip.dev);
return 0;
}
diff --git a/drivers/pwm/pwm-tiehrpwm.c b/drivers/pwm/pwm-tiehrpwm.c
index c3756d1..b1996bc 100644
--- a/drivers/pwm/pwm-tiehrpwm.c
+++ b/drivers/pwm/pwm-tiehrpwm.c
@@ -104,6 +104,7 @@
struct pwm_chip chip;
unsigned int clk_rate;
void __iomem *mmio_base;
+ unsigned long period_cycles[NUM_PWM_CHANNEL];
};
static inline struct ehrpwm_pwm_chip *to_ehrpwm_pwm_chip(struct pwm_chip *chip)
@@ -210,6 +211,7 @@
unsigned long long c;
unsigned long period_cycles, duty_cycles;
unsigned short ps_divval, tb_divval;
+ int i;
if (period_ns < 0 || duty_ns < 0 || period_ns > NSEC_PER_SEC)
return -ERANGE;
@@ -229,6 +231,28 @@
duty_cycles = (unsigned long)c;
}
+ /*
+ * Period values should be same for multiple PWM channels as IP uses
+ * same period register for multiple channels.
+ */
+ for (i = 0; i < NUM_PWM_CHANNEL; i++) {
+ if (pc->period_cycles[i] &&
+ (pc->period_cycles[i] != period_cycles)) {
+ /*
+ * Allow channel to reconfigure period if no other
+ * channels being configured.
+ */
+ if (i == pwm->hwpwm)
+ continue;
+
+ dev_err(chip->dev, "Period value conflicts with channel %d\n",
+ i);
+ return -EINVAL;
+ }
+ }
+
+ pc->period_cycles[pwm->hwpwm] = period_cycles;
+
/* Configure clock prescaler to support Low frequency PWM wave */
if (set_prescale_div(period_cycles/PERIOD_MAX, &ps_divval,
&tb_divval)) {
@@ -320,10 +344,15 @@
static void ehrpwm_pwm_free(struct pwm_chip *chip, struct pwm_device *pwm)
{
+ struct ehrpwm_pwm_chip *pc = to_ehrpwm_pwm_chip(chip);
+
if (test_bit(PWMF_ENABLED, &pwm->flags)) {
dev_warn(chip->dev, "Removing PWM device without disabling\n");
pm_runtime_put_sync(chip->dev);
}
+
+ /* set period value to zero on free */
+ pc->period_cycles[pwm->hwpwm] = 0;
}
static const struct pwm_ops ehrpwm_pwm_ops = {
diff --git a/drivers/rapidio/devices/tsi721.c b/drivers/rapidio/devices/tsi721.c
index 5d44252..d5e1625 100644
--- a/drivers/rapidio/devices/tsi721.c
+++ b/drivers/rapidio/devices/tsi721.c
@@ -2219,9 +2219,7 @@
const struct pci_device_id *id)
{
struct tsi721_device *priv;
- int cap;
int err;
- u32 regval;
priv = kzalloc(sizeof(struct tsi721_device), GFP_KERNEL);
if (priv == NULL) {
@@ -2330,20 +2328,16 @@
dev_info(&pdev->dev, "Unable to set consistent DMA mask\n");
}
- cap = pci_pcie_cap(pdev);
- BUG_ON(cap == 0);
+ BUG_ON(!pci_is_pcie(pdev));
/* Clear "no snoop" and "relaxed ordering" bits, use default MRRS. */
- pci_read_config_dword(pdev, cap + PCI_EXP_DEVCTL, ®val);
- regval &= ~(PCI_EXP_DEVCTL_READRQ | PCI_EXP_DEVCTL_RELAX_EN |
- PCI_EXP_DEVCTL_NOSNOOP_EN);
- regval |= 0x2 << MAX_READ_REQUEST_SZ_SHIFT;
- pci_write_config_dword(pdev, cap + PCI_EXP_DEVCTL, regval);
+ pcie_capability_clear_and_set_word(pdev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_READRQ | PCI_EXP_DEVCTL_RELAX_EN |
+ PCI_EXP_DEVCTL_NOSNOOP_EN,
+ 0x2 << MAX_READ_REQUEST_SZ_SHIFT);
/* Adjust PCIe completion timeout. */
- pci_read_config_dword(pdev, cap + PCI_EXP_DEVCTL2, ®val);
- regval &= ~(0x0f);
- pci_write_config_dword(pdev, cap + PCI_EXP_DEVCTL2, regval | 0x2);
+ pcie_capability_clear_and_set_word(pdev, PCI_EXP_DEVCTL2, 0xf, 0x2);
/*
* FIXUP: correct offsets of MSI-X tables in the MSI-X Capability Block
diff --git a/drivers/regulator/Kconfig b/drivers/regulator/Kconfig
index 4e932cc..e98a5e7 100644
--- a/drivers/regulator/Kconfig
+++ b/drivers/regulator/Kconfig
@@ -33,9 +33,8 @@
help
If this option is enabled then when a regulator lookup fails
and the board has not specified that it has provided full
- constraints then the regulator core will provide an always
- enabled dummy regulator will be provided, allowing consumer
- drivers to continue.
+ constraints the regulator core will provide an always
+ enabled dummy regulator, allowing consumer drivers to continue.
A warning will be generated when this substitution is done.
@@ -50,11 +49,11 @@
tristate "Virtual regulator consumer support"
help
This driver provides a virtual consumer for the voltage and
- current regulator API which provides sysfs controls for
- configuring the supplies requested. This is mainly useful
- for test purposes.
+ current regulator API which provides sysfs controls for
+ configuring the supplies requested. This is mainly useful
+ for test purposes.
- If unsure, say no.
+ If unsure, say no.
config REGULATOR_USERSPACE_CONSUMER
tristate "Userspace regulator consumer support"
@@ -63,7 +62,7 @@
from user space. Userspace consumer driver provides ability to
control power supplies for such devices.
- If unsure, say no.
+ If unsure, say no.
config REGULATOR_GPIO
tristate "GPIO regulator support"
@@ -110,6 +109,17 @@
This driver supports the voltage regulators of DA9052-BC and
DA9053-AA/Bx PMIC.
+config REGULATOR_FAN53555
+ tristate "Fairchild FAN53555 Regulator"
+ depends on I2C
+ select REGMAP_I2C
+ help
+ This driver supports Fairchild FAN53555 Digitally Programmable
+ TinyBuck Regulator. The FAN53555 is a step-down switching voltage
+ regulator that delivers a digitally programmable output from an
+ input voltage supply of 2.5V to 5.5V. The output voltage is
+ programmed through an I2C interface.
+
config REGULATOR_ANATOP
tristate "Freescale i.MX on-chip ANATOP LDO regulators"
depends on MFD_ANATOP
@@ -172,6 +182,14 @@
This driver controls a Maxim 8660/8661 voltage output
regulator via I2C bus.
+config REGULATOR_MAX8907
+ tristate "Maxim 8907 voltage regulator"
+ depends on MFD_MAX8907
+ help
+ This driver controls a Maxim 8907 voltage output regulator
+ via I2C bus. The provided regulator is suitable for Tegra
+ chip to control Step-Down DC-DC and LDOs.
+
config REGULATOR_MAX8925
tristate "Maxim MAX8925 Power Management IC"
depends on MFD_MAX8925
@@ -247,7 +265,7 @@
config REGULATOR_PCF50633
tristate "NXP PCF50633 regulator driver"
- depends on MFD_PCF50633
+ depends on MFD_PCF50633
help
Say Y here to support the voltage regulators and convertors
on PCF50633
@@ -416,7 +434,7 @@
depends on MFD_WM8350
help
This driver provides support for the voltage and current regulators
- of the WM8350 AudioPlus PMIC.
+ of the WM8350 AudioPlus PMIC.
config REGULATOR_WM8400
tristate "Wolfson Microelectronics WM8400 AudioPlus PMIC"
diff --git a/drivers/regulator/Makefile b/drivers/regulator/Makefile
index 3342615..e431eed 100644
--- a/drivers/regulator/Makefile
+++ b/drivers/regulator/Makefile
@@ -20,6 +20,7 @@
obj-$(CONFIG_REGULATOR_DA9052) += da9052-regulator.o
obj-$(CONFIG_REGULATOR_DBX500_PRCMU) += dbx500-prcmu.o
obj-$(CONFIG_REGULATOR_DB8500_PRCMU) += db8500-prcmu.o
+obj-$(CONFIG_REGULATOR_FAN53555) += fan53555.o
obj-$(CONFIG_REGULATOR_GPIO) += gpio-regulator.o
obj-$(CONFIG_REGULATOR_ISL6271A) += isl6271a-regulator.o
obj-$(CONFIG_REGULATOR_LP3971) += lp3971.o
@@ -30,6 +31,7 @@
obj-$(CONFIG_REGULATOR_MAX1586) += max1586.o
obj-$(CONFIG_REGULATOR_MAX8649) += max8649.o
obj-$(CONFIG_REGULATOR_MAX8660) += max8660.o
+obj-$(CONFIG_REGULATOR_MAX8907) += max8907-regulator.o
obj-$(CONFIG_REGULATOR_MAX8925) += max8925-regulator.o
obj-$(CONFIG_REGULATOR_MAX8952) += max8952.o
obj-$(CONFIG_REGULATOR_MAX8997) += max8997.o
diff --git a/drivers/regulator/aat2870-regulator.c b/drivers/regulator/aat2870-regulator.c
index 6f45bfd..167c93f 100644
--- a/drivers/regulator/aat2870-regulator.c
+++ b/drivers/regulator/aat2870-regulator.c
@@ -162,7 +162,7 @@
static int aat2870_regulator_probe(struct platform_device *pdev)
{
struct aat2870_regulator *ri;
- struct regulator_config config = { 0 };
+ struct regulator_config config = { };
struct regulator_dev *rdev;
ri = aat2870_get_regulator(pdev->id);
diff --git a/drivers/regulator/ab3100.c b/drivers/regulator/ab3100.c
index c151fd5..65ad2b3 100644
--- a/drivers/regulator/ab3100.c
+++ b/drivers/regulator/ab3100.c
@@ -347,17 +347,11 @@
return abreg->plfdata->external_voltage;
}
-static int ab3100_get_fixed_voltage_regulator(struct regulator_dev *reg)
-{
- return reg->desc->min_uV;
-}
-
static struct regulator_ops regulator_ops_fixed = {
.list_voltage = regulator_list_voltage_linear,
.enable = ab3100_enable_regulator,
.disable = ab3100_disable_regulator,
.is_enabled = ab3100_is_enabled_regulator,
- .get_voltage = ab3100_get_fixed_voltage_regulator,
};
static struct regulator_ops regulator_ops_variable = {
diff --git a/drivers/regulator/ab8500.c b/drivers/regulator/ab8500.c
index 10f2f4d..e3d1d06 100644
--- a/drivers/regulator/ab8500.c
+++ b/drivers/regulator/ab8500.c
@@ -37,6 +37,7 @@
* @voltage_bank: bank to control regulator voltage
* @voltage_reg: register to control regulator voltage
* @voltage_mask: mask to control regulator voltage
+ * @voltage_shift: shift to control regulator voltage
* @delay: startup/set voltage delay in us
*/
struct ab8500_regulator_info {
@@ -50,6 +51,7 @@
u8 voltage_bank;
u8 voltage_reg;
u8 voltage_mask;
+ u8 voltage_shift;
unsigned int delay;
};
@@ -195,17 +197,14 @@
}
dev_vdbg(rdev_get_dev(rdev),
- "%s-get_voltage (bank, reg, mask, value): 0x%x, 0x%x, 0x%x,"
- " 0x%x\n",
- info->desc.name, info->voltage_bank, info->voltage_reg,
- info->voltage_mask, regval);
+ "%s-get_voltage (bank, reg, mask, shift, value): "
+ "0x%x, 0x%x, 0x%x, 0x%x, 0x%x\n",
+ info->desc.name, info->voltage_bank,
+ info->voltage_reg, info->voltage_mask,
+ info->voltage_shift, regval);
- /* vintcore has a different layout */
val = regval & info->voltage_mask;
- if (info->desc.id == AB8500_LDO_INTCORE)
- return val >> 0x3;
- else
- return val;
+ return val >> info->voltage_shift;
}
static int ab8500_regulator_set_voltage_sel(struct regulator_dev *rdev,
@@ -221,7 +220,7 @@
}
/* set the registers for the request */
- regval = (u8)selector;
+ regval = (u8)selector << info->voltage_shift;
ret = abx500_mask_and_set_register_interruptible(info->dev,
info->voltage_bank, info->voltage_reg,
info->voltage_mask, regval);
@@ -238,13 +237,6 @@
return ret;
}
-static int ab8500_regulator_enable_time(struct regulator_dev *rdev)
-{
- struct ab8500_regulator_info *info = rdev_get_drvdata(rdev);
-
- return info->delay;
-}
-
static int ab8500_regulator_set_voltage_time_sel(struct regulator_dev *rdev,
unsigned int old_sel,
unsigned int new_sel)
@@ -261,22 +253,14 @@
.get_voltage_sel = ab8500_regulator_get_voltage_sel,
.set_voltage_sel = ab8500_regulator_set_voltage_sel,
.list_voltage = regulator_list_voltage_table,
- .enable_time = ab8500_regulator_enable_time,
.set_voltage_time_sel = ab8500_regulator_set_voltage_time_sel,
};
-static int ab8500_fixed_get_voltage(struct regulator_dev *rdev)
-{
- return rdev->desc->min_uV;
-}
-
static struct regulator_ops ab8500_regulator_fixed_ops = {
.enable = ab8500_regulator_enable,
.disable = ab8500_regulator_disable,
.is_enabled = ab8500_regulator_is_enabled,
- .get_voltage = ab8500_fixed_get_voltage,
.list_voltage = regulator_list_voltage_linear,
- .enable_time = ab8500_regulator_enable_time,
};
static struct ab8500_regulator_info
@@ -358,6 +342,7 @@
.voltage_bank = 0x03,
.voltage_reg = 0x80,
.voltage_mask = 0x38,
+ .voltage_shift = 3,
},
/*
@@ -374,6 +359,7 @@
.owner = THIS_MODULE,
.n_voltages = 1,
.min_uV = 2000000,
+ .enable_time = 10000,
},
.delay = 10000,
.update_bank = 0x03,
diff --git a/drivers/regulator/arizona-ldo1.c b/drivers/regulator/arizona-ldo1.c
index c8f95c0..d184aa3 100644
--- a/drivers/regulator/arizona-ldo1.c
+++ b/drivers/regulator/arizona-ldo1.c
@@ -39,6 +39,8 @@
.map_voltage = regulator_map_voltage_linear,
.get_voltage_sel = regulator_get_voltage_sel_regmap,
.set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_bypass = regulator_get_bypass_regmap,
+ .set_bypass = regulator_set_bypass_regmap,
};
static const struct regulator_desc arizona_ldo1 = {
@@ -49,9 +51,11 @@
.vsel_reg = ARIZONA_LDO1_CONTROL_1,
.vsel_mask = ARIZONA_LDO1_VSEL_MASK,
+ .bypass_reg = ARIZONA_LDO1_CONTROL_1,
+ .bypass_mask = ARIZONA_LDO1_BYPASS,
.min_uV = 900000,
.uV_step = 50000,
- .n_voltages = 7,
+ .n_voltages = 6,
.owner = THIS_MODULE,
};
diff --git a/drivers/regulator/arizona-micsupp.c b/drivers/regulator/arizona-micsupp.c
index 450a069..d9b1f82 100644
--- a/drivers/regulator/arizona-micsupp.c
+++ b/drivers/regulator/arizona-micsupp.c
@@ -82,6 +82,9 @@
.get_voltage_sel = regulator_get_voltage_sel_regmap,
.set_voltage_sel = regulator_set_voltage_sel_regmap,
+
+ .get_bypass = regulator_get_bypass_regmap,
+ .set_bypass = regulator_set_bypass_regmap,
};
static const struct regulator_desc arizona_micsupp = {
@@ -95,6 +98,8 @@
.vsel_mask = ARIZONA_LDO2_VSEL_MASK,
.enable_reg = ARIZONA_MIC_CHARGE_PUMP_1,
.enable_mask = ARIZONA_CPMIC_ENA,
+ .bypass_reg = ARIZONA_MIC_CHARGE_PUMP_1,
+ .bypass_mask = ARIZONA_CPMIC_BYPASS,
.owner = THIS_MODULE,
};
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index 4838531..2e0352d 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -77,6 +77,7 @@
struct device *dev;
struct list_head list;
unsigned int always_on:1;
+ unsigned int bypass:1;
int uA_load;
int min_uV;
int max_uV;
@@ -394,6 +395,9 @@
case REGULATOR_STATUS_STANDBY:
label = "standby";
break;
+ case REGULATOR_STATUS_BYPASS:
+ label = "bypass";
+ break;
case REGULATOR_STATUS_UNDEFINED:
label = "undefined";
break;
@@ -585,6 +589,27 @@
static DEVICE_ATTR(suspend_standby_state, 0444,
regulator_suspend_standby_state_show, NULL);
+static ssize_t regulator_bypass_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct regulator_dev *rdev = dev_get_drvdata(dev);
+ const char *report;
+ bool bypass;
+ int ret;
+
+ ret = rdev->desc->ops->get_bypass(rdev, &bypass);
+
+ if (ret != 0)
+ report = "unknown";
+ else if (bypass)
+ report = "enabled";
+ else
+ report = "disabled";
+
+ return sprintf(buf, "%s\n", report);
+}
+static DEVICE_ATTR(bypass, 0444,
+ regulator_bypass_show, NULL);
/*
* These are the only attributes are present for all regulators.
@@ -778,6 +803,9 @@
if (constraints->valid_modes_mask & REGULATOR_MODE_STANDBY)
count += sprintf(buf + count, "standby");
+ if (!count)
+ sprintf(buf, "no parameters");
+
rdev_info(rdev, "%s\n", buf);
if ((constraints->min_uV != constraints->max_uV) &&
@@ -974,6 +1002,7 @@
err = -ENOMEM;
return err;
}
+ supply_rdev->open_count++;
return 0;
}
@@ -1720,6 +1749,9 @@
if (regulator->always_on)
return 0;
+ if (!ms)
+ return regulator_disable(regulator);
+
mutex_lock(&rdev->mutex);
rdev->deferred_disables++;
mutex_unlock(&rdev->mutex);
@@ -2178,9 +2210,12 @@
}
}
- if (ret == 0 && best_val >= 0)
+ if (ret == 0 && best_val >= 0) {
+ unsigned long data = best_val;
+
_notifier_call_chain(rdev, REGULATOR_EVENT_VOLTAGE_CHANGE,
- (void *)best_val);
+ (void *)data);
+ }
trace_regulator_set_voltage_complete(rdev_get_name(rdev), best_val);
@@ -2291,8 +2326,8 @@
EXPORT_SYMBOL_GPL(regulator_set_voltage_time);
/**
- *regulator_set_voltage_time_sel - get raise/fall time
- * @regulator: regulator source
+ * regulator_set_voltage_time_sel - get raise/fall time
+ * @rdev: regulator source device
* @old_selector: selector for starting voltage
* @new_selector: selector for target voltage
*
@@ -2388,6 +2423,8 @@
ret = rdev->desc->ops->list_voltage(rdev, sel);
} else if (rdev->desc->ops->get_voltage) {
ret = rdev->desc->ops->get_voltage(rdev);
+ } else if (rdev->desc->ops->list_voltage) {
+ ret = rdev->desc->ops->list_voltage(rdev, 0);
} else {
return -EINVAL;
}
@@ -2674,6 +2711,100 @@
EXPORT_SYMBOL_GPL(regulator_set_optimum_mode);
/**
+ * regulator_set_bypass_regmap - Default set_bypass() using regmap
+ *
+ * @rdev: device to operate on.
+ * @enable: state to set.
+ */
+int regulator_set_bypass_regmap(struct regulator_dev *rdev, bool enable)
+{
+ unsigned int val;
+
+ if (enable)
+ val = rdev->desc->bypass_mask;
+ else
+ val = 0;
+
+ return regmap_update_bits(rdev->regmap, rdev->desc->bypass_reg,
+ rdev->desc->bypass_mask, val);
+}
+EXPORT_SYMBOL_GPL(regulator_set_bypass_regmap);
+
+/**
+ * regulator_get_bypass_regmap - Default get_bypass() using regmap
+ *
+ * @rdev: device to operate on.
+ * @enable: current state.
+ */
+int regulator_get_bypass_regmap(struct regulator_dev *rdev, bool *enable)
+{
+ unsigned int val;
+ int ret;
+
+ ret = regmap_read(rdev->regmap, rdev->desc->bypass_reg, &val);
+ if (ret != 0)
+ return ret;
+
+ *enable = val & rdev->desc->bypass_mask;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(regulator_get_bypass_regmap);
+
+/**
+ * regulator_allow_bypass - allow the regulator to go into bypass mode
+ *
+ * @regulator: Regulator to configure
+ * @allow: enable or disable bypass mode
+ *
+ * Allow the regulator to go into bypass mode if all other consumers
+ * for the regulator also enable bypass mode and the machine
+ * constraints allow this. Bypass mode means that the regulator is
+ * simply passing the input directly to the output with no regulation.
+ */
+int regulator_allow_bypass(struct regulator *regulator, bool enable)
+{
+ struct regulator_dev *rdev = regulator->rdev;
+ int ret = 0;
+
+ if (!rdev->desc->ops->set_bypass)
+ return 0;
+
+ if (rdev->constraints &&
+ !(rdev->constraints->valid_ops_mask & REGULATOR_CHANGE_BYPASS))
+ return 0;
+
+ mutex_lock(&rdev->mutex);
+
+ if (enable && !regulator->bypass) {
+ rdev->bypass_count++;
+
+ if (rdev->bypass_count == rdev->open_count) {
+ ret = rdev->desc->ops->set_bypass(rdev, enable);
+ if (ret != 0)
+ rdev->bypass_count--;
+ }
+
+ } else if (!enable && regulator->bypass) {
+ rdev->bypass_count--;
+
+ if (rdev->bypass_count != rdev->open_count) {
+ ret = rdev->desc->ops->set_bypass(rdev, enable);
+ if (ret != 0)
+ rdev->bypass_count++;
+ }
+ }
+
+ if (ret == 0)
+ regulator->bypass = enable;
+
+ mutex_unlock(&rdev->mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(regulator_allow_bypass);
+
+/**
* regulator_register_notifier - register regulator event notifier
* @regulator: regulator source
* @nb: notifier block
@@ -3011,7 +3142,8 @@
/* some attributes need specific methods to be displayed */
if ((ops->get_voltage && ops->get_voltage(rdev) >= 0) ||
- (ops->get_voltage_sel && ops->get_voltage_sel(rdev) >= 0)) {
+ (ops->get_voltage_sel && ops->get_voltage_sel(rdev) >= 0) ||
+ (ops->list_voltage && ops->list_voltage(rdev, 0) >= 0)) {
status = device_create_file(dev, &dev_attr_microvolts);
if (status < 0)
return status;
@@ -3036,6 +3168,11 @@
if (status < 0)
return status;
}
+ if (ops->get_bypass) {
+ status = device_create_file(dev, &dev_attr_bypass);
+ if (status < 0)
+ return status;
+ }
/* some attributes are type-specific */
if (rdev->desc->type == REGULATOR_CURRENT) {
@@ -3124,6 +3261,8 @@
&rdev->use_count);
debugfs_create_u32("open_count", 0444, rdev->debugfs,
&rdev->open_count);
+ debugfs_create_u32("bypass_count", 0444, rdev->debugfs,
+ &rdev->bypass_count);
}
/**
@@ -3189,8 +3328,10 @@
rdev->desc = regulator_desc;
if (config->regmap)
rdev->regmap = config->regmap;
- else
+ else if (dev_get_regmap(dev, NULL))
rdev->regmap = dev_get_regmap(dev, NULL);
+ else if (dev->parent)
+ rdev->regmap = dev_get_regmap(dev->parent, NULL);
INIT_LIST_HEAD(&rdev->consumer_list);
INIT_LIST_HEAD(&rdev->list);
BLOCKING_INIT_NOTIFIER_HEAD(&rdev->notifier);
diff --git a/drivers/regulator/da9052-regulator.c b/drivers/regulator/da9052-regulator.c
index 903299c..27355b1 100644
--- a/drivers/regulator/da9052-regulator.c
+++ b/drivers/regulator/da9052-regulator.c
@@ -133,8 +133,8 @@
max_uA < da9052_current_limits[row][DA9052_MIN_UA])
return -EINVAL;
- for (i = 0; i < DA9052_CURRENT_RANGE; i++) {
- if (min_uA <= da9052_current_limits[row][i]) {
+ for (i = DA9052_CURRENT_RANGE - 1; i >= 0; i--) {
+ if (da9052_current_limits[row][i] <= max_uA) {
reg_val = i;
break;
}
diff --git a/drivers/regulator/dummy.c b/drivers/regulator/dummy.c
index 86f655c..03a1d7c 100644
--- a/drivers/regulator/dummy.c
+++ b/drivers/regulator/dummy.c
@@ -30,7 +30,7 @@
static struct regulator_ops dummy_ops;
static struct regulator_desc dummy_desc = {
- .name = "dummy",
+ .name = "regulator-dummy",
.id = -1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
diff --git a/drivers/regulator/fan53555.c b/drivers/regulator/fan53555.c
new file mode 100644
index 0000000..339f4d7
--- /dev/null
+++ b/drivers/regulator/fan53555.c
@@ -0,0 +1,322 @@
+/*
+ * FAN53555 Fairchild Digitally Programmable TinyBuck Regulator Driver.
+ *
+ * Supported Part Numbers:
+ * FAN53555UC00X/01X/03X/04X/05X
+ *
+ * Copyright (c) 2012 Marvell Technology Ltd.
+ * Yunfan Zhang <yfzhang@marvell.com>
+ *
+ * This package is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+#include <linux/module.h>
+#include <linux/param.h>
+#include <linux/err.h>
+#include <linux/platform_device.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/i2c.h>
+#include <linux/slab.h>
+#include <linux/regmap.h>
+#include <linux/regulator/fan53555.h>
+
+/* Voltage setting */
+#define FAN53555_VSEL0 0x00
+#define FAN53555_VSEL1 0x01
+/* Control register */
+#define FAN53555_CONTROL 0x02
+/* IC Type */
+#define FAN53555_ID1 0x03
+/* IC mask version */
+#define FAN53555_ID2 0x04
+/* Monitor register */
+#define FAN53555_MONITOR 0x05
+
+/* VSEL bit definitions */
+#define VSEL_BUCK_EN (1 << 7)
+#define VSEL_MODE (1 << 6)
+#define VSEL_NSEL_MASK 0x3F
+/* Chip ID and Verison */
+#define DIE_ID 0x0F /* ID1 */
+#define DIE_REV 0x0F /* ID2 */
+/* Control bit definitions */
+#define CTL_OUTPUT_DISCHG (1 << 7)
+#define CTL_SLEW_MASK (0x7 << 4)
+#define CTL_SLEW_SHIFT 4
+#define CTL_RESET (1 << 2)
+
+#define FAN53555_NVOLTAGES 64 /* Numbers of voltages */
+
+/* IC Type */
+enum {
+ FAN53555_CHIP_ID_00 = 0,
+ FAN53555_CHIP_ID_01,
+ FAN53555_CHIP_ID_02,
+ FAN53555_CHIP_ID_03,
+ FAN53555_CHIP_ID_04,
+ FAN53555_CHIP_ID_05,
+};
+
+struct fan53555_device_info {
+ struct regmap *regmap;
+ struct device *dev;
+ struct regulator_desc desc;
+ struct regulator_dev *rdev;
+ struct regulator_init_data *regulator;
+ /* IC Type and Rev */
+ int chip_id;
+ int chip_rev;
+ /* Voltage setting register */
+ unsigned int vol_reg;
+ unsigned int sleep_reg;
+ /* Voltage range and step(linear) */
+ unsigned int vsel_min;
+ unsigned int vsel_step;
+ /* Voltage slew rate limiting */
+ unsigned int slew_rate;
+ /* Sleep voltage cache */
+ unsigned int sleep_vol_cache;
+};
+
+static int fan53555_set_suspend_voltage(struct regulator_dev *rdev, int uV)
+{
+ struct fan53555_device_info *di = rdev_get_drvdata(rdev);
+ int ret;
+
+ if (di->sleep_vol_cache == uV)
+ return 0;
+ ret = regulator_map_voltage_linear(rdev, uV, uV);
+ if (ret < 0)
+ return -EINVAL;
+ ret = regmap_update_bits(di->regmap, di->sleep_reg,
+ VSEL_NSEL_MASK, ret);
+ if (ret < 0)
+ return -EINVAL;
+ /* Cache the sleep voltage setting.
+ * Might not be the real voltage which is rounded */
+ di->sleep_vol_cache = uV;
+
+ return 0;
+}
+
+static int fan53555_set_mode(struct regulator_dev *rdev, unsigned int mode)
+{
+ struct fan53555_device_info *di = rdev_get_drvdata(rdev);
+
+ switch (mode) {
+ case REGULATOR_MODE_FAST:
+ regmap_update_bits(di->regmap, di->vol_reg,
+ VSEL_MODE, VSEL_MODE);
+ break;
+ case REGULATOR_MODE_NORMAL:
+ regmap_update_bits(di->regmap, di->vol_reg, VSEL_MODE, 0);
+ break;
+ default:
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static unsigned int fan53555_get_mode(struct regulator_dev *rdev)
+{
+ struct fan53555_device_info *di = rdev_get_drvdata(rdev);
+ unsigned int val;
+ int ret = 0;
+
+ ret = regmap_read(di->regmap, di->vol_reg, &val);
+ if (ret < 0)
+ return ret;
+ if (val & VSEL_MODE)
+ return REGULATOR_MODE_FAST;
+ else
+ return REGULATOR_MODE_NORMAL;
+}
+
+static struct regulator_ops fan53555_regulator_ops = {
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .map_voltage = regulator_map_voltage_linear,
+ .list_voltage = regulator_list_voltage_linear,
+ .set_suspend_voltage = fan53555_set_suspend_voltage,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+ .set_mode = fan53555_set_mode,
+ .get_mode = fan53555_get_mode,
+};
+
+/* For 00,01,03,05 options:
+ * VOUT = 0.60V + NSELx * 10mV, from 0.60 to 1.23V.
+ * For 04 option:
+ * VOUT = 0.603V + NSELx * 12.826mV, from 0.603 to 1.411V.
+ * */
+static int fan53555_device_setup(struct fan53555_device_info *di,
+ struct fan53555_platform_data *pdata)
+{
+ unsigned int reg, data, mask;
+
+ /* Setup voltage control register */
+ switch (pdata->sleep_vsel_id) {
+ case FAN53555_VSEL_ID_0:
+ di->sleep_reg = FAN53555_VSEL0;
+ di->vol_reg = FAN53555_VSEL1;
+ break;
+ case FAN53555_VSEL_ID_1:
+ di->sleep_reg = FAN53555_VSEL1;
+ di->vol_reg = FAN53555_VSEL0;
+ break;
+ default:
+ dev_err(di->dev, "Invalid VSEL ID!\n");
+ return -EINVAL;
+ }
+ /* Init voltage range and step */
+ switch (di->chip_id) {
+ case FAN53555_CHIP_ID_00:
+ case FAN53555_CHIP_ID_01:
+ case FAN53555_CHIP_ID_03:
+ case FAN53555_CHIP_ID_05:
+ di->vsel_min = 600000;
+ di->vsel_step = 10000;
+ break;
+ case FAN53555_CHIP_ID_04:
+ di->vsel_min = 603000;
+ di->vsel_step = 12826;
+ break;
+ default:
+ dev_err(di->dev,
+ "Chip ID[%d]\n not supported!\n", di->chip_id);
+ return -EINVAL;
+ }
+ /* Init slew rate */
+ if (pdata->slew_rate & 0x7)
+ di->slew_rate = pdata->slew_rate;
+ else
+ di->slew_rate = FAN53555_SLEW_RATE_64MV;
+ reg = FAN53555_CONTROL;
+ data = di->slew_rate << CTL_SLEW_SHIFT;
+ mask = CTL_SLEW_MASK;
+ return regmap_update_bits(di->regmap, reg, mask, data);
+}
+
+static int fan53555_regulator_register(struct fan53555_device_info *di,
+ struct regulator_config *config)
+{
+ struct regulator_desc *rdesc = &di->desc;
+
+ rdesc->name = "fan53555-reg";
+ rdesc->ops = &fan53555_regulator_ops;
+ rdesc->type = REGULATOR_VOLTAGE;
+ rdesc->n_voltages = FAN53555_NVOLTAGES;
+ rdesc->enable_reg = di->vol_reg;
+ rdesc->enable_mask = VSEL_BUCK_EN;
+ rdesc->min_uV = di->vsel_min;
+ rdesc->uV_step = di->vsel_step;
+ rdesc->vsel_reg = di->vol_reg;
+ rdesc->vsel_mask = VSEL_NSEL_MASK;
+ rdesc->owner = THIS_MODULE;
+
+ di->rdev = regulator_register(&di->desc, config);
+ if (IS_ERR(di->rdev))
+ return PTR_ERR(di->rdev);
+ return 0;
+
+}
+
+static struct regmap_config fan53555_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+};
+
+static int __devinit fan53555_regulator_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct fan53555_device_info *di;
+ struct fan53555_platform_data *pdata;
+ struct regulator_config config = { };
+ unsigned int val;
+ int ret;
+
+ pdata = client->dev.platform_data;
+ if (!pdata || !pdata->regulator) {
+ dev_err(&client->dev, "Platform data not found!\n");
+ return -ENODEV;
+ }
+
+ di = devm_kzalloc(&client->dev, sizeof(struct fan53555_device_info),
+ GFP_KERNEL);
+ if (!di) {
+ dev_err(&client->dev, "Failed to allocate device info data!\n");
+ return -ENOMEM;
+ }
+ di->regmap = devm_regmap_init_i2c(client, &fan53555_regmap_config);
+ if (IS_ERR(di->regmap)) {
+ dev_err(&client->dev, "Failed to allocate regmap!\n");
+ return PTR_ERR(di->regmap);
+ }
+ di->dev = &client->dev;
+ di->regulator = pdata->regulator;
+ i2c_set_clientdata(client, di);
+ /* Get chip ID */
+ ret = regmap_read(di->regmap, FAN53555_ID1, &val);
+ if (ret < 0) {
+ dev_err(&client->dev, "Failed to get chip ID!\n");
+ return -ENODEV;
+ }
+ di->chip_id = val & DIE_ID;
+ /* Get chip revision */
+ ret = regmap_read(di->regmap, FAN53555_ID2, &val);
+ if (ret < 0) {
+ dev_err(&client->dev, "Failed to get chip Rev!\n");
+ return -ENODEV;
+ }
+ di->chip_rev = val & DIE_REV;
+ dev_info(&client->dev, "FAN53555 Option[%d] Rev[%d] Detected!\n",
+ di->chip_id, di->chip_rev);
+ /* Device init */
+ ret = fan53555_device_setup(di, pdata);
+ if (ret < 0) {
+ dev_err(&client->dev, "Failed to setup device!\n");
+ return ret;
+ }
+ /* Register regulator */
+ config.dev = di->dev;
+ config.init_data = di->regulator;
+ config.regmap = di->regmap;
+ config.driver_data = di;
+ ret = fan53555_regulator_register(di, &config);
+ if (ret < 0)
+ dev_err(&client->dev, "Failed to register regulator!\n");
+ return ret;
+
+}
+
+static int __devexit fan53555_regulator_remove(struct i2c_client *client)
+{
+ struct fan53555_device_info *di = i2c_get_clientdata(client);
+
+ regulator_unregister(di->rdev);
+ return 0;
+}
+
+static const struct i2c_device_id fan53555_id[] = {
+ {"fan53555", -1},
+ { },
+};
+
+static struct i2c_driver fan53555_regulator_driver = {
+ .driver = {
+ .name = "fan53555-regulator",
+ },
+ .probe = fan53555_regulator_probe,
+ .remove = __devexit_p(fan53555_regulator_remove),
+ .id_table = fan53555_id,
+};
+
+module_i2c_driver(fan53555_regulator_driver);
+
+MODULE_AUTHOR("Yunfan Zhang <yfzhang@marvell.com>");
+MODULE_DESCRIPTION("FAN53555 regulator driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/regulator/isl6271a-regulator.c b/drivers/regulator/isl6271a-regulator.c
index 1d145a0..d8ecf49a 100644
--- a/drivers/regulator/isl6271a-regulator.c
+++ b/drivers/regulator/isl6271a-regulator.c
@@ -73,13 +73,7 @@
.map_voltage = regulator_map_voltage_linear,
};
-static int isl6271a_get_fixed_voltage(struct regulator_dev *dev)
-{
- return dev->desc->min_uV;
-}
-
static struct regulator_ops isl_fixed_ops = {
- .get_voltage = isl6271a_get_fixed_voltage,
.list_voltage = regulator_list_voltage_linear,
};
diff --git a/drivers/regulator/lp872x.c b/drivers/regulator/lp872x.c
index 212c38e..708f4b6 100644
--- a/drivers/regulator/lp872x.c
+++ b/drivers/regulator/lp872x.c
@@ -86,6 +86,10 @@
#define EXTERN_DVS_USED 0
#define MAX_DELAY 6
+/* Default DVS Mode */
+#define LP8720_DEFAULT_DVS 0
+#define LP8725_DEFAULT_DVS BIT(2)
+
/* dump registers in regmap-debugfs */
#define MAX_REGISTERS 0x0F
@@ -269,9 +273,9 @@
return val > MAX_DELAY ? 0 : val * time_step_us;
}
-static void lp872x_set_dvs(struct lp872x *lp, int gpio)
+static void lp872x_set_dvs(struct lp872x *lp, enum lp872x_dvs_sel dvs_sel,
+ int gpio)
{
- enum lp872x_dvs_sel dvs_sel = lp->pdata->dvs->vsel;
enum lp872x_dvs_state state;
state = dvs_sel == SEL_V1 ? DVS_HIGH : DVS_LOW;
@@ -339,10 +343,10 @@
struct lp872x *lp = rdev_get_drvdata(rdev);
enum lp872x_regulator_id buck = rdev_get_id(rdev);
u8 addr, mask = LP872X_VOUT_M;
- struct lp872x_dvs *dvs = lp->pdata->dvs;
+ struct lp872x_dvs *dvs = lp->pdata ? lp->pdata->dvs : NULL;
if (dvs && gpio_is_valid(dvs->gpio))
- lp872x_set_dvs(lp, dvs->gpio);
+ lp872x_set_dvs(lp, dvs->vsel, dvs->gpio);
addr = lp872x_select_buck_vout_addr(lp, buck);
if (!lp872x_is_valid_buck_addr(addr))
@@ -374,8 +378,8 @@
{
struct lp872x *lp = rdev_get_drvdata(rdev);
enum lp872x_regulator_id buck = rdev_get_id(rdev);
- int i, max = ARRAY_SIZE(lp8725_buck_uA);
- u8 addr, val;
+ int i;
+ u8 addr;
switch (buck) {
case LP8725_ID_BUCK1:
@@ -388,17 +392,15 @@
return -EINVAL;
}
- for (i = 0 ; i < max ; i++)
+ for (i = ARRAY_SIZE(lp8725_buck_uA) - 1 ; i >= 0; i--) {
if (lp8725_buck_uA[i] >= min_uA &&
lp8725_buck_uA[i] <= max_uA)
- break;
+ return lp872x_update_bits(lp, addr,
+ LP8725_BUCK_CL_M,
+ i << LP8725_BUCK_CL_S);
+ }
- if (i == max)
- return -EINVAL;
-
- val = i << LP8725_BUCK_CL_S;
-
- return lp872x_update_bits(lp, addr, LP8725_BUCK_CL_M, val);
+ return -EINVAL;
}
static int lp8725_buck_get_current_limit(struct regulator_dev *rdev)
@@ -727,39 +729,16 @@
},
};
-static int lp872x_check_dvs_validity(struct lp872x *lp)
-{
- struct lp872x_dvs *dvs = lp->pdata->dvs;
- u8 val = 0;
- int ret;
-
- ret = lp872x_read_byte(lp, LP872X_GENERAL_CFG, &val);
- if (ret)
- return ret;
-
- ret = 0;
- if (lp->chipid == LP8720) {
- if (val & LP8720_EXT_DVS_M)
- ret = dvs ? 0 : -EINVAL;
- } else {
- if ((val & LP8725_DVS1_M) == EXTERN_DVS_USED)
- ret = dvs ? 0 : -EINVAL;
- }
-
- return ret;
-}
-
static int lp872x_init_dvs(struct lp872x *lp)
{
int ret, gpio;
- struct lp872x_dvs *dvs = lp->pdata->dvs;
+ struct lp872x_dvs *dvs = lp->pdata ? lp->pdata->dvs : NULL;
enum lp872x_dvs_state pinstate;
+ u8 mask[] = { LP8720_EXT_DVS_M, LP8725_DVS1_M | LP8725_DVS2_M };
+ u8 default_dvs_mode[] = { LP8720_DEFAULT_DVS, LP8725_DEFAULT_DVS };
- ret = lp872x_check_dvs_validity(lp);
- if (ret) {
- dev_warn(lp->dev, "invalid dvs data: %d\n", ret);
- return ret;
- }
+ if (!dvs)
+ goto set_default_dvs_mode;
gpio = dvs->gpio;
if (!gpio_is_valid(gpio)) {
@@ -778,6 +757,10 @@
lp->dvs_gpio = gpio;
return 0;
+
+set_default_dvs_mode:
+ return lp872x_update_bits(lp, LP872X_GENERAL_CFG, mask[lp->chipid],
+ default_dvs_mode[lp->chipid]);
}
static int lp872x_config(struct lp872x *lp)
@@ -785,24 +768,29 @@
struct lp872x_platform_data *pdata = lp->pdata;
int ret;
- if (!pdata->update_config)
- return 0;
+ if (!pdata || !pdata->update_config)
+ goto init_dvs;
ret = lp872x_write_byte(lp, LP872X_GENERAL_CFG, pdata->general_config);
if (ret)
return ret;
+init_dvs:
return lp872x_init_dvs(lp);
}
static struct regulator_init_data
*lp872x_find_regulator_init_data(int id, struct lp872x *lp)
{
+ struct lp872x_platform_data *pdata = lp->pdata;
int i;
+ if (!pdata)
+ return NULL;
+
for (i = 0; i < lp->num_regulators; i++) {
- if (lp->pdata->regulator_data[i].id == id)
- return lp->pdata->regulator_data[i].init_data;
+ if (pdata->regulator_data[i].id == id)
+ return pdata->regulator_data[i].init_data;
}
return NULL;
@@ -863,18 +851,12 @@
static int lp872x_probe(struct i2c_client *cl, const struct i2c_device_id *id)
{
struct lp872x *lp;
- struct lp872x_platform_data *pdata = cl->dev.platform_data;
int ret, size, num_regulators;
const int lp872x_num_regulators[] = {
[LP8720] = LP8720_NUM_REGULATORS,
[LP8725] = LP8725_NUM_REGULATORS,
};
- if (!pdata) {
- dev_err(&cl->dev, "no platform data\n");
- return -EINVAL;
- }
-
lp = devm_kzalloc(&cl->dev, sizeof(struct lp872x), GFP_KERNEL);
if (!lp)
goto err_mem;
@@ -894,7 +876,7 @@
}
lp->dev = &cl->dev;
- lp->pdata = pdata;
+ lp->pdata = cl->dev.platform_data;
lp->chipid = id->driver_data;
lp->num_regulators = num_regulators;
i2c_set_clientdata(cl, lp);
diff --git a/drivers/regulator/lp8788-buck.c b/drivers/regulator/lp8788-buck.c
index 6356e82..ba3e0aa 100644
--- a/drivers/regulator/lp8788-buck.c
+++ b/drivers/regulator/lp8788-buck.c
@@ -69,6 +69,9 @@
#define PIN_HIGH 1
#define ENABLE_TIME_USEC 32
+#define BUCK_FPWM_MASK(x) (1 << (x))
+#define BUCK_FPWM_SHIFT(x) (x)
+
enum lp8788_dvs_state {
DVS_LOW = GPIOF_OUT_INIT_LOW,
DVS_HIGH = GPIOF_OUT_INIT_HIGH,
@@ -86,15 +89,9 @@
BUCK4,
};
-struct lp8788_pwm_map {
- u8 mask;
- u8 shift;
-};
-
struct lp8788_buck {
struct lp8788 *lp;
struct regulator_dev *regulator;
- struct lp8788_pwm_map *pmap;
void *dvs;
};
@@ -106,29 +103,6 @@
1950000, 2000000,
};
-/* buck pwm mode selection : used for set/get_mode in regulator ops
- * @forced pwm : fast mode
- * @auto pwm : normal mode
- */
-static struct lp8788_pwm_map buck_pmap[] = {
- [BUCK1] = {
- .mask = LP8788_FPWM_BUCK1_M,
- .shift = LP8788_FPWM_BUCK1_S,
- },
- [BUCK2] = {
- .mask = LP8788_FPWM_BUCK2_M,
- .shift = LP8788_FPWM_BUCK2_S,
- },
- [BUCK3] = {
- .mask = LP8788_FPWM_BUCK3_M,
- .shift = LP8788_FPWM_BUCK3_S,
- },
- [BUCK4] = {
- .mask = LP8788_FPWM_BUCK4_M,
- .shift = LP8788_FPWM_BUCK4_S,
- },
-};
-
static const u8 buck1_vout_addr[] = {
LP8788_BUCK1_VOUT0, LP8788_BUCK1_VOUT1,
LP8788_BUCK1_VOUT2, LP8788_BUCK1_VOUT3,
@@ -347,41 +321,37 @@
static int lp8788_buck_set_mode(struct regulator_dev *rdev, unsigned int mode)
{
struct lp8788_buck *buck = rdev_get_drvdata(rdev);
- struct lp8788_pwm_map *pmap = buck->pmap;
- u8 val;
+ enum lp8788_buck_id id = rdev_get_id(rdev);
+ u8 mask, val;
- if (!pmap)
- return -EINVAL;
-
+ mask = BUCK_FPWM_MASK(id);
switch (mode) {
case REGULATOR_MODE_FAST:
- val = LP8788_FORCE_PWM << pmap->shift;
+ val = LP8788_FORCE_PWM << BUCK_FPWM_SHIFT(id);
break;
case REGULATOR_MODE_NORMAL:
- val = LP8788_AUTO_PWM << pmap->shift;
+ val = LP8788_AUTO_PWM << BUCK_FPWM_SHIFT(id);
break;
default:
return -EINVAL;
}
- return lp8788_update_bits(buck->lp, LP8788_BUCK_PWM, pmap->mask, val);
+ return lp8788_update_bits(buck->lp, LP8788_BUCK_PWM, mask, val);
}
static unsigned int lp8788_buck_get_mode(struct regulator_dev *rdev)
{
struct lp8788_buck *buck = rdev_get_drvdata(rdev);
- struct lp8788_pwm_map *pmap = buck->pmap;
+ enum lp8788_buck_id id = rdev_get_id(rdev);
u8 val;
int ret;
- if (!pmap)
- return -EINVAL;
-
ret = lp8788_read_byte(buck->lp, LP8788_BUCK_PWM, &val);
if (ret)
return ret;
- return val & pmap->mask ? REGULATOR_MODE_FAST : REGULATOR_MODE_NORMAL;
+ return val & BUCK_FPWM_MASK(id) ?
+ REGULATOR_MODE_FAST : REGULATOR_MODE_NORMAL;
}
static struct regulator_ops lp8788_buck12_ops = {
@@ -459,27 +429,6 @@
},
};
-static int lp8788_set_default_dvs_ctrl_mode(struct lp8788 *lp,
- enum lp8788_buck_id id)
-{
- u8 mask, val;
-
- switch (id) {
- case BUCK1:
- mask = LP8788_BUCK1_DVS_SEL_M;
- val = LP8788_BUCK1_DVS_I2C;
- break;
- case BUCK2:
- mask = LP8788_BUCK2_DVS_SEL_M;
- val = LP8788_BUCK2_DVS_I2C;
- break;
- default:
- return 0;
- }
-
- return lp8788_update_bits(lp, LP8788_BUCK_DVS_SEL, mask, val);
-}
-
static int _gpio_request(struct lp8788_buck *buck, int gpio, char *name)
{
struct device *dev = buck->lp->dev;
@@ -530,6 +479,7 @@
struct lp8788_platform_data *pdata = buck->lp->pdata;
u8 mask[] = { LP8788_BUCK1_DVS_SEL_M, LP8788_BUCK2_DVS_SEL_M };
u8 val[] = { LP8788_BUCK1_DVS_PIN, LP8788_BUCK2_DVS_PIN };
+ u8 default_dvs_mode[] = { LP8788_BUCK1_DVS_I2C, LP8788_BUCK2_DVS_I2C };
/* no dvs for buck3, 4 */
if (id == BUCK3 || id == BUCK4)
@@ -550,7 +500,8 @@
val[id]);
set_default_dvs_mode:
- return lp8788_set_default_dvs_ctrl_mode(buck->lp, id);
+ return lp8788_update_bits(buck->lp, LP8788_BUCK_DVS_SEL, mask[id],
+ default_dvs_mode[id]);
}
static __devinit int lp8788_buck_probe(struct platform_device *pdev)
@@ -567,7 +518,6 @@
return -ENOMEM;
buck->lp = lp;
- buck->pmap = &buck_pmap[id];
ret = lp8788_init_dvs(buck, id);
if (ret)
diff --git a/drivers/regulator/lp8788-ldo.c b/drivers/regulator/lp8788-ldo.c
index d2122e4..6796eeb 100644
--- a/drivers/regulator/lp8788-ldo.c
+++ b/drivers/regulator/lp8788-ldo.c
@@ -496,6 +496,7 @@
.name = "dldo12",
.id = DLDO12,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_B,
@@ -521,6 +522,7 @@
.name = "aldo2",
.id = ALDO2,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_B,
@@ -530,6 +532,7 @@
.name = "aldo3",
.id = ALDO3,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_B,
@@ -539,6 +542,7 @@
.name = "aldo4",
.id = ALDO4,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_B,
@@ -548,6 +552,7 @@
.name = "aldo5",
.id = ALDO5,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_C,
@@ -583,6 +588,7 @@
.name = "aldo8",
.id = ALDO8,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_C,
@@ -592,6 +598,7 @@
.name = "aldo9",
.id = ALDO9,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_C,
@@ -601,6 +608,7 @@
.name = "aldo10",
.id = ALDO10,
.ops = &lp8788_ldo_voltage_fixed_ops,
+ .n_voltages = 1,
.type = REGULATOR_VOLTAGE,
.owner = THIS_MODULE,
.enable_reg = LP8788_EN_LDO_C,
diff --git a/drivers/regulator/max77686.c b/drivers/regulator/max77686.c
index c564af6..2a67d08 100644
--- a/drivers/regulator/max77686.c
+++ b/drivers/regulator/max77686.c
@@ -66,7 +66,7 @@
};
struct max77686_data {
- struct regulator_dev **rdev;
+ struct regulator_dev *rdev[MAX77686_REGULATORS];
};
static int max77686_set_ramp_delay(struct regulator_dev *rdev, int ramp_delay)
@@ -265,6 +265,7 @@
rmatch.of_node = NULL;
of_regulator_match(iodev->dev, regulators_np, &rmatch, 1);
rdata[i].initdata = rmatch.init_data;
+ rdata[i].of_node = rmatch.of_node;
}
pdata->regulators = rdata;
@@ -283,10 +284,8 @@
{
struct max77686_dev *iodev = dev_get_drvdata(pdev->dev.parent);
struct max77686_platform_data *pdata = dev_get_platdata(iodev->dev);
- struct regulator_dev **rdev;
struct max77686_data *max77686;
- int i, size;
- int ret = 0;
+ int i, ret = 0;
struct regulator_config config = { };
dev_dbg(&pdev->dev, "%s\n", __func__);
@@ -313,45 +312,38 @@
if (!max77686)
return -ENOMEM;
- size = sizeof(struct regulator_dev *) * MAX77686_REGULATORS;
- max77686->rdev = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
- if (!max77686->rdev)
- return -ENOMEM;
-
- rdev = max77686->rdev;
config.dev = &pdev->dev;
config.regmap = iodev->regmap;
platform_set_drvdata(pdev, max77686);
for (i = 0; i < MAX77686_REGULATORS; i++) {
config.init_data = pdata->regulators[i].initdata;
+ config.of_node = pdata->regulators[i].of_node;
- rdev[i] = regulator_register(®ulators[i], &config);
- if (IS_ERR(rdev[i])) {
- ret = PTR_ERR(rdev[i]);
+ max77686->rdev[i] = regulator_register(®ulators[i], &config);
+ if (IS_ERR(max77686->rdev[i])) {
+ ret = PTR_ERR(max77686->rdev[i]);
dev_err(&pdev->dev,
"regulator init failed for %d\n", i);
- rdev[i] = NULL;
- goto err;
+ max77686->rdev[i] = NULL;
+ goto err;
}
}
return 0;
err:
while (--i >= 0)
- regulator_unregister(rdev[i]);
+ regulator_unregister(max77686->rdev[i]);
return ret;
}
static int __devexit max77686_pmic_remove(struct platform_device *pdev)
{
struct max77686_data *max77686 = platform_get_drvdata(pdev);
- struct regulator_dev **rdev = max77686->rdev;
int i;
for (i = 0; i < MAX77686_REGULATORS; i++)
- if (rdev[i])
- regulator_unregister(rdev[i]);
+ regulator_unregister(max77686->rdev[i]);
return 0;
}
diff --git a/drivers/regulator/max8907-regulator.c b/drivers/regulator/max8907-regulator.c
new file mode 100644
index 0000000..af76075
--- /dev/null
+++ b/drivers/regulator/max8907-regulator.c
@@ -0,0 +1,408 @@
+/*
+ * max8907-regulator.c -- support regulators in max8907
+ *
+ * Copyright (C) 2010 Gyungoh Yoo <jack.yoo@maxim-ic.com>
+ * Copyright (C) 2010-2012, NVIDIA CORPORATION. All rights reserved.
+ *
+ * Portions based on drivers/regulator/tps65910-regulator.c,
+ * Copyright 2010 Texas Instruments Inc.
+ * Author: Graeme Gregory <gg@slimlogic.co.uk>
+ * Author: Jorge Eduardo Candelaria <jedu@slimlogic.co.uk>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/mfd/core.h>
+#include <linux/mfd/max8907.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/regulator/driver.h>
+#include <linux/regulator/machine.h>
+#include <linux/regulator/of_regulator.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+
+#define MAX8907_II2RR_VERSION_MASK 0xF0
+#define MAX8907_II2RR_VERSION_REV_A 0x00
+#define MAX8907_II2RR_VERSION_REV_B 0x10
+#define MAX8907_II2RR_VERSION_REV_C 0x30
+
+struct max8907_regulator {
+ struct regulator_desc desc[MAX8907_NUM_REGULATORS];
+ struct regulator_dev *rdev[MAX8907_NUM_REGULATORS];
+};
+
+#define REG_MBATT() \
+ [MAX8907_MBATT] = { \
+ .name = "MBATT", \
+ .supply_name = "mbatt", \
+ .id = MAX8907_MBATT, \
+ .ops = &max8907_mbatt_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ }
+
+#define REG_LDO(ids, supply, base, min, max, step) \
+ [MAX8907_##ids] = { \
+ .name = #ids, \
+ .supply_name = supply, \
+ .id = MAX8907_##ids, \
+ .n_voltages = ((max) - (min)) / (step) + 1, \
+ .ops = &max8907_ldo_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = (min), \
+ .uV_step = (step), \
+ .vsel_reg = (base) + MAX8907_VOUT, \
+ .vsel_mask = 0x3f, \
+ .enable_reg = (base) + MAX8907_CTL, \
+ .enable_mask = MAX8907_MASK_LDO_EN, \
+ }
+
+#define REG_FIXED(ids, supply, voltage) \
+ [MAX8907_##ids] = { \
+ .name = #ids, \
+ .supply_name = supply, \
+ .id = MAX8907_##ids, \
+ .n_voltages = 1, \
+ .ops = &max8907_fixed_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = (voltage), \
+ }
+
+#define REG_OUT5V(ids, supply, base, voltage) \
+ [MAX8907_##ids] = { \
+ .name = #ids, \
+ .supply_name = supply, \
+ .id = MAX8907_##ids, \
+ .n_voltages = 1, \
+ .ops = &max8907_out5v_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = (voltage), \
+ .enable_reg = (base), \
+ .enable_mask = MAX8907_MASK_OUT5V_EN, \
+ }
+
+#define REG_BBAT(ids, supply, base, min, max, step) \
+ [MAX8907_##ids] = { \
+ .name = #ids, \
+ .supply_name = supply, \
+ .id = MAX8907_##ids, \
+ .n_voltages = ((max) - (min)) / (step) + 1, \
+ .ops = &max8907_bbat_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .owner = THIS_MODULE, \
+ .min_uV = (min), \
+ .uV_step = (step), \
+ .vsel_reg = (base), \
+ .vsel_mask = MAX8907_MASK_VBBATTCV, \
+ }
+
+#define LDO_750_50(id, supply, base) REG_LDO(id, supply, (base), \
+ 750000, 3900000, 50000)
+#define LDO_650_25(id, supply, base) REG_LDO(id, supply, (base), \
+ 650000, 2225000, 25000)
+
+static struct regulator_ops max8907_mbatt_ops = {
+};
+
+static struct regulator_ops max8907_ldo_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+};
+
+static struct regulator_ops max8907_ldo_hwctl_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+};
+
+static struct regulator_ops max8907_fixed_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+};
+
+static struct regulator_ops max8907_out5v_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+};
+
+static struct regulator_ops max8907_out5v_hwctl_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+};
+
+static struct regulator_ops max8907_bbat_ops = {
+ .list_voltage = regulator_list_voltage_linear,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+};
+
+static struct regulator_desc max8907_regulators[] = {
+ REG_MBATT(),
+ REG_LDO(SD1, "in-v1", MAX8907_REG_SDCTL1, 650000, 2225000, 25000),
+ REG_LDO(SD2, "in-v2", MAX8907_REG_SDCTL2, 637500, 1425000, 12500),
+ REG_LDO(SD3, "in-v3", MAX8907_REG_SDCTL3, 750000, 3900000, 50000),
+ LDO_750_50(LDO1, "in1", MAX8907_REG_LDOCTL1),
+ LDO_650_25(LDO2, "in2", MAX8907_REG_LDOCTL2),
+ LDO_650_25(LDO3, "in3", MAX8907_REG_LDOCTL3),
+ LDO_750_50(LDO4, "in4", MAX8907_REG_LDOCTL4),
+ LDO_750_50(LDO5, "in5", MAX8907_REG_LDOCTL5),
+ LDO_750_50(LDO6, "in6", MAX8907_REG_LDOCTL6),
+ LDO_750_50(LDO7, "in7", MAX8907_REG_LDOCTL7),
+ LDO_750_50(LDO8, "in8", MAX8907_REG_LDOCTL8),
+ LDO_750_50(LDO9, "in9", MAX8907_REG_LDOCTL9),
+ LDO_750_50(LDO10, "in10", MAX8907_REG_LDOCTL10),
+ LDO_750_50(LDO11, "in11", MAX8907_REG_LDOCTL11),
+ LDO_750_50(LDO12, "in12", MAX8907_REG_LDOCTL12),
+ LDO_750_50(LDO13, "in13", MAX8907_REG_LDOCTL13),
+ LDO_750_50(LDO14, "in14", MAX8907_REG_LDOCTL14),
+ LDO_750_50(LDO15, "in15", MAX8907_REG_LDOCTL15),
+ LDO_750_50(LDO16, "in16", MAX8907_REG_LDOCTL16),
+ LDO_650_25(LDO17, "in17", MAX8907_REG_LDOCTL17),
+ LDO_650_25(LDO18, "in18", MAX8907_REG_LDOCTL18),
+ LDO_750_50(LDO19, "in19", MAX8907_REG_LDOCTL19),
+ LDO_750_50(LDO20, "in20", MAX8907_REG_LDOCTL20),
+ REG_OUT5V(OUT5V, "mbatt", MAX8907_REG_OUT5VEN, 5000000),
+ REG_OUT5V(OUT33V, "mbatt", MAX8907_REG_OUT33VEN, 3300000),
+ REG_BBAT(BBAT, "MBATT", MAX8907_REG_BBAT_CNFG,
+ 2400000, 3000000, 200000),
+ REG_FIXED(SDBY, "MBATT", 1200000),
+ REG_FIXED(VRTC, "MBATT", 3300000),
+};
+
+#ifdef CONFIG_OF
+
+#define MATCH(_name, _id) \
+ [MAX8907_##_id] = { \
+ .name = #_name, \
+ .driver_data = (void *)&max8907_regulators[MAX8907_##_id], \
+ }
+
+static struct of_regulator_match max8907_matches[] = {
+ MATCH(mbatt, MBATT),
+ MATCH(sd1, SD1),
+ MATCH(sd2, SD2),
+ MATCH(sd3, SD3),
+ MATCH(ldo1, LDO1),
+ MATCH(ldo2, LDO2),
+ MATCH(ldo3, LDO3),
+ MATCH(ldo4, LDO4),
+ MATCH(ldo5, LDO5),
+ MATCH(ldo6, LDO6),
+ MATCH(ldo7, LDO7),
+ MATCH(ldo8, LDO8),
+ MATCH(ldo9, LDO9),
+ MATCH(ldo10, LDO10),
+ MATCH(ldo11, LDO11),
+ MATCH(ldo12, LDO12),
+ MATCH(ldo13, LDO13),
+ MATCH(ldo14, LDO14),
+ MATCH(ldo15, LDO15),
+ MATCH(ldo16, LDO16),
+ MATCH(ldo17, LDO17),
+ MATCH(ldo18, LDO18),
+ MATCH(ldo19, LDO19),
+ MATCH(ldo20, LDO20),
+ MATCH(out5v, OUT5V),
+ MATCH(out33v, OUT33V),
+ MATCH(bbat, BBAT),
+ MATCH(sdby, SDBY),
+ MATCH(vrtc, VRTC),
+};
+
+static int max8907_regulator_parse_dt(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.parent->of_node;
+ struct device_node *regulators;
+ int ret;
+
+ if (!pdev->dev.parent->of_node)
+ return 0;
+
+ regulators = of_find_node_by_name(np, "regulators");
+ if (!regulators) {
+ dev_err(&pdev->dev, "regulators node not found\n");
+ return -EINVAL;
+ }
+
+ ret = of_regulator_match(pdev->dev.parent, regulators,
+ max8907_matches,
+ ARRAY_SIZE(max8907_matches));
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Error parsing regulator init data: %d\n",
+ ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static inline struct regulator_init_data *match_init_data(int index)
+{
+ return max8907_matches[index].init_data;
+}
+
+static inline struct device_node *match_of_node(int index)
+{
+ return max8907_matches[index].of_node;
+}
+#else
+static int max8907_regulator_parse_dt(struct platform_device *pdev)
+{
+ return 0;
+}
+
+static inline struct regulator_init_data *match_init_data(int index)
+{
+ return NULL;
+}
+
+static inline struct device_node *match_of_node(int index)
+{
+ return NULL;
+}
+#endif
+
+static __devinit int max8907_regulator_probe(struct platform_device *pdev)
+{
+ struct max8907 *max8907 = dev_get_drvdata(pdev->dev.parent);
+ struct max8907_platform_data *pdata = dev_get_platdata(max8907->dev);
+ int ret;
+ struct max8907_regulator *pmic;
+ unsigned int val;
+ int i;
+ struct regulator_config config = {};
+ struct regulator_init_data *idata;
+ const char *mbatt_rail_name = NULL;
+
+ ret = max8907_regulator_parse_dt(pdev);
+ if (ret)
+ return ret;
+
+ pmic = devm_kzalloc(&pdev->dev, sizeof(*pmic), GFP_KERNEL);
+ if (!pmic) {
+ dev_err(&pdev->dev, "Failed to alloc pmic\n");
+ return -ENOMEM;
+ }
+ platform_set_drvdata(pdev, pmic);
+
+ memcpy(pmic->desc, max8907_regulators, sizeof(pmic->desc));
+
+ /* Backwards compatibility with MAX8907B; SD1 uses different voltages */
+ regmap_read(max8907->regmap_gen, MAX8907_REG_II2RR, &val);
+ if ((val & MAX8907_II2RR_VERSION_MASK) ==
+ MAX8907_II2RR_VERSION_REV_B) {
+ pmic->desc[MAX8907_SD1].min_uV = 637500;
+ pmic->desc[MAX8907_SD1].uV_step = 12500;
+ pmic->desc[MAX8907_SD1].n_voltages =
+ (1425000 - 637500) / 12500 + 1;
+ }
+
+ for (i = 0; i < MAX8907_NUM_REGULATORS; i++) {
+ config.dev = pdev->dev.parent;
+ if (pdata)
+ idata = pdata->init_data[i];
+ else
+ idata = match_init_data(i);
+ config.init_data = idata;
+ config.driver_data = pmic;
+ config.regmap = max8907->regmap_gen;
+ config.of_node = match_of_node(i);
+
+ switch (pmic->desc[i].id) {
+ case MAX8907_MBATT:
+ if (idata && idata->constraints.name)
+ mbatt_rail_name = idata->constraints.name;
+ else
+ mbatt_rail_name = pmic->desc[i].name;
+ break;
+ case MAX8907_BBAT:
+ case MAX8907_SDBY:
+ case MAX8907_VRTC:
+ idata->supply_regulator = mbatt_rail_name;
+ break;
+ }
+
+ if (pmic->desc[i].ops == &max8907_ldo_ops) {
+ regmap_read(config.regmap, pmic->desc[i].enable_reg,
+ &val);
+ if ((val & MAX8907_MASK_LDO_SEQ) !=
+ MAX8907_MASK_LDO_SEQ)
+ pmic->desc[i].ops = &max8907_ldo_hwctl_ops;
+ } else if (pmic->desc[i].ops == &max8907_out5v_ops) {
+ regmap_read(config.regmap, pmic->desc[i].enable_reg,
+ &val);
+ if ((val & (MAX8907_MASK_OUT5V_VINEN |
+ MAX8907_MASK_OUT5V_ENSRC)) !=
+ MAX8907_MASK_OUT5V_ENSRC)
+ pmic->desc[i].ops = &max8907_out5v_hwctl_ops;
+ }
+
+ pmic->rdev[i] = regulator_register(&pmic->desc[i], &config);
+ if (IS_ERR(pmic->rdev[i])) {
+ dev_err(&pdev->dev,
+ "failed to register %s regulator\n",
+ pmic->desc[i].name);
+ ret = PTR_ERR(pmic->rdev[i]);
+ goto err_unregister_regulator;
+ }
+ }
+
+ return 0;
+
+err_unregister_regulator:
+ while (--i >= 0)
+ regulator_unregister(pmic->rdev[i]);
+ return ret;
+}
+
+static __devexit int max8907_regulator_remove(struct platform_device *pdev)
+{
+ struct max8907_regulator *pmic = platform_get_drvdata(pdev);
+ int i;
+
+ for (i = 0; i < MAX8907_NUM_REGULATORS; i++)
+ regulator_unregister(pmic->rdev[i]);
+
+ return 0;
+}
+
+static struct platform_driver max8907_regulator_driver = {
+ .driver = {
+ .name = "max8907-regulator",
+ .owner = THIS_MODULE,
+ },
+ .probe = max8907_regulator_probe,
+ .remove = __devexit_p(max8907_regulator_remove),
+};
+
+static int __init max8907_regulator_init(void)
+{
+ return platform_driver_register(&max8907_regulator_driver);
+}
+
+subsys_initcall(max8907_regulator_init);
+
+static void __exit max8907_reg_exit(void)
+{
+ platform_driver_unregister(&max8907_regulator_driver);
+}
+
+module_exit(max8907_reg_exit);
+
+MODULE_DESCRIPTION("MAX8907 regulator driver");
+MODULE_AUTHOR("Gyungoh Yoo <jack.yoo@maxim-ic.com>");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:max8907-regulator");
diff --git a/drivers/regulator/mc13783-regulator.c b/drivers/regulator/mc13783-regulator.c
index 4932e34..0801a6d 100644
--- a/drivers/regulator/mc13783-regulator.c
+++ b/drivers/regulator/mc13783-regulator.c
@@ -21,6 +21,30 @@
#include <linux/module.h>
#include "mc13xxx.h"
+#define MC13783_REG_SWITCHERS0 24
+/* Enable does not exist for SW1A */
+#define MC13783_REG_SWITCHERS0_SW1AEN 0
+#define MC13783_REG_SWITCHERS0_SW1AVSEL 0
+#define MC13783_REG_SWITCHERS0_SW1AVSEL_M (63 << 0)
+
+#define MC13783_REG_SWITCHERS1 25
+/* Enable does not exist for SW1B */
+#define MC13783_REG_SWITCHERS1_SW1BEN 0
+#define MC13783_REG_SWITCHERS1_SW1BVSEL 0
+#define MC13783_REG_SWITCHERS1_SW1BVSEL_M (63 << 0)
+
+#define MC13783_REG_SWITCHERS2 26
+/* Enable does not exist for SW2A */
+#define MC13783_REG_SWITCHERS2_SW2AEN 0
+#define MC13783_REG_SWITCHERS2_SW2AVSEL 0
+#define MC13783_REG_SWITCHERS2_SW2AVSEL_M (63 << 0)
+
+#define MC13783_REG_SWITCHERS3 27
+/* Enable does not exist for SW2B */
+#define MC13783_REG_SWITCHERS3_SW2BEN 0
+#define MC13783_REG_SWITCHERS3_SW2BVSEL 0
+#define MC13783_REG_SWITCHERS3_SW2BVSEL_M (63 << 0)
+
#define MC13783_REG_SWITCHERS5 29
#define MC13783_REG_SWITCHERS5_SW3EN (1 << 20)
#define MC13783_REG_SWITCHERS5_SW3VSEL 18
@@ -93,6 +117,44 @@
/* Voltage Values */
+static const int mc13783_sw1x_val[] = {
+ 900000, 925000, 950000, 975000,
+ 1000000, 1025000, 1050000, 1075000,
+ 1100000, 1125000, 1150000, 1175000,
+ 1200000, 1225000, 1250000, 1275000,
+ 1300000, 1325000, 1350000, 1375000,
+ 1400000, 1425000, 1450000, 1475000,
+ 1500000, 1525000, 1550000, 1575000,
+ 1600000, 1625000, 1650000, 1675000,
+ 1700000, 1700000, 1700000, 1700000,
+ 1800000, 1800000, 1800000, 1800000,
+ 1850000, 1850000, 1850000, 1850000,
+ 2000000, 2000000, 2000000, 2000000,
+ 2100000, 2100000, 2100000, 2100000,
+ 2200000, 2200000, 2200000, 2200000,
+ 2200000, 2200000, 2200000, 2200000,
+ 2200000, 2200000, 2200000, 2200000,
+};
+
+static const int mc13783_sw2x_val[] = {
+ 900000, 925000, 950000, 975000,
+ 1000000, 1025000, 1050000, 1075000,
+ 1100000, 1125000, 1150000, 1175000,
+ 1200000, 1225000, 1250000, 1275000,
+ 1300000, 1325000, 1350000, 1375000,
+ 1400000, 1425000, 1450000, 1475000,
+ 1500000, 1525000, 1550000, 1575000,
+ 1600000, 1625000, 1650000, 1675000,
+ 1700000, 1700000, 1700000, 1700000,
+ 1800000, 1800000, 1800000, 1800000,
+ 1900000, 1900000, 1900000, 1900000,
+ 2000000, 2000000, 2000000, 2000000,
+ 2100000, 2100000, 2100000, 2100000,
+ 2200000, 2200000, 2200000, 2200000,
+ 2200000, 2200000, 2200000, 2200000,
+ 2200000, 2200000, 2200000, 2200000,
+};
+
static const unsigned int mc13783_sw3_val[] = {
5000000, 5000000, 5000000, 5500000,
};
@@ -188,6 +250,10 @@
MC13783_DEFINE(REG, _name, _reg, _vsel_reg, _voltages)
static struct mc13xxx_regulator mc13783_regulators[] = {
+ MC13783_DEFINE_SW(SW1A, SWITCHERS0, SWITCHERS0, mc13783_sw1x_val),
+ MC13783_DEFINE_SW(SW1B, SWITCHERS1, SWITCHERS1, mc13783_sw1x_val),
+ MC13783_DEFINE_SW(SW2A, SWITCHERS2, SWITCHERS2, mc13783_sw2x_val),
+ MC13783_DEFINE_SW(SW2B, SWITCHERS3, SWITCHERS3, mc13783_sw2x_val),
MC13783_DEFINE_SW(SW3, SWITCHERS5, SWITCHERS5, mc13783_sw3_val),
MC13783_FIXED_DEFINE(REG, VAUDIO, REGULATORMODE0, mc13783_vaudio_val),
@@ -238,9 +304,10 @@
BUG_ON(val & ~mask);
+ mc13xxx_lock(priv->mc13xxx);
ret = mc13xxx_reg_read(mc13783, MC13783_REG_POWERMISC, &valread);
if (ret)
- return ret;
+ goto out;
/* Update the stored state for Power Gates. */
priv->powermisc_pwgt_state =
@@ -253,7 +320,10 @@
valread = (valread & ~MC13783_REG_POWERMISC_PWGTSPI_M) |
priv->powermisc_pwgt_state;
- return mc13xxx_reg_write(mc13783, MC13783_REG_POWERMISC, valread);
+ ret = mc13xxx_reg_write(mc13783, MC13783_REG_POWERMISC, valread);
+out:
+ mc13xxx_unlock(priv->mc13xxx);
+ return ret;
}
static int mc13783_gpo_regulator_enable(struct regulator_dev *rdev)
@@ -261,7 +331,6 @@
struct mc13xxx_regulator_priv *priv = rdev_get_drvdata(rdev);
struct mc13xxx_regulator *mc13xxx_regulators = priv->mc13xxx_regulators;
int id = rdev_get_id(rdev);
- int ret;
u32 en_val = mc13xxx_regulators[id].enable_bit;
dev_dbg(rdev_get_dev(rdev), "%s id: %d\n", __func__, id);
@@ -271,12 +340,8 @@
id == MC13783_REG_PWGT2SPI)
en_val = 0;
- mc13xxx_lock(priv->mc13xxx);
- ret = mc13783_powermisc_rmw(priv, mc13xxx_regulators[id].enable_bit,
+ return mc13783_powermisc_rmw(priv, mc13xxx_regulators[id].enable_bit,
en_val);
- mc13xxx_unlock(priv->mc13xxx);
-
- return ret;
}
static int mc13783_gpo_regulator_disable(struct regulator_dev *rdev)
@@ -284,7 +349,6 @@
struct mc13xxx_regulator_priv *priv = rdev_get_drvdata(rdev);
struct mc13xxx_regulator *mc13xxx_regulators = priv->mc13xxx_regulators;
int id = rdev_get_id(rdev);
- int ret;
u32 dis_val = 0;
dev_dbg(rdev_get_dev(rdev), "%s id: %d\n", __func__, id);
@@ -294,12 +358,8 @@
id == MC13783_REG_PWGT2SPI)
dis_val = mc13xxx_regulators[id].enable_bit;
- mc13xxx_lock(priv->mc13xxx);
- ret = mc13783_powermisc_rmw(priv, mc13xxx_regulators[id].enable_bit,
+ return mc13783_powermisc_rmw(priv, mc13xxx_regulators[id].enable_bit,
dis_val);
- mc13xxx_unlock(priv->mc13xxx);
-
- return ret;
}
static int mc13783_gpo_regulator_is_enabled(struct regulator_dev *rdev)
@@ -330,7 +390,6 @@
.is_enabled = mc13783_gpo_regulator_is_enabled,
.list_voltage = regulator_list_voltage_table,
.set_voltage = mc13xxx_fixed_regulator_set_voltage,
- .get_voltage = mc13xxx_fixed_regulator_get_voltage,
};
static int __devinit mc13783_regulator_probe(struct platform_device *pdev)
diff --git a/drivers/regulator/mc13892-regulator.c b/drivers/regulator/mc13892-regulator.c
index b388b74..1fa6381 100644
--- a/drivers/regulator/mc13892-regulator.c
+++ b/drivers/regulator/mc13892-regulator.c
@@ -305,9 +305,10 @@
BUG_ON(val & ~mask);
+ mc13xxx_lock(priv->mc13xxx);
ret = mc13xxx_reg_read(mc13892, MC13892_POWERMISC, &valread);
if (ret)
- return ret;
+ goto out;
/* Update the stored state for Power Gates. */
priv->powermisc_pwgt_state =
@@ -320,14 +321,16 @@
valread = (valread & ~MC13892_POWERMISC_PWGTSPI_M) |
priv->powermisc_pwgt_state;
- return mc13xxx_reg_write(mc13892, MC13892_POWERMISC, valread);
+ ret = mc13xxx_reg_write(mc13892, MC13892_POWERMISC, valread);
+out:
+ mc13xxx_unlock(priv->mc13xxx);
+ return ret;
}
static int mc13892_gpo_regulator_enable(struct regulator_dev *rdev)
{
struct mc13xxx_regulator_priv *priv = rdev_get_drvdata(rdev);
int id = rdev_get_id(rdev);
- int ret;
u32 en_val = mc13892_regulators[id].enable_bit;
u32 mask = mc13892_regulators[id].enable_bit;
@@ -340,18 +343,13 @@
if (id == MC13892_GPO4)
mask |= MC13892_POWERMISC_GPO4ADINEN;
- mc13xxx_lock(priv->mc13xxx);
- ret = mc13892_powermisc_rmw(priv, mask, en_val);
- mc13xxx_unlock(priv->mc13xxx);
-
- return ret;
+ return mc13892_powermisc_rmw(priv, mask, en_val);
}
static int mc13892_gpo_regulator_disable(struct regulator_dev *rdev)
{
struct mc13xxx_regulator_priv *priv = rdev_get_drvdata(rdev);
int id = rdev_get_id(rdev);
- int ret;
u32 dis_val = 0;
dev_dbg(rdev_get_dev(rdev), "%s id: %d\n", __func__, id);
@@ -360,12 +358,8 @@
if (id == MC13892_PWGT1SPI || id == MC13892_PWGT2SPI)
dis_val = mc13892_regulators[id].enable_bit;
- mc13xxx_lock(priv->mc13xxx);
- ret = mc13892_powermisc_rmw(priv, mc13892_regulators[id].enable_bit,
+ return mc13892_powermisc_rmw(priv, mc13892_regulators[id].enable_bit,
dis_val);
- mc13xxx_unlock(priv->mc13xxx);
-
- return ret;
}
static int mc13892_gpo_regulator_is_enabled(struct regulator_dev *rdev)
@@ -396,14 +390,13 @@
.is_enabled = mc13892_gpo_regulator_is_enabled,
.list_voltage = regulator_list_voltage_table,
.set_voltage = mc13xxx_fixed_regulator_set_voltage,
- .get_voltage = mc13xxx_fixed_regulator_get_voltage,
};
-static int mc13892_sw_regulator_get_voltage(struct regulator_dev *rdev)
+static int mc13892_sw_regulator_get_voltage_sel(struct regulator_dev *rdev)
{
struct mc13xxx_regulator_priv *priv = rdev_get_drvdata(rdev);
int ret, id = rdev_get_id(rdev);
- unsigned int val, hi;
+ unsigned int val;
dev_dbg(rdev_get_dev(rdev), "%s id: %d\n", __func__, id);
@@ -414,17 +407,11 @@
if (ret)
return ret;
- hi = val & MC13892_SWITCHERS0_SWxHI;
val = (val & mc13892_regulators[id].vsel_mask)
>> mc13892_regulators[id].vsel_shift;
dev_dbg(rdev_get_dev(rdev), "%s id: %d val: %d\n", __func__, id, val);
- if (hi)
- val = (25000 * val) + 1100000;
- else
- val = (25000 * val) + 600000;
-
return val;
}
@@ -432,37 +419,25 @@
unsigned selector)
{
struct mc13xxx_regulator_priv *priv = rdev_get_drvdata(rdev);
- int hi, value, mask, id = rdev_get_id(rdev);
- u32 valread;
+ int volt, mask, id = rdev_get_id(rdev);
+ u32 reg_value;
int ret;
- value = rdev->desc->volt_table[selector];
+ volt = rdev->desc->volt_table[selector];
+ mask = mc13892_regulators[id].vsel_mask;
+ reg_value = selector << mc13892_regulators[id].vsel_shift;
+
+ if (volt > 1375000) {
+ mask |= MC13892_SWITCHERS0_SWxHI;
+ reg_value |= MC13892_SWITCHERS0_SWxHI;
+ } else if (volt < 1100000) {
+ mask |= MC13892_SWITCHERS0_SWxHI;
+ reg_value &= ~MC13892_SWITCHERS0_SWxHI;
+ }
mc13xxx_lock(priv->mc13xxx);
- ret = mc13xxx_reg_read(priv->mc13xxx,
- mc13892_regulators[id].vsel_reg, &valread);
- if (ret)
- goto err;
-
- if (value > 1375000)
- hi = 1;
- else if (value < 1100000)
- hi = 0;
- else
- hi = valread & MC13892_SWITCHERS0_SWxHI;
-
- if (hi) {
- value = (value - 1100000) / 25000;
- value |= MC13892_SWITCHERS0_SWxHI;
- } else
- value = (value - 600000) / 25000;
-
- mask = mc13892_regulators[id].vsel_mask | MC13892_SWITCHERS0_SWxHI;
- valread = (valread & ~mask) |
- (value << mc13892_regulators[id].vsel_shift);
- ret = mc13xxx_reg_write(priv->mc13xxx, mc13892_regulators[id].vsel_reg,
- valread);
-err:
+ ret = mc13xxx_reg_rmw(priv->mc13xxx, mc13892_regulators[id].reg, mask,
+ reg_value);
mc13xxx_unlock(priv->mc13xxx);
return ret;
@@ -471,7 +446,7 @@
static struct regulator_ops mc13892_sw_regulator_ops = {
.list_voltage = regulator_list_voltage_table,
.set_voltage_sel = mc13892_sw_regulator_set_voltage_sel,
- .get_voltage = mc13892_sw_regulator_get_voltage,
+ .get_voltage_sel = mc13892_sw_regulator_get_voltage_sel,
};
static int mc13892_vcam_set_mode(struct regulator_dev *rdev, unsigned int mode)
diff --git a/drivers/regulator/mc13xxx-regulator-core.c b/drivers/regulator/mc13xxx-regulator-core.c
index d6eda28..88cbb83 100644
--- a/drivers/regulator/mc13xxx-regulator-core.c
+++ b/drivers/regulator/mc13xxx-regulator-core.c
@@ -143,30 +143,21 @@
__func__, id, min_uV, max_uV);
if (min_uV <= rdev->desc->volt_table[0] &&
- rdev->desc->volt_table[0] <= max_uV)
+ rdev->desc->volt_table[0] <= max_uV) {
+ *selector = 0;
return 0;
- else
+ } else {
return -EINVAL;
+ }
}
EXPORT_SYMBOL_GPL(mc13xxx_fixed_regulator_set_voltage);
-int mc13xxx_fixed_regulator_get_voltage(struct regulator_dev *rdev)
-{
- int id = rdev_get_id(rdev);
-
- dev_dbg(rdev_get_dev(rdev), "%s id: %d\n", __func__, id);
-
- return rdev->desc->volt_table[0];
-}
-EXPORT_SYMBOL_GPL(mc13xxx_fixed_regulator_get_voltage);
-
struct regulator_ops mc13xxx_fixed_regulator_ops = {
.enable = mc13xxx_regulator_enable,
.disable = mc13xxx_regulator_disable,
.is_enabled = mc13xxx_regulator_is_enabled,
.list_voltage = regulator_list_voltage_table,
.set_voltage = mc13xxx_fixed_regulator_set_voltage,
- .get_voltage = mc13xxx_fixed_regulator_get_voltage,
};
EXPORT_SYMBOL_GPL(mc13xxx_fixed_regulator_ops);
diff --git a/drivers/regulator/mc13xxx.h b/drivers/regulator/mc13xxx.h
index eaff551..06c8903 100644
--- a/drivers/regulator/mc13xxx.h
+++ b/drivers/regulator/mc13xxx.h
@@ -34,7 +34,6 @@
extern int mc13xxx_fixed_regulator_set_voltage(struct regulator_dev *rdev,
int min_uV, int max_uV, unsigned *selector);
-extern int mc13xxx_fixed_regulator_get_voltage(struct regulator_dev *rdev);
#ifdef CONFIG_OF
extern int mc13xxx_get_num_regulators_dt(struct platform_device *pdev);
diff --git a/drivers/regulator/of_regulator.c b/drivers/regulator/of_regulator.c
index 3e4106f..6f68491 100644
--- a/drivers/regulator/of_regulator.c
+++ b/drivers/regulator/of_regulator.c
@@ -92,16 +92,18 @@
EXPORT_SYMBOL_GPL(of_get_regulator_init_data);
/**
- * of_regulator_match - extract regulator init data when node
- * property "regulator-compatible" matches with the regulator name.
+ * of_regulator_match - extract multiple regulator init data from device tree.
* @dev: device requesting the data
* @node: parent device node of the regulators
* @matches: match table for the regulators
* @num_matches: number of entries in match table
*
- * This function uses a match table specified by the regulator driver and
- * looks up the corresponding init data in the device tree if
- * regulator-compatible matches. Note that the match table is modified
+ * This function uses a match table specified by the regulator driver to
+ * parse regulator init data from the device tree. @node is expected to
+ * contain a set of child nodes, each providing the init data for one
+ * regulator. The data parsed from a child node will be matched to a regulator
+ * based on either the deprecated property regulator-compatible if present,
+ * or otherwise the child node's name. Note that the match table is modified
* in place.
*
* Returns the number of matches found or a negative error code on failure.
@@ -112,26 +114,23 @@
{
unsigned int count = 0;
unsigned int i;
- const char *regulator_comp;
+ const char *name;
struct device_node *child;
if (!dev || !node)
return -EINVAL;
for_each_child_of_node(node, child) {
- regulator_comp = of_get_property(child,
+ name = of_get_property(child,
"regulator-compatible", NULL);
- if (!regulator_comp) {
- dev_err(dev, "regulator-compatible is missing for node %s\n",
- child->name);
- continue;
- }
+ if (!name)
+ name = child->name;
for (i = 0; i < num_matches; i++) {
struct of_regulator_match *match = &matches[i];
if (match->of_node)
continue;
- if (strcmp(match->name, regulator_comp))
+ if (strcmp(match->name, name))
continue;
match->init_data =
diff --git a/drivers/regulator/palmas-regulator.c b/drivers/regulator/palmas-regulator.c
index 46c7e88..2ba7502 100644
--- a/drivers/regulator/palmas-regulator.c
+++ b/drivers/regulator/palmas-regulator.c
@@ -443,44 +443,6 @@
return 850000 + (selector * 50000);
}
-static int palmas_get_voltage_ldo_sel(struct regulator_dev *dev)
-{
- struct palmas_pmic *pmic = rdev_get_drvdata(dev);
- int id = rdev_get_id(dev);
- int selector;
- unsigned int reg;
- unsigned int addr;
-
- addr = palmas_regs_info[id].vsel_addr;
-
- palmas_ldo_read(pmic->palmas, addr, ®);
-
- selector = reg & PALMAS_LDO1_VOLTAGE_VSEL_MASK;
-
- /* Adjust selector to match list_voltage ranges */
- if (selector > 49)
- selector = 49;
-
- return selector;
-}
-
-static int palmas_set_voltage_ldo_sel(struct regulator_dev *dev,
- unsigned selector)
-{
- struct palmas_pmic *pmic = rdev_get_drvdata(dev);
- int id = rdev_get_id(dev);
- unsigned int reg = 0;
- unsigned int addr;
-
- addr = palmas_regs_info[id].vsel_addr;
-
- reg = selector;
-
- palmas_ldo_write(pmic->palmas, addr, reg);
-
- return 0;
-}
-
static int palmas_map_voltage_ldo(struct regulator_dev *rdev,
int min_uV, int max_uV)
{
@@ -505,8 +467,8 @@
.is_enabled = palmas_is_enabled_ldo,
.enable = regulator_enable_regmap,
.disable = regulator_disable_regmap,
- .get_voltage_sel = palmas_get_voltage_ldo_sel,
- .set_voltage_sel = palmas_set_voltage_ldo_sel,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
+ .set_voltage_sel = regulator_set_voltage_sel_regmap,
.list_voltage = palmas_list_voltage_ldo,
.map_voltage = palmas_map_voltage_ldo,
};
@@ -757,6 +719,9 @@
pmic->desc[id].type = REGULATOR_VOLTAGE;
pmic->desc[id].owner = THIS_MODULE;
+ pmic->desc[id].vsel_reg = PALMAS_BASE_TO_REG(PALMAS_LDO_BASE,
+ palmas_regs_info[id].vsel_addr);
+ pmic->desc[id].vsel_mask = PALMAS_LDO1_VOLTAGE_VSEL_MASK;
pmic->desc[id].enable_reg = PALMAS_BASE_TO_REG(PALMAS_LDO_BASE,
palmas_regs_info[id].ctrl_addr);
pmic->desc[id].enable_mask = PALMAS_LDO1_CTRL_MODE_ACTIVE;
diff --git a/drivers/regulator/s2mps11.c b/drivers/regulator/s2mps11.c
index 4669dc9..926f9c8 100644
--- a/drivers/regulator/s2mps11.c
+++ b/drivers/regulator/s2mps11.c
@@ -24,7 +24,7 @@
#include <linux/mfd/samsung/s2mps11.h>
struct s2mps11_info {
- struct regulator_dev **rdev;
+ struct regulator_dev *rdev[S2MPS11_REGULATOR_MAX];
int ramp_delay2;
int ramp_delay34;
@@ -236,9 +236,8 @@
struct sec_pmic_dev *iodev = dev_get_drvdata(pdev->dev.parent);
struct sec_platform_data *pdata = dev_get_platdata(iodev->dev);
struct regulator_config config = { };
- struct regulator_dev **rdev;
struct s2mps11_info *s2mps11;
- int i, ret, size;
+ int i, ret;
unsigned char ramp_enable, ramp_reg = 0;
if (!pdata) {
@@ -251,13 +250,6 @@
if (!s2mps11)
return -ENOMEM;
- size = sizeof(struct regulator_dev *) * S2MPS11_REGULATOR_MAX;
- s2mps11->rdev = devm_kzalloc(&pdev->dev, size, GFP_KERNEL);
- if (!s2mps11->rdev) {
- return -ENOMEM;
- }
-
- rdev = s2mps11->rdev;
platform_set_drvdata(pdev, s2mps11);
s2mps11->ramp_delay2 = pdata->buck2_ramp_delay;
@@ -297,12 +289,12 @@
config.init_data = pdata->regulators[i].initdata;
config.driver_data = s2mps11;
- rdev[i] = regulator_register(®ulators[i], &config);
- if (IS_ERR(rdev[i])) {
- ret = PTR_ERR(rdev[i]);
+ s2mps11->rdev[i] = regulator_register(®ulators[i], &config);
+ if (IS_ERR(s2mps11->rdev[i])) {
+ ret = PTR_ERR(s2mps11->rdev[i]);
dev_err(&pdev->dev, "regulator init failed for %d\n",
i);
- rdev[i] = NULL;
+ s2mps11->rdev[i] = NULL;
goto err;
}
}
@@ -310,8 +302,7 @@
return 0;
err:
for (i = 0; i < S2MPS11_REGULATOR_MAX; i++)
- if (rdev[i])
- regulator_unregister(rdev[i]);
+ regulator_unregister(s2mps11->rdev[i]);
return ret;
}
@@ -319,12 +310,10 @@
static int __devexit s2mps11_pmic_remove(struct platform_device *pdev)
{
struct s2mps11_info *s2mps11 = platform_get_drvdata(pdev);
- struct regulator_dev **rdev = s2mps11->rdev;
int i;
for (i = 0; i < S2MPS11_REGULATOR_MAX; i++)
- if (rdev[i])
- regulator_unregister(rdev[i]);
+ regulator_unregister(s2mps11->rdev[i]);
return 0;
}
diff --git a/drivers/regulator/tps65217-regulator.c b/drivers/regulator/tps65217-regulator.c
index 6caa222..ab00cab 100644
--- a/drivers/regulator/tps65217-regulator.c
+++ b/drivers/regulator/tps65217-regulator.c
@@ -22,6 +22,7 @@
#include <linux/err.h>
#include <linux/platform_device.h>
+#include <linux/regulator/of_regulator.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
#include <linux/mfd/tps65217.h>
@@ -281,37 +282,130 @@
NULL),
};
+#ifdef CONFIG_OF
+static struct of_regulator_match reg_matches[] = {
+ { .name = "dcdc1", .driver_data = (void *)TPS65217_DCDC_1 },
+ { .name = "dcdc2", .driver_data = (void *)TPS65217_DCDC_2 },
+ { .name = "dcdc3", .driver_data = (void *)TPS65217_DCDC_3 },
+ { .name = "ldo1", .driver_data = (void *)TPS65217_LDO_1 },
+ { .name = "ldo2", .driver_data = (void *)TPS65217_LDO_2 },
+ { .name = "ldo3", .driver_data = (void *)TPS65217_LDO_3 },
+ { .name = "ldo4", .driver_data = (void *)TPS65217_LDO_4 },
+};
+
+static struct tps65217_board *tps65217_parse_dt(struct platform_device *pdev)
+{
+ struct tps65217 *tps = dev_get_drvdata(pdev->dev.parent);
+ struct device_node *node = tps->dev->of_node;
+ struct tps65217_board *pdata;
+ struct device_node *regs;
+ int i, count;
+
+ regs = of_find_node_by_name(node, "regulators");
+ if (!regs)
+ return NULL;
+
+ count = of_regulator_match(pdev->dev.parent, regs,
+ reg_matches, TPS65217_NUM_REGULATOR);
+ of_node_put(regs);
+ if ((count < 0) || (count > TPS65217_NUM_REGULATOR))
+ return NULL;
+
+ pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
+ if (!pdata)
+ return NULL;
+
+ for (i = 0; i < count; i++) {
+ if (!reg_matches[i].init_data || !reg_matches[i].of_node)
+ continue;
+
+ pdata->tps65217_init_data[i] = reg_matches[i].init_data;
+ pdata->of_node[i] = reg_matches[i].of_node;
+ }
+
+ return pdata;
+}
+#else
+static struct tps65217_board *tps65217_parse_dt(struct platform_device *pdev)
+{
+ return NULL;
+}
+#endif
+
static int __devinit tps65217_regulator_probe(struct platform_device *pdev)
{
+ struct tps65217 *tps = dev_get_drvdata(pdev->dev.parent);
+ struct tps65217_board *pdata = dev_get_platdata(tps->dev);
+ struct regulator_init_data *reg_data;
struct regulator_dev *rdev;
- struct tps65217 *tps;
- struct tps_info *info = &tps65217_pmic_regs[pdev->id];
struct regulator_config config = { };
+ int i, ret;
- /* Already set by core driver */
- tps = dev_to_tps65217(pdev->dev.parent);
- tps->info[pdev->id] = info;
+ if (tps->dev->of_node)
+ pdata = tps65217_parse_dt(pdev);
- config.dev = &pdev->dev;
- config.of_node = pdev->dev.of_node;
- config.init_data = pdev->dev.platform_data;
- config.driver_data = tps;
+ if (!pdata) {
+ dev_err(&pdev->dev, "Platform data not found\n");
+ return -EINVAL;
+ }
- rdev = regulator_register(®ulators[pdev->id], &config);
- if (IS_ERR(rdev))
- return PTR_ERR(rdev);
+ if (tps65217_chip_id(tps) != TPS65217) {
+ dev_err(&pdev->dev, "Invalid tps chip version\n");
+ return -ENODEV;
+ }
- platform_set_drvdata(pdev, rdev);
+ platform_set_drvdata(pdev, tps);
+ for (i = 0; i < TPS65217_NUM_REGULATOR; i++) {
+
+ reg_data = pdata->tps65217_init_data[i];
+
+ /*
+ * Regulator API handles empty constraints but not NULL
+ * constraints
+ */
+ if (!reg_data)
+ continue;
+
+ /* Register the regulators */
+ tps->info[i] = &tps65217_pmic_regs[i];
+
+ config.dev = tps->dev;
+ config.init_data = reg_data;
+ config.driver_data = tps;
+ config.regmap = tps->regmap;
+ if (tps->dev->of_node)
+ config.of_node = pdata->of_node[i];
+
+ rdev = regulator_register(®ulators[i], &config);
+ if (IS_ERR(rdev)) {
+ dev_err(tps->dev, "failed to register %s regulator\n",
+ pdev->name);
+ ret = PTR_ERR(rdev);
+ goto err_unregister_regulator;
+ }
+
+ /* Save regulator for cleanup */
+ tps->rdev[i] = rdev;
+ }
return 0;
+
+err_unregister_regulator:
+ while (--i >= 0)
+ regulator_unregister(tps->rdev[i]);
+
+ return ret;
}
static int __devexit tps65217_regulator_remove(struct platform_device *pdev)
{
- struct regulator_dev *rdev = platform_get_drvdata(pdev);
+ struct tps65217 *tps = platform_get_drvdata(pdev);
+ unsigned int i;
+
+ for (i = 0; i < TPS65217_NUM_REGULATOR; i++)
+ regulator_unregister(tps->rdev[i]);
platform_set_drvdata(pdev, NULL);
- regulator_unregister(rdev);
return 0;
}
diff --git a/drivers/regulator/tps6524x-regulator.c b/drivers/regulator/tps6524x-regulator.c
index 947ece9..058d2f2 100644
--- a/drivers/regulator/tps6524x-regulator.c
+++ b/drivers/regulator/tps6524x-regulator.c
@@ -502,15 +502,13 @@
if (info->n_ilimsels == 1)
return -EINVAL;
- for (i = 0; i < info->n_ilimsels; i++)
+ for (i = info->n_ilimsels - 1; i >= 0; i--) {
if (min_uA <= info->ilimsels[i] &&
max_uA >= info->ilimsels[i])
- break;
+ return write_field(hw, &info->ilimsel, i);
+ }
- if (i >= info->n_ilimsels)
- return -EINVAL;
-
- return write_field(hw, &info->ilimsel, i);
+ return -EINVAL;
}
static int get_current_limit(struct regulator_dev *rdev)
diff --git a/drivers/regulator/tps6586x-regulator.c b/drivers/regulator/tps6586x-regulator.c
index 19241fc..ce1e7cb 100644
--- a/drivers/regulator/tps6586x-regulator.c
+++ b/drivers/regulator/tps6586x-regulator.c
@@ -57,9 +57,6 @@
struct tps6586x_regulator {
struct regulator_desc desc;
- int volt_reg;
- int volt_shift;
- int volt_nbits;
int enable_bit[2];
int enable_reg[2];
@@ -81,10 +78,10 @@
int ret, val, rid = rdev_get_id(rdev);
uint8_t mask;
- val = selector << ri->volt_shift;
- mask = ((1 << ri->volt_nbits) - 1) << ri->volt_shift;
+ val = selector << (ffs(rdev->desc->vsel_mask) - 1);
+ mask = rdev->desc->vsel_mask;
- ret = tps6586x_update(parent, ri->volt_reg, val, mask);
+ ret = tps6586x_update(parent, rdev->desc->vsel_reg, val, mask);
if (ret)
return ret;
@@ -100,66 +97,17 @@
return ret;
}
-static int tps6586x_get_voltage_sel(struct regulator_dev *rdev)
-{
- struct tps6586x_regulator *ri = rdev_get_drvdata(rdev);
- struct device *parent = to_tps6586x_dev(rdev);
- uint8_t val, mask;
- int ret;
-
- ret = tps6586x_read(parent, ri->volt_reg, &val);
- if (ret)
- return ret;
-
- mask = ((1 << ri->volt_nbits) - 1) << ri->volt_shift;
- val = (val & mask) >> ri->volt_shift;
-
- if (val >= ri->desc.n_voltages)
- BUG();
-
- return val;
-}
-
-static int tps6586x_regulator_enable(struct regulator_dev *rdev)
-{
- struct tps6586x_regulator *ri = rdev_get_drvdata(rdev);
- struct device *parent = to_tps6586x_dev(rdev);
-
- return tps6586x_set_bits(parent, ri->enable_reg[0],
- 1 << ri->enable_bit[0]);
-}
-
-static int tps6586x_regulator_disable(struct regulator_dev *rdev)
-{
- struct tps6586x_regulator *ri = rdev_get_drvdata(rdev);
- struct device *parent = to_tps6586x_dev(rdev);
-
- return tps6586x_clr_bits(parent, ri->enable_reg[0],
- 1 << ri->enable_bit[0]);
-}
-
-static int tps6586x_regulator_is_enabled(struct regulator_dev *rdev)
-{
- struct tps6586x_regulator *ri = rdev_get_drvdata(rdev);
- struct device *parent = to_tps6586x_dev(rdev);
- uint8_t reg_val;
- int ret;
-
- ret = tps6586x_read(parent, ri->enable_reg[0], ®_val);
- if (ret)
- return ret;
-
- return !!(reg_val & (1 << ri->enable_bit[0]));
-}
-
static struct regulator_ops tps6586x_regulator_ops = {
.list_voltage = regulator_list_voltage_table,
- .get_voltage_sel = tps6586x_get_voltage_sel,
+ .get_voltage_sel = regulator_get_voltage_sel_regmap,
.set_voltage_sel = tps6586x_set_voltage_sel,
- .is_enabled = tps6586x_regulator_is_enabled,
- .enable = tps6586x_regulator_enable,
- .disable = tps6586x_regulator_disable,
+ .is_enabled = regulator_is_enabled_regmap,
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+};
+
+static struct regulator_ops tps6586x_sys_regulator_ops = {
};
static const unsigned int tps6586x_ldo0_voltages[] = {
@@ -202,10 +150,11 @@
.n_voltages = ARRAY_SIZE(tps6586x_##vdata##_voltages), \
.volt_table = tps6586x_##vdata##_voltages, \
.owner = THIS_MODULE, \
+ .enable_reg = TPS6586X_SUPPLY##ereg0, \
+ .enable_mask = 1 << (ebit0), \
+ .vsel_reg = TPS6586X_##vreg, \
+ .vsel_mask = ((1 << (nbits)) - 1) << (shift), \
}, \
- .volt_reg = TPS6586X_##vreg, \
- .volt_shift = (shift), \
- .volt_nbits = (nbits), \
.enable_reg[0] = TPS6586X_SUPPLY##ereg0, \
.enable_bit[0] = (ebit0), \
.enable_reg[1] = TPS6586X_SUPPLY##ereg1, \
@@ -230,15 +179,28 @@
TPS6586X_REGULATOR_DVM_GOREG(goreg, gobit) \
}
+#define TPS6586X_SYS_REGULATOR() \
+{ \
+ .desc = { \
+ .supply_name = "sys", \
+ .name = "REG-SYS", \
+ .ops = &tps6586x_sys_regulator_ops, \
+ .type = REGULATOR_VOLTAGE, \
+ .id = TPS6586X_ID_SYS, \
+ .owner = THIS_MODULE, \
+ }, \
+}
+
static struct tps6586x_regulator tps6586x_regulator[] = {
+ TPS6586X_SYS_REGULATOR(),
TPS6586X_LDO(LDO_0, "vinldo01", ldo0, SUPPLYV1, 5, 3, ENC, 0, END, 0),
TPS6586X_LDO(LDO_3, "vinldo23", ldo, SUPPLYV4, 0, 3, ENC, 2, END, 2),
- TPS6586X_LDO(LDO_5, NULL, ldo, SUPPLYV6, 0, 3, ENE, 6, ENE, 6),
+ TPS6586X_LDO(LDO_5, "REG-SYS", ldo, SUPPLYV6, 0, 3, ENE, 6, ENE, 6),
TPS6586X_LDO(LDO_6, "vinldo678", ldo, SUPPLYV3, 0, 3, ENC, 4, END, 4),
TPS6586X_LDO(LDO_7, "vinldo678", ldo, SUPPLYV3, 3, 3, ENC, 5, END, 5),
TPS6586X_LDO(LDO_8, "vinldo678", ldo, SUPPLYV2, 5, 3, ENC, 6, END, 6),
TPS6586X_LDO(LDO_9, "vinldo9", ldo, SUPPLYV6, 3, 3, ENE, 7, ENE, 7),
- TPS6586X_LDO(LDO_RTC, NULL, ldo, SUPPLYV4, 3, 3, V4, 7, V4, 7),
+ TPS6586X_LDO(LDO_RTC, "REG-SYS", ldo, SUPPLYV4, 3, 3, V4, 7, V4, 7),
TPS6586X_LDO(LDO_1, "vinldo01", dvm, SUPPLYV1, 0, 5, ENC, 1, END, 1),
TPS6586X_LDO(SM_2, "vin-sm2", sm2, SUPPLYV2, 0, 5, ENC, 7, END, 7),
diff --git a/drivers/regulator/twl-regulator.c b/drivers/regulator/twl-regulator.c
index 77a71a5..7eb986a 100644
--- a/drivers/regulator/twl-regulator.c
+++ b/drivers/regulator/twl-regulator.c
@@ -10,6 +10,8 @@
*/
#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/platform_device.h>
@@ -624,18 +626,9 @@
return info->min_mV * 1000;
}
-static int twlfixed_get_voltage(struct regulator_dev *rdev)
-{
- struct twlreg_info *info = rdev_get_drvdata(rdev);
-
- return info->min_mV * 1000;
-}
-
static struct regulator_ops twl4030fixed_ops = {
.list_voltage = twlfixed_list_voltage,
- .get_voltage = twlfixed_get_voltage,
-
.enable = twl4030reg_enable,
.disable = twl4030reg_disable,
.is_enabled = twl4030reg_is_enabled,
@@ -648,8 +641,6 @@
static struct regulator_ops twl6030fixed_ops = {
.list_voltage = twlfixed_list_voltage,
- .get_voltage = twlfixed_get_voltage,
-
.enable = twl6030reg_enable,
.disable = twl6030reg_disable,
.is_enabled = twl6030reg_is_enabled,
@@ -659,13 +650,6 @@
.get_status = twl6030reg_get_status,
};
-static struct regulator_ops twl6030_fixed_resource = {
- .enable = twl6030reg_enable,
- .disable = twl6030reg_disable,
- .is_enabled = twl6030reg_is_enabled,
- .get_status = twl6030reg_get_status,
-};
-
/*
* SMPS status and control
*/
@@ -757,37 +741,32 @@
return voltage;
}
-static int
-twl6030smps_set_voltage(struct regulator_dev *rdev, int min_uV, int max_uV,
- unsigned int *selector)
+static int twl6030smps_map_voltage(struct regulator_dev *rdev, int min_uV,
+ int max_uV)
{
- struct twlreg_info *info = rdev_get_drvdata(rdev);
- int vsel = 0;
+ struct twlreg_info *info = rdev_get_drvdata(rdev);
+ int vsel = 0;
switch (info->flags) {
case 0:
if (min_uV == 0)
vsel = 0;
else if ((min_uV >= 600000) && (min_uV <= 1300000)) {
- int calc_uV;
vsel = DIV_ROUND_UP(min_uV - 600000, 12500);
vsel++;
- calc_uV = twl6030smps_list_voltage(rdev, vsel);
- if (calc_uV > max_uV)
- return -EINVAL;
}
/* Values 1..57 for vsel are linear and can be calculated
* values 58..62 are non linear.
*/
- else if ((min_uV > 1900000) && (max_uV >= 2100000))
+ else if ((min_uV > 1900000) && (min_uV <= 2100000))
vsel = 62;
- else if ((min_uV > 1800000) && (max_uV >= 1900000))
+ else if ((min_uV > 1800000) && (min_uV <= 1900000))
vsel = 61;
- else if ((min_uV > 1500000) && (max_uV >= 1800000))
+ else if ((min_uV > 1500000) && (min_uV <= 1800000))
vsel = 60;
- else if ((min_uV > 1350000) && (max_uV >= 1500000))
+ else if ((min_uV > 1350000) && (min_uV <= 1500000))
vsel = 59;
- else if ((min_uV > 1300000) && (max_uV >= 1350000))
+ else if ((min_uV > 1300000) && (min_uV <= 1350000))
vsel = 58;
else
return -EINVAL;
@@ -796,25 +775,21 @@
if (min_uV == 0)
vsel = 0;
else if ((min_uV >= 700000) && (min_uV <= 1420000)) {
- int calc_uV;
vsel = DIV_ROUND_UP(min_uV - 700000, 12500);
vsel++;
- calc_uV = twl6030smps_list_voltage(rdev, vsel);
- if (calc_uV > max_uV)
- return -EINVAL;
}
/* Values 1..57 for vsel are linear and can be calculated
* values 58..62 are non linear.
*/
- else if ((min_uV > 1900000) && (max_uV >= 2100000))
+ else if ((min_uV > 1900000) && (min_uV <= 2100000))
vsel = 62;
- else if ((min_uV > 1800000) && (max_uV >= 1900000))
+ else if ((min_uV > 1800000) && (min_uV <= 1900000))
vsel = 61;
- else if ((min_uV > 1350000) && (max_uV >= 1800000))
+ else if ((min_uV > 1350000) && (min_uV <= 1800000))
vsel = 60;
- else if ((min_uV > 1350000) && (max_uV >= 1500000))
+ else if ((min_uV > 1350000) && (min_uV <= 1500000))
vsel = 59;
- else if ((min_uV > 1300000) && (max_uV >= 1350000))
+ else if ((min_uV > 1300000) && (min_uV <= 1350000))
vsel = 58;
else
return -EINVAL;
@@ -830,17 +805,23 @@
case SMPS_OFFSET_EN|SMPS_EXTENDED_EN:
if (min_uV == 0) {
vsel = 0;
- } else if ((min_uV >= 2161000) && (max_uV <= 4321000)) {
+ } else if ((min_uV >= 2161000) && (min_uV <= 4321000)) {
vsel = DIV_ROUND_UP(min_uV - 2161000, 38600);
vsel++;
}
break;
}
- *selector = vsel;
+ return vsel;
+}
+
+static int twl6030smps_set_voltage_sel(struct regulator_dev *rdev,
+ unsigned int selector)
+{
+ struct twlreg_info *info = rdev_get_drvdata(rdev);
return twlreg_write(info, TWL_MODULE_PM_RECEIVER, VREG_VOLTAGE_SMPS,
- vsel);
+ selector);
}
static int twl6030smps_get_voltage_sel(struct regulator_dev *rdev)
@@ -852,8 +833,9 @@
static struct regulator_ops twlsmps_ops = {
.list_voltage = twl6030smps_list_voltage,
+ .map_voltage = twl6030smps_map_voltage,
- .set_voltage = twl6030smps_set_voltage,
+ .set_voltage_sel = twl6030smps_set_voltage_sel,
.get_voltage_sel = twl6030smps_get_voltage_sel,
.enable = twl6030reg_enable,
@@ -876,7 +858,7 @@
0x0, TWL6030, twl6030fixed_ops)
#define TWL4030_ADJUSTABLE_LDO(label, offset, num, turnon_delay, remap_conf) \
-static struct twlreg_info TWL4030_INFO_##label = { \
+static const struct twlreg_info TWL4030_INFO_##label = { \
.base = offset, \
.id = num, \
.table_len = ARRAY_SIZE(label##_VSEL_table), \
@@ -894,7 +876,7 @@
}
#define TWL4030_ADJUSTABLE_SMPS(label, offset, num, turnon_delay, remap_conf) \
-static struct twlreg_info TWL4030_INFO_##label = { \
+static const struct twlreg_info TWL4030_INFO_##label = { \
.base = offset, \
.id = num, \
.remap = remap_conf, \
@@ -909,7 +891,7 @@
}
#define TWL6030_ADJUSTABLE_SMPS(label) \
-static struct twlreg_info TWL6030_INFO_##label = { \
+static const struct twlreg_info TWL6030_INFO_##label = { \
.desc = { \
.name = #label, \
.id = TWL6030_REG_##label, \
@@ -920,7 +902,7 @@
}
#define TWL6030_ADJUSTABLE_LDO(label, offset, min_mVolts, max_mVolts) \
-static struct twlreg_info TWL6030_INFO_##label = { \
+static const struct twlreg_info TWL6030_INFO_##label = { \
.base = offset, \
.min_mV = min_mVolts, \
.max_mV = max_mVolts, \
@@ -935,7 +917,7 @@
}
#define TWL6025_ADJUSTABLE_LDO(label, offset, min_mVolts, max_mVolts) \
-static struct twlreg_info TWL6025_INFO_##label = { \
+static const struct twlreg_info TWL6025_INFO_##label = { \
.base = offset, \
.min_mV = min_mVolts, \
.max_mV = max_mVolts, \
@@ -951,7 +933,7 @@
#define TWL_FIXED_LDO(label, offset, mVolts, num, turnon_delay, remap_conf, \
family, operations) \
-static struct twlreg_info TWLFIXED_INFO_##label = { \
+static const struct twlreg_info TWLFIXED_INFO_##label = { \
.base = offset, \
.id = num, \
.min_mV = mVolts, \
@@ -981,7 +963,7 @@
}
#define TWL6025_ADJUSTABLE_SMPS(label, offset) \
-static struct twlreg_info TWLSMPS_INFO_##label = { \
+static const struct twlreg_info TWLSMPS_INFO_##label = { \
.base = offset, \
.min_mV = 600, \
.max_mV = 2100, \
@@ -1138,6 +1120,7 @@
{
int i, id;
struct twlreg_info *info;
+ const struct twlreg_info *template;
struct regulator_init_data *initdata;
struct regulation_constraints *c;
struct regulator_dev *rdev;
@@ -1147,17 +1130,17 @@
match = of_match_device(twl_of_match, &pdev->dev);
if (match) {
- info = match->data;
- id = info->desc.id;
+ template = match->data;
+ id = template->desc.id;
initdata = of_get_regulator_init_data(&pdev->dev,
pdev->dev.of_node);
drvdata = NULL;
} else {
id = pdev->id;
initdata = pdev->dev.platform_data;
- for (i = 0, info = NULL; i < ARRAY_SIZE(twl_of_match); i++) {
- info = twl_of_match[i].data;
- if (info && info->desc.id == id)
+ for (i = 0, template = NULL; i < ARRAY_SIZE(twl_of_match); i++) {
+ template = twl_of_match[i].data;
+ if (template && template->desc.id == id)
break;
}
if (i == ARRAY_SIZE(twl_of_match))
@@ -1168,12 +1151,16 @@
return -EINVAL;
}
- if (!info)
+ if (!template)
return -ENODEV;
if (!initdata)
return -EINVAL;
+ info = kmemdup(template, sizeof (*info), GFP_KERNEL);
+ if (!info)
+ return -ENOMEM;
+
if (drvdata) {
/* copy the driver data into regulator data */
info->features = drvdata->features;
@@ -1234,6 +1221,7 @@
if (IS_ERR(rdev)) {
dev_err(&pdev->dev, "can't register %s, %ld\n",
info->desc.name, PTR_ERR(rdev));
+ kfree(info);
return PTR_ERR(rdev);
}
platform_set_drvdata(pdev, rdev);
@@ -1255,7 +1243,11 @@
static int __devexit twlreg_remove(struct platform_device *pdev)
{
- regulator_unregister(platform_get_drvdata(pdev));
+ struct regulator_dev *rdev = platform_get_drvdata(pdev);
+ struct twlreg_info *info = rdev->reg_data;
+
+ regulator_unregister(rdev);
+ kfree(info);
return 0;
}
diff --git a/drivers/regulator/wm831x-dcdc.c b/drivers/regulator/wm831x-dcdc.c
index 7413885..90cbcc6 100644
--- a/drivers/regulator/wm831x-dcdc.c
+++ b/drivers/regulator/wm831x-dcdc.c
@@ -339,16 +339,15 @@
u16 reg = dcdc->base + WM831X_DCDC_CONTROL_2;
int i;
- for (i = 0; i < ARRAY_SIZE(wm831x_dcdc_ilim); i++) {
+ for (i = ARRAY_SIZE(wm831x_dcdc_ilim) - 1; i >= 0; i--) {
if ((min_uA <= wm831x_dcdc_ilim[i]) &&
(wm831x_dcdc_ilim[i] <= max_uA))
- break;
+ return wm831x_set_bits(wm831x, reg,
+ WM831X_DC1_HC_THR_MASK,
+ i << WM831X_DC1_HC_THR_SHIFT);
}
- if (i == ARRAY_SIZE(wm831x_dcdc_ilim))
- return -EINVAL;
- return wm831x_set_bits(wm831x, reg, WM831X_DC1_HC_THR_MASK,
- i << WM831X_DC1_HC_THR_SHIFT);
+ return -EINVAL;
}
static int wm831x_buckv_get_current_limit(struct regulator_dev *rdev)
diff --git a/drivers/regulator/wm831x-ldo.c b/drivers/regulator/wm831x-ldo.c
index 5cb70ca..9af5126 100644
--- a/drivers/regulator/wm831x-ldo.c
+++ b/drivers/regulator/wm831x-ldo.c
@@ -205,6 +205,8 @@
/* Is it reporting under voltage? */
ret = wm831x_reg_read(wm831x, WM831X_LDO_UV_STATUS);
+ if (ret < 0)
+ return ret;
if (ret & mask)
return REGULATOR_STATUS_ERROR;
@@ -237,6 +239,8 @@
.set_mode = wm831x_gp_ldo_set_mode,
.get_status = wm831x_gp_ldo_get_status,
.get_optimum_mode = wm831x_gp_ldo_get_optimum_mode,
+ .get_bypass = regulator_get_bypass_regmap,
+ .set_bypass = regulator_set_bypass_regmap,
.is_enabled = regulator_is_enabled_regmap,
.enable = regulator_enable_regmap,
@@ -293,6 +297,8 @@
ldo->desc.vsel_mask = WM831X_LDO1_ON_VSEL_MASK;
ldo->desc.enable_reg = WM831X_LDO_ENABLE;
ldo->desc.enable_mask = 1 << id;
+ ldo->desc.bypass_reg = ldo->base;
+ ldo->desc.bypass_mask = WM831X_LDO1_SWI;
config.dev = pdev->dev.parent;
if (pdata)
@@ -469,6 +475,8 @@
/* Is it reporting under voltage? */
ret = wm831x_reg_read(wm831x, WM831X_LDO_UV_STATUS);
+ if (ret < 0)
+ return ret;
if (ret & mask)
return REGULATOR_STATUS_ERROR;
@@ -488,6 +496,8 @@
.get_mode = wm831x_aldo_get_mode,
.set_mode = wm831x_aldo_set_mode,
.get_status = wm831x_aldo_get_status,
+ .set_bypass = regulator_set_bypass_regmap,
+ .get_bypass = regulator_get_bypass_regmap,
.is_enabled = regulator_is_enabled_regmap,
.enable = regulator_enable_regmap,
@@ -544,6 +554,8 @@
ldo->desc.vsel_mask = WM831X_LDO7_ON_VSEL_MASK;
ldo->desc.enable_reg = WM831X_LDO_ENABLE;
ldo->desc.enable_mask = 1 << id;
+ ldo->desc.bypass_reg = ldo->base;
+ ldo->desc.bypass_mask = WM831X_LDO7_SWI;
config.dev = pdev->dev.parent;
if (pdata)
diff --git a/drivers/regulator/wm8400-regulator.c b/drivers/regulator/wm8400-regulator.c
index 9035dd0..27c746e 100644
--- a/drivers/regulator/wm8400-regulator.c
+++ b/drivers/regulator/wm8400-regulator.c
@@ -120,13 +120,8 @@
case REGULATOR_MODE_IDLE:
/* Datasheet: standby */
- ret = wm8400_set_bits(wm8400, WM8400_DCDC1_CONTROL_1 + offset,
- WM8400_DC1_ACTIVE, 0);
- if (ret != 0)
- return ret;
return wm8400_set_bits(wm8400, WM8400_DCDC1_CONTROL_1 + offset,
- WM8400_DC1_SLEEP, 0);
-
+ WM8400_DC1_ACTIVE | WM8400_DC1_SLEEP, 0);
default:
return -EINVAL;
}
diff --git a/drivers/rpmsg/virtio_rpmsg_bus.c b/drivers/rpmsg/virtio_rpmsg_bus.c
index 590cfaf..1859f71 100644
--- a/drivers/rpmsg/virtio_rpmsg_bus.c
+++ b/drivers/rpmsg/virtio_rpmsg_bus.c
@@ -1008,8 +1008,8 @@
return 0;
free_coherent:
- dma_free_coherent(vdev->dev.parent, RPMSG_TOTAL_BUF_SPACE, bufs_va,
- vrp->bufs_dma);
+ dma_free_coherent(vdev->dev.parent->parent, RPMSG_TOTAL_BUF_SPACE,
+ bufs_va, vrp->bufs_dma);
vqs_del:
vdev->config->del_vqs(vrp->vdev);
free_vrp:
@@ -1043,7 +1043,7 @@
vdev->config->del_vqs(vrp->vdev);
- dma_free_coherent(vdev->dev.parent, RPMSG_TOTAL_BUF_SPACE,
+ dma_free_coherent(vdev->dev.parent->parent, RPMSG_TOTAL_BUF_SPACE,
vrp->rbufs, vrp->bufs_dma);
kfree(vrp);
diff --git a/drivers/rtc/rtc-at91sam9.c b/drivers/rtc/rtc-at91sam9.c
index 8318689..1dd61f4 100644
--- a/drivers/rtc/rtc-at91sam9.c
+++ b/drivers/rtc/rtc-at91sam9.c
@@ -58,6 +58,7 @@
struct rtc_device *rtcdev;
u32 imr;
void __iomem *gpbr;
+ int irq;
};
#define rtt_readl(rtc, field) \
@@ -292,7 +293,7 @@
{
struct resource *r, *r_gpbr;
struct sam9_rtc *rtc;
- int ret;
+ int ret, irq;
u32 mr;
r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -302,10 +303,18 @@
return -ENODEV;
}
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ dev_err(&pdev->dev, "failed to get interrupt resource\n");
+ return irq;
+ }
+
rtc = kzalloc(sizeof *rtc, GFP_KERNEL);
if (!rtc)
return -ENOMEM;
+ rtc->irq = irq;
+
/* platform setup code should have handled this; sigh */
if (!device_can_wakeup(&pdev->dev))
device_init_wakeup(&pdev->dev, 1);
@@ -345,11 +354,10 @@
}
/* register irq handler after we know what name we'll use */
- ret = request_irq(AT91_ID_SYS, at91_rtc_interrupt,
- IRQF_SHARED,
+ ret = request_irq(rtc->irq, at91_rtc_interrupt, IRQF_SHARED,
dev_name(&rtc->rtcdev->dev), rtc);
if (ret) {
- dev_dbg(&pdev->dev, "can't share IRQ %d?\n", AT91_ID_SYS);
+ dev_dbg(&pdev->dev, "can't share IRQ %d?\n", rtc->irq);
rtc_device_unregister(rtc->rtcdev);
goto fail_register;
}
@@ -386,7 +394,7 @@
/* disable all interrupts */
rtt_writel(rtc, MR, mr & ~(AT91_RTT_ALMIEN | AT91_RTT_RTTINCIEN));
- free_irq(AT91_ID_SYS, rtc);
+ free_irq(rtc->irq, rtc);
rtc_device_unregister(rtc->rtcdev);
@@ -423,7 +431,7 @@
rtc->imr = mr & (AT91_RTT_ALMIEN | AT91_RTT_RTTINCIEN);
if (rtc->imr) {
if (device_may_wakeup(&pdev->dev) && (mr & AT91_RTT_ALMIEN)) {
- enable_irq_wake(AT91_ID_SYS);
+ enable_irq_wake(rtc->irq);
/* don't let RTTINC cause wakeups */
if (mr & AT91_RTT_RTTINCIEN)
rtt_writel(rtc, MR, mr & ~AT91_RTT_RTTINCIEN);
@@ -441,7 +449,7 @@
if (rtc->imr) {
if (device_may_wakeup(&pdev->dev))
- disable_irq_wake(AT91_ID_SYS);
+ disable_irq_wake(rtc->irq);
mr = rtt_readl(rtc, MR);
rtt_writel(rtc, MR, mr | rtc->imr);
}
diff --git a/drivers/rtc/rtc-twl.c b/drivers/rtc/rtc-twl.c
index c5d06fe..9277d94 100644
--- a/drivers/rtc/rtc-twl.c
+++ b/drivers/rtc/rtc-twl.c
@@ -495,6 +495,11 @@
if (ret < 0)
goto out1;
+ /* ensure interrupts are disabled, bootloaders can be strange */
+ ret = twl_rtc_write_u8(0, REG_RTC_INTERRUPTS_REG);
+ if (ret < 0)
+ dev_warn(&pdev->dev, "unable to disable interrupt\n");
+
/* init cached IRQ enable bits */
ret = twl_rtc_read_u8(&rtc_irq_bits, REG_RTC_INTERRUPTS_REG);
if (ret < 0)
diff --git a/drivers/s390/block/Kconfig b/drivers/s390/block/Kconfig
index 8e477bb..4a3b623 100644
--- a/drivers/s390/block/Kconfig
+++ b/drivers/s390/block/Kconfig
@@ -70,3 +70,21 @@
This driver provides a character device interface to the
DASD extended error reporting. This is only needed if you want to
use applications written for the EER facility.
+
+config SCM_BLOCK
+ def_tristate m
+ prompt "Support for Storage Class Memory"
+ depends on S390 && BLOCK && EADM_SCH && SCM_BUS
+ help
+ Block device driver for Storage Class Memory (SCM). This driver
+ provides a block device interface for each available SCM increment.
+
+ To compile this driver as a module, choose M here: the
+ module will be called scm_block.
+
+config SCM_BLOCK_CLUSTER_WRITE
+ def_bool y
+ prompt "SCM force cluster writes"
+ depends on SCM_BLOCK
+ help
+ Force writes to Storage Class Memory (SCM) to be in done in clusters.
diff --git a/drivers/s390/block/Makefile b/drivers/s390/block/Makefile
index 0a89e08..c2f4e67 100644
--- a/drivers/s390/block/Makefile
+++ b/drivers/s390/block/Makefile
@@ -17,3 +17,9 @@
obj-$(CONFIG_DASD_FBA) += dasd_fba_mod.o
obj-$(CONFIG_BLK_DEV_XPRAM) += xpram.o
obj-$(CONFIG_DCSSBLK) += dcssblk.o
+
+scm_block-objs := scm_drv.o scm_blk.o
+ifdef CONFIG_SCM_BLOCK_CLUSTER_WRITE
+scm_block-objs += scm_blk_cluster.o
+endif
+obj-$(CONFIG_SCM_BLOCK) += scm_block.o
diff --git a/drivers/s390/block/dasd.c b/drivers/s390/block/dasd.c
index 15370a2..0595c76 100644
--- a/drivers/s390/block/dasd.c
+++ b/drivers/s390/block/dasd.c
@@ -534,11 +534,11 @@
if (rc)
device->target = device->state;
- if (device->state == device->target)
- wake_up(&dasd_init_waitq);
-
/* let user-space know that the device status changed */
kobject_uevent(&device->cdev->dev.kobj, KOBJ_CHANGE);
+
+ if (device->state == device->target)
+ wake_up(&dasd_init_waitq);
}
/*
@@ -2157,6 +2157,7 @@
test_bit(DASD_CQR_FLAGS_FAILFAST, &cqr->flags) &&
(!dasd_eer_enabled(device))) {
cqr->status = DASD_CQR_FAILED;
+ cqr->intrc = -EAGAIN;
continue;
}
/* Don't try to start requests if device is stopped */
@@ -3270,6 +3271,16 @@
dasd_schedule_device_bh(device);
}
if (path_event[chp] & PE_PATHGROUP_ESTABLISHED) {
+ if (!(device->path_data.opm & eventlpm) &&
+ !(device->path_data.tbvpm & eventlpm)) {
+ /*
+ * we can not establish a pathgroup on an
+ * unavailable path, so trigger a path
+ * verification first
+ */
+ device->path_data.tbvpm |= eventlpm;
+ dasd_schedule_device_bh(device);
+ }
DBF_DEV_EVENT(DBF_WARNING, device, "%s",
"Pathgroup re-established\n");
if (device->discipline->kick_validate)
diff --git a/drivers/s390/block/dasd_alias.c b/drivers/s390/block/dasd_alias.c
index 157defe..6b55699 100644
--- a/drivers/s390/block/dasd_alias.c
+++ b/drivers/s390/block/dasd_alias.c
@@ -384,6 +384,29 @@
group->next = NULL;
};
+static int
+suborder_not_supported(struct dasd_ccw_req *cqr)
+{
+ char *sense;
+ char reason;
+ char msg_format;
+ char msg_no;
+
+ sense = dasd_get_sense(&cqr->irb);
+ if (!sense)
+ return 0;
+
+ reason = sense[0];
+ msg_format = (sense[7] & 0xF0);
+ msg_no = (sense[7] & 0x0F);
+
+ /* command reject, Format 0 MSG 4 - invalid parameter */
+ if ((reason == 0x80) && (msg_format == 0x00) && (msg_no == 0x04))
+ return 1;
+
+ return 0;
+}
+
static int read_unit_address_configuration(struct dasd_device *device,
struct alias_lcu *lcu)
{
@@ -435,6 +458,8 @@
do {
rc = dasd_sleep_on(cqr);
+ if (rc && suborder_not_supported(cqr))
+ return -EOPNOTSUPP;
} while (rc && (cqr->retries > 0));
if (rc) {
spin_lock_irqsave(&lcu->lock, flags);
@@ -521,7 +546,7 @@
* processing the data
*/
spin_lock_irqsave(&lcu->lock, flags);
- if (rc || (lcu->flags & NEED_UAC_UPDATE)) {
+ if ((rc && (rc != -EOPNOTSUPP)) || (lcu->flags & NEED_UAC_UPDATE)) {
DBF_DEV_EVENT(DBF_WARNING, device, "could not update"
" alias data in lcu (rc = %d), retry later", rc);
schedule_delayed_work(&lcu->ruac_data.dwork, 30*HZ);
diff --git a/drivers/s390/block/dasd_eckd.c b/drivers/s390/block/dasd_eckd.c
index 40a826a..108332b 100644
--- a/drivers/s390/block/dasd_eckd.c
+++ b/drivers/s390/block/dasd_eckd.c
@@ -20,6 +20,7 @@
#include <linux/compat.h>
#include <linux/init.h>
+#include <asm/css_chars.h>
#include <asm/debug.h>
#include <asm/idals.h>
#include <asm/ebcdic.h>
@@ -31,8 +32,6 @@
#include "dasd_int.h"
#include "dasd_eckd.h"
-#include "../cio/chsc.h"
-
#ifdef PRINTK_HEADER
#undef PRINTK_HEADER
@@ -140,6 +139,10 @@
static const int sizes_trk0[] = { 28, 148, 84 };
#define LABEL_SIZE 140
+/* head and record addresses of count_area read in analysis ccw */
+static const int count_area_head[] = { 0, 0, 0, 0, 2 };
+static const int count_area_rec[] = { 1, 2, 3, 4, 1 };
+
static inline unsigned int
round_up_multiple(unsigned int no, unsigned int mult)
{
@@ -212,7 +215,7 @@
rc = get_sync_clock(&data->ep_sys_time);
/* Ignore return code if sync clock is switched off. */
- if (rc == -ENOSYS || rc == -EACCES)
+ if (rc == -EOPNOTSUPP || rc == -EACCES)
rc = 0;
de_ccw->count = sizeof(struct DE_eckd_data);
@@ -323,7 +326,7 @@
rc = get_sync_clock(&pfxdata->define_extent.ep_sys_time);
/* Ignore return code if sync clock is switched off. */
- if (rc == -ENOSYS || rc == -EACCES)
+ if (rc == -EOPNOTSUPP || rc == -EACCES)
rc = 0;
return rc;
}
@@ -1507,7 +1510,8 @@
* call might change behaviour of DASD devices.
*/
static int
-dasd_eckd_psf_ssc(struct dasd_device *device, int enable_pav)
+dasd_eckd_psf_ssc(struct dasd_device *device, int enable_pav,
+ unsigned long flags)
{
struct dasd_ccw_req *cqr;
int rc;
@@ -1516,10 +1520,19 @@
if (IS_ERR(cqr))
return PTR_ERR(cqr);
+ /*
+ * set flags e.g. turn on failfast, to prevent blocking
+ * the calling function should handle failed requests
+ */
+ cqr->flags |= flags;
+
rc = dasd_sleep_on(cqr);
if (!rc)
/* trigger CIO to reprobe devices */
css_schedule_reprobe();
+ else if (cqr->intrc == -EAGAIN)
+ rc = -EAGAIN;
+
dasd_sfree_request(cqr, cqr->memdev);
return rc;
}
@@ -1527,7 +1540,8 @@
/*
* Valide storage server of current device.
*/
-static void dasd_eckd_validate_server(struct dasd_device *device)
+static int dasd_eckd_validate_server(struct dasd_device *device,
+ unsigned long flags)
{
int rc;
struct dasd_eckd_private *private;
@@ -1536,17 +1550,18 @@
private = (struct dasd_eckd_private *) device->private;
if (private->uid.type == UA_BASE_PAV_ALIAS ||
private->uid.type == UA_HYPER_PAV_ALIAS)
- return;
+ return 0;
if (dasd_nopav || MACHINE_IS_VM)
enable_pav = 0;
else
enable_pav = 1;
- rc = dasd_eckd_psf_ssc(device, enable_pav);
+ rc = dasd_eckd_psf_ssc(device, enable_pav, flags);
/* may be requested feature is not available on server,
* therefore just report error and go ahead */
DBF_EVENT_DEVID(DBF_WARNING, device->cdev, "PSF-SSC for SSID %04x "
"returned rc=%d", private->uid.ssid, rc);
+ return rc;
}
/*
@@ -1556,7 +1571,13 @@
{
struct dasd_device *device = container_of(work, struct dasd_device,
kick_validate);
- dasd_eckd_validate_server(device);
+ if (dasd_eckd_validate_server(device, DASD_CQR_FLAGS_FAILFAST)
+ == -EAGAIN) {
+ /* schedule worker again if failed */
+ schedule_work(&device->kick_validate);
+ return;
+ }
+
dasd_put_device(device);
}
@@ -1685,7 +1706,7 @@
if (rc)
goto out_err2;
- dasd_eckd_validate_server(device);
+ dasd_eckd_validate_server(device, 0);
/* device may report different configuration data after LCU setup */
rc = dasd_eckd_read_conf(device);
@@ -1922,7 +1943,10 @@
count_area = NULL;
for (i = 0; i < 3; i++) {
if (private->count_area[i].kl != 4 ||
- private->count_area[i].dl != dasd_eckd_cdl_reclen(i) - 4) {
+ private->count_area[i].dl != dasd_eckd_cdl_reclen(i) - 4 ||
+ private->count_area[i].cyl != 0 ||
+ private->count_area[i].head != count_area_head[i] ||
+ private->count_area[i].record != count_area_rec[i]) {
private->uses_cdl = 0;
break;
}
@@ -1934,7 +1958,10 @@
for (i = 0; i < 5; i++) {
if ((private->count_area[i].kl != 0) ||
(private->count_area[i].dl !=
- private->count_area[0].dl))
+ private->count_area[0].dl) ||
+ private->count_area[i].cyl != 0 ||
+ private->count_area[i].head != count_area_head[i] ||
+ private->count_area[i].record != count_area_rec[i])
break;
}
if (i == 5)
@@ -3804,7 +3831,7 @@
case BIODASDSYMMIO:
return dasd_symm_io(device, argp);
default:
- return -ENOIOCTLCMD;
+ return -ENOTTY;
}
}
@@ -4153,7 +4180,7 @@
rc = dasd_alias_make_device_known_to_lcu(device);
if (rc)
return rc;
- dasd_eckd_validate_server(device);
+ dasd_eckd_validate_server(device, DASD_CQR_FLAGS_FAILFAST);
/* RE-Read Configuration Data */
rc = dasd_eckd_read_conf(device);
diff --git a/drivers/s390/block/dasd_ioctl.c b/drivers/s390/block/dasd_ioctl.c
index cceae70..8252f37 100644
--- a/drivers/s390/block/dasd_ioctl.c
+++ b/drivers/s390/block/dasd_ioctl.c
@@ -292,12 +292,12 @@
#else
static int dasd_ioctl_reset_profile(struct dasd_block *block)
{
- return -ENOSYS;
+ return -ENOTTY;
}
static int dasd_ioctl_read_profile(struct dasd_block *block, void __user *argp)
{
- return -ENOSYS;
+ return -ENOTTY;
}
#endif
@@ -498,12 +498,9 @@
break;
default:
/* if the discipline has an ioctl method try it. */
- if (base->discipline->ioctl) {
+ rc = -ENOTTY;
+ if (base->discipline->ioctl)
rc = base->discipline->ioctl(block, cmd, argp);
- if (rc == -ENOIOCTLCMD)
- rc = -EINVAL;
- } else
- rc = -EINVAL;
}
dasd_put_device(base);
return rc;
diff --git a/drivers/s390/block/scm_blk.c b/drivers/s390/block/scm_blk.c
new file mode 100644
index 0000000..9978ad4
--- /dev/null
+++ b/drivers/s390/block/scm_blk.c
@@ -0,0 +1,445 @@
+/*
+ * Block driver for s390 storage class memory.
+ *
+ * Copyright IBM Corp. 2012
+ * Author(s): Sebastian Ott <sebott@linux.vnet.ibm.com>
+ */
+
+#define KMSG_COMPONENT "scm_block"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/module.h>
+#include <linux/blkdev.h>
+#include <linux/genhd.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <asm/eadm.h>
+#include "scm_blk.h"
+
+debug_info_t *scm_debug;
+static int scm_major;
+static DEFINE_SPINLOCK(list_lock);
+static LIST_HEAD(inactive_requests);
+static unsigned int nr_requests = 64;
+static atomic_t nr_devices = ATOMIC_INIT(0);
+module_param(nr_requests, uint, S_IRUGO);
+MODULE_PARM_DESC(nr_requests, "Number of parallel requests.");
+
+MODULE_DESCRIPTION("Block driver for s390 storage class memory.");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("scm:scmdev*");
+
+static void __scm_free_rq(struct scm_request *scmrq)
+{
+ struct aob_rq_header *aobrq = to_aobrq(scmrq);
+
+ free_page((unsigned long) scmrq->aob);
+ free_page((unsigned long) scmrq->aidaw);
+ __scm_free_rq_cluster(scmrq);
+ kfree(aobrq);
+}
+
+static void scm_free_rqs(void)
+{
+ struct list_head *iter, *safe;
+ struct scm_request *scmrq;
+
+ spin_lock_irq(&list_lock);
+ list_for_each_safe(iter, safe, &inactive_requests) {
+ scmrq = list_entry(iter, struct scm_request, list);
+ list_del(&scmrq->list);
+ __scm_free_rq(scmrq);
+ }
+ spin_unlock_irq(&list_lock);
+}
+
+static int __scm_alloc_rq(void)
+{
+ struct aob_rq_header *aobrq;
+ struct scm_request *scmrq;
+
+ aobrq = kzalloc(sizeof(*aobrq) + sizeof(*scmrq), GFP_KERNEL);
+ if (!aobrq)
+ return -ENOMEM;
+
+ scmrq = (void *) aobrq->data;
+ scmrq->aidaw = (void *) get_zeroed_page(GFP_DMA);
+ scmrq->aob = (void *) get_zeroed_page(GFP_DMA);
+ if (!scmrq->aob || !scmrq->aidaw) {
+ __scm_free_rq(scmrq);
+ return -ENOMEM;
+ }
+
+ if (__scm_alloc_rq_cluster(scmrq)) {
+ __scm_free_rq(scmrq);
+ return -ENOMEM;
+ }
+
+ INIT_LIST_HEAD(&scmrq->list);
+ spin_lock_irq(&list_lock);
+ list_add(&scmrq->list, &inactive_requests);
+ spin_unlock_irq(&list_lock);
+
+ return 0;
+}
+
+static int scm_alloc_rqs(unsigned int nrqs)
+{
+ int ret = 0;
+
+ while (nrqs-- && !ret)
+ ret = __scm_alloc_rq();
+
+ return ret;
+}
+
+static struct scm_request *scm_request_fetch(void)
+{
+ struct scm_request *scmrq = NULL;
+
+ spin_lock(&list_lock);
+ if (list_empty(&inactive_requests))
+ goto out;
+ scmrq = list_first_entry(&inactive_requests, struct scm_request, list);
+ list_del(&scmrq->list);
+out:
+ spin_unlock(&list_lock);
+ return scmrq;
+}
+
+static void scm_request_done(struct scm_request *scmrq)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&list_lock, flags);
+ list_add(&scmrq->list, &inactive_requests);
+ spin_unlock_irqrestore(&list_lock, flags);
+}
+
+static int scm_open(struct block_device *blkdev, fmode_t mode)
+{
+ return scm_get_ref();
+}
+
+static int scm_release(struct gendisk *gendisk, fmode_t mode)
+{
+ scm_put_ref();
+ return 0;
+}
+
+static const struct block_device_operations scm_blk_devops = {
+ .owner = THIS_MODULE,
+ .open = scm_open,
+ .release = scm_release,
+};
+
+static void scm_request_prepare(struct scm_request *scmrq)
+{
+ struct scm_blk_dev *bdev = scmrq->bdev;
+ struct scm_device *scmdev = bdev->gendisk->private_data;
+ struct aidaw *aidaw = scmrq->aidaw;
+ struct msb *msb = &scmrq->aob->msb[0];
+ struct req_iterator iter;
+ struct bio_vec *bv;
+
+ msb->bs = MSB_BS_4K;
+ scmrq->aob->request.msb_count = 1;
+ msb->scm_addr = scmdev->address +
+ ((u64) blk_rq_pos(scmrq->request) << 9);
+ msb->oc = (rq_data_dir(scmrq->request) == READ) ?
+ MSB_OC_READ : MSB_OC_WRITE;
+ msb->flags |= MSB_FLAG_IDA;
+ msb->data_addr = (u64) aidaw;
+
+ rq_for_each_segment(bv, scmrq->request, iter) {
+ WARN_ON(bv->bv_offset);
+ msb->blk_count += bv->bv_len >> 12;
+ aidaw->data_addr = (u64) page_address(bv->bv_page);
+ aidaw++;
+ }
+}
+
+static inline void scm_request_init(struct scm_blk_dev *bdev,
+ struct scm_request *scmrq,
+ struct request *req)
+{
+ struct aob_rq_header *aobrq = to_aobrq(scmrq);
+ struct aob *aob = scmrq->aob;
+
+ memset(aob, 0, sizeof(*aob));
+ memset(scmrq->aidaw, 0, PAGE_SIZE);
+ aobrq->scmdev = bdev->scmdev;
+ aob->request.cmd_code = ARQB_CMD_MOVE;
+ aob->request.data = (u64) aobrq;
+ scmrq->request = req;
+ scmrq->bdev = bdev;
+ scmrq->retries = 4;
+ scmrq->error = 0;
+ scm_request_cluster_init(scmrq);
+}
+
+static void scm_ensure_queue_restart(struct scm_blk_dev *bdev)
+{
+ if (atomic_read(&bdev->queued_reqs)) {
+ /* Queue restart is triggered by the next interrupt. */
+ return;
+ }
+ blk_delay_queue(bdev->rq, SCM_QUEUE_DELAY);
+}
+
+void scm_request_requeue(struct scm_request *scmrq)
+{
+ struct scm_blk_dev *bdev = scmrq->bdev;
+
+ scm_release_cluster(scmrq);
+ blk_requeue_request(bdev->rq, scmrq->request);
+ scm_request_done(scmrq);
+ scm_ensure_queue_restart(bdev);
+}
+
+void scm_request_finish(struct scm_request *scmrq)
+{
+ scm_release_cluster(scmrq);
+ blk_end_request_all(scmrq->request, scmrq->error);
+ scm_request_done(scmrq);
+}
+
+static void scm_blk_request(struct request_queue *rq)
+{
+ struct scm_device *scmdev = rq->queuedata;
+ struct scm_blk_dev *bdev = dev_get_drvdata(&scmdev->dev);
+ struct scm_request *scmrq;
+ struct request *req;
+ int ret;
+
+ while ((req = blk_peek_request(rq))) {
+ if (req->cmd_type != REQ_TYPE_FS)
+ continue;
+
+ scmrq = scm_request_fetch();
+ if (!scmrq) {
+ SCM_LOG(5, "no request");
+ scm_ensure_queue_restart(bdev);
+ return;
+ }
+ scm_request_init(bdev, scmrq, req);
+ if (!scm_reserve_cluster(scmrq)) {
+ SCM_LOG(5, "cluster busy");
+ scm_request_done(scmrq);
+ return;
+ }
+ if (scm_need_cluster_request(scmrq)) {
+ blk_start_request(req);
+ scm_initiate_cluster_request(scmrq);
+ return;
+ }
+ scm_request_prepare(scmrq);
+ blk_start_request(req);
+
+ ret = scm_start_aob(scmrq->aob);
+ if (ret) {
+ SCM_LOG(5, "no subchannel");
+ scm_request_requeue(scmrq);
+ return;
+ }
+ atomic_inc(&bdev->queued_reqs);
+ }
+}
+
+static void __scmrq_log_error(struct scm_request *scmrq)
+{
+ struct aob *aob = scmrq->aob;
+
+ if (scmrq->error == -ETIMEDOUT)
+ SCM_LOG(1, "Request timeout");
+ else {
+ SCM_LOG(1, "Request error");
+ SCM_LOG_HEX(1, &aob->response, sizeof(aob->response));
+ }
+ if (scmrq->retries)
+ SCM_LOG(1, "Retry request");
+ else
+ pr_err("An I/O operation to SCM failed with rc=%d\n",
+ scmrq->error);
+}
+
+void scm_blk_irq(struct scm_device *scmdev, void *data, int error)
+{
+ struct scm_request *scmrq = data;
+ struct scm_blk_dev *bdev = scmrq->bdev;
+
+ scmrq->error = error;
+ if (error)
+ __scmrq_log_error(scmrq);
+
+ spin_lock(&bdev->lock);
+ list_add_tail(&scmrq->list, &bdev->finished_requests);
+ spin_unlock(&bdev->lock);
+ tasklet_hi_schedule(&bdev->tasklet);
+}
+
+static void scm_blk_tasklet(struct scm_blk_dev *bdev)
+{
+ struct scm_request *scmrq;
+ unsigned long flags;
+
+ spin_lock_irqsave(&bdev->lock, flags);
+ while (!list_empty(&bdev->finished_requests)) {
+ scmrq = list_first_entry(&bdev->finished_requests,
+ struct scm_request, list);
+ list_del(&scmrq->list);
+ spin_unlock_irqrestore(&bdev->lock, flags);
+
+ if (scmrq->error && scmrq->retries-- > 0) {
+ if (scm_start_aob(scmrq->aob)) {
+ spin_lock_irqsave(&bdev->rq_lock, flags);
+ scm_request_requeue(scmrq);
+ spin_unlock_irqrestore(&bdev->rq_lock, flags);
+ }
+ /* Request restarted or requeued, handle next. */
+ spin_lock_irqsave(&bdev->lock, flags);
+ continue;
+ }
+
+ if (scm_test_cluster_request(scmrq)) {
+ scm_cluster_request_irq(scmrq);
+ spin_lock_irqsave(&bdev->lock, flags);
+ continue;
+ }
+
+ scm_request_finish(scmrq);
+ atomic_dec(&bdev->queued_reqs);
+ spin_lock_irqsave(&bdev->lock, flags);
+ }
+ spin_unlock_irqrestore(&bdev->lock, flags);
+ /* Look out for more requests. */
+ blk_run_queue(bdev->rq);
+}
+
+int scm_blk_dev_setup(struct scm_blk_dev *bdev, struct scm_device *scmdev)
+{
+ struct request_queue *rq;
+ int len, ret = -ENOMEM;
+ unsigned int devindex, nr_max_blk;
+
+ devindex = atomic_inc_return(&nr_devices) - 1;
+ /* scma..scmz + scmaa..scmzz */
+ if (devindex > 701) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ bdev->scmdev = scmdev;
+ spin_lock_init(&bdev->rq_lock);
+ spin_lock_init(&bdev->lock);
+ INIT_LIST_HEAD(&bdev->finished_requests);
+ atomic_set(&bdev->queued_reqs, 0);
+ tasklet_init(&bdev->tasklet,
+ (void (*)(unsigned long)) scm_blk_tasklet,
+ (unsigned long) bdev);
+
+ rq = blk_init_queue(scm_blk_request, &bdev->rq_lock);
+ if (!rq)
+ goto out;
+
+ bdev->rq = rq;
+ nr_max_blk = min(scmdev->nr_max_block,
+ (unsigned int) (PAGE_SIZE / sizeof(struct aidaw)));
+
+ blk_queue_logical_block_size(rq, 1 << 12);
+ blk_queue_max_hw_sectors(rq, nr_max_blk << 3); /* 8 * 512 = blk_size */
+ blk_queue_max_segments(rq, nr_max_blk);
+ queue_flag_set_unlocked(QUEUE_FLAG_NONROT, rq);
+ scm_blk_dev_cluster_setup(bdev);
+
+ bdev->gendisk = alloc_disk(SCM_NR_PARTS);
+ if (!bdev->gendisk)
+ goto out_queue;
+
+ rq->queuedata = scmdev;
+ bdev->gendisk->driverfs_dev = &scmdev->dev;
+ bdev->gendisk->private_data = scmdev;
+ bdev->gendisk->fops = &scm_blk_devops;
+ bdev->gendisk->queue = rq;
+ bdev->gendisk->major = scm_major;
+ bdev->gendisk->first_minor = devindex * SCM_NR_PARTS;
+
+ len = snprintf(bdev->gendisk->disk_name, DISK_NAME_LEN, "scm");
+ if (devindex > 25) {
+ len += snprintf(bdev->gendisk->disk_name + len,
+ DISK_NAME_LEN - len, "%c",
+ 'a' + (devindex / 26) - 1);
+ devindex = devindex % 26;
+ }
+ snprintf(bdev->gendisk->disk_name + len, DISK_NAME_LEN - len, "%c",
+ 'a' + devindex);
+
+ /* 512 byte sectors */
+ set_capacity(bdev->gendisk, scmdev->size >> 9);
+ add_disk(bdev->gendisk);
+ return 0;
+
+out_queue:
+ blk_cleanup_queue(rq);
+out:
+ atomic_dec(&nr_devices);
+ return ret;
+}
+
+void scm_blk_dev_cleanup(struct scm_blk_dev *bdev)
+{
+ tasklet_kill(&bdev->tasklet);
+ del_gendisk(bdev->gendisk);
+ blk_cleanup_queue(bdev->gendisk->queue);
+ put_disk(bdev->gendisk);
+}
+
+static int __init scm_blk_init(void)
+{
+ int ret = -EINVAL;
+
+ if (!scm_cluster_size_valid())
+ goto out;
+
+ ret = register_blkdev(0, "scm");
+ if (ret < 0)
+ goto out;
+
+ scm_major = ret;
+ if (scm_alloc_rqs(nr_requests))
+ goto out_unreg;
+
+ scm_debug = debug_register("scm_log", 16, 1, 16);
+ if (!scm_debug)
+ goto out_free;
+
+ debug_register_view(scm_debug, &debug_hex_ascii_view);
+ debug_set_level(scm_debug, 2);
+
+ ret = scm_drv_init();
+ if (ret)
+ goto out_dbf;
+
+ return ret;
+
+out_dbf:
+ debug_unregister(scm_debug);
+out_free:
+ scm_free_rqs();
+out_unreg:
+ unregister_blkdev(scm_major, "scm");
+out:
+ return ret;
+}
+module_init(scm_blk_init);
+
+static void __exit scm_blk_cleanup(void)
+{
+ scm_drv_cleanup();
+ debug_unregister(scm_debug);
+ scm_free_rqs();
+ unregister_blkdev(scm_major, "scm");
+}
+module_exit(scm_blk_cleanup);
diff --git a/drivers/s390/block/scm_blk.h b/drivers/s390/block/scm_blk.h
new file mode 100644
index 0000000..7ac6bad
--- /dev/null
+++ b/drivers/s390/block/scm_blk.h
@@ -0,0 +1,117 @@
+#ifndef SCM_BLK_H
+#define SCM_BLK_H
+
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/blkdev.h>
+#include <linux/genhd.h>
+#include <linux/list.h>
+
+#include <asm/debug.h>
+#include <asm/eadm.h>
+
+#define SCM_NR_PARTS 8
+#define SCM_QUEUE_DELAY 5
+
+struct scm_blk_dev {
+ struct tasklet_struct tasklet;
+ struct request_queue *rq;
+ struct gendisk *gendisk;
+ struct scm_device *scmdev;
+ spinlock_t rq_lock; /* guard the request queue */
+ spinlock_t lock; /* guard the rest of the blockdev */
+ atomic_t queued_reqs;
+ struct list_head finished_requests;
+#ifdef CONFIG_SCM_BLOCK_CLUSTER_WRITE
+ struct list_head cluster_list;
+#endif
+};
+
+struct scm_request {
+ struct scm_blk_dev *bdev;
+ struct request *request;
+ struct aidaw *aidaw;
+ struct aob *aob;
+ struct list_head list;
+ u8 retries;
+ int error;
+#ifdef CONFIG_SCM_BLOCK_CLUSTER_WRITE
+ struct {
+ enum {CLUSTER_NONE, CLUSTER_READ, CLUSTER_WRITE} state;
+ struct list_head list;
+ void **buf;
+ } cluster;
+#endif
+};
+
+#define to_aobrq(rq) container_of((void *) rq, struct aob_rq_header, data)
+
+int scm_blk_dev_setup(struct scm_blk_dev *, struct scm_device *);
+void scm_blk_dev_cleanup(struct scm_blk_dev *);
+void scm_blk_irq(struct scm_device *, void *, int);
+
+void scm_request_finish(struct scm_request *);
+void scm_request_requeue(struct scm_request *);
+
+int scm_drv_init(void);
+void scm_drv_cleanup(void);
+
+#ifdef CONFIG_SCM_BLOCK_CLUSTER_WRITE
+void __scm_free_rq_cluster(struct scm_request *);
+int __scm_alloc_rq_cluster(struct scm_request *);
+void scm_request_cluster_init(struct scm_request *);
+bool scm_reserve_cluster(struct scm_request *);
+void scm_release_cluster(struct scm_request *);
+void scm_blk_dev_cluster_setup(struct scm_blk_dev *);
+bool scm_need_cluster_request(struct scm_request *);
+void scm_initiate_cluster_request(struct scm_request *);
+void scm_cluster_request_irq(struct scm_request *);
+bool scm_test_cluster_request(struct scm_request *);
+bool scm_cluster_size_valid(void);
+#else
+#define __scm_free_rq_cluster(scmrq) {}
+#define __scm_alloc_rq_cluster(scmrq) 0
+#define scm_request_cluster_init(scmrq) {}
+#define scm_reserve_cluster(scmrq) true
+#define scm_release_cluster(scmrq) {}
+#define scm_blk_dev_cluster_setup(bdev) {}
+#define scm_need_cluster_request(scmrq) false
+#define scm_initiate_cluster_request(scmrq) {}
+#define scm_cluster_request_irq(scmrq) {}
+#define scm_test_cluster_request(scmrq) false
+#define scm_cluster_size_valid() true
+#endif
+
+extern debug_info_t *scm_debug;
+
+#define SCM_LOG(imp, txt) do { \
+ debug_text_event(scm_debug, imp, txt); \
+ } while (0)
+
+static inline void SCM_LOG_HEX(int level, void *data, int length)
+{
+ if (level > scm_debug->level)
+ return;
+ while (length > 0) {
+ debug_event(scm_debug, level, data, length);
+ length -= scm_debug->buf_size;
+ data += scm_debug->buf_size;
+ }
+}
+
+static inline void SCM_LOG_STATE(int level, struct scm_device *scmdev)
+{
+ struct {
+ u64 address;
+ u8 oper_state;
+ u8 rank;
+ } __packed data = {
+ .address = scmdev->address,
+ .oper_state = scmdev->attrs.oper_state,
+ .rank = scmdev->attrs.rank,
+ };
+
+ SCM_LOG_HEX(level, &data, sizeof(data));
+}
+
+#endif /* SCM_BLK_H */
diff --git a/drivers/s390/block/scm_blk_cluster.c b/drivers/s390/block/scm_blk_cluster.c
new file mode 100644
index 0000000..f4bb61b
--- /dev/null
+++ b/drivers/s390/block/scm_blk_cluster.c
@@ -0,0 +1,228 @@
+/*
+ * Block driver for s390 storage class memory.
+ *
+ * Copyright IBM Corp. 2012
+ * Author(s): Sebastian Ott <sebott@linux.vnet.ibm.com>
+ */
+
+#include <linux/spinlock.h>
+#include <linux/module.h>
+#include <linux/blkdev.h>
+#include <linux/genhd.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <asm/eadm.h>
+#include "scm_blk.h"
+
+static unsigned int write_cluster_size = 64;
+module_param(write_cluster_size, uint, S_IRUGO);
+MODULE_PARM_DESC(write_cluster_size,
+ "Number of pages used for contiguous writes.");
+
+#define CLUSTER_SIZE (write_cluster_size * PAGE_SIZE)
+
+void __scm_free_rq_cluster(struct scm_request *scmrq)
+{
+ int i;
+
+ if (!scmrq->cluster.buf)
+ return;
+
+ for (i = 0; i < 2 * write_cluster_size; i++)
+ free_page((unsigned long) scmrq->cluster.buf[i]);
+
+ kfree(scmrq->cluster.buf);
+}
+
+int __scm_alloc_rq_cluster(struct scm_request *scmrq)
+{
+ int i;
+
+ scmrq->cluster.buf = kzalloc(sizeof(void *) * 2 * write_cluster_size,
+ GFP_KERNEL);
+ if (!scmrq->cluster.buf)
+ return -ENOMEM;
+
+ for (i = 0; i < 2 * write_cluster_size; i++) {
+ scmrq->cluster.buf[i] = (void *) get_zeroed_page(GFP_DMA);
+ if (!scmrq->cluster.buf[i])
+ return -ENOMEM;
+ }
+ INIT_LIST_HEAD(&scmrq->cluster.list);
+ return 0;
+}
+
+void scm_request_cluster_init(struct scm_request *scmrq)
+{
+ scmrq->cluster.state = CLUSTER_NONE;
+}
+
+static bool clusters_intersect(struct scm_request *A, struct scm_request *B)
+{
+ unsigned long firstA, lastA, firstB, lastB;
+
+ firstA = ((u64) blk_rq_pos(A->request) << 9) / CLUSTER_SIZE;
+ lastA = (((u64) blk_rq_pos(A->request) << 9) +
+ blk_rq_bytes(A->request) - 1) / CLUSTER_SIZE;
+
+ firstB = ((u64) blk_rq_pos(B->request) << 9) / CLUSTER_SIZE;
+ lastB = (((u64) blk_rq_pos(B->request) << 9) +
+ blk_rq_bytes(B->request) - 1) / CLUSTER_SIZE;
+
+ return (firstB <= lastA && firstA <= lastB);
+}
+
+bool scm_reserve_cluster(struct scm_request *scmrq)
+{
+ struct scm_blk_dev *bdev = scmrq->bdev;
+ struct scm_request *iter;
+
+ if (write_cluster_size == 0)
+ return true;
+
+ spin_lock(&bdev->lock);
+ list_for_each_entry(iter, &bdev->cluster_list, cluster.list) {
+ if (clusters_intersect(scmrq, iter) &&
+ (rq_data_dir(scmrq->request) == WRITE ||
+ rq_data_dir(iter->request) == WRITE)) {
+ spin_unlock(&bdev->lock);
+ return false;
+ }
+ }
+ list_add(&scmrq->cluster.list, &bdev->cluster_list);
+ spin_unlock(&bdev->lock);
+
+ return true;
+}
+
+void scm_release_cluster(struct scm_request *scmrq)
+{
+ struct scm_blk_dev *bdev = scmrq->bdev;
+ unsigned long flags;
+
+ if (write_cluster_size == 0)
+ return;
+
+ spin_lock_irqsave(&bdev->lock, flags);
+ list_del(&scmrq->cluster.list);
+ spin_unlock_irqrestore(&bdev->lock, flags);
+}
+
+void scm_blk_dev_cluster_setup(struct scm_blk_dev *bdev)
+{
+ INIT_LIST_HEAD(&bdev->cluster_list);
+ blk_queue_io_opt(bdev->rq, CLUSTER_SIZE);
+}
+
+static void scm_prepare_cluster_request(struct scm_request *scmrq)
+{
+ struct scm_blk_dev *bdev = scmrq->bdev;
+ struct scm_device *scmdev = bdev->gendisk->private_data;
+ struct request *req = scmrq->request;
+ struct aidaw *aidaw = scmrq->aidaw;
+ struct msb *msb = &scmrq->aob->msb[0];
+ struct req_iterator iter;
+ struct bio_vec *bv;
+ int i = 0;
+ u64 addr;
+
+ switch (scmrq->cluster.state) {
+ case CLUSTER_NONE:
+ scmrq->cluster.state = CLUSTER_READ;
+ /* fall through */
+ case CLUSTER_READ:
+ scmrq->aob->request.msb_count = 1;
+ msb->bs = MSB_BS_4K;
+ msb->oc = MSB_OC_READ;
+ msb->flags = MSB_FLAG_IDA;
+ msb->data_addr = (u64) aidaw;
+ msb->blk_count = write_cluster_size;
+
+ addr = scmdev->address + ((u64) blk_rq_pos(req) << 9);
+ msb->scm_addr = round_down(addr, CLUSTER_SIZE);
+
+ if (msb->scm_addr !=
+ round_down(addr + (u64) blk_rq_bytes(req) - 1,
+ CLUSTER_SIZE))
+ msb->blk_count = 2 * write_cluster_size;
+
+ for (i = 0; i < msb->blk_count; i++) {
+ aidaw->data_addr = (u64) scmrq->cluster.buf[i];
+ aidaw++;
+ }
+
+ break;
+ case CLUSTER_WRITE:
+ msb->oc = MSB_OC_WRITE;
+
+ for (addr = msb->scm_addr;
+ addr < scmdev->address + ((u64) blk_rq_pos(req) << 9);
+ addr += PAGE_SIZE) {
+ aidaw->data_addr = (u64) scmrq->cluster.buf[i];
+ aidaw++;
+ i++;
+ }
+ rq_for_each_segment(bv, req, iter) {
+ aidaw->data_addr = (u64) page_address(bv->bv_page);
+ aidaw++;
+ i++;
+ }
+ for (; i < msb->blk_count; i++) {
+ aidaw->data_addr = (u64) scmrq->cluster.buf[i];
+ aidaw++;
+ }
+ break;
+ }
+}
+
+bool scm_need_cluster_request(struct scm_request *scmrq)
+{
+ if (rq_data_dir(scmrq->request) == READ)
+ return false;
+
+ return blk_rq_bytes(scmrq->request) < CLUSTER_SIZE;
+}
+
+/* Called with queue lock held. */
+void scm_initiate_cluster_request(struct scm_request *scmrq)
+{
+ scm_prepare_cluster_request(scmrq);
+ if (scm_start_aob(scmrq->aob))
+ scm_request_requeue(scmrq);
+}
+
+bool scm_test_cluster_request(struct scm_request *scmrq)
+{
+ return scmrq->cluster.state != CLUSTER_NONE;
+}
+
+void scm_cluster_request_irq(struct scm_request *scmrq)
+{
+ struct scm_blk_dev *bdev = scmrq->bdev;
+ unsigned long flags;
+
+ switch (scmrq->cluster.state) {
+ case CLUSTER_NONE:
+ BUG();
+ break;
+ case CLUSTER_READ:
+ if (scmrq->error) {
+ scm_request_finish(scmrq);
+ break;
+ }
+ scmrq->cluster.state = CLUSTER_WRITE;
+ spin_lock_irqsave(&bdev->rq_lock, flags);
+ scm_initiate_cluster_request(scmrq);
+ spin_unlock_irqrestore(&bdev->rq_lock, flags);
+ break;
+ case CLUSTER_WRITE:
+ scm_request_finish(scmrq);
+ break;
+ }
+}
+
+bool scm_cluster_size_valid(void)
+{
+ return write_cluster_size == 0 || write_cluster_size == 32 ||
+ write_cluster_size == 64 || write_cluster_size == 128;
+}
diff --git a/drivers/s390/block/scm_drv.c b/drivers/s390/block/scm_drv.c
new file mode 100644
index 0000000..9fa0a90
--- /dev/null
+++ b/drivers/s390/block/scm_drv.c
@@ -0,0 +1,81 @@
+/*
+ * Device driver for s390 storage class memory.
+ *
+ * Copyright IBM Corp. 2012
+ * Author(s): Sebastian Ott <sebott@linux.vnet.ibm.com>
+ */
+
+#define KMSG_COMPONENT "scm_block"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <asm/eadm.h>
+#include "scm_blk.h"
+
+static void notify(struct scm_device *scmdev)
+{
+ pr_info("%lu: The capabilities of the SCM increment changed\n",
+ (unsigned long) scmdev->address);
+ SCM_LOG(2, "State changed");
+ SCM_LOG_STATE(2, scmdev);
+}
+
+static int scm_probe(struct scm_device *scmdev)
+{
+ struct scm_blk_dev *bdev;
+ int ret;
+
+ SCM_LOG(2, "probe");
+ SCM_LOG_STATE(2, scmdev);
+
+ if (scmdev->attrs.oper_state != OP_STATE_GOOD)
+ return -EINVAL;
+
+ bdev = kzalloc(sizeof(*bdev), GFP_KERNEL);
+ if (!bdev)
+ return -ENOMEM;
+
+ dev_set_drvdata(&scmdev->dev, bdev);
+ ret = scm_blk_dev_setup(bdev, scmdev);
+ if (ret) {
+ dev_set_drvdata(&scmdev->dev, NULL);
+ kfree(bdev);
+ goto out;
+ }
+
+out:
+ return ret;
+}
+
+static int scm_remove(struct scm_device *scmdev)
+{
+ struct scm_blk_dev *bdev = dev_get_drvdata(&scmdev->dev);
+
+ scm_blk_dev_cleanup(bdev);
+ dev_set_drvdata(&scmdev->dev, NULL);
+ kfree(bdev);
+
+ return 0;
+}
+
+static struct scm_driver scm_drv = {
+ .drv = {
+ .name = "scm_block",
+ .owner = THIS_MODULE,
+ },
+ .notify = notify,
+ .probe = scm_probe,
+ .remove = scm_remove,
+ .handler = scm_blk_irq,
+};
+
+int __init scm_drv_init(void)
+{
+ return scm_driver_register(&scm_drv);
+}
+
+void scm_drv_cleanup(void)
+{
+ scm_driver_unregister(&scm_drv);
+}
diff --git a/drivers/s390/char/con3270.c b/drivers/s390/char/con3270.c
index bb07577..699fd3e 100644
--- a/drivers/s390/char/con3270.c
+++ b/drivers/s390/char/con3270.c
@@ -35,7 +35,6 @@
*/
struct con3270 {
struct raw3270_view view;
- spinlock_t lock;
struct list_head freemem; /* list of free memory for strings. */
/* Output stuff. */
diff --git a/drivers/s390/char/monreader.c b/drivers/s390/char/monreader.c
index 5b8b859..f4ff515 100644
--- a/drivers/s390/char/monreader.c
+++ b/drivers/s390/char/monreader.c
@@ -571,8 +571,11 @@
if (rc)
goto out_iucv;
monreader_device = kzalloc(sizeof(struct device), GFP_KERNEL);
- if (!monreader_device)
+ if (!monreader_device) {
+ rc = -ENOMEM;
goto out_driver;
+ }
+
dev_set_name(monreader_device, "monreader-dev");
monreader_device->bus = &iucv_bus;
monreader_device->parent = iucv_root;
diff --git a/drivers/s390/char/sclp.c b/drivers/s390/char/sclp.c
index 3fcc000..4fa21f7 100644
--- a/drivers/s390/char/sclp.c
+++ b/drivers/s390/char/sclp.c
@@ -334,7 +334,7 @@
reg->receiver_fn(evbuf);
spin_lock_irqsave(&sclp_lock, flags);
} else if (reg == NULL)
- rc = -ENOSYS;
+ rc = -EOPNOTSUPP;
}
spin_unlock_irqrestore(&sclp_lock, flags);
return rc;
diff --git a/drivers/s390/char/sclp_rw.c b/drivers/s390/char/sclp_rw.c
index 4be63be..3b13d58 100644
--- a/drivers/s390/char/sclp_rw.c
+++ b/drivers/s390/char/sclp_rw.c
@@ -463,7 +463,7 @@
/* Use write priority message */
sccb->msg_buf.header.type = EVTYP_PMSGCMD;
else
- return -ENOSYS;
+ return -EOPNOTSUPP;
buffer->request.command = SCLP_CMDW_WRITE_EVENT_DATA;
buffer->request.status = SCLP_REQ_FILLED;
buffer->request.callback = sclp_writedata_callback;
diff --git a/drivers/s390/char/tape.h b/drivers/s390/char/tape.h
index c06be6c..ea664dd 100644
--- a/drivers/s390/char/tape.h
+++ b/drivers/s390/char/tape.h
@@ -15,7 +15,6 @@
#include <asm/ccwdev.h>
#include <asm/debug.h>
#include <asm/idals.h>
-#include <linux/blkdev.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mtio.h>
diff --git a/drivers/s390/char/tape_std.h b/drivers/s390/char/tape_std.h
index c5816ad..8c760c0 100644
--- a/drivers/s390/char/tape_std.h
+++ b/drivers/s390/char/tape_std.h
@@ -100,11 +100,7 @@
void tape_std_read_backward(struct tape_device *device,
struct tape_request *request);
struct tape_request *tape_std_write_block(struct tape_device *, size_t);
-struct tape_request *tape_std_bread(struct tape_device *, struct request *);
-void tape_std_free_bread(struct tape_request *);
void tape_std_check_locate(struct tape_device *, struct tape_request *);
-struct tape_request *tape_std_bwrite(struct request *,
- struct tape_device *, int);
/* Some non-mtop commands. */
int tape_std_assign(struct tape_device *);
diff --git a/drivers/s390/char/vmlogrdr.c b/drivers/s390/char/vmlogrdr.c
index c131bc4..9b3a24e 100644
--- a/drivers/s390/char/vmlogrdr.c
+++ b/drivers/s390/char/vmlogrdr.c
@@ -321,7 +321,7 @@
* only allow for blocking reads to be open
*/
if (filp->f_flags & O_NONBLOCK)
- return -ENOSYS;
+ return -EOPNOTSUPP;
/* Besure this device hasn't already been opened */
spin_lock_bh(&logptr->priv_lock);
diff --git a/drivers/s390/cio/Makefile b/drivers/s390/cio/Makefile
index e1b700a..8c4a386 100644
--- a/drivers/s390/cio/Makefile
+++ b/drivers/s390/cio/Makefile
@@ -8,6 +8,8 @@
ccw_device-objs += device_id.o device_pgid.o device_status.o
obj-y += ccw_device.o cmf.o
obj-$(CONFIG_CHSC_SCH) += chsc_sch.o
+obj-$(CONFIG_EADM_SCH) += eadm_sch.o
+obj-$(CONFIG_SCM_BUS) += scm.o
obj-$(CONFIG_CCWGROUP) += ccwgroup.o
qdio-objs := qdio_main.o qdio_thinint.o qdio_debug.o qdio_setup.o
diff --git a/drivers/s390/cio/chsc.c b/drivers/s390/cio/chsc.c
index cfe0c08..4d51a7c 100644
--- a/drivers/s390/cio/chsc.c
+++ b/drivers/s390/cio/chsc.c
@@ -52,6 +52,11 @@
return -EINVAL;
case 0x0004:
return -EOPNOTSUPP;
+ case 0x000b:
+ return -EBUSY;
+ case 0x0100:
+ case 0x0102:
+ return -ENOMEM;
default:
return -EIO;
}
@@ -393,6 +398,20 @@
}
}
+static void chsc_process_sei_scm_change(struct chsc_sei_area *sei_area)
+{
+ int ret;
+
+ CIO_CRW_EVENT(4, "chsc: scm change notification\n");
+ if (sei_area->rs != 7)
+ return;
+
+ ret = scm_update_information();
+ if (ret)
+ CIO_CRW_EVENT(0, "chsc: updating change notification"
+ " failed (rc=%d).\n", ret);
+}
+
static void chsc_process_sei(struct chsc_sei_area *sei_area)
{
/* Check if we might have lost some information. */
@@ -414,6 +433,9 @@
case 8: /* channel-path-configuration notification */
chsc_process_sei_chp_config(sei_area);
break;
+ case 12: /* scm change notification */
+ chsc_process_sei_scm_change(sei_area);
+ break;
default: /* other stuff */
CIO_CRW_EVENT(4, "chsc: unhandled sei content code %d\n",
sei_area->cc);
@@ -1047,3 +1069,33 @@
return rc;
}
EXPORT_SYMBOL_GPL(chsc_siosl);
+
+/**
+ * chsc_scm_info() - store SCM information (SSI)
+ * @scm_area: request and response block for SSI
+ * @token: continuation token
+ *
+ * Returns 0 on success.
+ */
+int chsc_scm_info(struct chsc_scm_info *scm_area, u64 token)
+{
+ int ccode, ret;
+
+ memset(scm_area, 0, sizeof(*scm_area));
+ scm_area->request.length = 0x0020;
+ scm_area->request.code = 0x004C;
+ scm_area->reqtok = token;
+
+ ccode = chsc(scm_area);
+ if (ccode > 0) {
+ ret = (ccode == 3) ? -ENODEV : -EBUSY;
+ goto out;
+ }
+ ret = chsc_error_from_response(scm_area->response.code);
+ if (ret != 0)
+ CIO_MSG_EVENT(2, "chsc: scm info failed (rc=%04x)\n",
+ scm_area->response.code);
+out:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(chsc_scm_info);
diff --git a/drivers/s390/cio/chsc.h b/drivers/s390/cio/chsc.h
index 3f15b2a..662dab4 100644
--- a/drivers/s390/cio/chsc.h
+++ b/drivers/s390/cio/chsc.h
@@ -3,6 +3,7 @@
#include <linux/types.h>
#include <linux/device.h>
+#include <asm/css_chars.h>
#include <asm/chpid.h>
#include <asm/chsc.h>
#include <asm/schid.h>
@@ -118,4 +119,46 @@
int chsc_siosl(struct subchannel_id schid);
+/* Functions and definitions to query storage-class memory. */
+struct sale {
+ u64 sa;
+ u32 p:4;
+ u32 op_state:4;
+ u32 data_state:4;
+ u32 rank:4;
+ u32 r:1;
+ u32:7;
+ u32 rid:8;
+ u32:32;
+} __packed;
+
+struct chsc_scm_info {
+ struct chsc_header request;
+ u32:32;
+ u64 reqtok;
+ u32 reserved1[4];
+ struct chsc_header response;
+ u64:56;
+ u8 rq;
+ u32 mbc;
+ u64 msa;
+ u16 is;
+ u16 mmc;
+ u32 mci;
+ u64 nr_scm_ini;
+ u64 nr_scm_unini;
+ u32 reserved2[10];
+ u64 restok;
+ struct sale scmal[248];
+} __packed;
+
+int chsc_scm_info(struct chsc_scm_info *scm_area, u64 token);
+
+#ifdef CONFIG_SCM_BUS
+int scm_update_information(void);
+#else /* CONFIG_SCM_BUS */
+#define scm_update_information() 0
+#endif /* CONFIG_SCM_BUS */
+
+
#endif
diff --git a/drivers/s390/cio/cio.c b/drivers/s390/cio/cio.c
index 33d1ef7..8e927b9 100644
--- a/drivers/s390/cio/cio.c
+++ b/drivers/s390/cio/cio.c
@@ -1029,7 +1029,7 @@
/* Make sure all subchannels are quiet before we re-ipl an lpar. */
void reipl_ccw_dev(struct ccw_dev_id *devid)
{
- struct subchannel_id schid;
+ struct subchannel_id uninitialized_var(schid);
s390_reset_system(NULL, NULL);
if (reipl_find_schid(devid, &schid) != 0)
diff --git a/drivers/s390/cio/css.c b/drivers/s390/cio/css.c
index 21908e6..b4d572f 100644
--- a/drivers/s390/cio/css.c
+++ b/drivers/s390/cio/css.c
@@ -445,6 +445,7 @@
put_device(&sch->dev);
}
}
+EXPORT_SYMBOL_GPL(css_sched_sch_todo);
static void css_sch_todo(struct work_struct *work)
{
diff --git a/drivers/s390/cio/device.c b/drivers/s390/cio/device.c
index ed25c87..fc916f5 100644
--- a/drivers/s390/cio/device.c
+++ b/drivers/s390/cio/device.c
@@ -1426,6 +1426,8 @@
return IO_SCH_REPROBE;
if (cdev->online)
return IO_SCH_VERIFY;
+ if (cdev->private->state == DEV_STATE_NOT_OPER)
+ return IO_SCH_UNREG_ATTACH;
return IO_SCH_NOP;
}
@@ -1519,11 +1521,14 @@
goto out;
break;
case IO_SCH_UNREG_ATTACH:
+ spin_lock_irqsave(sch->lock, flags);
if (cdev->private->flags.resuming) {
/* Device will be handled later. */
rc = 0;
- goto out;
+ goto out_unlock;
}
+ sch_set_cdev(sch, NULL);
+ spin_unlock_irqrestore(sch->lock, flags);
/* Unregister ccw device. */
ccw_device_unregister(cdev);
break;
diff --git a/drivers/s390/cio/eadm_sch.c b/drivers/s390/cio/eadm_sch.c
new file mode 100644
index 0000000..6c96734
--- /dev/null
+++ b/drivers/s390/cio/eadm_sch.c
@@ -0,0 +1,401 @@
+/*
+ * Driver for s390 eadm subchannels
+ *
+ * Copyright IBM Corp. 2012
+ * Author(s): Sebastian Ott <sebott@linux.vnet.ibm.com>
+ */
+
+#include <linux/kernel_stat.h>
+#include <linux/workqueue.h>
+#include <linux/spinlock.h>
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/timer.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+
+#include <asm/css_chars.h>
+#include <asm/debug.h>
+#include <asm/isc.h>
+#include <asm/cio.h>
+#include <asm/scsw.h>
+#include <asm/eadm.h>
+
+#include "eadm_sch.h"
+#include "ioasm.h"
+#include "cio.h"
+#include "css.h"
+#include "orb.h"
+
+MODULE_DESCRIPTION("driver for s390 eadm subchannels");
+MODULE_LICENSE("GPL");
+
+#define EADM_TIMEOUT (5 * HZ)
+static DEFINE_SPINLOCK(list_lock);
+static LIST_HEAD(eadm_list);
+
+static debug_info_t *eadm_debug;
+
+#define EADM_LOG(imp, txt) do { \
+ debug_text_event(eadm_debug, imp, txt); \
+ } while (0)
+
+static void EADM_LOG_HEX(int level, void *data, int length)
+{
+ if (level > eadm_debug->level)
+ return;
+ while (length > 0) {
+ debug_event(eadm_debug, level, data, length);
+ length -= eadm_debug->buf_size;
+ data += eadm_debug->buf_size;
+ }
+}
+
+static void orb_init(union orb *orb)
+{
+ memset(orb, 0, sizeof(union orb));
+ orb->eadm.compat1 = 1;
+ orb->eadm.compat2 = 1;
+ orb->eadm.fmt = 1;
+ orb->eadm.x = 1;
+}
+
+static int eadm_subchannel_start(struct subchannel *sch, struct aob *aob)
+{
+ union orb *orb = &get_eadm_private(sch)->orb;
+ int cc;
+
+ orb_init(orb);
+ orb->eadm.aob = (u32)__pa(aob);
+ orb->eadm.intparm = (u32)(addr_t)sch;
+ orb->eadm.key = PAGE_DEFAULT_KEY >> 4;
+
+ EADM_LOG(6, "start");
+ EADM_LOG_HEX(6, &sch->schid, sizeof(sch->schid));
+
+ cc = ssch(sch->schid, orb);
+ switch (cc) {
+ case 0:
+ sch->schib.scsw.eadm.actl |= SCSW_ACTL_START_PEND;
+ break;
+ case 1: /* status pending */
+ case 2: /* busy */
+ return -EBUSY;
+ case 3: /* not operational */
+ return -ENODEV;
+ }
+ return 0;
+}
+
+static int eadm_subchannel_clear(struct subchannel *sch)
+{
+ int cc;
+
+ cc = csch(sch->schid);
+ if (cc)
+ return -ENODEV;
+
+ sch->schib.scsw.eadm.actl |= SCSW_ACTL_CLEAR_PEND;
+ return 0;
+}
+
+static void eadm_subchannel_timeout(unsigned long data)
+{
+ struct subchannel *sch = (struct subchannel *) data;
+
+ spin_lock_irq(sch->lock);
+ EADM_LOG(1, "timeout");
+ EADM_LOG_HEX(1, &sch->schid, sizeof(sch->schid));
+ if (eadm_subchannel_clear(sch))
+ EADM_LOG(0, "clear failed");
+ spin_unlock_irq(sch->lock);
+}
+
+static void eadm_subchannel_set_timeout(struct subchannel *sch, int expires)
+{
+ struct eadm_private *private = get_eadm_private(sch);
+
+ if (expires == 0) {
+ del_timer(&private->timer);
+ return;
+ }
+ if (timer_pending(&private->timer)) {
+ if (mod_timer(&private->timer, jiffies + expires))
+ return;
+ }
+ private->timer.function = eadm_subchannel_timeout;
+ private->timer.data = (unsigned long) sch;
+ private->timer.expires = jiffies + expires;
+ add_timer(&private->timer);
+}
+
+static void eadm_subchannel_irq(struct subchannel *sch)
+{
+ struct eadm_private *private = get_eadm_private(sch);
+ struct eadm_scsw *scsw = &sch->schib.scsw.eadm;
+ struct irb *irb = (struct irb *)&S390_lowcore.irb;
+ int error = 0;
+
+ EADM_LOG(6, "irq");
+ EADM_LOG_HEX(6, irb, sizeof(*irb));
+
+ kstat_cpu(smp_processor_id()).irqs[IOINT_ADM]++;
+
+ if ((scsw->stctl & (SCSW_STCTL_ALERT_STATUS | SCSW_STCTL_STATUS_PEND))
+ && scsw->eswf == 1 && irb->esw.eadm.erw.r)
+ error = -EIO;
+
+ if (scsw->fctl & SCSW_FCTL_CLEAR_FUNC)
+ error = -ETIMEDOUT;
+
+ eadm_subchannel_set_timeout(sch, 0);
+
+ if (private->state != EADM_BUSY) {
+ EADM_LOG(1, "irq unsol");
+ EADM_LOG_HEX(1, irb, sizeof(*irb));
+ private->state = EADM_NOT_OPER;
+ css_sched_sch_todo(sch, SCH_TODO_EVAL);
+ return;
+ }
+ scm_irq_handler((struct aob *)(unsigned long)scsw->aob, error);
+ private->state = EADM_IDLE;
+}
+
+static struct subchannel *eadm_get_idle_sch(void)
+{
+ struct eadm_private *private;
+ struct subchannel *sch;
+ unsigned long flags;
+
+ spin_lock_irqsave(&list_lock, flags);
+ list_for_each_entry(private, &eadm_list, head) {
+ sch = private->sch;
+ spin_lock(sch->lock);
+ if (private->state == EADM_IDLE) {
+ private->state = EADM_BUSY;
+ list_move_tail(&private->head, &eadm_list);
+ spin_unlock(sch->lock);
+ spin_unlock_irqrestore(&list_lock, flags);
+
+ return sch;
+ }
+ spin_unlock(sch->lock);
+ }
+ spin_unlock_irqrestore(&list_lock, flags);
+
+ return NULL;
+}
+
+static int eadm_start_aob(struct aob *aob)
+{
+ struct eadm_private *private;
+ struct subchannel *sch;
+ unsigned long flags;
+ int ret;
+
+ sch = eadm_get_idle_sch();
+ if (!sch)
+ return -EBUSY;
+
+ spin_lock_irqsave(sch->lock, flags);
+ eadm_subchannel_set_timeout(sch, EADM_TIMEOUT);
+ ret = eadm_subchannel_start(sch, aob);
+ if (!ret)
+ goto out_unlock;
+
+ /* Handle start subchannel failure. */
+ eadm_subchannel_set_timeout(sch, 0);
+ private = get_eadm_private(sch);
+ private->state = EADM_NOT_OPER;
+ css_sched_sch_todo(sch, SCH_TODO_EVAL);
+
+out_unlock:
+ spin_unlock_irqrestore(sch->lock, flags);
+
+ return ret;
+}
+
+static int eadm_subchannel_probe(struct subchannel *sch)
+{
+ struct eadm_private *private;
+ int ret;
+
+ private = kzalloc(sizeof(*private), GFP_KERNEL | GFP_DMA);
+ if (!private)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&private->head);
+ init_timer(&private->timer);
+
+ spin_lock_irq(sch->lock);
+ set_eadm_private(sch, private);
+ private->state = EADM_IDLE;
+ private->sch = sch;
+ sch->isc = EADM_SCH_ISC;
+ ret = cio_enable_subchannel(sch, (u32)(unsigned long)sch);
+ if (ret) {
+ set_eadm_private(sch, NULL);
+ spin_unlock_irq(sch->lock);
+ kfree(private);
+ goto out;
+ }
+ spin_unlock_irq(sch->lock);
+
+ spin_lock_irq(&list_lock);
+ list_add(&private->head, &eadm_list);
+ spin_unlock_irq(&list_lock);
+
+ if (dev_get_uevent_suppress(&sch->dev)) {
+ dev_set_uevent_suppress(&sch->dev, 0);
+ kobject_uevent(&sch->dev.kobj, KOBJ_ADD);
+ }
+out:
+ return ret;
+}
+
+static void eadm_quiesce(struct subchannel *sch)
+{
+ int ret;
+
+ do {
+ spin_lock_irq(sch->lock);
+ ret = cio_disable_subchannel(sch);
+ spin_unlock_irq(sch->lock);
+ } while (ret == -EBUSY);
+}
+
+static int eadm_subchannel_remove(struct subchannel *sch)
+{
+ struct eadm_private *private = get_eadm_private(sch);
+
+ spin_lock_irq(&list_lock);
+ list_del(&private->head);
+ spin_unlock_irq(&list_lock);
+
+ eadm_quiesce(sch);
+
+ spin_lock_irq(sch->lock);
+ set_eadm_private(sch, NULL);
+ spin_unlock_irq(sch->lock);
+
+ kfree(private);
+
+ return 0;
+}
+
+static void eadm_subchannel_shutdown(struct subchannel *sch)
+{
+ eadm_quiesce(sch);
+}
+
+static int eadm_subchannel_freeze(struct subchannel *sch)
+{
+ return cio_disable_subchannel(sch);
+}
+
+static int eadm_subchannel_restore(struct subchannel *sch)
+{
+ return cio_enable_subchannel(sch, (u32)(unsigned long)sch);
+}
+
+/**
+ * eadm_subchannel_sch_event - process subchannel event
+ * @sch: subchannel
+ * @process: non-zero if function is called in process context
+ *
+ * An unspecified event occurred for this subchannel. Adjust data according
+ * to the current operational state of the subchannel. Return zero when the
+ * event has been handled sufficiently or -EAGAIN when this function should
+ * be called again in process context.
+ */
+static int eadm_subchannel_sch_event(struct subchannel *sch, int process)
+{
+ struct eadm_private *private;
+ unsigned long flags;
+ int ret = 0;
+
+ spin_lock_irqsave(sch->lock, flags);
+ if (!device_is_registered(&sch->dev))
+ goto out_unlock;
+
+ if (work_pending(&sch->todo_work))
+ goto out_unlock;
+
+ if (cio_update_schib(sch)) {
+ css_sched_sch_todo(sch, SCH_TODO_UNREG);
+ goto out_unlock;
+ }
+ private = get_eadm_private(sch);
+ if (private->state == EADM_NOT_OPER)
+ private->state = EADM_IDLE;
+
+out_unlock:
+ spin_unlock_irqrestore(sch->lock, flags);
+
+ return ret;
+}
+
+static struct css_device_id eadm_subchannel_ids[] = {
+ { .match_flags = 0x1, .type = SUBCHANNEL_TYPE_ADM, },
+ { /* end of list */ },
+};
+MODULE_DEVICE_TABLE(css, eadm_subchannel_ids);
+
+static struct css_driver eadm_subchannel_driver = {
+ .drv = {
+ .name = "eadm_subchannel",
+ .owner = THIS_MODULE,
+ },
+ .subchannel_type = eadm_subchannel_ids,
+ .irq = eadm_subchannel_irq,
+ .probe = eadm_subchannel_probe,
+ .remove = eadm_subchannel_remove,
+ .shutdown = eadm_subchannel_shutdown,
+ .sch_event = eadm_subchannel_sch_event,
+ .freeze = eadm_subchannel_freeze,
+ .thaw = eadm_subchannel_restore,
+ .restore = eadm_subchannel_restore,
+};
+
+static struct eadm_ops eadm_ops = {
+ .eadm_start = eadm_start_aob,
+ .owner = THIS_MODULE,
+};
+
+static int __init eadm_sch_init(void)
+{
+ int ret;
+
+ if (!css_general_characteristics.eadm)
+ return -ENXIO;
+
+ eadm_debug = debug_register("eadm_log", 16, 1, 16);
+ if (!eadm_debug)
+ return -ENOMEM;
+
+ debug_register_view(eadm_debug, &debug_hex_ascii_view);
+ debug_set_level(eadm_debug, 2);
+
+ isc_register(EADM_SCH_ISC);
+ ret = css_driver_register(&eadm_subchannel_driver);
+ if (ret)
+ goto cleanup;
+
+ register_eadm_ops(&eadm_ops);
+ return ret;
+
+cleanup:
+ isc_unregister(EADM_SCH_ISC);
+ debug_unregister(eadm_debug);
+ return ret;
+}
+
+static void __exit eadm_sch_exit(void)
+{
+ unregister_eadm_ops(&eadm_ops);
+ css_driver_unregister(&eadm_subchannel_driver);
+ isc_unregister(EADM_SCH_ISC);
+ debug_unregister(eadm_debug);
+}
+module_init(eadm_sch_init);
+module_exit(eadm_sch_exit);
diff --git a/drivers/s390/cio/eadm_sch.h b/drivers/s390/cio/eadm_sch.h
new file mode 100644
index 0000000..2779be0
--- /dev/null
+++ b/drivers/s390/cio/eadm_sch.h
@@ -0,0 +1,20 @@
+#ifndef EADM_SCH_H
+#define EADM_SCH_H
+
+#include <linux/device.h>
+#include <linux/timer.h>
+#include <linux/list.h>
+#include "orb.h"
+
+struct eadm_private {
+ union orb orb;
+ enum {EADM_IDLE, EADM_BUSY, EADM_NOT_OPER} state;
+ struct timer_list timer;
+ struct list_head head;
+ struct subchannel *sch;
+} __aligned(8);
+
+#define get_eadm_private(n) ((struct eadm_private *)dev_get_drvdata(&n->dev))
+#define set_eadm_private(n, p) (dev_set_drvdata(&n->dev, p))
+
+#endif
diff --git a/drivers/s390/cio/orb.h b/drivers/s390/cio/orb.h
index 45a9865..7a64053 100644
--- a/drivers/s390/cio/orb.h
+++ b/drivers/s390/cio/orb.h
@@ -59,9 +59,33 @@
u32:32;
} __packed __aligned(4);
+/*
+ * eadm operation request block
+ */
+struct eadm_orb {
+ u32 intparm;
+ u32 key:4;
+ u32:4;
+ u32 compat1:1;
+ u32 compat2:1;
+ u32:21;
+ u32 x:1;
+ u32 aob;
+ u32 css_prio:8;
+ u32:8;
+ u32 scm_prio:8;
+ u32:8;
+ u32:29;
+ u32 fmt:3;
+ u32:32;
+ u32:32;
+ u32:32;
+} __packed __aligned(4);
+
union orb {
struct cmd_orb cmd;
struct tm_orb tm;
+ struct eadm_orb eadm;
} __packed __aligned(4);
#endif /* S390_ORB_H */
diff --git a/drivers/s390/cio/qdio_debug.h b/drivers/s390/cio/qdio_debug.h
index e1f6468..7f8b973 100644
--- a/drivers/s390/cio/qdio_debug.h
+++ b/drivers/s390/cio/qdio_debug.h
@@ -37,10 +37,14 @@
debug_text_event(qdio_dbf_setup, DBF_ERR, debug_buffer); \
} while (0)
-#define DBF_HEX(addr, len) \
- do { \
- debug_event(qdio_dbf_setup, DBF_ERR, (void*)(addr), len); \
- } while (0)
+static inline void DBF_HEX(void *addr, int len)
+{
+ while (len > 0) {
+ debug_event(qdio_dbf_setup, DBF_ERR, addr, len);
+ len -= qdio_dbf_setup->buf_size;
+ addr += qdio_dbf_setup->buf_size;
+ }
+}
#define DBF_ERROR(text...) \
do { \
@@ -49,11 +53,14 @@
debug_text_event(qdio_dbf_error, DBF_ERR, debug_buffer); \
} while (0)
-#define DBF_ERROR_HEX(addr, len) \
- do { \
- debug_event(qdio_dbf_error, DBF_ERR, (void*)(addr), len); \
- } while (0)
-
+static inline void DBF_ERROR_HEX(void *addr, int len)
+{
+ while (len > 0) {
+ debug_event(qdio_dbf_error, DBF_ERR, addr, len);
+ len -= qdio_dbf_error->buf_size;
+ addr += qdio_dbf_error->buf_size;
+ }
+}
#define DBF_DEV_EVENT(level, device, text...) \
do { \
@@ -64,10 +71,15 @@
} \
} while (0)
-#define DBF_DEV_HEX(level, device, addr, len) \
- do { \
- debug_event(device->debug_area, level, (void*)(addr), len); \
- } while (0)
+static inline void DBF_DEV_HEX(struct qdio_irq *dev, void *addr,
+ int len, int level)
+{
+ while (len > 0) {
+ debug_event(dev->debug_area, level, addr, len);
+ len -= dev->debug_area->buf_size;
+ addr += dev->debug_area->buf_size;
+ }
+}
void qdio_allocate_dbf(struct qdio_initialize *init_data,
struct qdio_irq *irq_ptr);
diff --git a/drivers/s390/cio/scm.c b/drivers/s390/cio/scm.c
new file mode 100644
index 0000000..bcf20f3
--- /dev/null
+++ b/drivers/s390/cio/scm.c
@@ -0,0 +1,317 @@
+/*
+ * Recognize and maintain s390 storage class memory.
+ *
+ * Copyright IBM Corp. 2012
+ * Author(s): Sebastian Ott <sebott@linux.vnet.ibm.com>
+ */
+
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/err.h>
+#include <asm/eadm.h>
+#include "chsc.h"
+
+static struct device *scm_root;
+static struct eadm_ops *eadm_ops;
+static DEFINE_MUTEX(eadm_ops_mutex);
+
+#define to_scm_dev(n) container_of(n, struct scm_device, dev)
+#define to_scm_drv(d) container_of(d, struct scm_driver, drv)
+
+static int scmdev_probe(struct device *dev)
+{
+ struct scm_device *scmdev = to_scm_dev(dev);
+ struct scm_driver *scmdrv = to_scm_drv(dev->driver);
+
+ return scmdrv->probe ? scmdrv->probe(scmdev) : -ENODEV;
+}
+
+static int scmdev_remove(struct device *dev)
+{
+ struct scm_device *scmdev = to_scm_dev(dev);
+ struct scm_driver *scmdrv = to_scm_drv(dev->driver);
+
+ return scmdrv->remove ? scmdrv->remove(scmdev) : -ENODEV;
+}
+
+static int scmdev_uevent(struct device *dev, struct kobj_uevent_env *env)
+{
+ return add_uevent_var(env, "MODALIAS=scm:scmdev");
+}
+
+static struct bus_type scm_bus_type = {
+ .name = "scm",
+ .probe = scmdev_probe,
+ .remove = scmdev_remove,
+ .uevent = scmdev_uevent,
+};
+
+/**
+ * scm_driver_register() - register a scm driver
+ * @scmdrv: driver to be registered
+ */
+int scm_driver_register(struct scm_driver *scmdrv)
+{
+ struct device_driver *drv = &scmdrv->drv;
+
+ drv->bus = &scm_bus_type;
+
+ return driver_register(drv);
+}
+EXPORT_SYMBOL_GPL(scm_driver_register);
+
+/**
+ * scm_driver_unregister() - deregister a scm driver
+ * @scmdrv: driver to be deregistered
+ */
+void scm_driver_unregister(struct scm_driver *scmdrv)
+{
+ driver_unregister(&scmdrv->drv);
+}
+EXPORT_SYMBOL_GPL(scm_driver_unregister);
+
+int scm_get_ref(void)
+{
+ int ret = 0;
+
+ mutex_lock(&eadm_ops_mutex);
+ if (!eadm_ops || !try_module_get(eadm_ops->owner))
+ ret = -ENOENT;
+ mutex_unlock(&eadm_ops_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(scm_get_ref);
+
+void scm_put_ref(void)
+{
+ mutex_lock(&eadm_ops_mutex);
+ module_put(eadm_ops->owner);
+ mutex_unlock(&eadm_ops_mutex);
+}
+EXPORT_SYMBOL_GPL(scm_put_ref);
+
+void register_eadm_ops(struct eadm_ops *ops)
+{
+ mutex_lock(&eadm_ops_mutex);
+ eadm_ops = ops;
+ mutex_unlock(&eadm_ops_mutex);
+}
+EXPORT_SYMBOL_GPL(register_eadm_ops);
+
+void unregister_eadm_ops(struct eadm_ops *ops)
+{
+ mutex_lock(&eadm_ops_mutex);
+ eadm_ops = NULL;
+ mutex_unlock(&eadm_ops_mutex);
+}
+EXPORT_SYMBOL_GPL(unregister_eadm_ops);
+
+int scm_start_aob(struct aob *aob)
+{
+ return eadm_ops->eadm_start(aob);
+}
+EXPORT_SYMBOL_GPL(scm_start_aob);
+
+void scm_irq_handler(struct aob *aob, int error)
+{
+ struct aob_rq_header *aobrq = (void *) aob->request.data;
+ struct scm_device *scmdev = aobrq->scmdev;
+ struct scm_driver *scmdrv = to_scm_drv(scmdev->dev.driver);
+
+ scmdrv->handler(scmdev, aobrq->data, error);
+}
+EXPORT_SYMBOL_GPL(scm_irq_handler);
+
+#define scm_attr(name) \
+static ssize_t show_##name(struct device *dev, \
+ struct device_attribute *attr, char *buf) \
+{ \
+ struct scm_device *scmdev = to_scm_dev(dev); \
+ int ret; \
+ \
+ device_lock(dev); \
+ ret = sprintf(buf, "%u\n", scmdev->attrs.name); \
+ device_unlock(dev); \
+ \
+ return ret; \
+} \
+static DEVICE_ATTR(name, S_IRUGO, show_##name, NULL);
+
+scm_attr(persistence);
+scm_attr(oper_state);
+scm_attr(data_state);
+scm_attr(rank);
+scm_attr(release);
+scm_attr(res_id);
+
+static struct attribute *scmdev_attrs[] = {
+ &dev_attr_persistence.attr,
+ &dev_attr_oper_state.attr,
+ &dev_attr_data_state.attr,
+ &dev_attr_rank.attr,
+ &dev_attr_release.attr,
+ &dev_attr_res_id.attr,
+ NULL,
+};
+
+static struct attribute_group scmdev_attr_group = {
+ .attrs = scmdev_attrs,
+};
+
+static const struct attribute_group *scmdev_attr_groups[] = {
+ &scmdev_attr_group,
+ NULL,
+};
+
+static void scmdev_release(struct device *dev)
+{
+ struct scm_device *scmdev = to_scm_dev(dev);
+
+ kfree(scmdev);
+}
+
+static void scmdev_setup(struct scm_device *scmdev, struct sale *sale,
+ unsigned int size, unsigned int max_blk_count)
+{
+ dev_set_name(&scmdev->dev, "%016llx", (unsigned long long) sale->sa);
+ scmdev->nr_max_block = max_blk_count;
+ scmdev->address = sale->sa;
+ scmdev->size = 1UL << size;
+ scmdev->attrs.rank = sale->rank;
+ scmdev->attrs.persistence = sale->p;
+ scmdev->attrs.oper_state = sale->op_state;
+ scmdev->attrs.data_state = sale->data_state;
+ scmdev->attrs.rank = sale->rank;
+ scmdev->attrs.release = sale->r;
+ scmdev->attrs.res_id = sale->rid;
+ scmdev->dev.parent = scm_root;
+ scmdev->dev.bus = &scm_bus_type;
+ scmdev->dev.release = scmdev_release;
+ scmdev->dev.groups = scmdev_attr_groups;
+}
+
+/*
+ * Check for state-changes, notify the driver and userspace.
+ */
+static void scmdev_update(struct scm_device *scmdev, struct sale *sale)
+{
+ struct scm_driver *scmdrv;
+ bool changed;
+
+ device_lock(&scmdev->dev);
+ changed = scmdev->attrs.rank != sale->rank ||
+ scmdev->attrs.oper_state != sale->op_state;
+ scmdev->attrs.rank = sale->rank;
+ scmdev->attrs.oper_state = sale->op_state;
+ if (!scmdev->dev.driver)
+ goto out;
+ scmdrv = to_scm_drv(scmdev->dev.driver);
+ if (changed && scmdrv->notify)
+ scmdrv->notify(scmdev);
+out:
+ device_unlock(&scmdev->dev);
+ if (changed)
+ kobject_uevent(&scmdev->dev.kobj, KOBJ_CHANGE);
+}
+
+static int check_address(struct device *dev, void *data)
+{
+ struct scm_device *scmdev = to_scm_dev(dev);
+ struct sale *sale = data;
+
+ return scmdev->address == sale->sa;
+}
+
+static struct scm_device *scmdev_find(struct sale *sale)
+{
+ struct device *dev;
+
+ dev = bus_find_device(&scm_bus_type, NULL, sale, check_address);
+
+ return dev ? to_scm_dev(dev) : NULL;
+}
+
+static int scm_add(struct chsc_scm_info *scm_info, size_t num)
+{
+ struct sale *sale, *scmal = scm_info->scmal;
+ struct scm_device *scmdev;
+ int ret;
+
+ for (sale = scmal; sale < scmal + num; sale++) {
+ scmdev = scmdev_find(sale);
+ if (scmdev) {
+ scmdev_update(scmdev, sale);
+ /* Release reference from scm_find(). */
+ put_device(&scmdev->dev);
+ continue;
+ }
+ scmdev = kzalloc(sizeof(*scmdev), GFP_KERNEL);
+ if (!scmdev)
+ return -ENODEV;
+ scmdev_setup(scmdev, sale, scm_info->is, scm_info->mbc);
+ ret = device_register(&scmdev->dev);
+ if (ret) {
+ /* Release reference from device_initialize(). */
+ put_device(&scmdev->dev);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+int scm_update_information(void)
+{
+ struct chsc_scm_info *scm_info;
+ u64 token = 0;
+ size_t num;
+ int ret;
+
+ scm_info = (void *)__get_free_page(GFP_KERNEL | GFP_DMA);
+ if (!scm_info)
+ return -ENOMEM;
+
+ do {
+ ret = chsc_scm_info(scm_info, token);
+ if (ret)
+ break;
+
+ num = (scm_info->response.length -
+ (offsetof(struct chsc_scm_info, scmal) -
+ offsetof(struct chsc_scm_info, response))
+ ) / sizeof(struct sale);
+
+ ret = scm_add(scm_info, num);
+ if (ret)
+ break;
+
+ token = scm_info->restok;
+ } while (token);
+
+ free_page((unsigned long)scm_info);
+
+ return ret;
+}
+
+static int __init scm_init(void)
+{
+ int ret;
+
+ ret = bus_register(&scm_bus_type);
+ if (ret)
+ return ret;
+
+ scm_root = root_device_register("scm");
+ if (IS_ERR(scm_root)) {
+ bus_unregister(&scm_bus_type);
+ return PTR_ERR(scm_root);
+ }
+
+ scm_update_information();
+ return 0;
+}
+subsys_initcall_sync(scm_init);
diff --git a/drivers/s390/crypto/Makefile b/drivers/s390/crypto/Makefile
index af3c7f1..771faf70 100644
--- a/drivers/s390/crypto/Makefile
+++ b/drivers/s390/crypto/Makefile
@@ -4,4 +4,5 @@
ap-objs := ap_bus.o
obj-$(CONFIG_ZCRYPT) += ap.o zcrypt_api.o zcrypt_pcicc.o zcrypt_pcixcc.o
-obj-$(CONFIG_ZCRYPT) += zcrypt_pcica.o zcrypt_cex2a.o
+obj-$(CONFIG_ZCRYPT) += zcrypt_pcica.o zcrypt_cex2a.o zcrypt_cex4.o
+obj-$(CONFIG_ZCRYPT) += zcrypt_msgtype6.o zcrypt_msgtype50.o
diff --git a/drivers/s390/crypto/ap_bus.c b/drivers/s390/crypto/ap_bus.c
index ae258a4..7b865a7 100644
--- a/drivers/s390/crypto/ap_bus.c
+++ b/drivers/s390/crypto/ap_bus.c
@@ -1,5 +1,5 @@
/*
- * Copyright IBM Corp. 2006
+ * Copyright IBM Corp. 2006, 2012
* Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
* Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
@@ -62,13 +62,14 @@
static void ap_reset(struct ap_device *ap_dev);
static void ap_config_timeout(unsigned long ptr);
static int ap_select_domain(void);
+static void ap_query_configuration(void);
/*
* Module description.
*/
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("Adjunct Processor Bus driver, "
- "Copyright IBM Corp. 2006");
+MODULE_DESCRIPTION("Adjunct Processor Bus driver, " \
+ "Copyright IBM Corp. 2006, 2012");
MODULE_LICENSE("GPL");
/*
@@ -84,6 +85,7 @@
MODULE_PARM_DESC(poll_thread, "Turn on/off poll thread, default is 0 (off).");
static struct device *ap_root_device = NULL;
+static struct ap_config_info *ap_configuration;
static DEFINE_SPINLOCK(ap_device_list_lock);
static LIST_HEAD(ap_device_list);
@@ -158,6 +160,19 @@
}
/**
+ * ap_configuration_available(): Test if AP configuration
+ * information is available.
+ *
+ * Returns 1 if AP configuration information is available.
+ */
+#ifdef CONFIG_64BIT
+static int ap_configuration_available(void)
+{
+ return test_facility(2) && test_facility(12);
+}
+#endif
+
+/**
* ap_test_queue(): Test adjunct processor queue.
* @qid: The AP queue number
* @queue_depth: Pointer to queue depth value
@@ -242,6 +257,26 @@
}
#endif
+#ifdef CONFIG_64BIT
+static inline int __ap_query_configuration(struct ap_config_info *config)
+{
+ register unsigned long reg0 asm ("0") = 0x04000000UL;
+ register unsigned long reg1 asm ("1") = -EINVAL;
+ register unsigned char *reg2 asm ("2") = (unsigned char *)config;
+
+ asm volatile(
+ ".long 0xb2af0000\n" /* PQAP(QCI) */
+ "0: la %1,0\n"
+ "1:\n"
+ EX_TABLE(0b, 1b)
+ : "+d" (reg0), "+d" (reg1), "+d" (reg2)
+ :
+ : "cc");
+
+ return reg1;
+}
+#endif
+
/**
* ap_query_functions(): Query supported functions.
* @qid: The AP queue number
@@ -292,25 +327,6 @@
}
/**
- * ap_4096_commands_availablen(): Check for availability of 4096 bit RSA
- * support.
- * @qid: The AP queue number
- *
- * Returns 1 if 4096 bit RSA keys are support fo the AP, returns 0 if not.
- */
-int ap_4096_commands_available(ap_qid_t qid)
-{
- unsigned int functions;
-
- if (ap_query_functions(qid, &functions))
- return 0;
-
- return test_ap_facility(functions, 1) &&
- test_ap_facility(functions, 2);
-}
-EXPORT_SYMBOL(ap_4096_commands_available);
-
-/**
* ap_queue_enable_interruption(): Enable interruption on an AP.
* @qid: The AP queue number
* @ind: the notification indicator byte
@@ -657,6 +673,34 @@
static DEVICE_ATTR(request_count, 0444, ap_request_count_show, NULL);
+static ssize_t ap_requestq_count_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ap_device *ap_dev = to_ap_dev(dev);
+ int rc;
+
+ spin_lock_bh(&ap_dev->lock);
+ rc = snprintf(buf, PAGE_SIZE, "%d\n", ap_dev->requestq_count);
+ spin_unlock_bh(&ap_dev->lock);
+ return rc;
+}
+
+static DEVICE_ATTR(requestq_count, 0444, ap_requestq_count_show, NULL);
+
+static ssize_t ap_pendingq_count_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ap_device *ap_dev = to_ap_dev(dev);
+ int rc;
+
+ spin_lock_bh(&ap_dev->lock);
+ rc = snprintf(buf, PAGE_SIZE, "%d\n", ap_dev->pendingq_count);
+ spin_unlock_bh(&ap_dev->lock);
+ return rc;
+}
+
+static DEVICE_ATTR(pendingq_count, 0444, ap_pendingq_count_show, NULL);
+
static ssize_t ap_modalias_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -665,11 +709,23 @@
static DEVICE_ATTR(modalias, 0444, ap_modalias_show, NULL);
+static ssize_t ap_functions_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ap_device *ap_dev = to_ap_dev(dev);
+ return snprintf(buf, PAGE_SIZE, "0x%08X\n", ap_dev->functions);
+}
+
+static DEVICE_ATTR(ap_functions, 0444, ap_functions_show, NULL);
+
static struct attribute *ap_dev_attrs[] = {
&dev_attr_hwtype.attr,
&dev_attr_depth.attr,
&dev_attr_request_count.attr,
+ &dev_attr_requestq_count.attr,
+ &dev_attr_pendingq_count.attr,
&dev_attr_modalias.attr,
+ &dev_attr_ap_functions.attr,
NULL
};
static struct attribute_group ap_dev_attr_group = {
@@ -772,6 +828,7 @@
ap_suspend_flag = 0;
if (!ap_interrupts_available())
ap_interrupt_indicator = NULL;
+ ap_query_configuration();
if (!user_set_domain) {
ap_domain_index = -1;
ap_select_domain();
@@ -895,6 +952,20 @@
}
EXPORT_SYMBOL(ap_driver_unregister);
+void ap_bus_force_rescan(void)
+{
+ /* Delete the AP bus rescan timer. */
+ del_timer(&ap_config_timer);
+
+ /* processing a synchonuous bus rescan */
+ ap_scan_bus(NULL);
+
+ /* Setup the AP bus rescan timer again. */
+ ap_config_timer.expires = jiffies + ap_config_time * HZ;
+ add_timer(&ap_config_timer);
+}
+EXPORT_SYMBOL(ap_bus_force_rescan);
+
/*
* AP bus attributes.
*/
@@ -997,6 +1068,65 @@
NULL,
};
+static inline int ap_test_config(unsigned int *field, unsigned int nr)
+{
+ if (nr > 0xFFu)
+ return 0;
+ return ap_test_bit((field + (nr >> 5)), (nr & 0x1f));
+}
+
+/*
+ * ap_test_config_card_id(): Test, whether an AP card ID is configured.
+ * @id AP card ID
+ *
+ * Returns 0 if the card is not configured
+ * 1 if the card is configured or
+ * if the configuration information is not available
+ */
+static inline int ap_test_config_card_id(unsigned int id)
+{
+ if (!ap_configuration)
+ return 1;
+ return ap_test_config(ap_configuration->apm, id);
+}
+
+/*
+ * ap_test_config_domain(): Test, whether an AP usage domain is configured.
+ * @domain AP usage domain ID
+ *
+ * Returns 0 if the usage domain is not configured
+ * 1 if the usage domain is configured or
+ * if the configuration information is not available
+ */
+static inline int ap_test_config_domain(unsigned int domain)
+{
+ if (!ap_configuration)
+ return 1;
+ return ap_test_config(ap_configuration->aqm, domain);
+}
+
+/**
+ * ap_query_configuration(): Query AP configuration information.
+ *
+ * Query information of installed cards and configured domains from AP.
+ */
+static void ap_query_configuration(void)
+{
+#ifdef CONFIG_64BIT
+ if (ap_configuration_available()) {
+ if (!ap_configuration)
+ ap_configuration =
+ kzalloc(sizeof(struct ap_config_info),
+ GFP_KERNEL);
+ if (ap_configuration)
+ __ap_query_configuration(ap_configuration);
+ } else
+ ap_configuration = NULL;
+#else
+ ap_configuration = NULL;
+#endif
+}
+
/**
* ap_select_domain(): Select an AP domain.
*
@@ -1005,6 +1135,7 @@
static int ap_select_domain(void)
{
int queue_depth, device_type, count, max_count, best_domain;
+ ap_qid_t qid;
int rc, i, j;
/*
@@ -1018,9 +1149,13 @@
best_domain = -1;
max_count = 0;
for (i = 0; i < AP_DOMAINS; i++) {
+ if (!ap_test_config_domain(i))
+ continue;
count = 0;
for (j = 0; j < AP_DEVICES; j++) {
- ap_qid_t qid = AP_MKQID(j, i);
+ if (!ap_test_config_card_id(j))
+ continue;
+ qid = AP_MKQID(j, i);
rc = ap_query_queue(qid, &queue_depth, &device_type);
if (rc)
continue;
@@ -1169,6 +1304,7 @@
unsigned int device_functions;
int rc, i;
+ ap_query_configuration();
if (ap_select_domain() != 0)
return;
for (i = 0; i < AP_DEVICES; i++) {
@@ -1176,7 +1312,10 @@
dev = bus_find_device(&ap_bus_type, NULL,
(void *)(unsigned long)qid,
__ap_scan_bus);
- rc = ap_query_queue(qid, &queue_depth, &device_type);
+ if (ap_test_config_card_id(i))
+ rc = ap_query_queue(qid, &queue_depth, &device_type);
+ else
+ rc = -ENODEV;
if (dev) {
if (rc == -EBUSY) {
set_current_state(TASK_UNINTERRUPTIBLE);
@@ -1217,29 +1356,22 @@
(unsigned long) ap_dev);
switch (device_type) {
case 0:
+ /* device type probing for old cards */
if (ap_probe_device_type(ap_dev)) {
kfree(ap_dev);
continue;
}
break;
- case 10:
- if (ap_query_functions(qid, &device_functions)) {
- kfree(ap_dev);
- continue;
- }
- if (test_ap_facility(device_functions, 3))
- ap_dev->device_type = AP_DEVICE_TYPE_CEX3C;
- else if (test_ap_facility(device_functions, 4))
- ap_dev->device_type = AP_DEVICE_TYPE_CEX3A;
- else {
- kfree(ap_dev);
- continue;
- }
- break;
default:
ap_dev->device_type = device_type;
}
+ rc = ap_query_functions(qid, &device_functions);
+ if (!rc)
+ ap_dev->functions = device_functions;
+ else
+ ap_dev->functions = 0u;
+
ap_dev->device.bus = &ap_bus_type;
ap_dev->device.parent = ap_root_device;
if (dev_set_name(&ap_dev->device, "card%02x",
@@ -1785,6 +1917,7 @@
goto out_root;
}
+ ap_query_configuration();
if (ap_select_domain() == 0)
ap_scan_bus(NULL);
diff --git a/drivers/s390/crypto/ap_bus.h b/drivers/s390/crypto/ap_bus.h
index 52d6199..685f6cc0 100644
--- a/drivers/s390/crypto/ap_bus.h
+++ b/drivers/s390/crypto/ap_bus.h
@@ -1,5 +1,5 @@
/*
- * Copyright IBM Corp. 2006
+ * Copyright IBM Corp. 2006, 2012
* Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
* Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
@@ -83,13 +83,12 @@
return !(memcmp(status, &invalid, sizeof(struct ap_queue_status)));
}
-#define MAX_AP_FACILITY 31
-
-static inline int test_ap_facility(unsigned int function, unsigned int nr)
+#define AP_MAX_BITS 31
+static inline int ap_test_bit(unsigned int *ptr, unsigned int nr)
{
- if (nr > MAX_AP_FACILITY)
+ if (nr > AP_MAX_BITS)
return 0;
- return function & (unsigned int)(0x80000000 >> nr);
+ return (*ptr & (0x80000000u >> nr)) != 0;
}
#define AP_RESPONSE_NORMAL 0x00
@@ -117,6 +116,15 @@
#define AP_DEVICE_TYPE_CEX2C 7
#define AP_DEVICE_TYPE_CEX3A 8
#define AP_DEVICE_TYPE_CEX3C 9
+#define AP_DEVICE_TYPE_CEX4 10
+
+/*
+ * Known function facilities
+ */
+#define AP_FUNC_MEX4K 1
+#define AP_FUNC_CRT4K 2
+#define AP_FUNC_COPRO 3
+#define AP_FUNC_ACCEL 4
/*
* AP reset flag states
@@ -151,6 +159,7 @@
ap_qid_t qid; /* AP queue id. */
int queue_depth; /* AP queue depth.*/
int device_type; /* AP device type. */
+ unsigned int functions; /* AP device function bitfield. */
int unregistered; /* marks AP device as unregistered */
struct timer_list timeout; /* Timer for request timeouts. */
int reset; /* Reset required after req. timeout. */
@@ -183,6 +192,17 @@
struct ap_message *);
};
+struct ap_config_info {
+ unsigned int special_command:1;
+ unsigned int ap_extended:1;
+ unsigned char reserved1:6;
+ unsigned char reserved2[15];
+ unsigned int apm[8]; /* AP ID mask */
+ unsigned int aqm[8]; /* AP queue mask */
+ unsigned int adm[8]; /* AP domain mask */
+ unsigned char reserved4[16];
+} __packed;
+
#define AP_DEVICE(dt) \
.dev_type=(dt), \
.match_flags=AP_DEVICE_ID_MATCH_DEVICE_TYPE,
@@ -211,10 +231,9 @@
void ap_queue_message(struct ap_device *ap_dev, struct ap_message *ap_msg);
void ap_cancel_message(struct ap_device *ap_dev, struct ap_message *ap_msg);
void ap_flush_queue(struct ap_device *ap_dev);
+void ap_bus_force_rescan(void);
int ap_module_init(void);
void ap_module_exit(void);
-int ap_4096_commands_available(ap_qid_t qid);
-
#endif /* _AP_BUS_H_ */
diff --git a/drivers/s390/crypto/zcrypt_api.c b/drivers/s390/crypto/zcrypt_api.c
index 2f94132..31cfaa5 100644
--- a/drivers/s390/crypto/zcrypt_api.c
+++ b/drivers/s390/crypto/zcrypt_api.c
@@ -1,7 +1,7 @@
/*
* zcrypt 2.1.0
*
- * Copyright IBM Corp. 2001, 2006
+ * Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
* Cornelia Huck <cornelia.huck@de.ibm.com>
@@ -9,6 +9,7 @@
* Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
* Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -37,25 +38,39 @@
#include <linux/atomic.h>
#include <asm/uaccess.h>
#include <linux/hw_random.h>
+#include <linux/debugfs.h>
+#include <asm/debug.h>
+#include "zcrypt_debug.h"
#include "zcrypt_api.h"
/*
* Module description.
*/
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("Cryptographic Coprocessor interface, "
- "Copyright IBM Corp. 2001, 2006");
+MODULE_DESCRIPTION("Cryptographic Coprocessor interface, " \
+ "Copyright IBM Corp. 2001, 2012");
MODULE_LICENSE("GPL");
static DEFINE_SPINLOCK(zcrypt_device_lock);
static LIST_HEAD(zcrypt_device_list);
static int zcrypt_device_count = 0;
static atomic_t zcrypt_open_count = ATOMIC_INIT(0);
+static atomic_t zcrypt_rescan_count = ATOMIC_INIT(0);
+
+atomic_t zcrypt_rescan_req = ATOMIC_INIT(0);
+EXPORT_SYMBOL(zcrypt_rescan_req);
static int zcrypt_rng_device_add(void);
static void zcrypt_rng_device_remove(void);
+static DEFINE_SPINLOCK(zcrypt_ops_list_lock);
+static LIST_HEAD(zcrypt_ops_list);
+
+static debug_info_t *zcrypt_dbf_common;
+static debug_info_t *zcrypt_dbf_devices;
+static struct dentry *debugfs_root;
+
/*
* Device attributes common for all crypto devices.
*/
@@ -85,6 +100,8 @@
if (sscanf(buf, "%d\n", &online) != 1 || online < 0 || online > 1)
return -EINVAL;
zdev->online = online;
+ ZCRYPT_DBF_DEV(DBF_INFO, zdev, "dev%04xo%dman", zdev->ap_dev->qid,
+ zdev->online);
if (!online)
ap_flush_queue(zdev->ap_dev);
return count;
@@ -103,6 +120,24 @@
};
/**
+ * Process a rescan of the transport layer.
+ *
+ * Returns 1, if the rescan has been processed, otherwise 0.
+ */
+static inline int zcrypt_process_rescan(void)
+{
+ if (atomic_read(&zcrypt_rescan_req)) {
+ atomic_set(&zcrypt_rescan_req, 0);
+ atomic_inc(&zcrypt_rescan_count);
+ ap_bus_force_rescan();
+ ZCRYPT_DBF_COMMON(DBF_INFO, "rescan%07d",
+ atomic_inc_return(&zcrypt_rescan_count));
+ return 1;
+ }
+ return 0;
+}
+
+/**
* __zcrypt_increase_preference(): Increase preference of a crypto device.
* @zdev: Pointer the crypto device
*
@@ -190,6 +225,7 @@
zdev->reply.length = max_response_size;
spin_lock_init(&zdev->lock);
INIT_LIST_HEAD(&zdev->list);
+ zdev->dbf_area = zcrypt_dbf_devices;
return zdev;
out_free:
@@ -215,6 +251,8 @@
{
int rc;
+ if (!zdev->ops)
+ return -ENODEV;
rc = sysfs_create_group(&zdev->ap_dev->device.kobj,
&zcrypt_device_attr_group);
if (rc)
@@ -223,6 +261,8 @@
kref_init(&zdev->refcount);
spin_lock_bh(&zcrypt_device_lock);
zdev->online = 1; /* New devices are online by default. */
+ ZCRYPT_DBF_DEV(DBF_INFO, zdev, "dev%04xo%dreg", zdev->ap_dev->qid,
+ zdev->online);
list_add_tail(&zdev->list, &zcrypt_device_list);
__zcrypt_increase_preference(zdev);
zcrypt_device_count++;
@@ -269,6 +309,67 @@
}
EXPORT_SYMBOL(zcrypt_device_unregister);
+void zcrypt_msgtype_register(struct zcrypt_ops *zops)
+{
+ if (zops->owner) {
+ spin_lock_bh(&zcrypt_ops_list_lock);
+ list_add_tail(&zops->list, &zcrypt_ops_list);
+ spin_unlock_bh(&zcrypt_ops_list_lock);
+ }
+}
+EXPORT_SYMBOL(zcrypt_msgtype_register);
+
+void zcrypt_msgtype_unregister(struct zcrypt_ops *zops)
+{
+ spin_lock_bh(&zcrypt_ops_list_lock);
+ list_del_init(&zops->list);
+ spin_unlock_bh(&zcrypt_ops_list_lock);
+}
+EXPORT_SYMBOL(zcrypt_msgtype_unregister);
+
+static inline
+struct zcrypt_ops *__ops_lookup(unsigned char *name, int variant)
+{
+ struct zcrypt_ops *zops;
+ int found = 0;
+
+ spin_lock_bh(&zcrypt_ops_list_lock);
+ list_for_each_entry(zops, &zcrypt_ops_list, list) {
+ if ((zops->variant == variant) &&
+ (!strncmp(zops->owner->name, name, MODULE_NAME_LEN))) {
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock_bh(&zcrypt_ops_list_lock);
+
+ if (!found)
+ return NULL;
+ return zops;
+}
+
+struct zcrypt_ops *zcrypt_msgtype_request(unsigned char *name, int variant)
+{
+ struct zcrypt_ops *zops = NULL;
+
+ zops = __ops_lookup(name, variant);
+ if (!zops) {
+ request_module(name);
+ zops = __ops_lookup(name, variant);
+ }
+ if ((!zops) || (!try_module_get(zops->owner)))
+ return NULL;
+ return zops;
+}
+EXPORT_SYMBOL(zcrypt_msgtype_request);
+
+void zcrypt_msgtype_release(struct zcrypt_ops *zops)
+{
+ if (zops)
+ module_put(zops->owner);
+}
+EXPORT_SYMBOL(zcrypt_msgtype_release);
+
/**
* zcrypt_read (): Not supported beyond zcrypt 1.3.1.
*
@@ -640,6 +741,11 @@
do {
rc = zcrypt_rsa_modexpo(&mex);
} while (rc == -EAGAIN);
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_rsa_modexpo(&mex);
+ } while (rc == -EAGAIN);
if (rc)
return rc;
return put_user(mex.outputdatalength, &umex->outputdatalength);
@@ -652,6 +758,11 @@
do {
rc = zcrypt_rsa_crt(&crt);
} while (rc == -EAGAIN);
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_rsa_crt(&crt);
+ } while (rc == -EAGAIN);
if (rc)
return rc;
return put_user(crt.outputdatalength, &ucrt->outputdatalength);
@@ -664,6 +775,11 @@
do {
rc = zcrypt_send_cprb(&xcRB);
} while (rc == -EAGAIN);
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_send_cprb(&xcRB);
+ } while (rc == -EAGAIN);
if (copy_to_user(uxcRB, &xcRB, sizeof(xcRB)))
return -EFAULT;
return rc;
@@ -770,10 +886,15 @@
do {
rc = zcrypt_rsa_modexpo(&mex64);
} while (rc == -EAGAIN);
- if (!rc)
- rc = put_user(mex64.outputdatalength,
- &umex32->outputdatalength);
- return rc;
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_rsa_modexpo(&mex64);
+ } while (rc == -EAGAIN);
+ if (rc)
+ return rc;
+ return put_user(mex64.outputdatalength,
+ &umex32->outputdatalength);
}
struct compat_ica_rsa_modexpo_crt {
@@ -810,10 +931,15 @@
do {
rc = zcrypt_rsa_crt(&crt64);
} while (rc == -EAGAIN);
- if (!rc)
- rc = put_user(crt64.outputdatalength,
- &ucrt32->outputdatalength);
- return rc;
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_rsa_crt(&crt64);
+ } while (rc == -EAGAIN);
+ if (rc)
+ return rc;
+ return put_user(crt64.outputdatalength,
+ &ucrt32->outputdatalength);
}
struct compat_ica_xcRB {
@@ -869,6 +995,11 @@
do {
rc = zcrypt_send_cprb(&xcRB64);
} while (rc == -EAGAIN);
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ do {
+ rc = zcrypt_send_cprb(&xcRB64);
+ } while (rc == -EAGAIN);
xcRB32.reply_control_blk_length = xcRB64.reply_control_blk_length;
xcRB32.reply_data_length = xcRB64.reply_data_length;
xcRB32.status = xcRB64.status;
@@ -1126,6 +1257,9 @@
*/
if (zcrypt_rng_buffer_index == 0) {
rc = zcrypt_rng((char *) zcrypt_rng_buffer);
+ /* on failure: retry once again after a requested rescan */
+ if ((rc == -ENODEV) && (zcrypt_process_rescan()))
+ rc = zcrypt_rng((char *) zcrypt_rng_buffer);
if (rc < 0)
return -EIO;
zcrypt_rng_buffer_index = rc / sizeof *data;
@@ -1178,6 +1312,30 @@
mutex_unlock(&zcrypt_rng_mutex);
}
+int __init zcrypt_debug_init(void)
+{
+ debugfs_root = debugfs_create_dir("zcrypt", NULL);
+
+ zcrypt_dbf_common = debug_register("zcrypt_common", 1, 1, 16);
+ debug_register_view(zcrypt_dbf_common, &debug_hex_ascii_view);
+ debug_set_level(zcrypt_dbf_common, DBF_ERR);
+
+ zcrypt_dbf_devices = debug_register("zcrypt_devices", 1, 1, 16);
+ debug_register_view(zcrypt_dbf_devices, &debug_hex_ascii_view);
+ debug_set_level(zcrypt_dbf_devices, DBF_ERR);
+
+ return 0;
+}
+
+void zcrypt_debug_exit(void)
+{
+ debugfs_remove(debugfs_root);
+ if (zcrypt_dbf_common)
+ debug_unregister(zcrypt_dbf_common);
+ if (zcrypt_dbf_devices)
+ debug_unregister(zcrypt_dbf_devices);
+}
+
/**
* zcrypt_api_init(): Module initialization.
*
@@ -1187,6 +1345,12 @@
{
int rc;
+ rc = zcrypt_debug_init();
+ if (rc)
+ goto out;
+
+ atomic_set(&zcrypt_rescan_req, 0);
+
/* Register the request sprayer. */
rc = misc_register(&zcrypt_misc_device);
if (rc < 0)
@@ -1216,6 +1380,7 @@
{
remove_proc_entry("driver/z90crypt", NULL);
misc_deregister(&zcrypt_misc_device);
+ zcrypt_debug_exit();
}
module_init(zcrypt_api_init);
diff --git a/drivers/s390/crypto/zcrypt_api.h b/drivers/s390/crypto/zcrypt_api.h
index 7a32c4b..8963291 100644
--- a/drivers/s390/crypto/zcrypt_api.h
+++ b/drivers/s390/crypto/zcrypt_api.h
@@ -1,7 +1,7 @@
/*
* zcrypt 2.1.0
*
- * Copyright IBM Corp. 2001, 2006
+ * Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
* Cornelia Huck <cornelia.huck@de.ibm.com>
@@ -9,6 +9,7 @@
* Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
* Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -28,8 +29,10 @@
#ifndef _ZCRYPT_API_H_
#define _ZCRYPT_API_H_
-#include "ap_bus.h"
+#include <linux/atomic.h>
+#include <asm/debug.h>
#include <asm/zcrypt.h>
+#include "ap_bus.h"
/* deprecated status calls */
#define ICAZ90STATUS _IOR(ZCRYPT_IOCTL_MAGIC, 0x10, struct ica_z90_status)
@@ -87,6 +90,9 @@
struct ica_rsa_modexpo_crt *);
long (*send_cprb)(struct zcrypt_device *, struct ica_xcRB *);
long (*rng)(struct zcrypt_device *, char *);
+ struct list_head list; /* zcrypt ops list. */
+ struct module *owner;
+ int variant;
};
struct zcrypt_device {
@@ -108,14 +114,23 @@
struct ap_message reply; /* Per-device reply structure. */
int max_exp_bit_length;
+
+ debug_info_t *dbf_area; /* debugging */
};
+/* transport layer rescanning */
+extern atomic_t zcrypt_rescan_req;
+
struct zcrypt_device *zcrypt_device_alloc(size_t);
void zcrypt_device_free(struct zcrypt_device *);
void zcrypt_device_get(struct zcrypt_device *);
int zcrypt_device_put(struct zcrypt_device *);
int zcrypt_device_register(struct zcrypt_device *);
void zcrypt_device_unregister(struct zcrypt_device *);
+void zcrypt_msgtype_register(struct zcrypt_ops *);
+void zcrypt_msgtype_unregister(struct zcrypt_ops *);
+struct zcrypt_ops *zcrypt_msgtype_request(unsigned char *, int);
+void zcrypt_msgtype_release(struct zcrypt_ops *);
int zcrypt_api_init(void);
void zcrypt_api_exit(void);
diff --git a/drivers/s390/crypto/zcrypt_cex2a.c b/drivers/s390/crypto/zcrypt_cex2a.c
index 744c668..1e849d6 100644
--- a/drivers/s390/crypto/zcrypt_cex2a.c
+++ b/drivers/s390/crypto/zcrypt_cex2a.c
@@ -1,13 +1,14 @@
/*
* zcrypt 2.1.0
*
- * Copyright IBM Corp. 2001, 2006
+ * Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
*
* Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
* Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -35,6 +36,7 @@
#include "zcrypt_api.h"
#include "zcrypt_error.h"
#include "zcrypt_cex2a.h"
+#include "zcrypt_msgtype50.h"
#define CEX2A_MIN_MOD_SIZE 1 /* 8 bits */
#define CEX2A_MAX_MOD_SIZE 256 /* 2048 bits */
@@ -63,14 +65,12 @@
MODULE_DEVICE_TABLE(ap, zcrypt_cex2a_ids);
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("CEX2A Cryptographic Coprocessor device driver, "
- "Copyright IBM Corp. 2001, 2006");
+MODULE_DESCRIPTION("CEX2A Cryptographic Coprocessor device driver, " \
+ "Copyright IBM Corp. 2001, 2012");
MODULE_LICENSE("GPL");
static int zcrypt_cex2a_probe(struct ap_device *ap_dev);
static void zcrypt_cex2a_remove(struct ap_device *ap_dev);
-static void zcrypt_cex2a_receive(struct ap_device *, struct ap_message *,
- struct ap_message *);
static struct ap_driver zcrypt_cex2a_driver = {
.probe = zcrypt_cex2a_probe,
@@ -80,344 +80,6 @@
};
/**
- * Convert a ICAMEX message to a type50 MEX message.
- *
- * @zdev: crypto device pointer
- * @zreq: crypto request pointer
- * @mex: pointer to user input data
- *
- * Returns 0 on success or -EFAULT.
- */
-static int ICAMEX_msg_to_type50MEX_msg(struct zcrypt_device *zdev,
- struct ap_message *ap_msg,
- struct ica_rsa_modexpo *mex)
-{
- unsigned char *mod, *exp, *inp;
- int mod_len;
-
- mod_len = mex->inputdatalength;
-
- if (mod_len <= 128) {
- struct type50_meb1_msg *meb1 = ap_msg->message;
- memset(meb1, 0, sizeof(*meb1));
- ap_msg->length = sizeof(*meb1);
- meb1->header.msg_type_code = TYPE50_TYPE_CODE;
- meb1->header.msg_len = sizeof(*meb1);
- meb1->keyblock_type = TYPE50_MEB1_FMT;
- mod = meb1->modulus + sizeof(meb1->modulus) - mod_len;
- exp = meb1->exponent + sizeof(meb1->exponent) - mod_len;
- inp = meb1->message + sizeof(meb1->message) - mod_len;
- } else if (mod_len <= 256) {
- struct type50_meb2_msg *meb2 = ap_msg->message;
- memset(meb2, 0, sizeof(*meb2));
- ap_msg->length = sizeof(*meb2);
- meb2->header.msg_type_code = TYPE50_TYPE_CODE;
- meb2->header.msg_len = sizeof(*meb2);
- meb2->keyblock_type = TYPE50_MEB2_FMT;
- mod = meb2->modulus + sizeof(meb2->modulus) - mod_len;
- exp = meb2->exponent + sizeof(meb2->exponent) - mod_len;
- inp = meb2->message + sizeof(meb2->message) - mod_len;
- } else {
- /* mod_len > 256 = 4096 bit RSA Key */
- struct type50_meb3_msg *meb3 = ap_msg->message;
- memset(meb3, 0, sizeof(*meb3));
- ap_msg->length = sizeof(*meb3);
- meb3->header.msg_type_code = TYPE50_TYPE_CODE;
- meb3->header.msg_len = sizeof(*meb3);
- meb3->keyblock_type = TYPE50_MEB3_FMT;
- mod = meb3->modulus + sizeof(meb3->modulus) - mod_len;
- exp = meb3->exponent + sizeof(meb3->exponent) - mod_len;
- inp = meb3->message + sizeof(meb3->message) - mod_len;
- }
-
- if (copy_from_user(mod, mex->n_modulus, mod_len) ||
- copy_from_user(exp, mex->b_key, mod_len) ||
- copy_from_user(inp, mex->inputdata, mod_len))
- return -EFAULT;
- return 0;
-}
-
-/**
- * Convert a ICACRT message to a type50 CRT message.
- *
- * @zdev: crypto device pointer
- * @zreq: crypto request pointer
- * @crt: pointer to user input data
- *
- * Returns 0 on success or -EFAULT.
- */
-static int ICACRT_msg_to_type50CRT_msg(struct zcrypt_device *zdev,
- struct ap_message *ap_msg,
- struct ica_rsa_modexpo_crt *crt)
-{
- int mod_len, short_len, long_len, long_offset, limit;
- unsigned char *p, *q, *dp, *dq, *u, *inp;
-
- mod_len = crt->inputdatalength;
- short_len = mod_len / 2;
- long_len = mod_len / 2 + 8;
-
- /*
- * CEX2A cannot handle p, dp, or U > 128 bytes.
- * If we have one of these, we need to do extra checking.
- * For CEX3A the limit is 256 bytes.
- */
- if (zdev->max_mod_size == CEX3A_MAX_MOD_SIZE)
- limit = 256;
- else
- limit = 128;
-
- if (long_len > limit) {
- /*
- * zcrypt_rsa_crt already checked for the leading
- * zeroes of np_prime, bp_key and u_mult_inc.
- */
- long_offset = long_len - limit;
- long_len = limit;
- } else
- long_offset = 0;
-
- /*
- * Instead of doing extra work for p, dp, U > 64 bytes, we'll just use
- * the larger message structure.
- */
- if (long_len <= 64) {
- struct type50_crb1_msg *crb1 = ap_msg->message;
- memset(crb1, 0, sizeof(*crb1));
- ap_msg->length = sizeof(*crb1);
- crb1->header.msg_type_code = TYPE50_TYPE_CODE;
- crb1->header.msg_len = sizeof(*crb1);
- crb1->keyblock_type = TYPE50_CRB1_FMT;
- p = crb1->p + sizeof(crb1->p) - long_len;
- q = crb1->q + sizeof(crb1->q) - short_len;
- dp = crb1->dp + sizeof(crb1->dp) - long_len;
- dq = crb1->dq + sizeof(crb1->dq) - short_len;
- u = crb1->u + sizeof(crb1->u) - long_len;
- inp = crb1->message + sizeof(crb1->message) - mod_len;
- } else if (long_len <= 128) {
- struct type50_crb2_msg *crb2 = ap_msg->message;
- memset(crb2, 0, sizeof(*crb2));
- ap_msg->length = sizeof(*crb2);
- crb2->header.msg_type_code = TYPE50_TYPE_CODE;
- crb2->header.msg_len = sizeof(*crb2);
- crb2->keyblock_type = TYPE50_CRB2_FMT;
- p = crb2->p + sizeof(crb2->p) - long_len;
- q = crb2->q + sizeof(crb2->q) - short_len;
- dp = crb2->dp + sizeof(crb2->dp) - long_len;
- dq = crb2->dq + sizeof(crb2->dq) - short_len;
- u = crb2->u + sizeof(crb2->u) - long_len;
- inp = crb2->message + sizeof(crb2->message) - mod_len;
- } else {
- /* long_len >= 256 */
- struct type50_crb3_msg *crb3 = ap_msg->message;
- memset(crb3, 0, sizeof(*crb3));
- ap_msg->length = sizeof(*crb3);
- crb3->header.msg_type_code = TYPE50_TYPE_CODE;
- crb3->header.msg_len = sizeof(*crb3);
- crb3->keyblock_type = TYPE50_CRB3_FMT;
- p = crb3->p + sizeof(crb3->p) - long_len;
- q = crb3->q + sizeof(crb3->q) - short_len;
- dp = crb3->dp + sizeof(crb3->dp) - long_len;
- dq = crb3->dq + sizeof(crb3->dq) - short_len;
- u = crb3->u + sizeof(crb3->u) - long_len;
- inp = crb3->message + sizeof(crb3->message) - mod_len;
- }
-
- if (copy_from_user(p, crt->np_prime + long_offset, long_len) ||
- copy_from_user(q, crt->nq_prime, short_len) ||
- copy_from_user(dp, crt->bp_key + long_offset, long_len) ||
- copy_from_user(dq, crt->bq_key, short_len) ||
- copy_from_user(u, crt->u_mult_inv + long_offset, long_len) ||
- copy_from_user(inp, crt->inputdata, mod_len))
- return -EFAULT;
-
- return 0;
-}
-
-/**
- * Copy results from a type 80 reply message back to user space.
- *
- * @zdev: crypto device pointer
- * @reply: reply AP message.
- * @data: pointer to user output data
- * @length: size of user output data
- *
- * Returns 0 on success or -EFAULT.
- */
-static int convert_type80(struct zcrypt_device *zdev,
- struct ap_message *reply,
- char __user *outputdata,
- unsigned int outputdatalength)
-{
- struct type80_hdr *t80h = reply->message;
- unsigned char *data;
-
- if (t80h->len < sizeof(*t80h) + outputdatalength) {
- /* The result is too short, the CEX2A card may not do that.. */
- zdev->online = 0;
- return -EAGAIN; /* repeat the request on a different device. */
- }
- if (zdev->user_space_type == ZCRYPT_CEX2A)
- BUG_ON(t80h->len > CEX2A_MAX_RESPONSE_SIZE);
- else
- BUG_ON(t80h->len > CEX3A_MAX_RESPONSE_SIZE);
- data = reply->message + t80h->len - outputdatalength;
- if (copy_to_user(outputdata, data, outputdatalength))
- return -EFAULT;
- return 0;
-}
-
-static int convert_response(struct zcrypt_device *zdev,
- struct ap_message *reply,
- char __user *outputdata,
- unsigned int outputdatalength)
-{
- /* Response type byte is the second byte in the response. */
- switch (((unsigned char *) reply->message)[1]) {
- case TYPE82_RSP_CODE:
- case TYPE88_RSP_CODE:
- return convert_error(zdev, reply);
- case TYPE80_RSP_CODE:
- return convert_type80(zdev, reply,
- outputdata, outputdatalength);
- default: /* Unknown response type, this should NEVER EVER happen */
- zdev->online = 0;
- return -EAGAIN; /* repeat the request on a different device. */
- }
-}
-
-/**
- * This function is called from the AP bus code after a crypto request
- * "msg" has finished with the reply message "reply".
- * It is called from tasklet context.
- * @ap_dev: pointer to the AP device
- * @msg: pointer to the AP message
- * @reply: pointer to the AP reply message
- */
-static void zcrypt_cex2a_receive(struct ap_device *ap_dev,
- struct ap_message *msg,
- struct ap_message *reply)
-{
- static struct error_hdr error_reply = {
- .type = TYPE82_RSP_CODE,
- .reply_code = REP82_ERROR_MACHINE_FAILURE,
- };
- struct type80_hdr *t80h;
- int length;
-
- /* Copy the reply message to the request message buffer. */
- if (IS_ERR(reply)) {
- memcpy(msg->message, &error_reply, sizeof(error_reply));
- goto out;
- }
- t80h = reply->message;
- if (t80h->type == TYPE80_RSP_CODE) {
- if (ap_dev->device_type == AP_DEVICE_TYPE_CEX2A)
- length = min(CEX2A_MAX_RESPONSE_SIZE, (int) t80h->len);
- else
- length = min(CEX3A_MAX_RESPONSE_SIZE, (int) t80h->len);
- memcpy(msg->message, reply->message, length);
- } else
- memcpy(msg->message, reply->message, sizeof error_reply);
-out:
- complete((struct completion *) msg->private);
-}
-
-static atomic_t zcrypt_step = ATOMIC_INIT(0);
-
-/**
- * The request distributor calls this function if it picked the CEX2A
- * device to handle a modexpo request.
- * @zdev: pointer to zcrypt_device structure that identifies the
- * CEX2A device to the request distributor
- * @mex: pointer to the modexpo request buffer
- */
-static long zcrypt_cex2a_modexpo(struct zcrypt_device *zdev,
- struct ica_rsa_modexpo *mex)
-{
- struct ap_message ap_msg;
- struct completion work;
- int rc;
-
- ap_init_message(&ap_msg);
- if (zdev->user_space_type == ZCRYPT_CEX2A)
- ap_msg.message = kmalloc(CEX2A_MAX_MESSAGE_SIZE, GFP_KERNEL);
- else
- ap_msg.message = kmalloc(CEX3A_MAX_MESSAGE_SIZE, GFP_KERNEL);
- if (!ap_msg.message)
- return -ENOMEM;
- ap_msg.receive = zcrypt_cex2a_receive;
- ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
- atomic_inc_return(&zcrypt_step);
- ap_msg.private = &work;
- rc = ICAMEX_msg_to_type50MEX_msg(zdev, &ap_msg, mex);
- if (rc)
- goto out_free;
- init_completion(&work);
- ap_queue_message(zdev->ap_dev, &ap_msg);
- rc = wait_for_completion_interruptible(&work);
- if (rc == 0)
- rc = convert_response(zdev, &ap_msg, mex->outputdata,
- mex->outputdatalength);
- else
- /* Signal pending. */
- ap_cancel_message(zdev->ap_dev, &ap_msg);
-out_free:
- kfree(ap_msg.message);
- return rc;
-}
-
-/**
- * The request distributor calls this function if it picked the CEX2A
- * device to handle a modexpo_crt request.
- * @zdev: pointer to zcrypt_device structure that identifies the
- * CEX2A device to the request distributor
- * @crt: pointer to the modexpoc_crt request buffer
- */
-static long zcrypt_cex2a_modexpo_crt(struct zcrypt_device *zdev,
- struct ica_rsa_modexpo_crt *crt)
-{
- struct ap_message ap_msg;
- struct completion work;
- int rc;
-
- ap_init_message(&ap_msg);
- if (zdev->user_space_type == ZCRYPT_CEX2A)
- ap_msg.message = kmalloc(CEX2A_MAX_MESSAGE_SIZE, GFP_KERNEL);
- else
- ap_msg.message = kmalloc(CEX3A_MAX_MESSAGE_SIZE, GFP_KERNEL);
- if (!ap_msg.message)
- return -ENOMEM;
- ap_msg.receive = zcrypt_cex2a_receive;
- ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
- atomic_inc_return(&zcrypt_step);
- ap_msg.private = &work;
- rc = ICACRT_msg_to_type50CRT_msg(zdev, &ap_msg, crt);
- if (rc)
- goto out_free;
- init_completion(&work);
- ap_queue_message(zdev->ap_dev, &ap_msg);
- rc = wait_for_completion_interruptible(&work);
- if (rc == 0)
- rc = convert_response(zdev, &ap_msg, crt->outputdata,
- crt->outputdatalength);
- else
- /* Signal pending. */
- ap_cancel_message(zdev->ap_dev, &ap_msg);
-out_free:
- kfree(ap_msg.message);
- return rc;
-}
-
-/**
- * The crypto operations for a CEX2A card.
- */
-static struct zcrypt_ops zcrypt_cex2a_ops = {
- .rsa_modexpo = zcrypt_cex2a_modexpo,
- .rsa_modexpo_crt = zcrypt_cex2a_modexpo_crt,
-};
-
-/**
* Probe function for CEX2A cards. It always accepts the AP device
* since the bus_match already checked the hardware type.
* @ap_dev: pointer to the AP device.
@@ -449,7 +111,8 @@
zdev->min_mod_size = CEX2A_MIN_MOD_SIZE;
zdev->max_mod_size = CEX2A_MAX_MOD_SIZE;
zdev->max_exp_bit_length = CEX2A_MAX_MOD_SIZE;
- if (ap_4096_commands_available(ap_dev->qid)) {
+ if (ap_test_bit(&ap_dev->functions, AP_FUNC_MEX4K) &&
+ ap_test_bit(&ap_dev->functions, AP_FUNC_CRT4K)) {
zdev->max_mod_size = CEX3A_MAX_MOD_SIZE;
zdev->max_exp_bit_length = CEX3A_MAX_MOD_SIZE;
}
@@ -457,16 +120,18 @@
zdev->speed_rating = CEX3A_SPEED_RATING;
break;
}
- if (zdev != NULL) {
- zdev->ap_dev = ap_dev;
- zdev->ops = &zcrypt_cex2a_ops;
- zdev->online = 1;
- ap_dev->reply = &zdev->reply;
- ap_dev->private = zdev;
- rc = zcrypt_device_register(zdev);
- }
+ if (!zdev)
+ return -ENODEV;
+ zdev->ops = zcrypt_msgtype_request(MSGTYPE50_NAME,
+ MSGTYPE50_VARIANT_DEFAULT);
+ zdev->ap_dev = ap_dev;
+ zdev->online = 1;
+ ap_dev->reply = &zdev->reply;
+ ap_dev->private = zdev;
+ rc = zcrypt_device_register(zdev);
if (rc) {
ap_dev->private = NULL;
+ zcrypt_msgtype_release(zdev->ops);
zcrypt_device_free(zdev);
}
return rc;
@@ -479,8 +144,10 @@
static void zcrypt_cex2a_remove(struct ap_device *ap_dev)
{
struct zcrypt_device *zdev = ap_dev->private;
+ struct zcrypt_ops *zops = zdev->ops;
zcrypt_device_unregister(zdev);
+ zcrypt_msgtype_release(zops);
}
int __init zcrypt_cex2a_init(void)
diff --git a/drivers/s390/crypto/zcrypt_cex4.c b/drivers/s390/crypto/zcrypt_cex4.c
new file mode 100644
index 0000000..ce12263
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_cex4.c
@@ -0,0 +1,149 @@
+/*
+ * Copyright IBM Corp. 2012
+ * Author(s): Holger Dengler <hd@linux.vnet.ibm.com>
+ */
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/err.h>
+#include <linux/atomic.h>
+#include <linux/uaccess.h>
+
+#include "ap_bus.h"
+#include "zcrypt_api.h"
+#include "zcrypt_msgtype6.h"
+#include "zcrypt_msgtype50.h"
+#include "zcrypt_error.h"
+#include "zcrypt_cex4.h"
+
+#define CEX4A_MIN_MOD_SIZE 1 /* 8 bits */
+#define CEX4A_MAX_MOD_SIZE_2K 256 /* 2048 bits */
+#define CEX4A_MAX_MOD_SIZE_4K 512 /* 4096 bits */
+
+#define CEX4C_MIN_MOD_SIZE 16 /* 256 bits */
+#define CEX4C_MAX_MOD_SIZE 512 /* 4096 bits */
+
+#define CEX4A_SPEED_RATING 900 /* TODO new card, new speed rating */
+#define CEX4C_SPEED_RATING 6500 /* TODO new card, new speed rating */
+
+#define CEX4A_MAX_MESSAGE_SIZE MSGTYPE50_CRB3_MAX_MSG_SIZE
+#define CEX4C_MAX_MESSAGE_SIZE MSGTYPE06_MAX_MSG_SIZE
+
+#define CEX4_CLEANUP_TIME (15*HZ)
+
+static struct ap_device_id zcrypt_cex4_ids[] = {
+ { AP_DEVICE(AP_DEVICE_TYPE_CEX4) },
+ { /* end of list */ },
+};
+
+MODULE_DEVICE_TABLE(ap, zcrypt_cex4_ids);
+MODULE_AUTHOR("IBM Corporation");
+MODULE_DESCRIPTION("CEX4 Cryptographic Card device driver, " \
+ "Copyright IBM Corp. 2012");
+MODULE_LICENSE("GPL");
+
+static int zcrypt_cex4_probe(struct ap_device *ap_dev);
+static void zcrypt_cex4_remove(struct ap_device *ap_dev);
+
+static struct ap_driver zcrypt_cex4_driver = {
+ .probe = zcrypt_cex4_probe,
+ .remove = zcrypt_cex4_remove,
+ .ids = zcrypt_cex4_ids,
+ .request_timeout = CEX4_CLEANUP_TIME,
+};
+
+/**
+ * Probe function for CEX4 cards. It always accepts the AP device
+ * since the bus_match already checked the hardware type.
+ * @ap_dev: pointer to the AP device.
+ */
+static int zcrypt_cex4_probe(struct ap_device *ap_dev)
+{
+ struct zcrypt_device *zdev = NULL;
+ int rc = 0;
+
+ switch (ap_dev->device_type) {
+ case AP_DEVICE_TYPE_CEX4:
+ if (ap_test_bit(&ap_dev->functions, AP_FUNC_ACCEL)) {
+ zdev = zcrypt_device_alloc(CEX4A_MAX_MESSAGE_SIZE);
+ if (!zdev)
+ return -ENOMEM;
+ zdev->type_string = "CEX4A";
+ zdev->user_space_type = ZCRYPT_CEX3A;
+ zdev->min_mod_size = CEX4A_MIN_MOD_SIZE;
+ if (ap_test_bit(&ap_dev->functions, AP_FUNC_MEX4K) &&
+ ap_test_bit(&ap_dev->functions, AP_FUNC_CRT4K)) {
+ zdev->max_mod_size =
+ CEX4A_MAX_MOD_SIZE_4K;
+ zdev->max_exp_bit_length =
+ CEX4A_MAX_MOD_SIZE_4K;
+ } else {
+ zdev->max_mod_size =
+ CEX4A_MAX_MOD_SIZE_2K;
+ zdev->max_exp_bit_length =
+ CEX4A_MAX_MOD_SIZE_2K;
+ }
+ zdev->short_crt = 1;
+ zdev->speed_rating = CEX4A_SPEED_RATING;
+ zdev->ops = zcrypt_msgtype_request(MSGTYPE50_NAME,
+ MSGTYPE50_VARIANT_DEFAULT);
+ } else if (ap_test_bit(&ap_dev->functions, AP_FUNC_COPRO)) {
+ zdev = zcrypt_device_alloc(CEX4C_MAX_MESSAGE_SIZE);
+ if (!zdev)
+ return -ENOMEM;
+ zdev->type_string = "CEX4C";
+ zdev->user_space_type = ZCRYPT_CEX3C;
+ zdev->min_mod_size = CEX4C_MIN_MOD_SIZE;
+ zdev->max_mod_size = CEX4C_MAX_MOD_SIZE;
+ zdev->max_exp_bit_length = CEX4C_MAX_MOD_SIZE;
+ zdev->short_crt = 0;
+ zdev->speed_rating = CEX4C_SPEED_RATING;
+ zdev->ops = zcrypt_msgtype_request(MSGTYPE06_NAME,
+ MSGTYPE06_VARIANT_DEFAULT);
+ }
+ break;
+ }
+ if (!zdev)
+ return -ENODEV;
+ zdev->ap_dev = ap_dev;
+ zdev->online = 1;
+ ap_dev->reply = &zdev->reply;
+ ap_dev->private = zdev;
+ rc = zcrypt_device_register(zdev);
+ if (rc) {
+ zcrypt_msgtype_release(zdev->ops);
+ ap_dev->private = NULL;
+ zcrypt_device_free(zdev);
+ }
+ return rc;
+}
+
+/**
+ * This is called to remove the extended CEX4 driver information
+ * if an AP device is removed.
+ */
+static void zcrypt_cex4_remove(struct ap_device *ap_dev)
+{
+ struct zcrypt_device *zdev = ap_dev->private;
+ struct zcrypt_ops *zops;
+
+ if (zdev) {
+ zops = zdev->ops;
+ zcrypt_device_unregister(zdev);
+ zcrypt_msgtype_release(zops);
+ }
+}
+
+int __init zcrypt_cex4_init(void)
+{
+ return ap_driver_register(&zcrypt_cex4_driver, THIS_MODULE, "cex4");
+}
+
+void __exit zcrypt_cex4_exit(void)
+{
+ ap_driver_unregister(&zcrypt_cex4_driver);
+}
+
+module_init(zcrypt_cex4_init);
+module_exit(zcrypt_cex4_exit);
diff --git a/drivers/s390/crypto/zcrypt_cex4.h b/drivers/s390/crypto/zcrypt_cex4.h
new file mode 100644
index 0000000..7195713
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_cex4.h
@@ -0,0 +1,12 @@
+/*
+ * Copyright IBM Corp. 2012
+ * Author(s): Holger Dengler <hd@linux.vnet.ibm.com>
+ */
+
+#ifndef _ZCRYPT_CEX4_H_
+#define _ZCRYPT_CEX4_H_
+
+int zcrypt_cex4_init(void);
+void zcrypt_cex4_exit(void);
+
+#endif /* _ZCRYPT_CEX4_H_ */
diff --git a/drivers/s390/crypto/zcrypt_debug.h b/drivers/s390/crypto/zcrypt_debug.h
new file mode 100644
index 0000000..841ea72
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_debug.h
@@ -0,0 +1,59 @@
+/*
+ * Copyright IBM Corp. 2012
+ * Author(s): Holger Dengler (hd@linux.vnet.ibm.com)
+ */
+#ifndef ZCRYPT_DEBUG_H
+#define ZCRYPT_DEBUG_H
+
+#include <asm/debug.h>
+#include "zcrypt_api.h"
+
+/* that gives us 15 characters in the text event views */
+#define ZCRYPT_DBF_LEN 16
+
+/* sort out low debug levels early to avoid wasted sprints */
+static inline int zcrypt_dbf_passes(debug_info_t *dbf_grp, int level)
+{
+ return (level <= dbf_grp->level);
+}
+
+#define DBF_ERR 3 /* error conditions */
+#define DBF_WARN 4 /* warning conditions */
+#define DBF_INFO 6 /* informational */
+
+#define RC2WARN(rc) ((rc) ? DBF_WARN : DBF_INFO)
+
+#define ZCRYPT_DBF_COMMON(level, text...) \
+ do { \
+ if (zcrypt_dbf_passes(zcrypt_dbf_common, level)) { \
+ char debug_buffer[ZCRYPT_DBF_LEN]; \
+ snprintf(debug_buffer, ZCRYPT_DBF_LEN, text); \
+ debug_text_event(zcrypt_dbf_common, level, \
+ debug_buffer); \
+ } \
+ } while (0)
+
+#define ZCRYPT_DBF_DEVICES(level, text...) \
+ do { \
+ if (zcrypt_dbf_passes(zcrypt_dbf_devices, level)) { \
+ char debug_buffer[ZCRYPT_DBF_LEN]; \
+ snprintf(debug_buffer, ZCRYPT_DBF_LEN, text); \
+ debug_text_event(zcrypt_dbf_devices, level, \
+ debug_buffer); \
+ } \
+ } while (0)
+
+#define ZCRYPT_DBF_DEV(level, device, text...) \
+ do { \
+ if (zcrypt_dbf_passes(device->dbf_area, level)) { \
+ char debug_buffer[ZCRYPT_DBF_LEN]; \
+ snprintf(debug_buffer, ZCRYPT_DBF_LEN, text); \
+ debug_text_event(device->dbf_area, level, \
+ debug_buffer); \
+ } \
+ } while (0)
+
+int zcrypt_debug_init(void);
+void zcrypt_debug_exit(void);
+
+#endif /* ZCRYPT_DEBUG_H */
diff --git a/drivers/s390/crypto/zcrypt_error.h b/drivers/s390/crypto/zcrypt_error.h
index 0965e26..0079b66 100644
--- a/drivers/s390/crypto/zcrypt_error.h
+++ b/drivers/s390/crypto/zcrypt_error.h
@@ -26,6 +26,8 @@
#ifndef _ZCRYPT_ERROR_H_
#define _ZCRYPT_ERROR_H_
+#include <linux/atomic.h>
+#include "zcrypt_debug.h"
#include "zcrypt_api.h"
/**
@@ -108,16 +110,27 @@
* and then repeat the request.
*/
WARN_ON(1);
+ atomic_set(&zcrypt_rescan_req, 1);
zdev->online = 0;
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid,
+ zdev->online, ehdr->reply_code);
return -EAGAIN;
case REP82_ERROR_TRANSPORT_FAIL:
case REP82_ERROR_MACHINE_FAILURE:
// REP88_ERROR_MODULE_FAILURE // '10' CEX2A
/* If a card fails disable it and repeat the request. */
+ atomic_set(&zcrypt_rescan_req, 1);
zdev->online = 0;
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid,
+ zdev->online, ehdr->reply_code);
return -EAGAIN;
default:
zdev->online = 0;
+ ZCRYPT_DBF_DEV(DBF_ERR, zdev, "dev%04xo%drc%d",
+ zdev->ap_dev->qid,
+ zdev->online, ehdr->reply_code);
return -EAGAIN; /* repeat the request on a different device. */
}
}
diff --git a/drivers/s390/crypto/zcrypt_msgtype50.c b/drivers/s390/crypto/zcrypt_msgtype50.c
new file mode 100644
index 0000000..035b6dc
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_msgtype50.c
@@ -0,0 +1,531 @@
+/*
+ * zcrypt 2.1.0
+ *
+ * Copyright IBM Corp. 2001, 2012
+ * Author(s): Robert Burroughs
+ * Eric Rossman (edrossma@us.ibm.com)
+ *
+ * Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
+ * Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
+ * Ralph Wuerthner <rwuerthn@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/err.h>
+#include <linux/atomic.h>
+#include <linux/uaccess.h>
+
+#include "ap_bus.h"
+#include "zcrypt_api.h"
+#include "zcrypt_error.h"
+#include "zcrypt_msgtype50.h"
+
+#define CEX3A_MAX_MOD_SIZE 512 /* 4096 bits */
+
+#define CEX2A_MAX_RESPONSE_SIZE 0x110 /* max outputdatalength + type80_hdr */
+
+#define CEX3A_MAX_RESPONSE_SIZE 0x210 /* 512 bit modulus
+ * (max outputdatalength) +
+ * type80_hdr*/
+
+MODULE_AUTHOR("IBM Corporation");
+MODULE_DESCRIPTION("Cryptographic Accelerator (message type 50), " \
+ "Copyright IBM Corp. 2001, 2012");
+MODULE_LICENSE("GPL");
+
+static void zcrypt_cex2a_receive(struct ap_device *, struct ap_message *,
+ struct ap_message *);
+
+/**
+ * The type 50 message family is associated with a CEX2A card.
+ *
+ * The four members of the family are described below.
+ *
+ * Note that all unsigned char arrays are right-justified and left-padded
+ * with zeroes.
+ *
+ * Note that all reserved fields must be zeroes.
+ */
+struct type50_hdr {
+ unsigned char reserved1;
+ unsigned char msg_type_code; /* 0x50 */
+ unsigned short msg_len;
+ unsigned char reserved2;
+ unsigned char ignored;
+ unsigned short reserved3;
+} __packed;
+
+#define TYPE50_TYPE_CODE 0x50
+
+#define TYPE50_MEB1_FMT 0x0001
+#define TYPE50_MEB2_FMT 0x0002
+#define TYPE50_MEB3_FMT 0x0003
+#define TYPE50_CRB1_FMT 0x0011
+#define TYPE50_CRB2_FMT 0x0012
+#define TYPE50_CRB3_FMT 0x0013
+
+/* Mod-Exp, with a small modulus */
+struct type50_meb1_msg {
+ struct type50_hdr header;
+ unsigned short keyblock_type; /* 0x0001 */
+ unsigned char reserved[6];
+ unsigned char exponent[128];
+ unsigned char modulus[128];
+ unsigned char message[128];
+} __packed;
+
+/* Mod-Exp, with a large modulus */
+struct type50_meb2_msg {
+ struct type50_hdr header;
+ unsigned short keyblock_type; /* 0x0002 */
+ unsigned char reserved[6];
+ unsigned char exponent[256];
+ unsigned char modulus[256];
+ unsigned char message[256];
+} __packed;
+
+/* Mod-Exp, with a larger modulus */
+struct type50_meb3_msg {
+ struct type50_hdr header;
+ unsigned short keyblock_type; /* 0x0003 */
+ unsigned char reserved[6];
+ unsigned char exponent[512];
+ unsigned char modulus[512];
+ unsigned char message[512];
+} __packed;
+
+/* CRT, with a small modulus */
+struct type50_crb1_msg {
+ struct type50_hdr header;
+ unsigned short keyblock_type; /* 0x0011 */
+ unsigned char reserved[6];
+ unsigned char p[64];
+ unsigned char q[64];
+ unsigned char dp[64];
+ unsigned char dq[64];
+ unsigned char u[64];
+ unsigned char message[128];
+} __packed;
+
+/* CRT, with a large modulus */
+struct type50_crb2_msg {
+ struct type50_hdr header;
+ unsigned short keyblock_type; /* 0x0012 */
+ unsigned char reserved[6];
+ unsigned char p[128];
+ unsigned char q[128];
+ unsigned char dp[128];
+ unsigned char dq[128];
+ unsigned char u[128];
+ unsigned char message[256];
+} __packed;
+
+/* CRT, with a larger modulus */
+struct type50_crb3_msg {
+ struct type50_hdr header;
+ unsigned short keyblock_type; /* 0x0013 */
+ unsigned char reserved[6];
+ unsigned char p[256];
+ unsigned char q[256];
+ unsigned char dp[256];
+ unsigned char dq[256];
+ unsigned char u[256];
+ unsigned char message[512];
+} __packed;
+
+/**
+ * The type 80 response family is associated with a CEX2A card.
+ *
+ * Note that all unsigned char arrays are right-justified and left-padded
+ * with zeroes.
+ *
+ * Note that all reserved fields must be zeroes.
+ */
+
+#define TYPE80_RSP_CODE 0x80
+
+struct type80_hdr {
+ unsigned char reserved1;
+ unsigned char type; /* 0x80 */
+ unsigned short len;
+ unsigned char code; /* 0x00 */
+ unsigned char reserved2[3];
+ unsigned char reserved3[8];
+} __packed;
+
+/**
+ * Convert a ICAMEX message to a type50 MEX message.
+ *
+ * @zdev: crypto device pointer
+ * @zreq: crypto request pointer
+ * @mex: pointer to user input data
+ *
+ * Returns 0 on success or -EFAULT.
+ */
+static int ICAMEX_msg_to_type50MEX_msg(struct zcrypt_device *zdev,
+ struct ap_message *ap_msg,
+ struct ica_rsa_modexpo *mex)
+{
+ unsigned char *mod, *exp, *inp;
+ int mod_len;
+
+ mod_len = mex->inputdatalength;
+
+ if (mod_len <= 128) {
+ struct type50_meb1_msg *meb1 = ap_msg->message;
+ memset(meb1, 0, sizeof(*meb1));
+ ap_msg->length = sizeof(*meb1);
+ meb1->header.msg_type_code = TYPE50_TYPE_CODE;
+ meb1->header.msg_len = sizeof(*meb1);
+ meb1->keyblock_type = TYPE50_MEB1_FMT;
+ mod = meb1->modulus + sizeof(meb1->modulus) - mod_len;
+ exp = meb1->exponent + sizeof(meb1->exponent) - mod_len;
+ inp = meb1->message + sizeof(meb1->message) - mod_len;
+ } else if (mod_len <= 256) {
+ struct type50_meb2_msg *meb2 = ap_msg->message;
+ memset(meb2, 0, sizeof(*meb2));
+ ap_msg->length = sizeof(*meb2);
+ meb2->header.msg_type_code = TYPE50_TYPE_CODE;
+ meb2->header.msg_len = sizeof(*meb2);
+ meb2->keyblock_type = TYPE50_MEB2_FMT;
+ mod = meb2->modulus + sizeof(meb2->modulus) - mod_len;
+ exp = meb2->exponent + sizeof(meb2->exponent) - mod_len;
+ inp = meb2->message + sizeof(meb2->message) - mod_len;
+ } else {
+ /* mod_len > 256 = 4096 bit RSA Key */
+ struct type50_meb3_msg *meb3 = ap_msg->message;
+ memset(meb3, 0, sizeof(*meb3));
+ ap_msg->length = sizeof(*meb3);
+ meb3->header.msg_type_code = TYPE50_TYPE_CODE;
+ meb3->header.msg_len = sizeof(*meb3);
+ meb3->keyblock_type = TYPE50_MEB3_FMT;
+ mod = meb3->modulus + sizeof(meb3->modulus) - mod_len;
+ exp = meb3->exponent + sizeof(meb3->exponent) - mod_len;
+ inp = meb3->message + sizeof(meb3->message) - mod_len;
+ }
+
+ if (copy_from_user(mod, mex->n_modulus, mod_len) ||
+ copy_from_user(exp, mex->b_key, mod_len) ||
+ copy_from_user(inp, mex->inputdata, mod_len))
+ return -EFAULT;
+ return 0;
+}
+
+/**
+ * Convert a ICACRT message to a type50 CRT message.
+ *
+ * @zdev: crypto device pointer
+ * @zreq: crypto request pointer
+ * @crt: pointer to user input data
+ *
+ * Returns 0 on success or -EFAULT.
+ */
+static int ICACRT_msg_to_type50CRT_msg(struct zcrypt_device *zdev,
+ struct ap_message *ap_msg,
+ struct ica_rsa_modexpo_crt *crt)
+{
+ int mod_len, short_len, long_len, long_offset, limit;
+ unsigned char *p, *q, *dp, *dq, *u, *inp;
+
+ mod_len = crt->inputdatalength;
+ short_len = mod_len / 2;
+ long_len = mod_len / 2 + 8;
+
+ /*
+ * CEX2A cannot handle p, dp, or U > 128 bytes.
+ * If we have one of these, we need to do extra checking.
+ * For CEX3A the limit is 256 bytes.
+ */
+ if (zdev->max_mod_size == CEX3A_MAX_MOD_SIZE)
+ limit = 256;
+ else
+ limit = 128;
+
+ if (long_len > limit) {
+ /*
+ * zcrypt_rsa_crt already checked for the leading
+ * zeroes of np_prime, bp_key and u_mult_inc.
+ */
+ long_offset = long_len - limit;
+ long_len = limit;
+ } else
+ long_offset = 0;
+
+ /*
+ * Instead of doing extra work for p, dp, U > 64 bytes, we'll just use
+ * the larger message structure.
+ */
+ if (long_len <= 64) {
+ struct type50_crb1_msg *crb1 = ap_msg->message;
+ memset(crb1, 0, sizeof(*crb1));
+ ap_msg->length = sizeof(*crb1);
+ crb1->header.msg_type_code = TYPE50_TYPE_CODE;
+ crb1->header.msg_len = sizeof(*crb1);
+ crb1->keyblock_type = TYPE50_CRB1_FMT;
+ p = crb1->p + sizeof(crb1->p) - long_len;
+ q = crb1->q + sizeof(crb1->q) - short_len;
+ dp = crb1->dp + sizeof(crb1->dp) - long_len;
+ dq = crb1->dq + sizeof(crb1->dq) - short_len;
+ u = crb1->u + sizeof(crb1->u) - long_len;
+ inp = crb1->message + sizeof(crb1->message) - mod_len;
+ } else if (long_len <= 128) {
+ struct type50_crb2_msg *crb2 = ap_msg->message;
+ memset(crb2, 0, sizeof(*crb2));
+ ap_msg->length = sizeof(*crb2);
+ crb2->header.msg_type_code = TYPE50_TYPE_CODE;
+ crb2->header.msg_len = sizeof(*crb2);
+ crb2->keyblock_type = TYPE50_CRB2_FMT;
+ p = crb2->p + sizeof(crb2->p) - long_len;
+ q = crb2->q + sizeof(crb2->q) - short_len;
+ dp = crb2->dp + sizeof(crb2->dp) - long_len;
+ dq = crb2->dq + sizeof(crb2->dq) - short_len;
+ u = crb2->u + sizeof(crb2->u) - long_len;
+ inp = crb2->message + sizeof(crb2->message) - mod_len;
+ } else {
+ /* long_len >= 256 */
+ struct type50_crb3_msg *crb3 = ap_msg->message;
+ memset(crb3, 0, sizeof(*crb3));
+ ap_msg->length = sizeof(*crb3);
+ crb3->header.msg_type_code = TYPE50_TYPE_CODE;
+ crb3->header.msg_len = sizeof(*crb3);
+ crb3->keyblock_type = TYPE50_CRB3_FMT;
+ p = crb3->p + sizeof(crb3->p) - long_len;
+ q = crb3->q + sizeof(crb3->q) - short_len;
+ dp = crb3->dp + sizeof(crb3->dp) - long_len;
+ dq = crb3->dq + sizeof(crb3->dq) - short_len;
+ u = crb3->u + sizeof(crb3->u) - long_len;
+ inp = crb3->message + sizeof(crb3->message) - mod_len;
+ }
+
+ if (copy_from_user(p, crt->np_prime + long_offset, long_len) ||
+ copy_from_user(q, crt->nq_prime, short_len) ||
+ copy_from_user(dp, crt->bp_key + long_offset, long_len) ||
+ copy_from_user(dq, crt->bq_key, short_len) ||
+ copy_from_user(u, crt->u_mult_inv + long_offset, long_len) ||
+ copy_from_user(inp, crt->inputdata, mod_len))
+ return -EFAULT;
+
+ return 0;
+}
+
+/**
+ * Copy results from a type 80 reply message back to user space.
+ *
+ * @zdev: crypto device pointer
+ * @reply: reply AP message.
+ * @data: pointer to user output data
+ * @length: size of user output data
+ *
+ * Returns 0 on success or -EFAULT.
+ */
+static int convert_type80(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ char __user *outputdata,
+ unsigned int outputdatalength)
+{
+ struct type80_hdr *t80h = reply->message;
+ unsigned char *data;
+
+ if (t80h->len < sizeof(*t80h) + outputdatalength) {
+ /* The result is too short, the CEX2A card may not do that.. */
+ zdev->online = 0;
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+ if (zdev->user_space_type == ZCRYPT_CEX2A)
+ BUG_ON(t80h->len > CEX2A_MAX_RESPONSE_SIZE);
+ else
+ BUG_ON(t80h->len > CEX3A_MAX_RESPONSE_SIZE);
+ data = reply->message + t80h->len - outputdatalength;
+ if (copy_to_user(outputdata, data, outputdatalength))
+ return -EFAULT;
+ return 0;
+}
+
+static int convert_response(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ char __user *outputdata,
+ unsigned int outputdatalength)
+{
+ /* Response type byte is the second byte in the response. */
+ switch (((unsigned char *) reply->message)[1]) {
+ case TYPE82_RSP_CODE:
+ case TYPE88_RSP_CODE:
+ return convert_error(zdev, reply);
+ case TYPE80_RSP_CODE:
+ return convert_type80(zdev, reply,
+ outputdata, outputdatalength);
+ default: /* Unknown response type, this should NEVER EVER happen */
+ zdev->online = 0;
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+}
+
+/**
+ * This function is called from the AP bus code after a crypto request
+ * "msg" has finished with the reply message "reply".
+ * It is called from tasklet context.
+ * @ap_dev: pointer to the AP device
+ * @msg: pointer to the AP message
+ * @reply: pointer to the AP reply message
+ */
+static void zcrypt_cex2a_receive(struct ap_device *ap_dev,
+ struct ap_message *msg,
+ struct ap_message *reply)
+{
+ static struct error_hdr error_reply = {
+ .type = TYPE82_RSP_CODE,
+ .reply_code = REP82_ERROR_MACHINE_FAILURE,
+ };
+ struct type80_hdr *t80h;
+ int length;
+
+ /* Copy the reply message to the request message buffer. */
+ if (IS_ERR(reply)) {
+ memcpy(msg->message, &error_reply, sizeof(error_reply));
+ goto out;
+ }
+ t80h = reply->message;
+ if (t80h->type == TYPE80_RSP_CODE) {
+ if (ap_dev->device_type == AP_DEVICE_TYPE_CEX2A)
+ length = min_t(int,
+ CEX2A_MAX_RESPONSE_SIZE, t80h->len);
+ else
+ length = min_t(int,
+ CEX3A_MAX_RESPONSE_SIZE, t80h->len);
+ memcpy(msg->message, reply->message, length);
+ } else
+ memcpy(msg->message, reply->message, sizeof(error_reply));
+out:
+ complete((struct completion *) msg->private);
+}
+
+static atomic_t zcrypt_step = ATOMIC_INIT(0);
+
+/**
+ * The request distributor calls this function if it picked the CEX2A
+ * device to handle a modexpo request.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * CEX2A device to the request distributor
+ * @mex: pointer to the modexpo request buffer
+ */
+static long zcrypt_cex2a_modexpo(struct zcrypt_device *zdev,
+ struct ica_rsa_modexpo *mex)
+{
+ struct ap_message ap_msg;
+ struct completion work;
+ int rc;
+
+ ap_init_message(&ap_msg);
+ if (zdev->user_space_type == ZCRYPT_CEX2A)
+ ap_msg.message = kmalloc(MSGTYPE50_CRB2_MAX_MSG_SIZE,
+ GFP_KERNEL);
+ else
+ ap_msg.message = kmalloc(MSGTYPE50_CRB3_MAX_MSG_SIZE,
+ GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_cex2a_receive;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &work;
+ rc = ICAMEX_msg_to_type50MEX_msg(zdev, &ap_msg, mex);
+ if (rc)
+ goto out_free;
+ init_completion(&work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&work);
+ if (rc == 0)
+ rc = convert_response(zdev, &ap_msg, mex->outputdata,
+ mex->outputdatalength);
+ else
+ /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+out_free:
+ kfree(ap_msg.message);
+ return rc;
+}
+
+/**
+ * The request distributor calls this function if it picked the CEX2A
+ * device to handle a modexpo_crt request.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * CEX2A device to the request distributor
+ * @crt: pointer to the modexpoc_crt request buffer
+ */
+static long zcrypt_cex2a_modexpo_crt(struct zcrypt_device *zdev,
+ struct ica_rsa_modexpo_crt *crt)
+{
+ struct ap_message ap_msg;
+ struct completion work;
+ int rc;
+
+ ap_init_message(&ap_msg);
+ if (zdev->user_space_type == ZCRYPT_CEX2A)
+ ap_msg.message = kmalloc(MSGTYPE50_CRB2_MAX_MSG_SIZE,
+ GFP_KERNEL);
+ else
+ ap_msg.message = kmalloc(MSGTYPE50_CRB3_MAX_MSG_SIZE,
+ GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_cex2a_receive;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &work;
+ rc = ICACRT_msg_to_type50CRT_msg(zdev, &ap_msg, crt);
+ if (rc)
+ goto out_free;
+ init_completion(&work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&work);
+ if (rc == 0)
+ rc = convert_response(zdev, &ap_msg, crt->outputdata,
+ crt->outputdatalength);
+ else
+ /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+out_free:
+ kfree(ap_msg.message);
+ return rc;
+}
+
+/**
+ * The crypto operations for message type 50.
+ */
+static struct zcrypt_ops zcrypt_msgtype50_ops = {
+ .rsa_modexpo = zcrypt_cex2a_modexpo,
+ .rsa_modexpo_crt = zcrypt_cex2a_modexpo_crt,
+ .owner = THIS_MODULE,
+ .variant = MSGTYPE50_VARIANT_DEFAULT,
+};
+
+int __init zcrypt_msgtype50_init(void)
+{
+ zcrypt_msgtype_register(&zcrypt_msgtype50_ops);
+ return 0;
+}
+
+void __exit zcrypt_msgtype50_exit(void)
+{
+ zcrypt_msgtype_unregister(&zcrypt_msgtype50_ops);
+}
+
+module_init(zcrypt_msgtype50_init);
+module_exit(zcrypt_msgtype50_exit);
diff --git a/drivers/s390/crypto/zcrypt_msgtype50.h b/drivers/s390/crypto/zcrypt_msgtype50.h
new file mode 100644
index 0000000..e56dc72
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_msgtype50.h
@@ -0,0 +1,39 @@
+/*
+ * zcrypt 2.1.0
+ *
+ * Copyright IBM Corp. 2001, 2012
+ * Author(s): Robert Burroughs
+ * Eric Rossman (edrossma@us.ibm.com)
+ *
+ * Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
+ * Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef _ZCRYPT_MSGTYPE50_H_
+#define _ZCRYPT_MSGTYPE50_H_
+
+#define MSGTYPE50_NAME "zcrypt_msgtype50"
+#define MSGTYPE50_VARIANT_DEFAULT 0
+
+#define MSGTYPE50_CRB2_MAX_MSG_SIZE 0x390 /*sizeof(struct type50_crb2_msg)*/
+#define MSGTYPE50_CRB3_MAX_MSG_SIZE 0x710 /*sizeof(struct type50_crb3_msg)*/
+
+int zcrypt_msgtype50_init(void);
+void zcrypt_msgtype50_exit(void);
+
+#endif /* _ZCRYPT_MSGTYPE50_H_ */
diff --git a/drivers/s390/crypto/zcrypt_msgtype6.c b/drivers/s390/crypto/zcrypt_msgtype6.c
new file mode 100644
index 0000000..7d97fa5
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_msgtype6.c
@@ -0,0 +1,856 @@
+/*
+ * zcrypt 2.1.0
+ *
+ * Copyright IBM Corp. 2001, 2012
+ * Author(s): Robert Burroughs
+ * Eric Rossman (edrossma@us.ibm.com)
+ *
+ * Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
+ * Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
+ * Ralph Wuerthner <rwuerthn@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/err.h>
+#include <linux/delay.h>
+#include <linux/slab.h>
+#include <linux/atomic.h>
+#include <linux/uaccess.h>
+
+#include "ap_bus.h"
+#include "zcrypt_api.h"
+#include "zcrypt_error.h"
+#include "zcrypt_msgtype6.h"
+#include "zcrypt_cca_key.h"
+
+#define PCIXCC_MIN_MOD_SIZE_OLD 64 /* 512 bits */
+#define PCIXCC_MAX_ICA_RESPONSE_SIZE 0x77c /* max size type86 v2 reply */
+
+#define CEIL4(x) ((((x)+3)/4)*4)
+
+struct response_type {
+ struct completion work;
+ int type;
+};
+#define PCIXCC_RESPONSE_TYPE_ICA 0
+#define PCIXCC_RESPONSE_TYPE_XCRB 1
+
+MODULE_AUTHOR("IBM Corporation");
+MODULE_DESCRIPTION("Cryptographic Coprocessor (message type 6), " \
+ "Copyright IBM Corp. 2001, 2012");
+MODULE_LICENSE("GPL");
+
+static void zcrypt_msgtype6_receive(struct ap_device *, struct ap_message *,
+ struct ap_message *);
+
+/**
+ * CPRB
+ * Note that all shorts, ints and longs are little-endian.
+ * All pointer fields are 32-bits long, and mean nothing
+ *
+ * A request CPRB is followed by a request_parameter_block.
+ *
+ * The request (or reply) parameter block is organized thus:
+ * function code
+ * VUD block
+ * key block
+ */
+struct CPRB {
+ unsigned short cprb_len; /* CPRB length */
+ unsigned char cprb_ver_id; /* CPRB version id. */
+ unsigned char pad_000; /* Alignment pad byte. */
+ unsigned char srpi_rtcode[4]; /* SRPI return code LELONG */
+ unsigned char srpi_verb; /* SRPI verb type */
+ unsigned char flags; /* flags */
+ unsigned char func_id[2]; /* function id */
+ unsigned char checkpoint_flag; /* */
+ unsigned char resv2; /* reserved */
+ unsigned short req_parml; /* request parameter buffer */
+ /* length 16-bit little endian */
+ unsigned char req_parmp[4]; /* request parameter buffer *
+ * pointer (means nothing: the *
+ * parameter buffer follows *
+ * the CPRB). */
+ unsigned char req_datal[4]; /* request data buffer */
+ /* length ULELONG */
+ unsigned char req_datap[4]; /* request data buffer */
+ /* pointer */
+ unsigned short rpl_parml; /* reply parameter buffer */
+ /* length 16-bit little endian */
+ unsigned char pad_001[2]; /* Alignment pad bytes. ULESHORT */
+ unsigned char rpl_parmp[4]; /* reply parameter buffer *
+ * pointer (means nothing: the *
+ * parameter buffer follows *
+ * the CPRB). */
+ unsigned char rpl_datal[4]; /* reply data buffer len ULELONG */
+ unsigned char rpl_datap[4]; /* reply data buffer */
+ /* pointer */
+ unsigned short ccp_rscode; /* server reason code ULESHORT */
+ unsigned short ccp_rtcode; /* server return code ULESHORT */
+ unsigned char repd_parml[2]; /* replied parameter len ULESHORT*/
+ unsigned char mac_data_len[2]; /* Mac Data Length ULESHORT */
+ unsigned char repd_datal[4]; /* replied data length ULELONG */
+ unsigned char req_pc[2]; /* PC identifier */
+ unsigned char res_origin[8]; /* resource origin */
+ unsigned char mac_value[8]; /* Mac Value */
+ unsigned char logon_id[8]; /* Logon Identifier */
+ unsigned char usage_domain[2]; /* cdx */
+ unsigned char resv3[18]; /* reserved for requestor */
+ unsigned short svr_namel; /* server name length ULESHORT */
+ unsigned char svr_name[8]; /* server name */
+} __packed;
+
+struct function_and_rules_block {
+ unsigned char function_code[2];
+ unsigned short ulen;
+ unsigned char only_rule[8];
+} __packed;
+
+/**
+ * The following is used to initialize the CPRBX passed to the PCIXCC/CEX2C
+ * card in a type6 message. The 3 fields that must be filled in at execution
+ * time are req_parml, rpl_parml and usage_domain.
+ * Everything about this interface is ascii/big-endian, since the
+ * device does *not* have 'Intel inside'.
+ *
+ * The CPRBX is followed immediately by the parm block.
+ * The parm block contains:
+ * - function code ('PD' 0x5044 or 'PK' 0x504B)
+ * - rule block (one of:)
+ * + 0x000A 'PKCS-1.2' (MCL2 'PD')
+ * + 0x000A 'ZERO-PAD' (MCL2 'PK')
+ * + 0x000A 'ZERO-PAD' (MCL3 'PD' or CEX2C 'PD')
+ * + 0x000A 'MRP ' (MCL3 'PK' or CEX2C 'PK')
+ * - VUD block
+ */
+static struct CPRBX static_cprbx = {
+ .cprb_len = 0x00DC,
+ .cprb_ver_id = 0x02,
+ .func_id = {0x54, 0x32},
+};
+
+/**
+ * Convert a ICAMEX message to a type6 MEX message.
+ *
+ * @zdev: crypto device pointer
+ * @ap_msg: pointer to AP message
+ * @mex: pointer to user input data
+ *
+ * Returns 0 on success or -EFAULT.
+ */
+static int ICAMEX_msg_to_type6MEX_msgX(struct zcrypt_device *zdev,
+ struct ap_message *ap_msg,
+ struct ica_rsa_modexpo *mex)
+{
+ static struct type6_hdr static_type6_hdrX = {
+ .type = 0x06,
+ .offset1 = 0x00000058,
+ .agent_id = {'C', 'A',},
+ .function_code = {'P', 'K'},
+ };
+ static struct function_and_rules_block static_pke_fnr = {
+ .function_code = {'P', 'K'},
+ .ulen = 10,
+ .only_rule = {'M', 'R', 'P', ' ', ' ', ' ', ' ', ' '}
+ };
+ static struct function_and_rules_block static_pke_fnr_MCL2 = {
+ .function_code = {'P', 'K'},
+ .ulen = 10,
+ .only_rule = {'Z', 'E', 'R', 'O', '-', 'P', 'A', 'D'}
+ };
+ struct {
+ struct type6_hdr hdr;
+ struct CPRBX cprbx;
+ struct function_and_rules_block fr;
+ unsigned short length;
+ char text[0];
+ } __packed * msg = ap_msg->message;
+ int size;
+
+ /* VUD.ciphertext */
+ msg->length = mex->inputdatalength + 2;
+ if (copy_from_user(msg->text, mex->inputdata, mex->inputdatalength))
+ return -EFAULT;
+
+ /* Set up key which is located after the variable length text. */
+ size = zcrypt_type6_mex_key_en(mex, msg->text+mex->inputdatalength, 1);
+ if (size < 0)
+ return size;
+ size += sizeof(*msg) + mex->inputdatalength;
+
+ /* message header, cprbx and f&r */
+ msg->hdr = static_type6_hdrX;
+ msg->hdr.ToCardLen1 = size - sizeof(msg->hdr);
+ msg->hdr.FromCardLen1 = PCIXCC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
+
+ msg->cprbx = static_cprbx;
+ msg->cprbx.domain = AP_QID_QUEUE(zdev->ap_dev->qid);
+ msg->cprbx.rpl_msgbl = msg->hdr.FromCardLen1;
+
+ msg->fr = (zdev->user_space_type == ZCRYPT_PCIXCC_MCL2) ?
+ static_pke_fnr_MCL2 : static_pke_fnr;
+
+ msg->cprbx.req_parml = size - sizeof(msg->hdr) - sizeof(msg->cprbx);
+
+ ap_msg->length = size;
+ return 0;
+}
+
+/**
+ * Convert a ICACRT message to a type6 CRT message.
+ *
+ * @zdev: crypto device pointer
+ * @ap_msg: pointer to AP message
+ * @crt: pointer to user input data
+ *
+ * Returns 0 on success or -EFAULT.
+ */
+static int ICACRT_msg_to_type6CRT_msgX(struct zcrypt_device *zdev,
+ struct ap_message *ap_msg,
+ struct ica_rsa_modexpo_crt *crt)
+{
+ static struct type6_hdr static_type6_hdrX = {
+ .type = 0x06,
+ .offset1 = 0x00000058,
+ .agent_id = {'C', 'A',},
+ .function_code = {'P', 'D'},
+ };
+ static struct function_and_rules_block static_pkd_fnr = {
+ .function_code = {'P', 'D'},
+ .ulen = 10,
+ .only_rule = {'Z', 'E', 'R', 'O', '-', 'P', 'A', 'D'}
+ };
+
+ static struct function_and_rules_block static_pkd_fnr_MCL2 = {
+ .function_code = {'P', 'D'},
+ .ulen = 10,
+ .only_rule = {'P', 'K', 'C', 'S', '-', '1', '.', '2'}
+ };
+ struct {
+ struct type6_hdr hdr;
+ struct CPRBX cprbx;
+ struct function_and_rules_block fr;
+ unsigned short length;
+ char text[0];
+ } __packed * msg = ap_msg->message;
+ int size;
+
+ /* VUD.ciphertext */
+ msg->length = crt->inputdatalength + 2;
+ if (copy_from_user(msg->text, crt->inputdata, crt->inputdatalength))
+ return -EFAULT;
+
+ /* Set up key which is located after the variable length text. */
+ size = zcrypt_type6_crt_key(crt, msg->text + crt->inputdatalength, 1);
+ if (size < 0)
+ return size;
+ size += sizeof(*msg) + crt->inputdatalength; /* total size of msg */
+
+ /* message header, cprbx and f&r */
+ msg->hdr = static_type6_hdrX;
+ msg->hdr.ToCardLen1 = size - sizeof(msg->hdr);
+ msg->hdr.FromCardLen1 = PCIXCC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
+
+ msg->cprbx = static_cprbx;
+ msg->cprbx.domain = AP_QID_QUEUE(zdev->ap_dev->qid);
+ msg->cprbx.req_parml = msg->cprbx.rpl_msgbl =
+ size - sizeof(msg->hdr) - sizeof(msg->cprbx);
+
+ msg->fr = (zdev->user_space_type == ZCRYPT_PCIXCC_MCL2) ?
+ static_pkd_fnr_MCL2 : static_pkd_fnr;
+
+ ap_msg->length = size;
+ return 0;
+}
+
+/**
+ * Convert a XCRB message to a type6 CPRB message.
+ *
+ * @zdev: crypto device pointer
+ * @ap_msg: pointer to AP message
+ * @xcRB: pointer to user input data
+ *
+ * Returns 0 on success or -EFAULT, -EINVAL.
+ */
+struct type86_fmt2_msg {
+ struct type86_hdr hdr;
+ struct type86_fmt2_ext fmt2;
+} __packed;
+
+static int XCRB_msg_to_type6CPRB_msgX(struct zcrypt_device *zdev,
+ struct ap_message *ap_msg,
+ struct ica_xcRB *xcRB)
+{
+ static struct type6_hdr static_type6_hdrX = {
+ .type = 0x06,
+ .offset1 = 0x00000058,
+ };
+ struct {
+ struct type6_hdr hdr;
+ struct CPRBX cprbx;
+ } __packed * msg = ap_msg->message;
+
+ int rcblen = CEIL4(xcRB->request_control_blk_length);
+ int replylen;
+ char *req_data = ap_msg->message + sizeof(struct type6_hdr) + rcblen;
+ char *function_code;
+
+ /* length checks */
+ ap_msg->length = sizeof(struct type6_hdr) +
+ CEIL4(xcRB->request_control_blk_length) +
+ xcRB->request_data_length;
+ if (ap_msg->length > MSGTYPE06_MAX_MSG_SIZE)
+ return -EINVAL;
+ replylen = sizeof(struct type86_fmt2_msg) +
+ CEIL4(xcRB->reply_control_blk_length) +
+ xcRB->reply_data_length;
+ if (replylen > MSGTYPE06_MAX_MSG_SIZE)
+ return -EINVAL;
+
+ /* prepare type6 header */
+ msg->hdr = static_type6_hdrX;
+ memcpy(msg->hdr.agent_id , &(xcRB->agent_ID), sizeof(xcRB->agent_ID));
+ msg->hdr.ToCardLen1 = xcRB->request_control_blk_length;
+ if (xcRB->request_data_length) {
+ msg->hdr.offset2 = msg->hdr.offset1 + rcblen;
+ msg->hdr.ToCardLen2 = xcRB->request_data_length;
+ }
+ msg->hdr.FromCardLen1 = xcRB->reply_control_blk_length;
+ msg->hdr.FromCardLen2 = xcRB->reply_data_length;
+
+ /* prepare CPRB */
+ if (copy_from_user(&(msg->cprbx), xcRB->request_control_blk_addr,
+ xcRB->request_control_blk_length))
+ return -EFAULT;
+ if (msg->cprbx.cprb_len + sizeof(msg->hdr.function_code) >
+ xcRB->request_control_blk_length)
+ return -EINVAL;
+ function_code = ((unsigned char *)&msg->cprbx) + msg->cprbx.cprb_len;
+ memcpy(msg->hdr.function_code, function_code,
+ sizeof(msg->hdr.function_code));
+
+ if (memcmp(function_code, "US", 2) == 0)
+ ap_msg->special = 1;
+ else
+ ap_msg->special = 0;
+
+ /* copy data block */
+ if (xcRB->request_data_length &&
+ copy_from_user(req_data, xcRB->request_data_address,
+ xcRB->request_data_length))
+ return -EFAULT;
+ return 0;
+}
+
+/**
+ * Copy results from a type 86 ICA reply message back to user space.
+ *
+ * @zdev: crypto device pointer
+ * @reply: reply AP message.
+ * @data: pointer to user output data
+ * @length: size of user output data
+ *
+ * Returns 0 on success or -EINVAL, -EFAULT, -EAGAIN in case of an error.
+ */
+struct type86x_reply {
+ struct type86_hdr hdr;
+ struct type86_fmt2_ext fmt2;
+ struct CPRBX cprbx;
+ unsigned char pad[4]; /* 4 byte function code/rules block ? */
+ unsigned short length;
+ char text[0];
+} __packed;
+
+static int convert_type86_ica(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ char __user *outputdata,
+ unsigned int outputdatalength)
+{
+ static unsigned char static_pad[] = {
+ 0x00, 0x02,
+ 0x1B, 0x7B, 0x5D, 0xB5, 0x75, 0x01, 0x3D, 0xFD,
+ 0x8D, 0xD1, 0xC7, 0x03, 0x2D, 0x09, 0x23, 0x57,
+ 0x89, 0x49, 0xB9, 0x3F, 0xBB, 0x99, 0x41, 0x5B,
+ 0x75, 0x21, 0x7B, 0x9D, 0x3B, 0x6B, 0x51, 0x39,
+ 0xBB, 0x0D, 0x35, 0xB9, 0x89, 0x0F, 0x93, 0xA5,
+ 0x0B, 0x47, 0xF1, 0xD3, 0xBB, 0xCB, 0xF1, 0x9D,
+ 0x23, 0x73, 0x71, 0xFF, 0xF3, 0xF5, 0x45, 0xFB,
+ 0x61, 0x29, 0x23, 0xFD, 0xF1, 0x29, 0x3F, 0x7F,
+ 0x17, 0xB7, 0x1B, 0xA9, 0x19, 0xBD, 0x57, 0xA9,
+ 0xD7, 0x95, 0xA3, 0xCB, 0xED, 0x1D, 0xDB, 0x45,
+ 0x7D, 0x11, 0xD1, 0x51, 0x1B, 0xED, 0x71, 0xE9,
+ 0xB1, 0xD1, 0xAB, 0xAB, 0x21, 0x2B, 0x1B, 0x9F,
+ 0x3B, 0x9F, 0xF7, 0xF7, 0xBD, 0x63, 0xEB, 0xAD,
+ 0xDF, 0xB3, 0x6F, 0x5B, 0xDB, 0x8D, 0xA9, 0x5D,
+ 0xE3, 0x7D, 0x77, 0x49, 0x47, 0xF5, 0xA7, 0xFD,
+ 0xAB, 0x2F, 0x27, 0x35, 0x77, 0xD3, 0x49, 0xC9,
+ 0x09, 0xEB, 0xB1, 0xF9, 0xBF, 0x4B, 0xCB, 0x2B,
+ 0xEB, 0xEB, 0x05, 0xFF, 0x7D, 0xC7, 0x91, 0x8B,
+ 0x09, 0x83, 0xB9, 0xB9, 0x69, 0x33, 0x39, 0x6B,
+ 0x79, 0x75, 0x19, 0xBF, 0xBB, 0x07, 0x1D, 0xBD,
+ 0x29, 0xBF, 0x39, 0x95, 0x93, 0x1D, 0x35, 0xC7,
+ 0xC9, 0x4D, 0xE5, 0x97, 0x0B, 0x43, 0x9B, 0xF1,
+ 0x16, 0x93, 0x03, 0x1F, 0xA5, 0xFB, 0xDB, 0xF3,
+ 0x27, 0x4F, 0x27, 0x61, 0x05, 0x1F, 0xB9, 0x23,
+ 0x2F, 0xC3, 0x81, 0xA9, 0x23, 0x71, 0x55, 0x55,
+ 0xEB, 0xED, 0x41, 0xE5, 0xF3, 0x11, 0xF1, 0x43,
+ 0x69, 0x03, 0xBD, 0x0B, 0x37, 0x0F, 0x51, 0x8F,
+ 0x0B, 0xB5, 0x89, 0x5B, 0x67, 0xA9, 0xD9, 0x4F,
+ 0x01, 0xF9, 0x21, 0x77, 0x37, 0x73, 0x79, 0xC5,
+ 0x7F, 0x51, 0xC1, 0xCF, 0x97, 0xA1, 0x75, 0xAD,
+ 0x35, 0x9D, 0xD3, 0xD3, 0xA7, 0x9D, 0x5D, 0x41,
+ 0x6F, 0x65, 0x1B, 0xCF, 0xA9, 0x87, 0x91, 0x09
+ };
+ struct type86x_reply *msg = reply->message;
+ unsigned short service_rc, service_rs;
+ unsigned int reply_len, pad_len;
+ char *data;
+
+ service_rc = msg->cprbx.ccp_rtcode;
+ if (unlikely(service_rc != 0)) {
+ service_rs = msg->cprbx.ccp_rscode;
+ if (service_rc == 8 && service_rs == 66)
+ return -EINVAL;
+ if (service_rc == 8 && service_rs == 65)
+ return -EINVAL;
+ if (service_rc == 8 && service_rs == 770)
+ return -EINVAL;
+ if (service_rc == 8 && service_rs == 783) {
+ zdev->min_mod_size = PCIXCC_MIN_MOD_SIZE_OLD;
+ return -EAGAIN;
+ }
+ if (service_rc == 12 && service_rs == 769)
+ return -EINVAL;
+ if (service_rc == 8 && service_rs == 72)
+ return -EINVAL;
+ zdev->online = 0;
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+ data = msg->text;
+ reply_len = msg->length - 2;
+ if (reply_len > outputdatalength)
+ return -EINVAL;
+ /*
+ * For all encipher requests, the length of the ciphertext (reply_len)
+ * will always equal the modulus length. For MEX decipher requests
+ * the output needs to get padded. Minimum pad size is 10.
+ *
+ * Currently, the cases where padding will be added is for:
+ * - PCIXCC_MCL2 using a CRT form token (since PKD didn't support
+ * ZERO-PAD and CRT is only supported for PKD requests)
+ * - PCICC, always
+ */
+ pad_len = outputdatalength - reply_len;
+ if (pad_len > 0) {
+ if (pad_len < 10)
+ return -EINVAL;
+ /* 'restore' padding left in the PCICC/PCIXCC card. */
+ if (copy_to_user(outputdata, static_pad, pad_len - 1))
+ return -EFAULT;
+ if (put_user(0, outputdata + pad_len - 1))
+ return -EFAULT;
+ }
+ /* Copy the crypto response to user space. */
+ if (copy_to_user(outputdata + pad_len, data, reply_len))
+ return -EFAULT;
+ return 0;
+}
+
+/**
+ * Copy results from a type 86 XCRB reply message back to user space.
+ *
+ * @zdev: crypto device pointer
+ * @reply: reply AP message.
+ * @xcRB: pointer to XCRB
+ *
+ * Returns 0 on success or -EINVAL, -EFAULT, -EAGAIN in case of an error.
+ */
+static int convert_type86_xcrb(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ struct ica_xcRB *xcRB)
+{
+ struct type86_fmt2_msg *msg = reply->message;
+ char *data = reply->message;
+
+ /* Copy CPRB to user */
+ if (copy_to_user(xcRB->reply_control_blk_addr,
+ data + msg->fmt2.offset1, msg->fmt2.count1))
+ return -EFAULT;
+ xcRB->reply_control_blk_length = msg->fmt2.count1;
+
+ /* Copy data buffer to user */
+ if (msg->fmt2.count2)
+ if (copy_to_user(xcRB->reply_data_addr,
+ data + msg->fmt2.offset2, msg->fmt2.count2))
+ return -EFAULT;
+ xcRB->reply_data_length = msg->fmt2.count2;
+ return 0;
+}
+
+static int convert_type86_rng(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ char *buffer)
+{
+ struct {
+ struct type86_hdr hdr;
+ struct type86_fmt2_ext fmt2;
+ struct CPRBX cprbx;
+ } __packed * msg = reply->message;
+ char *data = reply->message;
+
+ if (msg->cprbx.ccp_rtcode != 0 || msg->cprbx.ccp_rscode != 0)
+ return -EINVAL;
+ memcpy(buffer, data + msg->fmt2.offset2, msg->fmt2.count2);
+ return msg->fmt2.count2;
+}
+
+static int convert_response_ica(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ char __user *outputdata,
+ unsigned int outputdatalength)
+{
+ struct type86x_reply *msg = reply->message;
+
+ /* Response type byte is the second byte in the response. */
+ switch (((unsigned char *) reply->message)[1]) {
+ case TYPE82_RSP_CODE:
+ case TYPE88_RSP_CODE:
+ return convert_error(zdev, reply);
+ case TYPE86_RSP_CODE:
+ if (msg->cprbx.ccp_rtcode &&
+ (msg->cprbx.ccp_rscode == 0x14f) &&
+ (outputdatalength > 256)) {
+ if (zdev->max_exp_bit_length <= 17) {
+ zdev->max_exp_bit_length = 17;
+ return -EAGAIN;
+ } else
+ return -EINVAL;
+ }
+ if (msg->hdr.reply_code)
+ return convert_error(zdev, reply);
+ if (msg->cprbx.cprb_ver_id == 0x02)
+ return convert_type86_ica(zdev, reply,
+ outputdata, outputdatalength);
+ /* Fall through, no break, incorrect cprb version is an unknown
+ * response */
+ default: /* Unknown response type, this should NEVER EVER happen */
+ zdev->online = 0;
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+}
+
+static int convert_response_xcrb(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ struct ica_xcRB *xcRB)
+{
+ struct type86x_reply *msg = reply->message;
+
+ /* Response type byte is the second byte in the response. */
+ switch (((unsigned char *) reply->message)[1]) {
+ case TYPE82_RSP_CODE:
+ case TYPE88_RSP_CODE:
+ xcRB->status = 0x0008044DL; /* HDD_InvalidParm */
+ return convert_error(zdev, reply);
+ case TYPE86_RSP_CODE:
+ if (msg->hdr.reply_code) {
+ memcpy(&(xcRB->status), msg->fmt2.apfs, sizeof(u32));
+ return convert_error(zdev, reply);
+ }
+ if (msg->cprbx.cprb_ver_id == 0x02)
+ return convert_type86_xcrb(zdev, reply, xcRB);
+ /* Fall through, no break, incorrect cprb version is an unknown
+ * response */
+ default: /* Unknown response type, this should NEVER EVER happen */
+ xcRB->status = 0x0008044DL; /* HDD_InvalidParm */
+ zdev->online = 0;
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+}
+
+static int convert_response_rng(struct zcrypt_device *zdev,
+ struct ap_message *reply,
+ char *data)
+{
+ struct type86x_reply *msg = reply->message;
+
+ switch (msg->hdr.type) {
+ case TYPE82_RSP_CODE:
+ case TYPE88_RSP_CODE:
+ return -EINVAL;
+ case TYPE86_RSP_CODE:
+ if (msg->hdr.reply_code)
+ return -EINVAL;
+ if (msg->cprbx.cprb_ver_id == 0x02)
+ return convert_type86_rng(zdev, reply, data);
+ /* Fall through, no break, incorrect cprb version is an unknown
+ * response */
+ default: /* Unknown response type, this should NEVER EVER happen */
+ zdev->online = 0;
+ return -EAGAIN; /* repeat the request on a different device. */
+ }
+}
+
+/**
+ * This function is called from the AP bus code after a crypto request
+ * "msg" has finished with the reply message "reply".
+ * It is called from tasklet context.
+ * @ap_dev: pointer to the AP device
+ * @msg: pointer to the AP message
+ * @reply: pointer to the AP reply message
+ */
+static void zcrypt_msgtype6_receive(struct ap_device *ap_dev,
+ struct ap_message *msg,
+ struct ap_message *reply)
+{
+ static struct error_hdr error_reply = {
+ .type = TYPE82_RSP_CODE,
+ .reply_code = REP82_ERROR_MACHINE_FAILURE,
+ };
+ struct response_type *resp_type =
+ (struct response_type *) msg->private;
+ struct type86x_reply *t86r;
+ int length;
+
+ /* Copy the reply message to the request message buffer. */
+ if (IS_ERR(reply)) {
+ memcpy(msg->message, &error_reply, sizeof(error_reply));
+ goto out;
+ }
+ t86r = reply->message;
+ if (t86r->hdr.type == TYPE86_RSP_CODE &&
+ t86r->cprbx.cprb_ver_id == 0x02) {
+ switch (resp_type->type) {
+ case PCIXCC_RESPONSE_TYPE_ICA:
+ length = sizeof(struct type86x_reply)
+ + t86r->length - 2;
+ length = min(PCIXCC_MAX_ICA_RESPONSE_SIZE, length);
+ memcpy(msg->message, reply->message, length);
+ break;
+ case PCIXCC_RESPONSE_TYPE_XCRB:
+ length = t86r->fmt2.offset2 + t86r->fmt2.count2;
+ length = min(MSGTYPE06_MAX_MSG_SIZE, length);
+ memcpy(msg->message, reply->message, length);
+ break;
+ default:
+ memcpy(msg->message, &error_reply,
+ sizeof(error_reply));
+ }
+ } else
+ memcpy(msg->message, reply->message, sizeof(error_reply));
+out:
+ complete(&(resp_type->work));
+}
+
+static atomic_t zcrypt_step = ATOMIC_INIT(0);
+
+/**
+ * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * device to handle a modexpo request.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * PCIXCC/CEX2C device to the request distributor
+ * @mex: pointer to the modexpo request buffer
+ */
+static long zcrypt_msgtype6_modexpo(struct zcrypt_device *zdev,
+ struct ica_rsa_modexpo *mex)
+{
+ struct ap_message ap_msg;
+ struct response_type resp_type = {
+ .type = PCIXCC_RESPONSE_TYPE_ICA,
+ };
+ int rc;
+
+ ap_init_message(&ap_msg);
+ ap_msg.message = (void *) get_zeroed_page(GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_msgtype6_receive;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &resp_type;
+ rc = ICAMEX_msg_to_type6MEX_msgX(zdev, &ap_msg, mex);
+ if (rc)
+ goto out_free;
+ init_completion(&resp_type.work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&resp_type.work);
+ if (rc == 0)
+ rc = convert_response_ica(zdev, &ap_msg, mex->outputdata,
+ mex->outputdatalength);
+ else
+ /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+out_free:
+ free_page((unsigned long) ap_msg.message);
+ return rc;
+}
+
+/**
+ * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * device to handle a modexpo_crt request.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * PCIXCC/CEX2C device to the request distributor
+ * @crt: pointer to the modexpoc_crt request buffer
+ */
+static long zcrypt_msgtype6_modexpo_crt(struct zcrypt_device *zdev,
+ struct ica_rsa_modexpo_crt *crt)
+{
+ struct ap_message ap_msg;
+ struct response_type resp_type = {
+ .type = PCIXCC_RESPONSE_TYPE_ICA,
+ };
+ int rc;
+
+ ap_init_message(&ap_msg);
+ ap_msg.message = (void *) get_zeroed_page(GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_msgtype6_receive;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &resp_type;
+ rc = ICACRT_msg_to_type6CRT_msgX(zdev, &ap_msg, crt);
+ if (rc)
+ goto out_free;
+ init_completion(&resp_type.work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&resp_type.work);
+ if (rc == 0)
+ rc = convert_response_ica(zdev, &ap_msg, crt->outputdata,
+ crt->outputdatalength);
+ else
+ /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+out_free:
+ free_page((unsigned long) ap_msg.message);
+ return rc;
+}
+
+/**
+ * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * device to handle a send_cprb request.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * PCIXCC/CEX2C device to the request distributor
+ * @xcRB: pointer to the send_cprb request buffer
+ */
+static long zcrypt_msgtype6_send_cprb(struct zcrypt_device *zdev,
+ struct ica_xcRB *xcRB)
+{
+ struct ap_message ap_msg;
+ struct response_type resp_type = {
+ .type = PCIXCC_RESPONSE_TYPE_XCRB,
+ };
+ int rc;
+
+ ap_init_message(&ap_msg);
+ ap_msg.message = kmalloc(MSGTYPE06_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_msgtype6_receive;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &resp_type;
+ rc = XCRB_msg_to_type6CPRB_msgX(zdev, &ap_msg, xcRB);
+ if (rc)
+ goto out_free;
+ init_completion(&resp_type.work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&resp_type.work);
+ if (rc == 0)
+ rc = convert_response_xcrb(zdev, &ap_msg, xcRB);
+ else
+ /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+out_free:
+ kzfree(ap_msg.message);
+ return rc;
+}
+
+/**
+ * The request distributor calls this function if it picked the PCIXCC/CEX2C
+ * device to generate random data.
+ * @zdev: pointer to zcrypt_device structure that identifies the
+ * PCIXCC/CEX2C device to the request distributor
+ * @buffer: pointer to a memory page to return random data
+ */
+
+static long zcrypt_msgtype6_rng(struct zcrypt_device *zdev,
+ char *buffer)
+{
+ struct ap_message ap_msg;
+ struct response_type resp_type = {
+ .type = PCIXCC_RESPONSE_TYPE_XCRB,
+ };
+ int rc;
+
+ ap_init_message(&ap_msg);
+ ap_msg.message = kmalloc(MSGTYPE06_MAX_MSG_SIZE, GFP_KERNEL);
+ if (!ap_msg.message)
+ return -ENOMEM;
+ ap_msg.receive = zcrypt_msgtype6_receive;
+ ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
+ atomic_inc_return(&zcrypt_step);
+ ap_msg.private = &resp_type;
+ rng_type6CPRB_msgX(zdev->ap_dev, &ap_msg, ZCRYPT_RNG_BUFFER_SIZE);
+ init_completion(&resp_type.work);
+ ap_queue_message(zdev->ap_dev, &ap_msg);
+ rc = wait_for_completion_interruptible(&resp_type.work);
+ if (rc == 0)
+ rc = convert_response_rng(zdev, &ap_msg, buffer);
+ else
+ /* Signal pending. */
+ ap_cancel_message(zdev->ap_dev, &ap_msg);
+ kfree(ap_msg.message);
+ return rc;
+}
+
+/**
+ * The crypto operations for a PCIXCC/CEX2C card.
+ */
+static struct zcrypt_ops zcrypt_msgtype6_norng_ops = {
+ .owner = THIS_MODULE,
+ .variant = MSGTYPE06_VARIANT_NORNG,
+ .rsa_modexpo = zcrypt_msgtype6_modexpo,
+ .rsa_modexpo_crt = zcrypt_msgtype6_modexpo_crt,
+ .send_cprb = zcrypt_msgtype6_send_cprb,
+};
+
+static struct zcrypt_ops zcrypt_msgtype6_ops = {
+ .owner = THIS_MODULE,
+ .variant = MSGTYPE06_VARIANT_DEFAULT,
+ .rsa_modexpo = zcrypt_msgtype6_modexpo,
+ .rsa_modexpo_crt = zcrypt_msgtype6_modexpo_crt,
+ .send_cprb = zcrypt_msgtype6_send_cprb,
+ .rng = zcrypt_msgtype6_rng,
+};
+
+int __init zcrypt_msgtype6_init(void)
+{
+ zcrypt_msgtype_register(&zcrypt_msgtype6_norng_ops);
+ zcrypt_msgtype_register(&zcrypt_msgtype6_ops);
+ return 0;
+}
+
+void __exit zcrypt_msgtype6_exit(void)
+{
+ zcrypt_msgtype_unregister(&zcrypt_msgtype6_norng_ops);
+ zcrypt_msgtype_unregister(&zcrypt_msgtype6_ops);
+}
+
+module_init(zcrypt_msgtype6_init);
+module_exit(zcrypt_msgtype6_exit);
diff --git a/drivers/s390/crypto/zcrypt_msgtype6.h b/drivers/s390/crypto/zcrypt_msgtype6.h
new file mode 100644
index 0000000..1e500d3
--- /dev/null
+++ b/drivers/s390/crypto/zcrypt_msgtype6.h
@@ -0,0 +1,169 @@
+/*
+ * zcrypt 2.1.0
+ *
+ * Copyright IBM Corp. 2001, 2012
+ * Author(s): Robert Burroughs
+ * Eric Rossman (edrossma@us.ibm.com)
+ *
+ * Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
+ * Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#ifndef _ZCRYPT_MSGTYPE6_H_
+#define _ZCRYPT_MSGTYPE6_H_
+
+#include <asm/zcrypt.h>
+
+#define MSGTYPE06_NAME "zcrypt_msgtype6"
+#define MSGTYPE06_VARIANT_DEFAULT 0
+#define MSGTYPE06_VARIANT_NORNG 1
+
+#define MSGTYPE06_MAX_MSG_SIZE (12*1024)
+
+/**
+ * The type 6 message family is associated with PCICC or PCIXCC cards.
+ *
+ * It contains a message header followed by a CPRB, both of which
+ * are described below.
+ *
+ * Note that all reserved fields must be zeroes.
+ */
+struct type6_hdr {
+ unsigned char reserved1; /* 0x00 */
+ unsigned char type; /* 0x06 */
+ unsigned char reserved2[2]; /* 0x0000 */
+ unsigned char right[4]; /* 0x00000000 */
+ unsigned char reserved3[2]; /* 0x0000 */
+ unsigned char reserved4[2]; /* 0x0000 */
+ unsigned char apfs[4]; /* 0x00000000 */
+ unsigned int offset1; /* 0x00000058 (offset to CPRB) */
+ unsigned int offset2; /* 0x00000000 */
+ unsigned int offset3; /* 0x00000000 */
+ unsigned int offset4; /* 0x00000000 */
+ unsigned char agent_id[16]; /* PCICC: */
+ /* 0x0100 */
+ /* 0x4343412d4150504c202020 */
+ /* 0x010101 */
+ /* PCIXCC: */
+ /* 0x4341000000000000 */
+ /* 0x0000000000000000 */
+ unsigned char rqid[2]; /* rqid. internal to 603 */
+ unsigned char reserved5[2]; /* 0x0000 */
+ unsigned char function_code[2]; /* for PKD, 0x5044 (ascii 'PD') */
+ unsigned char reserved6[2]; /* 0x0000 */
+ unsigned int ToCardLen1; /* (request CPRB len + 3) & -4 */
+ unsigned int ToCardLen2; /* db len 0x00000000 for PKD */
+ unsigned int ToCardLen3; /* 0x00000000 */
+ unsigned int ToCardLen4; /* 0x00000000 */
+ unsigned int FromCardLen1; /* response buffer length */
+ unsigned int FromCardLen2; /* db len 0x00000000 for PKD */
+ unsigned int FromCardLen3; /* 0x00000000 */
+ unsigned int FromCardLen4; /* 0x00000000 */
+} __packed;
+
+/**
+ * The type 86 message family is associated with PCICC and PCIXCC cards.
+ *
+ * It contains a message header followed by a CPRB. The CPRB is
+ * the same as the request CPRB, which is described above.
+ *
+ * If format is 1, an error condition exists and no data beyond
+ * the 8-byte message header is of interest.
+ *
+ * The non-error message is shown below.
+ *
+ * Note that all reserved fields must be zeroes.
+ */
+struct type86_hdr {
+ unsigned char reserved1; /* 0x00 */
+ unsigned char type; /* 0x86 */
+ unsigned char format; /* 0x01 (error) or 0x02 (ok) */
+ unsigned char reserved2; /* 0x00 */
+ unsigned char reply_code; /* reply code (see above) */
+ unsigned char reserved3[3]; /* 0x000000 */
+} __packed;
+
+#define TYPE86_RSP_CODE 0x86
+#define TYPE86_FMT2 0x02
+
+struct type86_fmt2_ext {
+ unsigned char reserved[4]; /* 0x00000000 */
+ unsigned char apfs[4]; /* final status */
+ unsigned int count1; /* length of CPRB + parameters */
+ unsigned int offset1; /* offset to CPRB */
+ unsigned int count2; /* 0x00000000 */
+ unsigned int offset2; /* db offset 0x00000000 for PKD */
+ unsigned int count3; /* 0x00000000 */
+ unsigned int offset3; /* 0x00000000 */
+ unsigned int count4; /* 0x00000000 */
+ unsigned int offset4; /* 0x00000000 */
+} __packed;
+
+/**
+ * Prepare a type6 CPRB message for random number generation
+ *
+ * @ap_dev: AP device pointer
+ * @ap_msg: pointer to AP message
+ */
+static inline void rng_type6CPRB_msgX(struct ap_device *ap_dev,
+ struct ap_message *ap_msg,
+ unsigned random_number_length)
+{
+ struct {
+ struct type6_hdr hdr;
+ struct CPRBX cprbx;
+ char function_code[2];
+ short int rule_length;
+ char rule[8];
+ short int verb_length;
+ short int key_length;
+ } __packed * msg = ap_msg->message;
+ static struct type6_hdr static_type6_hdrX = {
+ .type = 0x06,
+ .offset1 = 0x00000058,
+ .agent_id = {'C', 'A'},
+ .function_code = {'R', 'L'},
+ .ToCardLen1 = sizeof(*msg) - sizeof(msg->hdr),
+ .FromCardLen1 = sizeof(*msg) - sizeof(msg->hdr),
+ };
+ static struct CPRBX local_cprbx = {
+ .cprb_len = 0x00dc,
+ .cprb_ver_id = 0x02,
+ .func_id = {0x54, 0x32},
+ .req_parml = sizeof(*msg) - sizeof(msg->hdr) -
+ sizeof(msg->cprbx),
+ .rpl_msgbl = sizeof(*msg) - sizeof(msg->hdr),
+ };
+
+ msg->hdr = static_type6_hdrX;
+ msg->hdr.FromCardLen2 = random_number_length,
+ msg->cprbx = local_cprbx;
+ msg->cprbx.rpl_datal = random_number_length,
+ msg->cprbx.domain = AP_QID_QUEUE(ap_dev->qid);
+ memcpy(msg->function_code, msg->hdr.function_code, 0x02);
+ msg->rule_length = 0x0a;
+ memcpy(msg->rule, "RANDOM ", 8);
+ msg->verb_length = 0x02;
+ msg->key_length = 0x02;
+ ap_msg->length = sizeof(*msg);
+}
+
+int zcrypt_msgtype6_init(void);
+void zcrypt_msgtype6_exit(void);
+
+#endif /* _ZCRYPT_MSGTYPE6_H_ */
diff --git a/drivers/s390/crypto/zcrypt_pcixcc.c b/drivers/s390/crypto/zcrypt_pcixcc.c
index ccb4f8b6..c7275e3 100644
--- a/drivers/s390/crypto/zcrypt_pcixcc.c
+++ b/drivers/s390/crypto/zcrypt_pcixcc.c
@@ -1,13 +1,14 @@
/*
* zcrypt 2.1.0
*
- * Copyright IBM Corp. 2001, 2006
+ * Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
*
* Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
* Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
* Ralph Wuerthner <rwuerthn@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -35,9 +36,10 @@
#include "ap_bus.h"
#include "zcrypt_api.h"
#include "zcrypt_error.h"
-#include "zcrypt_pcicc.h"
+#include "zcrypt_msgtype6.h"
#include "zcrypt_pcixcc.h"
#include "zcrypt_cca_key.h"
+#include "zcrypt_msgtype6.h"
#define PCIXCC_MIN_MOD_SIZE 16 /* 128 bits */
#define PCIXCC_MIN_MOD_SIZE_OLD 64 /* 512 bits */
@@ -75,14 +77,12 @@
MODULE_DEVICE_TABLE(ap, zcrypt_pcixcc_ids);
MODULE_AUTHOR("IBM Corporation");
-MODULE_DESCRIPTION("PCIXCC Cryptographic Coprocessor device driver, "
- "Copyright IBM Corp. 2001, 2006");
+MODULE_DESCRIPTION("PCIXCC Cryptographic Coprocessor device driver, " \
+ "Copyright IBM Corp. 2001, 2012");
MODULE_LICENSE("GPL");
static int zcrypt_pcixcc_probe(struct ap_device *ap_dev);
static void zcrypt_pcixcc_remove(struct ap_device *ap_dev);
-static void zcrypt_pcixcc_receive(struct ap_device *, struct ap_message *,
- struct ap_message *);
static struct ap_driver zcrypt_pcixcc_driver = {
.probe = zcrypt_pcixcc_probe,
@@ -92,766 +92,6 @@
};
/**
- * The following is used to initialize the CPRBX passed to the PCIXCC/CEX2C
- * card in a type6 message. The 3 fields that must be filled in at execution
- * time are req_parml, rpl_parml and usage_domain.
- * Everything about this interface is ascii/big-endian, since the
- * device does *not* have 'Intel inside'.
- *
- * The CPRBX is followed immediately by the parm block.
- * The parm block contains:
- * - function code ('PD' 0x5044 or 'PK' 0x504B)
- * - rule block (one of:)
- * + 0x000A 'PKCS-1.2' (MCL2 'PD')
- * + 0x000A 'ZERO-PAD' (MCL2 'PK')
- * + 0x000A 'ZERO-PAD' (MCL3 'PD' or CEX2C 'PD')
- * + 0x000A 'MRP ' (MCL3 'PK' or CEX2C 'PK')
- * - VUD block
- */
-static struct CPRBX static_cprbx = {
- .cprb_len = 0x00DC,
- .cprb_ver_id = 0x02,
- .func_id = {0x54,0x32},
-};
-
-/**
- * Convert a ICAMEX message to a type6 MEX message.
- *
- * @zdev: crypto device pointer
- * @ap_msg: pointer to AP message
- * @mex: pointer to user input data
- *
- * Returns 0 on success or -EFAULT.
- */
-static int ICAMEX_msg_to_type6MEX_msgX(struct zcrypt_device *zdev,
- struct ap_message *ap_msg,
- struct ica_rsa_modexpo *mex)
-{
- static struct type6_hdr static_type6_hdrX = {
- .type = 0x06,
- .offset1 = 0x00000058,
- .agent_id = {'C','A',},
- .function_code = {'P','K'},
- };
- static struct function_and_rules_block static_pke_fnr = {
- .function_code = {'P','K'},
- .ulen = 10,
- .only_rule = {'M','R','P',' ',' ',' ',' ',' '}
- };
- static struct function_and_rules_block static_pke_fnr_MCL2 = {
- .function_code = {'P','K'},
- .ulen = 10,
- .only_rule = {'Z','E','R','O','-','P','A','D'}
- };
- struct {
- struct type6_hdr hdr;
- struct CPRBX cprbx;
- struct function_and_rules_block fr;
- unsigned short length;
- char text[0];
- } __attribute__((packed)) *msg = ap_msg->message;
- int size;
-
- /* VUD.ciphertext */
- msg->length = mex->inputdatalength + 2;
- if (copy_from_user(msg->text, mex->inputdata, mex->inputdatalength))
- return -EFAULT;
-
- /* Set up key which is located after the variable length text. */
- size = zcrypt_type6_mex_key_en(mex, msg->text+mex->inputdatalength, 1);
- if (size < 0)
- return size;
- size += sizeof(*msg) + mex->inputdatalength;
-
- /* message header, cprbx and f&r */
- msg->hdr = static_type6_hdrX;
- msg->hdr.ToCardLen1 = size - sizeof(msg->hdr);
- msg->hdr.FromCardLen1 = PCIXCC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
-
- msg->cprbx = static_cprbx;
- msg->cprbx.domain = AP_QID_QUEUE(zdev->ap_dev->qid);
- msg->cprbx.rpl_msgbl = msg->hdr.FromCardLen1;
-
- msg->fr = (zdev->user_space_type == ZCRYPT_PCIXCC_MCL2) ?
- static_pke_fnr_MCL2 : static_pke_fnr;
-
- msg->cprbx.req_parml = size - sizeof(msg->hdr) - sizeof(msg->cprbx);
-
- ap_msg->length = size;
- return 0;
-}
-
-/**
- * Convert a ICACRT message to a type6 CRT message.
- *
- * @zdev: crypto device pointer
- * @ap_msg: pointer to AP message
- * @crt: pointer to user input data
- *
- * Returns 0 on success or -EFAULT.
- */
-static int ICACRT_msg_to_type6CRT_msgX(struct zcrypt_device *zdev,
- struct ap_message *ap_msg,
- struct ica_rsa_modexpo_crt *crt)
-{
- static struct type6_hdr static_type6_hdrX = {
- .type = 0x06,
- .offset1 = 0x00000058,
- .agent_id = {'C','A',},
- .function_code = {'P','D'},
- };
- static struct function_and_rules_block static_pkd_fnr = {
- .function_code = {'P','D'},
- .ulen = 10,
- .only_rule = {'Z','E','R','O','-','P','A','D'}
- };
-
- static struct function_and_rules_block static_pkd_fnr_MCL2 = {
- .function_code = {'P','D'},
- .ulen = 10,
- .only_rule = {'P','K','C','S','-','1','.','2'}
- };
- struct {
- struct type6_hdr hdr;
- struct CPRBX cprbx;
- struct function_and_rules_block fr;
- unsigned short length;
- char text[0];
- } __attribute__((packed)) *msg = ap_msg->message;
- int size;
-
- /* VUD.ciphertext */
- msg->length = crt->inputdatalength + 2;
- if (copy_from_user(msg->text, crt->inputdata, crt->inputdatalength))
- return -EFAULT;
-
- /* Set up key which is located after the variable length text. */
- size = zcrypt_type6_crt_key(crt, msg->text + crt->inputdatalength, 1);
- if (size < 0)
- return size;
- size += sizeof(*msg) + crt->inputdatalength; /* total size of msg */
-
- /* message header, cprbx and f&r */
- msg->hdr = static_type6_hdrX;
- msg->hdr.ToCardLen1 = size - sizeof(msg->hdr);
- msg->hdr.FromCardLen1 = PCIXCC_MAX_ICA_RESPONSE_SIZE - sizeof(msg->hdr);
-
- msg->cprbx = static_cprbx;
- msg->cprbx.domain = AP_QID_QUEUE(zdev->ap_dev->qid);
- msg->cprbx.req_parml = msg->cprbx.rpl_msgbl =
- size - sizeof(msg->hdr) - sizeof(msg->cprbx);
-
- msg->fr = (zdev->user_space_type == ZCRYPT_PCIXCC_MCL2) ?
- static_pkd_fnr_MCL2 : static_pkd_fnr;
-
- ap_msg->length = size;
- return 0;
-}
-
-/**
- * Convert a XCRB message to a type6 CPRB message.
- *
- * @zdev: crypto device pointer
- * @ap_msg: pointer to AP message
- * @xcRB: pointer to user input data
- *
- * Returns 0 on success or -EFAULT, -EINVAL.
- */
-struct type86_fmt2_msg {
- struct type86_hdr hdr;
- struct type86_fmt2_ext fmt2;
-} __attribute__((packed));
-
-static int XCRB_msg_to_type6CPRB_msgX(struct zcrypt_device *zdev,
- struct ap_message *ap_msg,
- struct ica_xcRB *xcRB)
-{
- static struct type6_hdr static_type6_hdrX = {
- .type = 0x06,
- .offset1 = 0x00000058,
- };
- struct {
- struct type6_hdr hdr;
- struct CPRBX cprbx;
- } __attribute__((packed)) *msg = ap_msg->message;
-
- int rcblen = CEIL4(xcRB->request_control_blk_length);
- int replylen;
- char *req_data = ap_msg->message + sizeof(struct type6_hdr) + rcblen;
- char *function_code;
-
- /* length checks */
- ap_msg->length = sizeof(struct type6_hdr) +
- CEIL4(xcRB->request_control_blk_length) +
- xcRB->request_data_length;
- if (ap_msg->length > PCIXCC_MAX_XCRB_MESSAGE_SIZE)
- return -EINVAL;
- replylen = sizeof(struct type86_fmt2_msg) +
- CEIL4(xcRB->reply_control_blk_length) +
- xcRB->reply_data_length;
- if (replylen > PCIXCC_MAX_XCRB_MESSAGE_SIZE)
- return -EINVAL;
-
- /* prepare type6 header */
- msg->hdr = static_type6_hdrX;
- memcpy(msg->hdr.agent_id , &(xcRB->agent_ID), sizeof(xcRB->agent_ID));
- msg->hdr.ToCardLen1 = xcRB->request_control_blk_length;
- if (xcRB->request_data_length) {
- msg->hdr.offset2 = msg->hdr.offset1 + rcblen;
- msg->hdr.ToCardLen2 = xcRB->request_data_length;
- }
- msg->hdr.FromCardLen1 = xcRB->reply_control_blk_length;
- msg->hdr.FromCardLen2 = xcRB->reply_data_length;
-
- /* prepare CPRB */
- if (copy_from_user(&(msg->cprbx), xcRB->request_control_blk_addr,
- xcRB->request_control_blk_length))
- return -EFAULT;
- if (msg->cprbx.cprb_len + sizeof(msg->hdr.function_code) >
- xcRB->request_control_blk_length)
- return -EINVAL;
- function_code = ((unsigned char *)&msg->cprbx) + msg->cprbx.cprb_len;
- memcpy(msg->hdr.function_code, function_code, sizeof(msg->hdr.function_code));
-
- if (memcmp(function_code, "US", 2) == 0)
- ap_msg->special = 1;
- else
- ap_msg->special = 0;
-
- /* copy data block */
- if (xcRB->request_data_length &&
- copy_from_user(req_data, xcRB->request_data_address,
- xcRB->request_data_length))
- return -EFAULT;
- return 0;
-}
-
-/**
- * Prepare a type6 CPRB message for random number generation
- *
- * @ap_dev: AP device pointer
- * @ap_msg: pointer to AP message
- */
-static void rng_type6CPRB_msgX(struct ap_device *ap_dev,
- struct ap_message *ap_msg,
- unsigned random_number_length)
-{
- struct {
- struct type6_hdr hdr;
- struct CPRBX cprbx;
- char function_code[2];
- short int rule_length;
- char rule[8];
- short int verb_length;
- short int key_length;
- } __attribute__((packed)) *msg = ap_msg->message;
- static struct type6_hdr static_type6_hdrX = {
- .type = 0x06,
- .offset1 = 0x00000058,
- .agent_id = {'C', 'A'},
- .function_code = {'R', 'L'},
- .ToCardLen1 = sizeof *msg - sizeof(msg->hdr),
- .FromCardLen1 = sizeof *msg - sizeof(msg->hdr),
- };
- static struct CPRBX local_cprbx = {
- .cprb_len = 0x00dc,
- .cprb_ver_id = 0x02,
- .func_id = {0x54, 0x32},
- .req_parml = sizeof *msg - sizeof(msg->hdr) -
- sizeof(msg->cprbx),
- .rpl_msgbl = sizeof *msg - sizeof(msg->hdr),
- };
-
- msg->hdr = static_type6_hdrX;
- msg->hdr.FromCardLen2 = random_number_length,
- msg->cprbx = local_cprbx;
- msg->cprbx.rpl_datal = random_number_length,
- msg->cprbx.domain = AP_QID_QUEUE(ap_dev->qid);
- memcpy(msg->function_code, msg->hdr.function_code, 0x02);
- msg->rule_length = 0x0a;
- memcpy(msg->rule, "RANDOM ", 8);
- msg->verb_length = 0x02;
- msg->key_length = 0x02;
- ap_msg->length = sizeof *msg;
-}
-
-/**
- * Copy results from a type 86 ICA reply message back to user space.
- *
- * @zdev: crypto device pointer
- * @reply: reply AP message.
- * @data: pointer to user output data
- * @length: size of user output data
- *
- * Returns 0 on success or -EINVAL, -EFAULT, -EAGAIN in case of an error.
- */
-struct type86x_reply {
- struct type86_hdr hdr;
- struct type86_fmt2_ext fmt2;
- struct CPRBX cprbx;
- unsigned char pad[4]; /* 4 byte function code/rules block ? */
- unsigned short length;
- char text[0];
-} __attribute__((packed));
-
-static int convert_type86_ica(struct zcrypt_device *zdev,
- struct ap_message *reply,
- char __user *outputdata,
- unsigned int outputdatalength)
-{
- static unsigned char static_pad[] = {
- 0x00,0x02,
- 0x1B,0x7B,0x5D,0xB5,0x75,0x01,0x3D,0xFD,
- 0x8D,0xD1,0xC7,0x03,0x2D,0x09,0x23,0x57,
- 0x89,0x49,0xB9,0x3F,0xBB,0x99,0x41,0x5B,
- 0x75,0x21,0x7B,0x9D,0x3B,0x6B,0x51,0x39,
- 0xBB,0x0D,0x35,0xB9,0x89,0x0F,0x93,0xA5,
- 0x0B,0x47,0xF1,0xD3,0xBB,0xCB,0xF1,0x9D,
- 0x23,0x73,0x71,0xFF,0xF3,0xF5,0x45,0xFB,
- 0x61,0x29,0x23,0xFD,0xF1,0x29,0x3F,0x7F,
- 0x17,0xB7,0x1B,0xA9,0x19,0xBD,0x57,0xA9,
- 0xD7,0x95,0xA3,0xCB,0xED,0x1D,0xDB,0x45,
- 0x7D,0x11,0xD1,0x51,0x1B,0xED,0x71,0xE9,
- 0xB1,0xD1,0xAB,0xAB,0x21,0x2B,0x1B,0x9F,
- 0x3B,0x9F,0xF7,0xF7,0xBD,0x63,0xEB,0xAD,
- 0xDF,0xB3,0x6F,0x5B,0xDB,0x8D,0xA9,0x5D,
- 0xE3,0x7D,0x77,0x49,0x47,0xF5,0xA7,0xFD,
- 0xAB,0x2F,0x27,0x35,0x77,0xD3,0x49,0xC9,
- 0x09,0xEB,0xB1,0xF9,0xBF,0x4B,0xCB,0x2B,
- 0xEB,0xEB,0x05,0xFF,0x7D,0xC7,0x91,0x8B,
- 0x09,0x83,0xB9,0xB9,0x69,0x33,0x39,0x6B,
- 0x79,0x75,0x19,0xBF,0xBB,0x07,0x1D,0xBD,
- 0x29,0xBF,0x39,0x95,0x93,0x1D,0x35,0xC7,
- 0xC9,0x4D,0xE5,0x97,0x0B,0x43,0x9B,0xF1,
- 0x16,0x93,0x03,0x1F,0xA5,0xFB,0xDB,0xF3,
- 0x27,0x4F,0x27,0x61,0x05,0x1F,0xB9,0x23,
- 0x2F,0xC3,0x81,0xA9,0x23,0x71,0x55,0x55,
- 0xEB,0xED,0x41,0xE5,0xF3,0x11,0xF1,0x43,
- 0x69,0x03,0xBD,0x0B,0x37,0x0F,0x51,0x8F,
- 0x0B,0xB5,0x89,0x5B,0x67,0xA9,0xD9,0x4F,
- 0x01,0xF9,0x21,0x77,0x37,0x73,0x79,0xC5,
- 0x7F,0x51,0xC1,0xCF,0x97,0xA1,0x75,0xAD,
- 0x35,0x9D,0xD3,0xD3,0xA7,0x9D,0x5D,0x41,
- 0x6F,0x65,0x1B,0xCF,0xA9,0x87,0x91,0x09
- };
- struct type86x_reply *msg = reply->message;
- unsigned short service_rc, service_rs;
- unsigned int reply_len, pad_len;
- char *data;
-
- service_rc = msg->cprbx.ccp_rtcode;
- if (unlikely(service_rc != 0)) {
- service_rs = msg->cprbx.ccp_rscode;
- if (service_rc == 8 && service_rs == 66)
- return -EINVAL;
- if (service_rc == 8 && service_rs == 65)
- return -EINVAL;
- if (service_rc == 8 && service_rs == 770)
- return -EINVAL;
- if (service_rc == 8 && service_rs == 783) {
- zdev->min_mod_size = PCIXCC_MIN_MOD_SIZE_OLD;
- return -EAGAIN;
- }
- if (service_rc == 12 && service_rs == 769)
- return -EINVAL;
- if (service_rc == 8 && service_rs == 72)
- return -EINVAL;
- zdev->online = 0;
- return -EAGAIN; /* repeat the request on a different device. */
- }
- data = msg->text;
- reply_len = msg->length - 2;
- if (reply_len > outputdatalength)
- return -EINVAL;
- /*
- * For all encipher requests, the length of the ciphertext (reply_len)
- * will always equal the modulus length. For MEX decipher requests
- * the output needs to get padded. Minimum pad size is 10.
- *
- * Currently, the cases where padding will be added is for:
- * - PCIXCC_MCL2 using a CRT form token (since PKD didn't support
- * ZERO-PAD and CRT is only supported for PKD requests)
- * - PCICC, always
- */
- pad_len = outputdatalength - reply_len;
- if (pad_len > 0) {
- if (pad_len < 10)
- return -EINVAL;
- /* 'restore' padding left in the PCICC/PCIXCC card. */
- if (copy_to_user(outputdata, static_pad, pad_len - 1))
- return -EFAULT;
- if (put_user(0, outputdata + pad_len - 1))
- return -EFAULT;
- }
- /* Copy the crypto response to user space. */
- if (copy_to_user(outputdata + pad_len, data, reply_len))
- return -EFAULT;
- return 0;
-}
-
-/**
- * Copy results from a type 86 XCRB reply message back to user space.
- *
- * @zdev: crypto device pointer
- * @reply: reply AP message.
- * @xcRB: pointer to XCRB
- *
- * Returns 0 on success or -EINVAL, -EFAULT, -EAGAIN in case of an error.
- */
-static int convert_type86_xcrb(struct zcrypt_device *zdev,
- struct ap_message *reply,
- struct ica_xcRB *xcRB)
-{
- struct type86_fmt2_msg *msg = reply->message;
- char *data = reply->message;
-
- /* Copy CPRB to user */
- if (copy_to_user(xcRB->reply_control_blk_addr,
- data + msg->fmt2.offset1, msg->fmt2.count1))
- return -EFAULT;
- xcRB->reply_control_blk_length = msg->fmt2.count1;
-
- /* Copy data buffer to user */
- if (msg->fmt2.count2)
- if (copy_to_user(xcRB->reply_data_addr,
- data + msg->fmt2.offset2, msg->fmt2.count2))
- return -EFAULT;
- xcRB->reply_data_length = msg->fmt2.count2;
- return 0;
-}
-
-static int convert_type86_rng(struct zcrypt_device *zdev,
- struct ap_message *reply,
- char *buffer)
-{
- struct {
- struct type86_hdr hdr;
- struct type86_fmt2_ext fmt2;
- struct CPRBX cprbx;
- } __attribute__((packed)) *msg = reply->message;
- char *data = reply->message;
-
- if (msg->cprbx.ccp_rtcode != 0 || msg->cprbx.ccp_rscode != 0)
- return -EINVAL;
- memcpy(buffer, data + msg->fmt2.offset2, msg->fmt2.count2);
- return msg->fmt2.count2;
-}
-
-static int convert_response_ica(struct zcrypt_device *zdev,
- struct ap_message *reply,
- char __user *outputdata,
- unsigned int outputdatalength)
-{
- struct type86x_reply *msg = reply->message;
-
- /* Response type byte is the second byte in the response. */
- switch (((unsigned char *) reply->message)[1]) {
- case TYPE82_RSP_CODE:
- case TYPE88_RSP_CODE:
- return convert_error(zdev, reply);
- case TYPE86_RSP_CODE:
- if (msg->cprbx.ccp_rtcode &&
- (msg->cprbx.ccp_rscode == 0x14f) &&
- (outputdatalength > 256)) {
- if (zdev->max_exp_bit_length <= 17) {
- zdev->max_exp_bit_length = 17;
- return -EAGAIN;
- } else
- return -EINVAL;
- }
- if (msg->hdr.reply_code)
- return convert_error(zdev, reply);
- if (msg->cprbx.cprb_ver_id == 0x02)
- return convert_type86_ica(zdev, reply,
- outputdata, outputdatalength);
- /* Fall through, no break, incorrect cprb version is an unknown
- * response */
- default: /* Unknown response type, this should NEVER EVER happen */
- zdev->online = 0;
- return -EAGAIN; /* repeat the request on a different device. */
- }
-}
-
-static int convert_response_xcrb(struct zcrypt_device *zdev,
- struct ap_message *reply,
- struct ica_xcRB *xcRB)
-{
- struct type86x_reply *msg = reply->message;
-
- /* Response type byte is the second byte in the response. */
- switch (((unsigned char *) reply->message)[1]) {
- case TYPE82_RSP_CODE:
- case TYPE88_RSP_CODE:
- xcRB->status = 0x0008044DL; /* HDD_InvalidParm */
- return convert_error(zdev, reply);
- case TYPE86_RSP_CODE:
- if (msg->hdr.reply_code) {
- memcpy(&(xcRB->status), msg->fmt2.apfs, sizeof(u32));
- return convert_error(zdev, reply);
- }
- if (msg->cprbx.cprb_ver_id == 0x02)
- return convert_type86_xcrb(zdev, reply, xcRB);
- /* Fall through, no break, incorrect cprb version is an unknown
- * response */
- default: /* Unknown response type, this should NEVER EVER happen */
- xcRB->status = 0x0008044DL; /* HDD_InvalidParm */
- zdev->online = 0;
- return -EAGAIN; /* repeat the request on a different device. */
- }
-}
-
-static int convert_response_rng(struct zcrypt_device *zdev,
- struct ap_message *reply,
- char *data)
-{
- struct type86x_reply *msg = reply->message;
-
- switch (msg->hdr.type) {
- case TYPE82_RSP_CODE:
- case TYPE88_RSP_CODE:
- return -EINVAL;
- case TYPE86_RSP_CODE:
- if (msg->hdr.reply_code)
- return -EINVAL;
- if (msg->cprbx.cprb_ver_id == 0x02)
- return convert_type86_rng(zdev, reply, data);
- /* Fall through, no break, incorrect cprb version is an unknown
- * response */
- default: /* Unknown response type, this should NEVER EVER happen */
- zdev->online = 0;
- return -EAGAIN; /* repeat the request on a different device. */
- }
-}
-
-/**
- * This function is called from the AP bus code after a crypto request
- * "msg" has finished with the reply message "reply".
- * It is called from tasklet context.
- * @ap_dev: pointer to the AP device
- * @msg: pointer to the AP message
- * @reply: pointer to the AP reply message
- */
-static void zcrypt_pcixcc_receive(struct ap_device *ap_dev,
- struct ap_message *msg,
- struct ap_message *reply)
-{
- static struct error_hdr error_reply = {
- .type = TYPE82_RSP_CODE,
- .reply_code = REP82_ERROR_MACHINE_FAILURE,
- };
- struct response_type *resp_type =
- (struct response_type *) msg->private;
- struct type86x_reply *t86r;
- int length;
-
- /* Copy the reply message to the request message buffer. */
- if (IS_ERR(reply)) {
- memcpy(msg->message, &error_reply, sizeof(error_reply));
- goto out;
- }
- t86r = reply->message;
- if (t86r->hdr.type == TYPE86_RSP_CODE &&
- t86r->cprbx.cprb_ver_id == 0x02) {
- switch (resp_type->type) {
- case PCIXCC_RESPONSE_TYPE_ICA:
- length = sizeof(struct type86x_reply)
- + t86r->length - 2;
- length = min(PCIXCC_MAX_ICA_RESPONSE_SIZE, length);
- memcpy(msg->message, reply->message, length);
- break;
- case PCIXCC_RESPONSE_TYPE_XCRB:
- length = t86r->fmt2.offset2 + t86r->fmt2.count2;
- length = min(PCIXCC_MAX_XCRB_MESSAGE_SIZE, length);
- memcpy(msg->message, reply->message, length);
- break;
- default:
- memcpy(msg->message, &error_reply, sizeof error_reply);
- }
- } else
- memcpy(msg->message, reply->message, sizeof error_reply);
-out:
- complete(&(resp_type->work));
-}
-
-static atomic_t zcrypt_step = ATOMIC_INIT(0);
-
-/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
- * device to handle a modexpo request.
- * @zdev: pointer to zcrypt_device structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
- * @mex: pointer to the modexpo request buffer
- */
-static long zcrypt_pcixcc_modexpo(struct zcrypt_device *zdev,
- struct ica_rsa_modexpo *mex)
-{
- struct ap_message ap_msg;
- struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_ICA,
- };
- int rc;
-
- ap_init_message(&ap_msg);
- ap_msg.message = (void *) get_zeroed_page(GFP_KERNEL);
- if (!ap_msg.message)
- return -ENOMEM;
- ap_msg.receive = zcrypt_pcixcc_receive;
- ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
- atomic_inc_return(&zcrypt_step);
- ap_msg.private = &resp_type;
- rc = ICAMEX_msg_to_type6MEX_msgX(zdev, &ap_msg, mex);
- if (rc)
- goto out_free;
- init_completion(&resp_type.work);
- ap_queue_message(zdev->ap_dev, &ap_msg);
- rc = wait_for_completion_interruptible(&resp_type.work);
- if (rc == 0)
- rc = convert_response_ica(zdev, &ap_msg, mex->outputdata,
- mex->outputdatalength);
- else
- /* Signal pending. */
- ap_cancel_message(zdev->ap_dev, &ap_msg);
-out_free:
- free_page((unsigned long) ap_msg.message);
- return rc;
-}
-
-/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
- * device to handle a modexpo_crt request.
- * @zdev: pointer to zcrypt_device structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
- * @crt: pointer to the modexpoc_crt request buffer
- */
-static long zcrypt_pcixcc_modexpo_crt(struct zcrypt_device *zdev,
- struct ica_rsa_modexpo_crt *crt)
-{
- struct ap_message ap_msg;
- struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_ICA,
- };
- int rc;
-
- ap_init_message(&ap_msg);
- ap_msg.message = (void *) get_zeroed_page(GFP_KERNEL);
- if (!ap_msg.message)
- return -ENOMEM;
- ap_msg.receive = zcrypt_pcixcc_receive;
- ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
- atomic_inc_return(&zcrypt_step);
- ap_msg.private = &resp_type;
- rc = ICACRT_msg_to_type6CRT_msgX(zdev, &ap_msg, crt);
- if (rc)
- goto out_free;
- init_completion(&resp_type.work);
- ap_queue_message(zdev->ap_dev, &ap_msg);
- rc = wait_for_completion_interruptible(&resp_type.work);
- if (rc == 0)
- rc = convert_response_ica(zdev, &ap_msg, crt->outputdata,
- crt->outputdatalength);
- else
- /* Signal pending. */
- ap_cancel_message(zdev->ap_dev, &ap_msg);
-out_free:
- free_page((unsigned long) ap_msg.message);
- return rc;
-}
-
-/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
- * device to handle a send_cprb request.
- * @zdev: pointer to zcrypt_device structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
- * @xcRB: pointer to the send_cprb request buffer
- */
-static long zcrypt_pcixcc_send_cprb(struct zcrypt_device *zdev,
- struct ica_xcRB *xcRB)
-{
- struct ap_message ap_msg;
- struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_XCRB,
- };
- int rc;
-
- ap_init_message(&ap_msg);
- ap_msg.message = kmalloc(PCIXCC_MAX_XCRB_MESSAGE_SIZE, GFP_KERNEL);
- if (!ap_msg.message)
- return -ENOMEM;
- ap_msg.receive = zcrypt_pcixcc_receive;
- ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
- atomic_inc_return(&zcrypt_step);
- ap_msg.private = &resp_type;
- rc = XCRB_msg_to_type6CPRB_msgX(zdev, &ap_msg, xcRB);
- if (rc)
- goto out_free;
- init_completion(&resp_type.work);
- ap_queue_message(zdev->ap_dev, &ap_msg);
- rc = wait_for_completion_interruptible(&resp_type.work);
- if (rc == 0)
- rc = convert_response_xcrb(zdev, &ap_msg, xcRB);
- else
- /* Signal pending. */
- ap_cancel_message(zdev->ap_dev, &ap_msg);
-out_free:
- kzfree(ap_msg.message);
- return rc;
-}
-
-/**
- * The request distributor calls this function if it picked the PCIXCC/CEX2C
- * device to generate random data.
- * @zdev: pointer to zcrypt_device structure that identifies the
- * PCIXCC/CEX2C device to the request distributor
- * @buffer: pointer to a memory page to return random data
- */
-
-static long zcrypt_pcixcc_rng(struct zcrypt_device *zdev,
- char *buffer)
-{
- struct ap_message ap_msg;
- struct response_type resp_type = {
- .type = PCIXCC_RESPONSE_TYPE_XCRB,
- };
- int rc;
-
- ap_init_message(&ap_msg);
- ap_msg.message = kmalloc(PCIXCC_MAX_XCRB_MESSAGE_SIZE, GFP_KERNEL);
- if (!ap_msg.message)
- return -ENOMEM;
- ap_msg.receive = zcrypt_pcixcc_receive;
- ap_msg.psmid = (((unsigned long long) current->pid) << 32) +
- atomic_inc_return(&zcrypt_step);
- ap_msg.private = &resp_type;
- rng_type6CPRB_msgX(zdev->ap_dev, &ap_msg, ZCRYPT_RNG_BUFFER_SIZE);
- init_completion(&resp_type.work);
- ap_queue_message(zdev->ap_dev, &ap_msg);
- rc = wait_for_completion_interruptible(&resp_type.work);
- if (rc == 0)
- rc = convert_response_rng(zdev, &ap_msg, buffer);
- else
- /* Signal pending. */
- ap_cancel_message(zdev->ap_dev, &ap_msg);
- kfree(ap_msg.message);
- return rc;
-}
-
-/**
- * The crypto operations for a PCIXCC/CEX2C card.
- */
-static struct zcrypt_ops zcrypt_pcixcc_ops = {
- .rsa_modexpo = zcrypt_pcixcc_modexpo,
- .rsa_modexpo_crt = zcrypt_pcixcc_modexpo_crt,
- .send_cprb = zcrypt_pcixcc_send_cprb,
-};
-
-static struct zcrypt_ops zcrypt_pcixcc_with_rng_ops = {
- .rsa_modexpo = zcrypt_pcixcc_modexpo,
- .rsa_modexpo_crt = zcrypt_pcixcc_modexpo_crt,
- .send_cprb = zcrypt_pcixcc_send_cprb,
- .rng = zcrypt_pcixcc_rng,
-};
-
-/**
* Micro-code detection function. Its sends a message to a pcixcc card
* to find out the microcode level.
* @ap_dev: pointer to the AP device.
@@ -1083,9 +323,11 @@
return rc;
}
if (rc)
- zdev->ops = &zcrypt_pcixcc_with_rng_ops;
+ zdev->ops = zcrypt_msgtype_request(MSGTYPE06_NAME,
+ MSGTYPE06_VARIANT_DEFAULT);
else
- zdev->ops = &zcrypt_pcixcc_ops;
+ zdev->ops = zcrypt_msgtype_request(MSGTYPE06_NAME,
+ MSGTYPE06_VARIANT_NORNG);
ap_dev->reply = &zdev->reply;
ap_dev->private = zdev;
rc = zcrypt_device_register(zdev);
@@ -1095,6 +337,7 @@
out_free:
ap_dev->private = NULL;
+ zcrypt_msgtype_release(zdev->ops);
zcrypt_device_free(zdev);
return rc;
}
@@ -1106,8 +349,10 @@
static void zcrypt_pcixcc_remove(struct ap_device *ap_dev)
{
struct zcrypt_device *zdev = ap_dev->private;
+ struct zcrypt_ops *zops = zdev->ops;
zcrypt_device_unregister(zdev);
+ zcrypt_msgtype_release(zops);
}
int __init zcrypt_pcixcc_init(void)
diff --git a/drivers/s390/crypto/zcrypt_pcixcc.h b/drivers/s390/crypto/zcrypt_pcixcc.h
index c7cdf59..eacafc8 100644
--- a/drivers/s390/crypto/zcrypt_pcixcc.h
+++ b/drivers/s390/crypto/zcrypt_pcixcc.h
@@ -1,12 +1,13 @@
/*
* zcrypt 2.1.0
*
- * Copyright IBM Corp. 2001, 2006
+ * Copyright IBM Corp. 2001, 2012
* Author(s): Robert Burroughs
* Eric Rossman (edrossma@us.ibm.com)
*
* Hotplug & misc device support: Jochen Roehrig (roehrig@de.ibm.com)
* Major cleanup & driver split: Martin Schwidefsky <schwidefsky@de.ibm.com>
+ * MSGTYPE restruct: Holger Dengler <hd@linux.vnet.ibm.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c
index 7a8b096..cf6da7f 100644
--- a/drivers/s390/net/qeth_core_main.c
+++ b/drivers/s390/net/qeth_core_main.c
@@ -2993,7 +2993,7 @@
struct sysinfo_2_2_2 *info222 = (struct sysinfo_2_2_2 *)info;
struct sysinfo_3_2_2 *info322 = (struct sysinfo_3_2_2 *)info;
struct ccw_dev_id ccwid;
- int level, rc;
+ int level;
tid->chpid = card->info.chpid;
ccw_device_get_id(CARD_RDEV(card), &ccwid);
@@ -3001,17 +3001,10 @@
tid->devno = ccwid.devno;
if (!info)
return;
-
- rc = stsi(NULL, 0, 0, 0);
- if (rc == -ENOSYS)
- level = rc;
- else
- level = (((unsigned int) rc) >> 28);
-
- if ((level >= 2) && (stsi(info222, 2, 2, 2) != -ENOSYS))
+ level = stsi(NULL, 0, 0, 0);
+ if ((level >= 2) && (stsi(info222, 2, 2, 2) == 0))
tid->lparnr = info222->lpar_number;
-
- if ((level >= 3) && (stsi(info322, 3, 2, 2) != -ENOSYS)) {
+ if ((level >= 3) && (stsi(info322, 3, 2, 2) == 0)) {
EBCASC(info322->vm[0].name, sizeof(info322->vm[0].name));
memcpy(tid->vmname, info322->vm[0].name, sizeof(tid->vmname));
}
diff --git a/drivers/scsi/aic7xxx/aic79xx_core.c b/drivers/scsi/aic7xxx/aic79xx_core.c
index 25417d0..0bcacf7 100644
--- a/drivers/scsi/aic7xxx/aic79xx_core.c
+++ b/drivers/scsi/aic7xxx/aic79xx_core.c
@@ -2888,7 +2888,7 @@
ahd_outb(ahd, CLRINT, CLRSCSIINT);
ahd_unpause(ahd);
} else {
- printk("Reseting Channel for LQI Phase error\n");
+ printk("Resetting Channel for LQI Phase error\n");
ahd_dump_card_state(ahd);
ahd_reset_channel(ahd, 'A', /*Initiate Reset*/TRUE);
}
diff --git a/drivers/scsi/bfa/bfa_ioc.c b/drivers/scsi/bfa/bfa_ioc.c
index 8cdb79c..21ad290 100644
--- a/drivers/scsi/bfa/bfa_ioc.c
+++ b/drivers/scsi/bfa/bfa_ioc.c
@@ -5587,7 +5587,7 @@
static void bfa_dconf_init_cb(void *arg, bfa_status_t status);
/*
- * Begining state of dconf module. Waiting for an event to start.
+ * Beginning state of dconf module. Waiting for an event to start.
*/
static void
bfa_dconf_sm_uninit(struct bfa_dconf_mod_s *dconf, enum bfa_dconf_event event)
diff --git a/drivers/scsi/bfa/bfa_ioc.h b/drivers/scsi/bfa/bfa_ioc.h
index 1a99d4b..7b916e0 100644
--- a/drivers/scsi/bfa/bfa_ioc.h
+++ b/drivers/scsi/bfa/bfa_ioc.h
@@ -530,7 +530,7 @@
struct bfa_diag_qtest_result_s {
u32 status;
- u16 count; /* sucessful queue test count */
+ u16 count; /* successful queue test count */
u8 queue;
u8 rsvd; /* 64-bit align */
};
diff --git a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c
index ae1cb76..e055865 100644
--- a/drivers/scsi/bnx2fc/bnx2fc_fcoe.c
+++ b/drivers/scsi/bnx2fc/bnx2fc_fcoe.c
@@ -908,7 +908,7 @@
return;
default:
- printk(KERN_ERR PFX "Unkonwn netevent %ld", event);
+ printk(KERN_ERR PFX "Unknown netevent %ld", event);
return;
}
@@ -1738,7 +1738,7 @@
/**
* bnx2fc_ulp_start - cnic callback to initialize & start adapter instance
*
- * @handle: transport handle pointing to adapter struture
+ * @handle: transport handle pointing to adapter structure
*
* This function maps adapter structure to pcidev structure and initiates
* firmware handshake to enable/initialize on-chip FCoE components.
diff --git a/drivers/scsi/bnx2i/bnx2i_hwi.c b/drivers/scsi/bnx2i/bnx2i_hwi.c
index 33d6630..91eec60 100644
--- a/drivers/scsi/bnx2i/bnx2i_hwi.c
+++ b/drivers/scsi/bnx2i/bnx2i_hwi.c
@@ -1264,6 +1264,9 @@
int rc = 0;
u64 mask64;
+ memset(&iscsi_init, 0x00, sizeof(struct iscsi_kwqe_init1));
+ memset(&iscsi_init2, 0x00, sizeof(struct iscsi_kwqe_init2));
+
bnx2i_adjust_qp_size(hba);
iscsi_init.flags =
diff --git a/drivers/scsi/gdth.h b/drivers/scsi/gdth.h
index d3e4d7c..fbf6f0f4 100644
--- a/drivers/scsi/gdth.h
+++ b/drivers/scsi/gdth.h
@@ -49,15 +49,6 @@
/* GDT_ISA */
#define GDT2_ID 0x0120941c /* GDT2000/2020 */
-/* vendor ID, device IDs (PCI) */
-/* these defines should already exist in <linux/pci.h> */
-#ifndef PCI_VENDOR_ID_VORTEX
-#define PCI_VENDOR_ID_VORTEX 0x1119 /* PCI controller vendor ID */
-#endif
-#ifndef PCI_VENDOR_ID_INTEL
-#define PCI_VENDOR_ID_INTEL 0x8086
-#endif
-
#ifndef PCI_DEVICE_ID_VORTEX_GDT60x0
/* GDT_PCI */
#define PCI_DEVICE_ID_VORTEX_GDT60x0 0 /* GDT6000/6020/6050 */
diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index 796482b..2b4261c 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -1315,8 +1315,9 @@
}
break;
case CMD_PROTOCOL_ERR:
+ cmd->result = DID_ERROR << 16;
dev_warn(&h->pdev->dev, "cp %p has "
- "protocol error \n", cp);
+ "protocol error\n", cp);
break;
case CMD_HARDWARE_ERR:
cmd->result = DID_ERROR << 16;
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index 467dc38..0a2c5a8 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -192,7 +192,7 @@
{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE, IPR_USE_MSI, IPR_SIS64, IPR_MMIO, &ipr_chip_cfg[2] }
};
-static int ipr_max_bus_speeds [] = {
+static int ipr_max_bus_speeds[] = {
IPR_80MBs_SCSI_RATE, IPR_U160_SCSI_RATE, IPR_U320_SCSI_RATE
};
@@ -562,7 +562,7 @@
trace_entry->u.add_data = add_data;
}
#else
-#define ipr_trc_hook(ipr_cmd, type, add_data) do { } while(0)
+#define ipr_trc_hook(ipr_cmd, type, add_data) do { } while (0)
#endif
/**
@@ -1002,7 +1002,7 @@
**/
static void ipr_update_ata_class(struct ipr_resource_entry *res, unsigned int proto)
{
- switch(proto) {
+ switch (proto) {
case IPR_PROTO_SATA:
case IPR_PROTO_SAS_STP:
res->ata_class = ATA_DEV_ATA;
@@ -3043,7 +3043,7 @@
}
#else
-#define ipr_get_ioa_dump(ioa_cfg, dump) do { } while(0)
+#define ipr_get_ioa_dump(ioa_cfg, dump) do { } while (0)
#endif
/**
@@ -3055,7 +3055,7 @@
**/
static void ipr_release_dump(struct kref *kref)
{
- struct ipr_dump *dump = container_of(kref,struct ipr_dump,kref);
+ struct ipr_dump *dump = container_of(kref, struct ipr_dump, kref);
struct ipr_ioa_cfg *ioa_cfg = dump->ioa_cfg;
unsigned long lock_flags = 0;
int i;
@@ -3142,7 +3142,7 @@
break;
}
}
- } while(did_work);
+ } while (did_work);
list_for_each_entry(res, &ioa_cfg->used_res_q, queue) {
if (res->add_to_ml) {
@@ -3268,7 +3268,7 @@
* number of bytes printed to buffer
**/
static ssize_t ipr_store_log_level(struct device *dev,
- struct device_attribute *attr,
+ struct device_attribute *attr,
const char *buf, size_t count)
{
struct Scsi_Host *shost = class_to_shost(dev);
@@ -3315,7 +3315,7 @@
return -EACCES;
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
@@ -3682,7 +3682,7 @@
unsigned long lock_flags;
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
@@ -3746,7 +3746,7 @@
len = snprintf(fname, 99, "%s", buf);
fname[len-1] = '\0';
- if(request_firmware(&fw_entry, fname, &ioa_cfg->pdev->dev)) {
+ if (request_firmware(&fw_entry, fname, &ioa_cfg->pdev->dev)) {
dev_err(&ioa_cfg->pdev->dev, "Firmware file %s not found\n", fname);
return -EIO;
}
@@ -4612,7 +4612,7 @@
* Return value:
* SUCCESS / FAILED
**/
-static int __ipr_eh_host_reset(struct scsi_cmnd * scsi_cmd)
+static int __ipr_eh_host_reset(struct scsi_cmnd *scsi_cmd)
{
struct ipr_ioa_cfg *ioa_cfg;
int rc;
@@ -4634,7 +4634,7 @@
return rc;
}
-static int ipr_eh_host_reset(struct scsi_cmnd * cmd)
+static int ipr_eh_host_reset(struct scsi_cmnd *cmd)
{
int rc;
@@ -4701,7 +4701,7 @@
}
LEAVE;
- return (IPR_IOASC_SENSE_KEY(ioasc) ? -EIO : 0);
+ return IPR_IOASC_SENSE_KEY(ioasc) ? -EIO : 0;
}
/**
@@ -4725,7 +4725,7 @@
ENTER;
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
@@ -4753,7 +4753,7 @@
* Return value:
* SUCCESS / FAILED
**/
-static int __ipr_eh_dev_reset(struct scsi_cmnd * scsi_cmd)
+static int __ipr_eh_dev_reset(struct scsi_cmnd *scsi_cmd)
{
struct ipr_cmnd *ipr_cmd;
struct ipr_ioa_cfg *ioa_cfg;
@@ -4811,10 +4811,10 @@
res->resetting_device = 0;
LEAVE;
- return (rc ? FAILED : SUCCESS);
+ return rc ? FAILED : SUCCESS;
}
-static int ipr_eh_dev_reset(struct scsi_cmnd * cmd)
+static int ipr_eh_dev_reset(struct scsi_cmnd *cmd)
{
int rc;
@@ -4910,7 +4910,7 @@
* Return value:
* SUCCESS / FAILED
**/
-static int ipr_cancel_op(struct scsi_cmnd * scsi_cmd)
+static int ipr_cancel_op(struct scsi_cmnd *scsi_cmd)
{
struct ipr_cmnd *ipr_cmd;
struct ipr_ioa_cfg *ioa_cfg;
@@ -4979,7 +4979,7 @@
res->needs_sync_complete = 1;
LEAVE;
- return (IPR_IOASC_SENSE_KEY(ioasc) ? FAILED : SUCCESS);
+ return IPR_IOASC_SENSE_KEY(ioasc) ? FAILED : SUCCESS;
}
/**
@@ -4989,7 +4989,7 @@
* Return value:
* SUCCESS / FAILED
**/
-static int ipr_eh_abort(struct scsi_cmnd * scsi_cmd)
+static int ipr_eh_abort(struct scsi_cmnd *scsi_cmd)
{
unsigned long flags;
int rc;
@@ -5907,7 +5907,7 @@
* Return value:
* pointer to buffer with description string
**/
-static const char * ipr_ioa_info(struct Scsi_Host *host)
+static const char *ipr_ioa_info(struct Scsi_Host *host)
{
static char buffer[512];
struct ipr_ioa_cfg *ioa_cfg;
@@ -5965,7 +5965,7 @@
ENTER;
spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
@@ -6005,7 +6005,7 @@
unsigned long flags;
spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
@@ -6330,7 +6330,7 @@
int i;
if ((ioa_cfg->type == 0x5702) && (ioa_cfg->pdev->revision < 4)) {
- for (i = 0; i < ARRAY_SIZE(ipr_blocked_processors); i++){
+ for (i = 0; i < ARRAY_SIZE(ipr_blocked_processors); i++) {
if (__is_processor(ipr_blocked_processors[i]))
return 1;
}
@@ -6608,7 +6608,7 @@
* none
**/
static void ipr_modify_ioafp_mode_page_28(struct ipr_ioa_cfg *ioa_cfg,
- struct ipr_mode_pages *mode_pages)
+ struct ipr_mode_pages *mode_pages)
{
int i, entry_length;
struct ipr_dev_bus_entry *bus;
@@ -8022,7 +8022,7 @@
ipr_reinit_ipr_cmnd(ipr_cmd);
ipr_cmd->job_step_failed = ipr_reset_cmd_failed;
rc = ipr_cmd->job_step(ipr_cmd);
- } while(rc == IPR_RC_JOB_CONTINUE);
+ } while (rc == IPR_RC_JOB_CONTINUE);
}
/**
@@ -8283,7 +8283,7 @@
}
if (ioa_cfg->ipr_cmd_pool)
- pci_pool_destroy (ioa_cfg->ipr_cmd_pool);
+ pci_pool_destroy(ioa_cfg->ipr_cmd_pool);
kfree(ioa_cfg->ipr_cmnd_list);
kfree(ioa_cfg->ipr_cmnd_list_dma);
@@ -8363,8 +8363,8 @@
dma_addr_t dma_addr;
int i;
- ioa_cfg->ipr_cmd_pool = pci_pool_create (IPR_NAME, ioa_cfg->pdev,
- sizeof(struct ipr_cmnd), 512, 0);
+ ioa_cfg->ipr_cmd_pool = pci_pool_create(IPR_NAME, ioa_cfg->pdev,
+ sizeof(struct ipr_cmnd), 512, 0);
if (!ioa_cfg->ipr_cmd_pool)
return -ENOMEM;
@@ -8378,7 +8378,7 @@
}
for (i = 0; i < IPR_NUM_CMD_BLKS; i++) {
- ipr_cmd = pci_pool_alloc (ioa_cfg->ipr_cmd_pool, GFP_KERNEL, &dma_addr);
+ ipr_cmd = pci_pool_alloc(ioa_cfg->ipr_cmd_pool, GFP_KERNEL, &dma_addr);
if (!ipr_cmd) {
ipr_free_cmd_blks(ioa_cfg);
@@ -8964,7 +8964,7 @@
int target, lun;
for (target = 0; target < IPR_MAX_NUM_TARGETS_PER_BUS; target++)
- for (lun = 0; lun < IPR_MAX_NUM_VSET_LUNS_PER_TARGET; lun++ )
+ for (lun = 0; lun < IPR_MAX_NUM_VSET_LUNS_PER_TARGET; lun++)
scsi_add_device(ioa_cfg->host, IPR_VSET_BUS, target, lun);
}
@@ -9010,7 +9010,7 @@
ENTER;
spin_lock_irqsave(ioa_cfg->host->host_lock, host_lock_flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, host_lock_flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, host_lock_flags);
@@ -9139,7 +9139,7 @@
unsigned long lock_flags = 0;
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
- while(ioa_cfg->in_reset_reload) {
+ while (ioa_cfg->in_reset_reload) {
spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags);
wait_event(ioa_cfg->reset_wait_q, !ioa_cfg->in_reset_reload);
spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags);
@@ -9228,7 +9228,7 @@
};
MODULE_DEVICE_TABLE(pci, ipr_pci_table);
-static struct pci_error_handlers ipr_err_handler = {
+static const struct pci_error_handlers ipr_err_handler = {
.error_detected = ipr_pci_error_detected,
.slot_reset = ipr_pci_slot_reset,
};
diff --git a/drivers/scsi/isci/host.c b/drivers/scsi/isci/host.c
index 45385f5..b334fdc 100644
--- a/drivers/scsi/isci/host.c
+++ b/drivers/scsi/isci/host.c
@@ -492,7 +492,7 @@
u32 event_cycle;
dev_dbg(&ihost->pdev->dev,
- "%s: completion queue begining get:0x%08x\n",
+ "%s: completion queue beginning get:0x%08x\n",
__func__,
ihost->completion_queue_get);
diff --git a/drivers/scsi/isci/init.c b/drivers/scsi/isci/init.c
index 92c1d86..9be45a2 100644
--- a/drivers/scsi/isci/init.c
+++ b/drivers/scsi/isci/init.c
@@ -222,7 +222,7 @@
* @isci_host: This parameter specifies the lldd specific wrapper for the
* libsas sas_ha struct.
*
- * This method returns an error code indicating sucess or failure. The user
+ * This method returns an error code indicating success or failure. The user
* should check for possible memory allocation error return otherwise, a zero
* indicates success.
*/
diff --git a/drivers/scsi/isci/port.c b/drivers/scsi/isci/port.c
index 2fb85bf..13098b0 100644
--- a/drivers/scsi/isci/port.c
+++ b/drivers/scsi/isci/port.c
@@ -212,7 +212,7 @@
memcpy(iphy->sas_phy.attached_sas_addr,
iphy->frame_rcvd.iaf.sas_addr, SAS_ADDR_SIZE);
} else {
- dev_err(&isci_host->pdev->dev, "%s: unkown target\n", __func__);
+ dev_err(&isci_host->pdev->dev, "%s: unknown target\n", __func__);
success = false;
}
diff --git a/drivers/scsi/isci/request.c b/drivers/scsi/isci/request.c
index 7a0431c..c1bafc3 100644
--- a/drivers/scsi/isci/request.c
+++ b/drivers/scsi/isci/request.c
@@ -2240,7 +2240,7 @@
status = ireq->sci_status;
sci_change_state(&idev->sm, SCI_STP_DEV_ATAPI_ERROR);
} else {
- /* If receiving any non-sucess TC status, no UF
+ /* If receiving any non-success TC status, no UF
* received yet, then an UF for the status fis
* is coming after (XXX: suspect this is
* actually a protocol error or a bug like the
diff --git a/drivers/scsi/isci/task.c b/drivers/scsi/isci/task.c
index 6bc74eb..b6f19a1 100644
--- a/drivers/scsi/isci/task.c
+++ b/drivers/scsi/isci/task.c
@@ -532,7 +532,7 @@
/* The request has already completed and there
* is nothing to do here other than to set the task
* done bit, and indicate that the task abort function
- * was sucessful.
+ * was successful.
*/
spin_lock_irqsave(&task->task_state_lock, flags);
task->task_state_flags |= SAS_TASK_STATE_DONE;
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 45c1520..628a703 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -6607,7 +6607,7 @@
* we just use some constant number as place holder.
*
* Return codes
- * 0 - sucessful
+ * 0 - successful
* -ENOMEM - No availble memory
* -EIO - The mailbox failed to complete successfully.
**/
@@ -10425,7 +10425,7 @@
MODULE_DEVICE_TABLE(pci, lpfc_id_table);
-static struct pci_error_handlers lpfc_err_handler = {
+static const struct pci_error_handlers lpfc_err_handler = {
.error_detected = lpfc_io_error_detected,
.slot_reset = lpfc_io_slot_reset,
.resume = lpfc_io_resume,
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 9cbd20b..0e7e144 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -4739,7 +4739,7 @@
* is attached to.
*
* Return codes
- * 0 - sucessful
+ * 0 - successful
* otherwise - failed to retrieve physical port name
**/
static int
@@ -15209,7 +15209,7 @@
/*
* if next_fcf_pri was not set above and the list is not empty then
* we have failed flogis on all of them. So reset flogi failed
- * and start at the begining.
+ * and start at the beginning.
*/
if (!next_fcf_pri && !list_empty(&phba->fcf.fcf_pri_list)) {
list_for_each_entry(fcf_pri, &phba->fcf.fcf_pri_list, list) {
diff --git a/drivers/scsi/megaraid.c b/drivers/scsi/megaraid.c
index 97825f1..76ad72d 100644
--- a/drivers/scsi/megaraid.c
+++ b/drivers/scsi/megaraid.c
@@ -305,12 +305,11 @@
adapter->host->sg_tablesize = adapter->sglen;
-
/* use HP firmware and bios version encoding
Note: fw_version[0|1] and bios_version[0|1] were originally shifted
right 8 bits making them zero. This 0 value was hardcoded to fix
sparse warnings. */
- if (adapter->product_info.subsysvid == HP_SUBSYS_VID) {
+ if (adapter->product_info.subsysvid == PCI_VENDOR_ID_HP) {
sprintf (adapter->fw_version, "%c%d%d.%d%d",
adapter->product_info.fw_version[2],
0,
@@ -4716,7 +4715,7 @@
* support, since this firmware cannot handle 64 bit
* addressing
*/
- if ((subsysvid == HP_SUBSYS_VID) &&
+ if ((subsysvid == PCI_VENDOR_ID_HP) &&
((subsysid == 0x60E7) || (subsysid == 0x60E8))) {
/*
* which firmware
diff --git a/drivers/scsi/megaraid.h b/drivers/scsi/megaraid.h
index 9a7897f..4fb2adf 100644
--- a/drivers/scsi/megaraid.h
+++ b/drivers/scsi/megaraid.h
@@ -45,45 +45,10 @@
#define MAX_DEV_TYPE 32
-#ifndef PCI_VENDOR_ID_LSI_LOGIC
-#define PCI_VENDOR_ID_LSI_LOGIC 0x1000
-#endif
-
-#ifndef PCI_VENDOR_ID_AMI
-#define PCI_VENDOR_ID_AMI 0x101E
-#endif
-
-#ifndef PCI_VENDOR_ID_DELL
-#define PCI_VENDOR_ID_DELL 0x1028
-#endif
-
-#ifndef PCI_VENDOR_ID_INTEL
-#define PCI_VENDOR_ID_INTEL 0x8086
-#endif
-
-#ifndef PCI_DEVICE_ID_AMI_MEGARAID
-#define PCI_DEVICE_ID_AMI_MEGARAID 0x9010
-#endif
-
-#ifndef PCI_DEVICE_ID_AMI_MEGARAID2
-#define PCI_DEVICE_ID_AMI_MEGARAID2 0x9060
-#endif
-
-#ifndef PCI_DEVICE_ID_AMI_MEGARAID3
-#define PCI_DEVICE_ID_AMI_MEGARAID3 0x1960
-#endif
-
#define PCI_DEVICE_ID_DISCOVERY 0x000E
#define PCI_DEVICE_ID_PERC4_DI 0x000F
#define PCI_DEVICE_ID_PERC4_QC_VERDE 0x0407
-/* Sub-System Vendor IDs */
-#define AMI_SUBSYS_VID 0x101E
-#define DELL_SUBSYS_VID 0x1028
-#define HP_SUBSYS_VID 0x103C
-#define LSI_SUBSYS_VID 0x1000
-#define INTEL_SUBSYS_VID 0x8086
-
#define HBA_SIGNATURE 0x3344
#define HBA_SIGNATURE_471 0xCCCC
#define HBA_SIGNATURE_64BIT 0x0299
diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
index dc27598..ed38454 100644
--- a/drivers/scsi/megaraid/megaraid_sas_base.c
+++ b/drivers/scsi/megaraid/megaraid_sas_base.c
@@ -4066,7 +4066,6 @@
spin_lock_init(&instance->cmd_pool_lock);
spin_lock_init(&instance->hba_lock);
spin_lock_init(&instance->completion_lock);
- spin_lock_init(&poll_aen_lock);
mutex_init(&instance->aen_mutex);
mutex_init(&instance->reset_mutex);
@@ -5392,6 +5391,8 @@
printk(KERN_INFO "megasas: %s %s\n", MEGASAS_VERSION,
MEGASAS_EXT_VERSION);
+ spin_lock_init(&poll_aen_lock);
+
support_poll_for_event = 2;
support_device_change = 1;
diff --git a/drivers/scsi/mpt2sas/mpt2sas_base.c b/drivers/scsi/mpt2sas/mpt2sas_base.c
index 9d46fcb..9d5a56c 100644
--- a/drivers/scsi/mpt2sas/mpt2sas_base.c
+++ b/drivers/scsi/mpt2sas/mpt2sas_base.c
@@ -1209,6 +1209,13 @@
u16 message_control;
+ /* Check whether controller SAS2008 B0 controller,
+ if it is SAS2008 B0 controller use IO-APIC instead of MSIX */
+ if (ioc->pdev->device == MPI2_MFGPAGE_DEVID_SAS2008 &&
+ ioc->pdev->revision == 0x01) {
+ return -EINVAL;
+ }
+
base = pci_find_capability(ioc->pdev, PCI_CAP_ID_MSIX);
if (!base) {
dfailprintk(ioc, printk(MPT2SAS_INFO_FMT "msix not "
@@ -2424,10 +2431,13 @@
}
/* command line tunables for max controller queue depth */
- if (max_queue_depth != -1)
- max_request_credit = (max_queue_depth < facts->RequestCredit)
- ? max_queue_depth : facts->RequestCredit;
- else
+ if (max_queue_depth != -1 && max_queue_depth != 0) {
+ max_request_credit = min_t(u16, max_queue_depth +
+ ioc->hi_priority_depth + ioc->internal_depth,
+ facts->RequestCredit);
+ if (max_request_credit > MAX_HBA_QUEUE_DEPTH)
+ max_request_credit = MAX_HBA_QUEUE_DEPTH;
+ } else
max_request_credit = min_t(u16, facts->RequestCredit,
MAX_HBA_QUEUE_DEPTH);
@@ -2502,7 +2512,7 @@
/* set the scsi host can_queue depth
* with some internal commands that could be outstanding
*/
- ioc->shost->can_queue = ioc->scsiio_depth - (2);
+ ioc->shost->can_queue = ioc->scsiio_depth;
dinitprintk(ioc, printk(MPT2SAS_INFO_FMT "scsi host: "
"can_queue depth (%d)\n", ioc->name, ioc->shost->can_queue));
diff --git a/drivers/scsi/mpt2sas/mpt2sas_scsih.c b/drivers/scsi/mpt2sas/mpt2sas_scsih.c
index b1ebd6f..1ccae45 100644
--- a/drivers/scsi/mpt2sas/mpt2sas_scsih.c
+++ b/drivers/scsi/mpt2sas/mpt2sas_scsih.c
@@ -8306,7 +8306,7 @@
return PCI_ERS_RESULT_NEED_RESET;
}
-static struct pci_error_handlers _scsih_err_handler = {
+static const struct pci_error_handlers _scsih_err_handler = {
.error_detected = _scsih_pci_error_detected,
.mmio_enabled = _scsih_pci_mmio_enabled,
.slot_reset = _scsih_pci_slot_reset,
diff --git a/drivers/scsi/mvumi.c b/drivers/scsi/mvumi.c
index 88cf1db..783edc7 100644
--- a/drivers/scsi/mvumi.c
+++ b/drivers/scsi/mvumi.c
@@ -122,7 +122,7 @@
if (!res) {
dev_err(&mhba->pdev->dev,
- "Failed to allocate memory for resouce manager.\n");
+ "Failed to allocate memory for resource manager.\n");
return NULL;
}
@@ -1007,13 +1007,13 @@
tmp |= INT_MAP_COMAOUT | INT_MAP_COMAERR;
iowrite32(tmp, regs + CPU_ENPOINTA_MASK_REG);
iowrite32(mhba->list_num_io, mhba->ib_shadow);
- /* Set InBound List Avaliable count shadow */
+ /* Set InBound List Available count shadow */
iowrite32(lower_32_bits(mhba->ib_shadow_phys),
regs + CLA_INB_AVAL_COUNT_BASEL);
iowrite32(upper_32_bits(mhba->ib_shadow_phys),
regs + CLA_INB_AVAL_COUNT_BASEH);
- /* Set OutBound List Avaliable count shadow */
+ /* Set OutBound List Available count shadow */
iowrite32((mhba->list_num_io-1) | CL_POINTER_TOGGLE,
mhba->ob_shadow);
iowrite32(lower_32_bits(mhba->ob_shadow_phys), regs + 0x5B0);
diff --git a/drivers/scsi/qla2xxx/qla_nx.c b/drivers/scsi/qla2xxx/qla_nx.c
index 9ce3a8f..7cfdf2b 100644
--- a/drivers/scsi/qla2xxx/qla_nx.c
+++ b/drivers/scsi/qla2xxx/qla_nx.c
@@ -1615,13 +1615,11 @@
char *
qla82xx_pci_info_str(struct scsi_qla_host *vha, char *str)
{
- int pcie_reg;
struct qla_hw_data *ha = vha->hw;
char lwstr[6];
uint16_t lnk;
- pcie_reg = pci_pcie_cap(ha->pdev);
- pci_read_config_word(ha->pdev, pcie_reg + PCI_EXP_LNKSTA, &lnk);
+ pcie_capability_read_word(ha->pdev, PCI_EXP_LNKSTA, &lnk);
ha->link_width = (lnk >> 4) & 0x3f;
strcpy(str, "PCIe (");
@@ -2497,7 +2495,6 @@
int
qla82xx_start_firmware(scsi_qla_host_t *vha)
{
- int pcie_cap;
uint16_t lnk;
struct qla_hw_data *ha = vha->hw;
@@ -2528,8 +2525,7 @@
}
/* Negotiated Link width */
- pcie_cap = pci_pcie_cap(ha->pdev);
- pci_read_config_word(ha->pdev, pcie_cap + PCI_EXP_LNKSTA, &lnk);
+ pcie_capability_read_word(ha->pdev, PCI_EXP_LNKSTA, &lnk);
ha->link_width = (lnk >> 4) & 0x3f;
/* Synchronize with Receive peg */
diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index fb8cd38..d305262 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -4471,7 +4471,7 @@
ha->flags.eeh_busy = 0;
}
-static struct pci_error_handlers qla2xxx_err_handler = {
+static const struct pci_error_handlers qla2xxx_err_handler = {
.error_detected = qla2xxx_pci_error_detected,
.mmio_enabled = qla2xxx_pci_mmio_enabled,
.slot_reset = qla2xxx_pci_slot_reset,
diff --git a/drivers/scsi/qla4xxx/ql4_nx.c b/drivers/scsi/qla4xxx/ql4_nx.c
index 939d726..807bf76 100644
--- a/drivers/scsi/qla4xxx/ql4_nx.c
+++ b/drivers/scsi/qla4xxx/ql4_nx.c
@@ -1566,7 +1566,6 @@
static int
qla4_8xxx_start_firmware(struct scsi_qla_host *ha, uint32_t image_start)
{
- int pcie_cap;
uint16_t lnk;
/* scrub dma mask expansion register */
@@ -1590,8 +1589,7 @@
}
/* Negotiated Link width */
- pcie_cap = pci_pcie_cap(ha->pdev);
- pci_read_config_word(ha->pdev, pcie_cap + PCI_EXP_LNKSTA, &lnk);
+ pcie_capability_read_word(ha->pdev, PCI_EXP_LNKSTA, &lnk);
ha->link_width = (lnk >> 4) & 0x3f;
/* Synchronize with Receive peg */
diff --git a/drivers/scsi/qla4xxx/ql4_os.c b/drivers/scsi/qla4xxx/ql4_os.c
index 9da4266..79243b7 100644
--- a/drivers/scsi/qla4xxx/ql4_os.c
+++ b/drivers/scsi/qla4xxx/ql4_os.c
@@ -803,7 +803,7 @@
iscsi_stats_dma);
if (ret != QLA_SUCCESS) {
ql4_printk(KERN_ERR, ha,
- "Unable to retreive iscsi stats\n");
+ "Unable to retrieve iscsi stats\n");
goto free_stats;
}
@@ -4338,7 +4338,7 @@
return QLA_ERROR;
/* For multi sessions, driver generates the ISID, so do not compare
- * ISID in reset path since it would be a comparision between the
+ * ISID in reset path since it would be a comparison between the
* driver generated ISID and firmware generated ISID. This could
* lead to adding duplicated DDBs in the list as driver generated
* ISID would not match firmware generated ISID.
@@ -5326,7 +5326,7 @@
}
}
/**
- * qla4xxx_remove_adapter - calback function to remove adapter.
+ * qla4xxx_remove_adapter - callback function to remove adapter.
* @pci_dev: PCI device pointer
**/
static void __devexit qla4xxx_remove_adapter(struct pci_dev *pdev)
@@ -6148,7 +6148,7 @@
clear_bit(AF_EEH_BUSY, &ha->flags);
}
-static struct pci_error_handlers qla4xxx_err_handler = {
+static const struct pci_error_handlers qla4xxx_err_handler = {
.error_detected = qla4xxx_pci_error_detected,
.mmio_enabled = qla4xxx_pci_mmio_enabled,
.slot_reset = qla4xxx_pci_slot_reset,
diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index 4a6381c..de2337f 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -42,6 +42,8 @@
#include <trace/events/scsi.h>
+static void scsi_eh_done(struct scsi_cmnd *scmd);
+
#define SENSE_TIMEOUT (10*HZ)
/*
@@ -241,6 +243,14 @@
if (! scsi_command_normalize_sense(scmd, &sshdr))
return FAILED; /* no valid sense data */
+ if (scmd->cmnd[0] == TEST_UNIT_READY && scmd->scsi_done != scsi_eh_done)
+ /*
+ * nasty: for mid-layer issued TURs, we need to return the
+ * actual sense data without any recovery attempt. For eh
+ * issued ones, we need to try to recover and interpret
+ */
+ return SUCCESS;
+
if (scsi_sense_is_deferred(&sshdr))
return NEEDS_RETRY;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index ffd7773..faa790f 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -776,7 +776,6 @@
}
if (req->cmd_type == REQ_TYPE_BLOCK_PC) { /* SG_IO ioctl from block level */
- req->errors = result;
if (result) {
if (sense_valid && req->sense) {
/*
@@ -792,6 +791,10 @@
if (!sense_deferred)
error = __scsi_error_from_host_byte(cmd, result);
}
+ /*
+ * __scsi_error_from_host_byte may have reset the host_byte
+ */
+ req->errors = cmd->result;
req->resid_len = scsi_get_resid(cmd);
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 56a9379..d947ffc 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -764,6 +764,16 @@
sdev->model = (char *) (sdev->inquiry + 16);
sdev->rev = (char *) (sdev->inquiry + 32);
+ if (strncmp(sdev->vendor, "ATA ", 8) == 0) {
+ /*
+ * sata emulation layer device. This is a hack to work around
+ * the SATL power management specifications which state that
+ * when the SATL detects the device has gone into standby
+ * mode, it shall respond with NOT READY.
+ */
+ sdev->allow_restart = 1;
+ }
+
if (*bflags & BLIST_ISROM) {
sdev->type = TYPE_ROM;
sdev->removable = 1;
diff --git a/drivers/scsi/sym53c8xx_2/sym_glue.c b/drivers/scsi/sym53c8xx_2/sym_glue.c
index 36d1ed7..e2b8e68 100644
--- a/drivers/scsi/sym53c8xx_2/sym_glue.c
+++ b/drivers/scsi/sym53c8xx_2/sym_glue.c
@@ -2117,7 +2117,7 @@
MODULE_DEVICE_TABLE(pci, sym2_id_table);
-static struct pci_error_handlers sym2_err_handler = {
+static const struct pci_error_handlers sym2_err_handler = {
.error_detected = sym2_io_error_detected,
.mmio_enabled = sym2_io_slot_dump,
.slot_reset = sym2_io_slot_reset,
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c
index c7030fb..3e79a2f 100644
--- a/drivers/scsi/virtio_scsi.c
+++ b/drivers/scsi/virtio_scsi.c
@@ -331,7 +331,7 @@
int i;
for_each_sg(table->sgl, sg_elem, table->nents, i)
- sg_set_buf(&sg[idx++], sg_virt(sg_elem), sg_elem->length);
+ sg[idx++] = *sg_elem;
*p_idx = idx;
}
diff --git a/drivers/scsi/vmw_pvscsi.c b/drivers/scsi/vmw_pvscsi.c
index 4411d42..20b3a48 100644
--- a/drivers/scsi/vmw_pvscsi.c
+++ b/drivers/scsi/vmw_pvscsi.c
@@ -295,7 +295,7 @@
static void ll_bus_reset(const struct pvscsi_adapter *adapter)
{
- dev_dbg(pvscsi_dev(adapter), "Reseting bus on %p\n", adapter);
+ dev_dbg(pvscsi_dev(adapter), "Resetting bus on %p\n", adapter);
pvscsi_write_cmd_desc(adapter, PVSCSI_CMD_RESET_BUS, NULL, 0);
}
@@ -304,7 +304,7 @@
{
struct PVSCSICmdDescResetDevice cmd = { 0 };
- dev_dbg(pvscsi_dev(adapter), "Reseting device: target=%u\n", target);
+ dev_dbg(pvscsi_dev(adapter), "Resetting device: target=%u\n", target);
cmd.target = target;
diff --git a/drivers/sh/intc/core.c b/drivers/sh/intc/core.c
index 32c26d7..8f32a13 100644
--- a/drivers/sh/intc/core.c
+++ b/drivers/sh/intc/core.c
@@ -355,7 +355,7 @@
if (unlikely(res)) {
if (res == -EEXIST) {
res = irq_domain_associate(d->domain,
- irq, irq);
+ irq2, irq2);
if (unlikely(res)) {
pr_err("domain association "
"failure\n");
diff --git a/drivers/sh/pfc/pinctrl.c b/drivers/sh/pfc/pinctrl.c
index 2804eaa..0646bf6 100644
--- a/drivers/sh/pfc/pinctrl.c
+++ b/drivers/sh/pfc/pinctrl.c
@@ -208,10 +208,13 @@
break;
case PINMUX_TYPE_GPIO:
+ case PINMUX_TYPE_INPUT:
+ case PINMUX_TYPE_OUTPUT:
break;
default:
pr_err("Unsupported mux type (%d), bailing...\n", pinmux_type);
- return -ENOTSUPP;
+ ret = -ENOTSUPP;
+ goto err;
}
ret = 0;
diff --git a/drivers/spi/spi-au1550.c b/drivers/spi/spi-au1550.c
index 5784c87..4de66d1 100644
--- a/drivers/spi/spi-au1550.c
+++ b/drivers/spi/spi-au1550.c
@@ -475,7 +475,7 @@
/*
* due to an spi error we consider transfer as done,
* so mask all events until before next transfer start
- * and stop the possibly running dma immediatelly
+ * and stop the possibly running dma immediately
*/
au1550_spi_mask_ack_all(hw);
au1xxx_dbdma_stop(hw->dma_rx_ch);
diff --git a/drivers/spi/spi-bcm63xx.c b/drivers/spi/spi-bcm63xx.c
index ea0aaa3..a9f4049 100644
--- a/drivers/spi/spi-bcm63xx.c
+++ b/drivers/spi/spi-bcm63xx.c
@@ -47,6 +47,8 @@
/* Platform data */
u32 speed_hz;
unsigned fifo_size;
+ unsigned int msg_type_shift;
+ unsigned int msg_ctl_width;
/* Data buffers */
const unsigned char *tx_ptr;
@@ -221,13 +223,20 @@
msg_ctl = (t->len << SPI_BYTE_CNT_SHIFT);
if (t->rx_buf && t->tx_buf)
- msg_ctl |= (SPI_FD_RW << SPI_MSG_TYPE_SHIFT);
+ msg_ctl |= (SPI_FD_RW << bs->msg_type_shift);
else if (t->rx_buf)
- msg_ctl |= (SPI_HD_R << SPI_MSG_TYPE_SHIFT);
+ msg_ctl |= (SPI_HD_R << bs->msg_type_shift);
else if (t->tx_buf)
- msg_ctl |= (SPI_HD_W << SPI_MSG_TYPE_SHIFT);
+ msg_ctl |= (SPI_HD_W << bs->msg_type_shift);
- bcm_spi_writew(bs, msg_ctl, SPI_MSG_CTL);
+ switch (bs->msg_ctl_width) {
+ case 8:
+ bcm_spi_writeb(bs, msg_ctl, SPI_MSG_CTL);
+ break;
+ case 16:
+ bcm_spi_writew(bs, msg_ctl, SPI_MSG_CTL);
+ break;
+ }
/* Issue the transfer */
cmd = SPI_CMD_START_IMMEDIATE;
@@ -406,9 +415,21 @@
master->transfer_one_message = bcm63xx_spi_transfer_one;
master->mode_bits = MODEBITS;
bs->speed_hz = pdata->speed_hz;
+ bs->msg_type_shift = pdata->msg_type_shift;
+ bs->msg_ctl_width = pdata->msg_ctl_width;
bs->tx_io = (u8 *)(bs->regs + bcm63xx_spireg(SPI_MSG_DATA));
bs->rx_io = (const u8 *)(bs->regs + bcm63xx_spireg(SPI_RX_DATA));
+ switch (bs->msg_ctl_width) {
+ case 8:
+ case 16:
+ break;
+ default:
+ dev_err(dev, "unsupported MSG_CTL width: %d\n",
+ bs->msg_ctl_width);
+ goto out_clk_disable;
+ }
+
/* Initialize hardware */
clk_enable(bs->clk);
bcm_spi_writeb(bs, SPI_INTR_CLEAR_ALL, SPI_INT_STATUS);
diff --git a/drivers/spi/spi-bfin-sport.c b/drivers/spi/spi-bfin-sport.c
index 1fe5119..6555ecd 100644
--- a/drivers/spi/spi-bfin-sport.c
+++ b/drivers/spi/spi-bfin-sport.c
@@ -467,7 +467,7 @@
dev_dbg(drv_data->dev, "IO write error!\n");
drv_data->state = ERROR_STATE;
} else {
- /* Update total byte transfered */
+ /* Update total byte transferred */
message->actual_length += transfer->len;
/* Move to next transfer of this msg */
drv_data->state = bfin_sport_spi_next_transfer(drv_data);
diff --git a/drivers/spi/spi-oc-tiny.c b/drivers/spi/spi-oc-tiny.c
index 698018f..9d9071b 100644
--- a/drivers/spi/spi-oc-tiny.c
+++ b/drivers/spi/spi-oc-tiny.c
@@ -129,7 +129,7 @@
unsigned int i;
if (hw->irq >= 0) {
- /* use intrrupt driven data transfer */
+ /* use interrupt driven data transfer */
hw->len = t->len;
hw->txp = t->tx_buf;
hw->rxp = t->rx_buf;
diff --git a/drivers/spi/spi-ppc4xx.c b/drivers/spi/spi-ppc4xx.c
index 75ac9d4..7a85f22 100644
--- a/drivers/spi/spi-ppc4xx.c
+++ b/drivers/spi/spi-ppc4xx.c
@@ -101,7 +101,7 @@
u8 dummy;
/*
* Clock divisor modulus register
- * This uses the follwing formula:
+ * This uses the following formula:
* SCPClkOut = OPBCLK/(4(CDM + 1))
* or
* CDM = (OPBCLK/4*SCPClkOut) - 1
@@ -201,7 +201,7 @@
return -EINVAL;
}
- /* Write new configration */
+ /* Write new configuration */
out_8(&hw->regs->mode, cs->mode);
/* Set the clock */
diff --git a/drivers/spi/spi-topcliff-pch.c b/drivers/spi/spi-topcliff-pch.c
index cd56dcf..1284c9b 100644
--- a/drivers/spi/spi-topcliff-pch.c
+++ b/drivers/spi/spi-topcliff-pch.c
@@ -505,7 +505,7 @@
}
if (unlikely(pspi->max_speed_hz == 0)) {
- dev_err(&pspi->dev, "%s pch_spi_tranfer maxspeed=%d\n",
+ dev_err(&pspi->dev, "%s pch_spi_transfer maxspeed=%d\n",
__func__, pspi->max_speed_hz);
retval = -EINVAL;
goto err_out;
diff --git a/drivers/staging/android/android_alarm.h b/drivers/staging/android/android_alarm.h
index d0cafd6..f2ffd96 100644
--- a/drivers/staging/android/android_alarm.h
+++ b/drivers/staging/android/android_alarm.h
@@ -51,10 +51,12 @@
#define ANDROID_ALARM_WAIT _IO('a', 1)
#define ALARM_IOW(c, type, size) _IOW('a', (c) | ((type) << 4), size)
+#define ALARM_IOR(c, type, size) _IOR('a', (c) | ((type) << 4), size)
+
/* Set alarm */
#define ANDROID_ALARM_SET(type) ALARM_IOW(2, type, struct timespec)
#define ANDROID_ALARM_SET_AND_WAIT(type) ALARM_IOW(3, type, struct timespec)
-#define ANDROID_ALARM_GET_TIME(type) ALARM_IOW(4, type, struct timespec)
+#define ANDROID_ALARM_GET_TIME(type) ALARM_IOR(4, type, struct timespec)
#define ANDROID_ALARM_SET_RTC _IOW('a', 5, struct timespec)
#define ANDROID_ALARM_BASE_CMD(cmd) (cmd & ~(_IOC(0, 0, 0xf0, 0)))
#define ANDROID_ALARM_IOCTL_TO_TYPE(cmd) (_IOC_NR(cmd) >> 4)
diff --git a/drivers/staging/comedi/drivers/amplc_dio200.c b/drivers/staging/comedi/drivers/amplc_dio200.c
index 6c81e377..cc8931f 100644
--- a/drivers/staging/comedi/drivers/amplc_dio200.c
+++ b/drivers/staging/comedi/drivers/amplc_dio200.c
@@ -1412,6 +1412,13 @@
dev_err(dev->class_dev, "BUG! cannot determine board type!\n");
return -EINVAL;
}
+ /*
+ * Need to 'get' the PCI device to match the 'put' in dio200_detach().
+ * TODO: Remove the pci_dev_get() and matching pci_dev_put() once
+ * support for manual attachment of PCI devices via dio200_attach()
+ * has been removed.
+ */
+ pci_dev_get(pci_dev);
return dio200_pci_common_attach(dev, pci_dev);
}
diff --git a/drivers/staging/comedi/drivers/amplc_pc236.c b/drivers/staging/comedi/drivers/amplc_pc236.c
index aabba98..f502879 100644
--- a/drivers/staging/comedi/drivers/amplc_pc236.c
+++ b/drivers/staging/comedi/drivers/amplc_pc236.c
@@ -565,6 +565,13 @@
dev_err(dev->class_dev, "BUG! cannot determine board type!\n");
return -EINVAL;
}
+ /*
+ * Need to 'get' the PCI device to match the 'put' in pc236_detach().
+ * TODO: Remove the pci_dev_get() and matching pci_dev_put() once
+ * support for manual attachment of PCI devices via pc236_attach()
+ * has been removed.
+ */
+ pci_dev_get(pci_dev);
return pc236_pci_common_attach(dev, pci_dev);
}
diff --git a/drivers/staging/comedi/drivers/amplc_pc263.c b/drivers/staging/comedi/drivers/amplc_pc263.c
index 40ec1ff..8191c4e 100644
--- a/drivers/staging/comedi/drivers/amplc_pc263.c
+++ b/drivers/staging/comedi/drivers/amplc_pc263.c
@@ -298,6 +298,13 @@
dev_err(dev->class_dev, "BUG! cannot determine board type!\n");
return -EINVAL;
}
+ /*
+ * Need to 'get' the PCI device to match the 'put' in pc263_detach().
+ * TODO: Remove the pci_dev_get() and matching pci_dev_put() once
+ * support for manual attachment of PCI devices via pc263_attach()
+ * has been removed.
+ */
+ pci_dev_get(pci_dev);
return pc263_pci_common_attach(dev, pci_dev);
}
diff --git a/drivers/staging/comedi/drivers/amplc_pci224.c b/drivers/staging/comedi/drivers/amplc_pci224.c
index 4e17f13..8bf109e 100644
--- a/drivers/staging/comedi/drivers/amplc_pci224.c
+++ b/drivers/staging/comedi/drivers/amplc_pci224.c
@@ -1503,6 +1503,13 @@
DRIVER_NAME ": BUG! cannot determine board type!\n");
return -EINVAL;
}
+ /*
+ * Need to 'get' the PCI device to match the 'put' in pci224_detach().
+ * TODO: Remove the pci_dev_get() and matching pci_dev_put() once
+ * support for manual attachment of PCI devices via pci224_attach()
+ * has been removed.
+ */
+ pci_dev_get(pci_dev);
return pci224_attach_common(dev, pci_dev, NULL);
}
diff --git a/drivers/staging/comedi/drivers/amplc_pci230.c b/drivers/staging/comedi/drivers/amplc_pci230.c
index 1b67d0c..66e74bd 100644
--- a/drivers/staging/comedi/drivers/amplc_pci230.c
+++ b/drivers/staging/comedi/drivers/amplc_pci230.c
@@ -2925,6 +2925,13 @@
"amplc_pci230: BUG! cannot determine board type!\n");
return -EINVAL;
}
+ /*
+ * Need to 'get' the PCI device to match the 'put' in pci230_detach().
+ * TODO: Remove the pci_dev_get() and matching pci_dev_put() once
+ * support for manual attachment of PCI devices via pci230_attach()
+ * has been removed.
+ */
+ pci_dev_get(pci_dev);
return pci230_attach_common(dev, pci_dev);
}
diff --git a/drivers/staging/comedi/drivers/das08.c b/drivers/staging/comedi/drivers/das08.c
index 874e02e..67a914a 100644
--- a/drivers/staging/comedi/drivers/das08.c
+++ b/drivers/staging/comedi/drivers/das08.c
@@ -378,7 +378,7 @@
int chan;
lsb = data[0] & 0xff;
- msb = (data[0] >> 8) & 0xf;
+ msb = (data[0] >> 8) & 0xff;
chan = CR_CHAN(insn->chanspec);
@@ -623,7 +623,7 @@
.ai = das08_ai_rinsn,
.ai_nbits = 16,
.ai_pg = das08_pg_none,
- .ai_encoding = das08_encode12,
+ .ai_encoding = das08_encode16,
.ao = das08jr_ao_winsn,
.ao_nbits = 16,
.di = das08jr_di_rbits,
@@ -922,6 +922,13 @@
dev_err(dev->class_dev, "BUG! cannot determine board type!\n");
return -EINVAL;
}
+ /*
+ * Need to 'get' the PCI device to match the 'put' in das08_detach().
+ * TODO: Remove the pci_dev_get() and matching pci_dev_put() once
+ * support for manual attachment of PCI devices via das08_attach()
+ * has been removed.
+ */
+ pci_dev_get(pdev);
return das08_pci_attach_common(dev, pdev);
}
diff --git a/drivers/staging/et131x/et131x.c b/drivers/staging/et131x/et131x.c
index 029725c..49553f8 100644
--- a/drivers/staging/et131x/et131x.c
+++ b/drivers/staging/et131x/et131x.c
@@ -3995,16 +3995,14 @@
static int et131x_pci_init(struct et131x_adapter *adapter,
struct pci_dev *pdev)
{
- int cap = pci_pcie_cap(pdev);
u16 max_payload;
- u16 ctl;
int i, rc;
rc = et131x_init_eeprom(adapter);
if (rc < 0)
goto out;
- if (!cap) {
+ if (!pci_is_pcie(pdev)) {
dev_err(&pdev->dev, "Missing PCIe capabilities\n");
goto err_out;
}
@@ -4012,7 +4010,7 @@
/* Let's set up the PORT LOGIC Register. First we need to know what
* the max_payload_size is
*/
- if (pci_read_config_word(pdev, cap + PCI_EXP_DEVCAP, &max_payload)) {
+ if (pcie_capability_read_word(pdev, PCI_EXP_DEVCAP, &max_payload)) {
dev_err(&pdev->dev,
"Could not read PCI config space for Max Payload Size\n");
goto err_out;
@@ -4049,17 +4047,10 @@
}
/* Change the max read size to 2k */
- if (pci_read_config_word(pdev, cap + PCI_EXP_DEVCTL, &ctl)) {
+ if (pcie_capability_clear_and_set_word(pdev, PCI_EXP_DEVCTL,
+ PCI_EXP_DEVCTL_READRQ, 0x4 << 12)) {
dev_err(&pdev->dev,
- "Could not read PCI config space for Max read size\n");
- goto err_out;
- }
-
- ctl = (ctl & ~PCI_EXP_DEVCTL_READRQ) | (0x04 << 12);
-
- if (pci_write_config_word(pdev, cap + PCI_EXP_DEVCTL, ctl)) {
- dev_err(&pdev->dev,
- "Could not write PCI config space for Max read size\n");
+ "Couldn't change PCI config space for Max read size\n");
goto err_out;
}
diff --git a/drivers/staging/iio/accel/lis3l02dq_ring.c b/drivers/staging/iio/accel/lis3l02dq_ring.c
index 18d108f..f3da590 100644
--- a/drivers/staging/iio/accel/lis3l02dq_ring.c
+++ b/drivers/staging/iio/accel/lis3l02dq_ring.c
@@ -121,8 +121,10 @@
if (rx_array == NULL)
return -ENOMEM;
ret = lis3l02dq_read_all(indio_dev, rx_array);
- if (ret < 0)
+ if (ret < 0) {
+ kfree(rx_array);
return ret;
+ }
for (i = 0; i < scan_count; i++)
data[i] = combine_8_to_16(rx_array[i*4+1],
rx_array[i*4+3]);
diff --git a/drivers/staging/iio/adc/ad7192.c b/drivers/staging/iio/adc/ad7192.c
index 0958372..19a064d 100644
--- a/drivers/staging/iio/adc/ad7192.c
+++ b/drivers/staging/iio/adc/ad7192.c
@@ -647,6 +647,8 @@
ret = strict_strtoul(buf, 10, &lval);
if (ret)
return ret;
+ if (lval == 0)
+ return -EINVAL;
mutex_lock(&indio_dev->mlock);
if (iio_buffer_enabled(indio_dev)) {
diff --git a/drivers/staging/iio/gyro/adis16260_core.c b/drivers/staging/iio/gyro/adis16260_core.c
index 93aa431..eb8e9d6 100644
--- a/drivers/staging/iio/gyro/adis16260_core.c
+++ b/drivers/staging/iio/gyro/adis16260_core.c
@@ -195,6 +195,8 @@
ret = strict_strtol(buf, 10, &val);
if (ret)
return ret;
+ if (val == 0)
+ return -EINVAL;
mutex_lock(&indio_dev->mlock);
if (spi_get_device_id(st->us)) {
diff --git a/drivers/staging/iio/imu/adis16400_core.c b/drivers/staging/iio/imu/adis16400_core.c
index 1f4c177..a618327 100644
--- a/drivers/staging/iio/imu/adis16400_core.c
+++ b/drivers/staging/iio/imu/adis16400_core.c
@@ -234,6 +234,8 @@
ret = strict_strtol(buf, 10, &val);
if (ret)
return ret;
+ if (val == 0)
+ return -EINVAL;
mutex_lock(&indio_dev->mlock);
diff --git a/drivers/staging/iio/meter/ade7753.c b/drivers/staging/iio/meter/ade7753.c
index f04ece7..3ccff18 100644
--- a/drivers/staging/iio/meter/ade7753.c
+++ b/drivers/staging/iio/meter/ade7753.c
@@ -425,6 +425,8 @@
ret = strict_strtol(buf, 10, &val);
if (ret)
return ret;
+ if (val == 0)
+ return -EINVAL;
mutex_lock(&indio_dev->mlock);
diff --git a/drivers/staging/iio/meter/ade7754.c b/drivers/staging/iio/meter/ade7754.c
index 6cee28a..abb1e9c 100644
--- a/drivers/staging/iio/meter/ade7754.c
+++ b/drivers/staging/iio/meter/ade7754.c
@@ -445,6 +445,8 @@
ret = strict_strtol(buf, 10, &val);
if (ret)
return ret;
+ if (val == 0)
+ return -EINVAL;
mutex_lock(&indio_dev->mlock);
diff --git a/drivers/staging/iio/meter/ade7759.c b/drivers/staging/iio/meter/ade7759.c
index b3f7e0fa9..eb0a2a9 100644
--- a/drivers/staging/iio/meter/ade7759.c
+++ b/drivers/staging/iio/meter/ade7759.c
@@ -385,6 +385,8 @@
ret = strict_strtol(buf, 10, &val);
if (ret)
return ret;
+ if (val == 0)
+ return -EINVAL;
mutex_lock(&indio_dev->mlock);
diff --git a/drivers/staging/nvec/nvec.c b/drivers/staging/nvec/nvec.c
index 695ea35..d0a7e40 100644
--- a/drivers/staging/nvec/nvec.c
+++ b/drivers/staging/nvec/nvec.c
@@ -837,7 +837,7 @@
}
ret = mfd_add_devices(nvec->dev, -1, nvec_devices,
- ARRAY_SIZE(nvec_devices), base, 0);
+ ARRAY_SIZE(nvec_devices), base, 0, NULL);
if (ret)
dev_err(nvec->dev, "error adding subdevices\n");
diff --git a/drivers/staging/omapdrm/omap_connector.c b/drivers/staging/omapdrm/omap_connector.c
index 5e2856c..55e9c86 100644
--- a/drivers/staging/omapdrm/omap_connector.c
+++ b/drivers/staging/omapdrm/omap_connector.c
@@ -48,13 +48,20 @@
mode->vsync_end = mode->vsync_start + timings->vsw;
mode->vtotal = mode->vsync_end + timings->vbp;
- /* note: whether or not it is interlaced, +/- h/vsync, etc,
- * which should be set in the mode flags, is not exposed in
- * the omap_video_timings struct.. but hdmi driver tracks
- * those separately so all we have to have to set the mode
- * is the way to recover these timings values, and the
- * omap_dss_driver would do the rest.
- */
+ mode->flags = 0;
+
+ if (timings->interlace)
+ mode->flags |= DRM_MODE_FLAG_INTERLACE;
+
+ if (timings->hsync_level == OMAPDSS_SIG_ACTIVE_HIGH)
+ mode->flags |= DRM_MODE_FLAG_PHSYNC;
+ else
+ mode->flags |= DRM_MODE_FLAG_NHSYNC;
+
+ if (timings->vsync_level == OMAPDSS_SIG_ACTIVE_HIGH)
+ mode->flags |= DRM_MODE_FLAG_PVSYNC;
+ else
+ mode->flags |= DRM_MODE_FLAG_NVSYNC;
}
static inline void copy_timings_drm_to_omap(struct omap_video_timings *timings,
@@ -71,6 +78,22 @@
timings->vfp = mode->vsync_start - mode->vdisplay;
timings->vsw = mode->vsync_end - mode->vsync_start;
timings->vbp = mode->vtotal - mode->vsync_end;
+
+ timings->interlace = !!(mode->flags & DRM_MODE_FLAG_INTERLACE);
+
+ if (mode->flags & DRM_MODE_FLAG_PHSYNC)
+ timings->hsync_level = OMAPDSS_SIG_ACTIVE_HIGH;
+ else
+ timings->hsync_level = OMAPDSS_SIG_ACTIVE_LOW;
+
+ if (mode->flags & DRM_MODE_FLAG_PVSYNC)
+ timings->vsync_level = OMAPDSS_SIG_ACTIVE_HIGH;
+ else
+ timings->vsync_level = OMAPDSS_SIG_ACTIVE_LOW;
+
+ timings->data_pclk_edge = OMAPDSS_DRIVE_SIG_RISING_EDGE;
+ timings->de_level = OMAPDSS_SIG_ACTIVE_HIGH;
+ timings->sync_pclk_edge = OMAPDSS_DRIVE_SIG_OPPOSITE_EDGES;
}
static void omap_connector_dpms(struct drm_connector *connector, int mode)
@@ -187,7 +210,7 @@
}
} else {
struct drm_display_mode *mode = drm_mode_create(dev);
- struct omap_video_timings timings;
+ struct omap_video_timings timings = {0};
dssdrv->get_timings(dssdev, &timings);
@@ -291,7 +314,7 @@
struct omap_connector *omap_connector = to_omap_connector(connector);
struct omap_dss_device *dssdev = omap_connector->dssdev;
struct omap_dss_driver *dssdrv = dssdev->driver;
- struct omap_video_timings timings;
+ struct omap_video_timings timings = {0};
copy_timings_drm_to_omap(&timings, mode);
diff --git a/drivers/staging/ozwpan/ozcdev.c b/drivers/staging/ozwpan/ozcdev.c
index d983219..758ce0a 100644
--- a/drivers/staging/ozwpan/ozcdev.c
+++ b/drivers/staging/ozwpan/ozcdev.c
@@ -8,6 +8,7 @@
#include <linux/cdev.h>
#include <linux/uaccess.h>
#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
#include <linux/poll.h>
#include <linux/sched.h>
#include "ozconfig.h"
@@ -213,7 +214,7 @@
if (old_pd)
oz_pd_put(old_pd);
} else {
- if (!memcmp(addr, "\0\0\0\0\0\0", sizeof(addr))) {
+ if (is_zero_ether_addr(addr)) {
spin_lock_bh(&g_cdev.lock);
pd = g_cdev.active_pd;
g_cdev.active_pd = 0;
diff --git a/drivers/staging/rtl8192e/rtl8192e/rtl_pci.c b/drivers/staging/rtl8192e/rtl8192e/rtl_pci.c
index ddadcc3..5abbee3 100644
--- a/drivers/staging/rtl8192e/rtl8192e/rtl_pci.c
+++ b/drivers/staging/rtl8192e/rtl8192e/rtl_pci.c
@@ -31,12 +31,10 @@
struct r8192_priv *priv = (struct r8192_priv *)rtllib_priv(dev);
u8 tmp;
- int pos;
- u8 LinkCtrlReg;
+ u16 LinkCtrlReg;
- pos = pci_find_capability(priv->pdev, PCI_CAP_ID_EXP);
- pci_read_config_byte(priv->pdev, pos + PCI_EXP_LNKCTL, &LinkCtrlReg);
- priv->NdisAdapter.LinkCtrlReg = LinkCtrlReg;
+ pcie_capability_read_word(priv->pdev, PCI_EXP_LNKCTL, &LinkCtrlReg);
+ priv->NdisAdapter.LinkCtrlReg = (u8)LinkCtrlReg;
RT_TRACE(COMP_INIT, "Link Control Register =%x\n",
priv->NdisAdapter.LinkCtrlReg);
diff --git a/drivers/staging/rtl8712/recv_linux.c b/drivers/staging/rtl8712/recv_linux.c
index 0e26d5f..495ee12 100644
--- a/drivers/staging/rtl8712/recv_linux.c
+++ b/drivers/staging/rtl8712/recv_linux.c
@@ -117,13 +117,8 @@
if (skb == NULL)
goto _recv_indicatepkt_drop;
skb->data = precv_frame->u.hdr.rx_data;
-#ifdef NET_SKBUFF_DATA_USES_OFFSET
- skb->tail = (sk_buff_data_t)(precv_frame->u.hdr.rx_tail -
- precv_frame->u.hdr.rx_head);
-#else
- skb->tail = (sk_buff_data_t)precv_frame->u.hdr.rx_tail;
-#endif
skb->len = precv_frame->u.hdr.len;
+ skb_set_tail_pointer(skb, skb->len);
if ((pattrib->tcpchk_valid == 1) && (pattrib->tcp_chkrpt == 1))
skb->ip_summed = CHECKSUM_UNNECESSARY;
else
diff --git a/drivers/staging/vt6656/dpc.c b/drivers/staging/vt6656/dpc.c
index e4bdf2a..3aa895e 100644
--- a/drivers/staging/vt6656/dpc.c
+++ b/drivers/staging/vt6656/dpc.c
@@ -200,7 +200,7 @@
} else if (!compare_ether_addr(pbyRxBuffer, &pDevice->abySNAP_RFC1042[0])) {
cbHeaderSize += 6;
pwType = (PWORD) (pbyRxBufferAddr + cbHeaderSize);
- if ((*pwType == cpu_to_le16(ETH_P_IPX)) ||
+ if ((*pwType == cpu_to_be16(ETH_P_IPX)) ||
(*pwType == cpu_to_le16(0xF380))) {
cbHeaderSize -= 8;
pwType = (PWORD) (pbyRxBufferAddr + cbHeaderSize);
diff --git a/drivers/staging/vt6656/rxtx.c b/drivers/staging/vt6656/rxtx.c
index bb46452..b6e04e7 100644
--- a/drivers/staging/vt6656/rxtx.c
+++ b/drivers/staging/vt6656/rxtx.c
@@ -1699,7 +1699,7 @@
// 802.1H
if (ntohs(psEthHeader->wType) > ETH_DATA_LEN) {
if (pDevice->dwDiagRefCount == 0) {
- if ((psEthHeader->wType == cpu_to_le16(ETH_P_IPX)) ||
+ if ((psEthHeader->wType == cpu_to_be16(ETH_P_IPX)) ||
(psEthHeader->wType == cpu_to_le16(0xF380))) {
memcpy((PBYTE) (pbyPayloadHead),
abySNAP_Bridgetunnel, 6);
@@ -2838,10 +2838,10 @@
Packet_Type = skb->data[ETH_HLEN+1];
Descriptor_type = skb->data[ETH_HLEN+1+1+2];
Key_info = (skb->data[ETH_HLEN+1+1+2+1] << 8)|(skb->data[ETH_HLEN+1+1+2+2]);
- if (pDevice->sTxEthHeader.wType == cpu_to_le16(ETH_P_PAE)) {
- /* 802.1x OR eapol-key challenge frame transfer */
- if (((Protocol_Version == 1) || (Protocol_Version == 2)) &&
- (Packet_Type == 3)) {
+ if (pDevice->sTxEthHeader.wType == cpu_to_be16(ETH_P_PAE)) {
+ /* 802.1x OR eapol-key challenge frame transfer */
+ if (((Protocol_Version == 1) || (Protocol_Version == 2)) &&
+ (Packet_Type == 3)) {
bTxeapol_key = TRUE;
if(!(Key_info & BIT3) && //WPA or RSN group-key challenge
(Key_info & BIT8) && (Key_info & BIT9)) { //send 2/2 key
@@ -2987,19 +2987,19 @@
}
}
- if (pDevice->sTxEthHeader.wType == cpu_to_le16(ETH_P_PAE)) {
- if (pDevice->byBBType != BB_TYPE_11A) {
- pDevice->wCurrentRate = RATE_1M;
- pDevice->byACKRate = RATE_1M;
- pDevice->byTopCCKBasicRate = RATE_1M;
- pDevice->byTopOFDMBasicRate = RATE_6M;
- } else {
- pDevice->wCurrentRate = RATE_6M;
- pDevice->byACKRate = RATE_6M;
- pDevice->byTopCCKBasicRate = RATE_1M;
- pDevice->byTopOFDMBasicRate = RATE_6M;
- }
- }
+ if (pDevice->sTxEthHeader.wType == cpu_to_be16(ETH_P_PAE)) {
+ if (pDevice->byBBType != BB_TYPE_11A) {
+ pDevice->wCurrentRate = RATE_1M;
+ pDevice->byACKRate = RATE_1M;
+ pDevice->byTopCCKBasicRate = RATE_1M;
+ pDevice->byTopOFDMBasicRate = RATE_6M;
+ } else {
+ pDevice->wCurrentRate = RATE_6M;
+ pDevice->byACKRate = RATE_6M;
+ pDevice->byTopCCKBasicRate = RATE_1M;
+ pDevice->byTopOFDMBasicRate = RATE_6M;
+ }
+ }
DBG_PRT(MSG_LEVEL_DEBUG,
KERN_INFO "dma_tx: pDevice->wCurrentRate = %d\n",
@@ -3015,7 +3015,7 @@
if (bNeedEncryption == TRUE) {
DBG_PRT(MSG_LEVEL_DEBUG, KERN_INFO"ntohs Pkt Type=%04x\n", ntohs(pDevice->sTxEthHeader.wType));
- if ((pDevice->sTxEthHeader.wType) == cpu_to_le16(ETH_P_PAE)) {
+ if ((pDevice->sTxEthHeader.wType) == cpu_to_be16(ETH_P_PAE)) {
bNeedEncryption = FALSE;
DBG_PRT(MSG_LEVEL_DEBUG, KERN_INFO"Pkt Type=%04x\n", (pDevice->sTxEthHeader.wType));
if ((pMgmt->eCurrMode == WMAC_MODE_ESS_STA) && (pMgmt->eCurrState == WMAC_STATE_ASSOC)) {
diff --git a/drivers/staging/wlan-ng/cfg80211.c b/drivers/staging/wlan-ng/cfg80211.c
index fabff4d..0970127 100644
--- a/drivers/staging/wlan-ng/cfg80211.c
+++ b/drivers/staging/wlan-ng/cfg80211.c
@@ -327,9 +327,9 @@
return result;
}
-int prism2_scan(struct wiphy *wiphy, struct net_device *dev,
- struct cfg80211_scan_request *request)
+int prism2_scan(struct wiphy *wiphy, struct cfg80211_scan_request *request)
{
+ struct net_device *dev = request->wdev->netdev;
struct prism2_wiphy_private *priv = wiphy_priv(wiphy);
wlandevice_t *wlandev = dev->ml_priv;
struct p80211msg_dot11req_scan msg1;
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index c214977..52b43b7 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -1251,13 +1251,12 @@
void *pampd, struct tmem_pool *pool,
struct tmem_oid *oid, uint32_t index)
{
- int ret = 0;
-
BUG_ON(!is_ephemeral(pool));
- zbud_decompress((struct page *)(data), pampd);
+ if (zbud_decompress((struct page *)(data), pampd) < 0)
+ return -EINVAL;
zbud_free_and_delist((struct zbud_hdr *)pampd);
atomic_dec(&zcache_curr_eph_pampd_count);
- return ret;
+ return 0;
}
/*
diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c
index 0694d9b..6aba439 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -221,6 +221,7 @@
{
struct iscsi_session *sess = NULL;
struct iscsi_login_req *pdu = (struct iscsi_login_req *)buf;
+ int ret;
sess = kzalloc(sizeof(struct iscsi_session), GFP_KERNEL);
if (!sess) {
@@ -257,9 +258,17 @@
return -ENOMEM;
}
spin_lock(&sess_idr_lock);
- idr_get_new(&sess_idr, NULL, &sess->session_index);
+ ret = idr_get_new(&sess_idr, NULL, &sess->session_index);
spin_unlock(&sess_idr_lock);
+ if (ret < 0) {
+ pr_err("idr_get_new() for sess_idr failed\n");
+ iscsit_tx_login_rsp(conn, ISCSI_STATUS_CLS_TARGET_ERR,
+ ISCSI_LOGIN_STATUS_NO_RESOURCES);
+ kfree(sess);
+ return -ENOMEM;
+ }
+
sess->creation_time = get_jiffies_64();
spin_lock_init(&sess->session_stats_lock);
/*
diff --git a/drivers/target/iscsi/iscsi_target_parameters.c b/drivers/target/iscsi/iscsi_target_parameters.c
index 0c4760f..240f7aa 100644
--- a/drivers/target/iscsi/iscsi_target_parameters.c
+++ b/drivers/target/iscsi/iscsi_target_parameters.c
@@ -662,7 +662,7 @@
{
*value = strchr(textbuf, '=');
if (!*value) {
- pr_err("Unable to locate \"=\" seperator for key,"
+ pr_err("Unable to locate \"=\" separator for key,"
" ignoring request.\n");
return -1;
}
@@ -1269,7 +1269,7 @@
comma_ptr = strchr(value, ',');
if (comma_ptr && !IS_TYPE_VALUE_LIST(param)) {
- pr_err("Detected value seperator \",\", but"
+ pr_err("Detected value separator \",\", but"
" key \"%s\" does not allow a value list,"
" protocol error.\n", param->name);
return -1;
diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c
index 9179997..41641ba 100644
--- a/drivers/target/target_core_alua.c
+++ b/drivers/target/target_core_alua.c
@@ -218,6 +218,13 @@
cmd->scsi_sense_reason = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
return -EINVAL;
}
+ if (cmd->data_length < 4) {
+ pr_warn("SET TARGET PORT GROUPS parameter list length %u too"
+ " small\n", cmd->data_length);
+ cmd->scsi_sense_reason = TCM_INVALID_PARAMETER_LIST;
+ return -EINVAL;
+ }
+
buf = transport_kmap_data_sg(cmd);
/*
diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index cf2c66f..9fc9a60 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -669,6 +669,13 @@
unsigned char *buf;
u32 lun_count = 0, offset = 8, i;
+ if (se_cmd->data_length < 16) {
+ pr_warn("REPORT LUNS allocation length %u too small\n",
+ se_cmd->data_length);
+ se_cmd->scsi_sense_reason = TCM_INVALID_CDB_FIELD;
+ return -EINVAL;
+ }
+
buf = transport_kmap_data_sg(se_cmd);
if (!buf)
return -ENOMEM;
diff --git a/drivers/target/target_core_iblock.c b/drivers/target/target_core_iblock.c
index 76db75e..9ba4954 100644
--- a/drivers/target/target_core_iblock.c
+++ b/drivers/target/target_core_iblock.c
@@ -325,17 +325,30 @@
struct iblock_dev *ibd = dev->dev_ptr;
unsigned char *buf, *ptr = NULL;
sector_t lba;
- int size = cmd->data_length;
+ int size;
u32 range;
int ret = 0;
int dl, bd_dl;
+ if (cmd->data_length < 8) {
+ pr_warn("UNMAP parameter list length %u too small\n",
+ cmd->data_length);
+ cmd->scsi_sense_reason = TCM_INVALID_PARAMETER_LIST;
+ return -EINVAL;
+ }
+
buf = transport_kmap_data_sg(cmd);
dl = get_unaligned_be16(&buf[0]);
bd_dl = get_unaligned_be16(&buf[2]);
- size = min(size - 8, bd_dl);
+ size = cmd->data_length - 8;
+ if (bd_dl > size)
+ pr_warn("UNMAP parameter list length %u too small, ignoring bd_dl %u\n",
+ cmd->data_length, bd_dl);
+ else
+ size = bd_dl;
+
if (size / 16 > dev->se_sub_dev->se_dev_attrib.max_unmap_block_desc_count) {
cmd->scsi_sense_reason = TCM_INVALID_PARAMETER_LIST;
ret = -EINVAL;
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 1e94650..956c84c 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -1540,6 +1540,14 @@
tidh_new->dest_local_nexus = 1;
list_add_tail(&tidh_new->dest_list, &tid_dest_list);
+ if (cmd->data_length < 28) {
+ pr_warn("SPC-PR: Received PR OUT parameter list"
+ " length too small: %u\n", cmd->data_length);
+ cmd->scsi_sense_reason = TCM_INVALID_PARAMETER_LIST;
+ ret = -EINVAL;
+ goto out;
+ }
+
buf = transport_kmap_data_sg(cmd);
/*
* For a PERSISTENT RESERVE OUT specify initiator ports payload,
diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
index 5552fa7..9d7ce3d 100644
--- a/drivers/target/target_core_pscsi.c
+++ b/drivers/target/target_core_pscsi.c
@@ -667,7 +667,8 @@
kfree(pdv);
}
-static int pscsi_transport_complete(struct se_cmd *cmd, struct scatterlist *sg)
+static void pscsi_transport_complete(struct se_cmd *cmd, struct scatterlist *sg,
+ unsigned char *sense_buffer)
{
struct pscsi_dev_virt *pdv = cmd->se_dev->dev_ptr;
struct scsi_device *sd = pdv->pdv_sd;
@@ -679,7 +680,7 @@
* not been allocated because TCM is handling the emulation directly.
*/
if (!pt)
- return 0;
+ return;
cdb = &pt->pscsi_cdb[0];
result = pt->pscsi_result;
@@ -687,11 +688,11 @@
* Hack to make sure that Write-Protect modepage is set if R/O mode is
* forced.
*/
+ if (!cmd->se_deve || !cmd->data_length)
+ goto after_mode_sense;
+
if (((cdb[0] == MODE_SENSE) || (cdb[0] == MODE_SENSE_10)) &&
(status_byte(result) << 1) == SAM_STAT_GOOD) {
- if (!cmd->se_deve)
- goto after_mode_sense;
-
if (cmd->se_deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY) {
unsigned char *buf = transport_kmap_data_sg(cmd);
@@ -708,7 +709,7 @@
}
after_mode_sense:
- if (sd->type != TYPE_TAPE)
+ if (sd->type != TYPE_TAPE || !cmd->data_length)
goto after_mode_select;
/*
@@ -750,10 +751,10 @@
}
after_mode_select:
- if (status_byte(result) & CHECK_CONDITION)
- return 1;
-
- return 0;
+ if (sense_buffer && (status_byte(result) & CHECK_CONDITION)) {
+ memcpy(sense_buffer, pt->pscsi_sense, TRANSPORT_SENSE_BUFFER);
+ cmd->se_cmd_flags |= SCF_TRANSPORT_TASK_SENSE;
+ }
}
enum {
@@ -1184,13 +1185,6 @@
return -ENOMEM;
}
-static unsigned char *pscsi_get_sense_buffer(struct se_cmd *cmd)
-{
- struct pscsi_plugin_task *pt = cmd->priv;
-
- return pt->pscsi_sense;
-}
-
/* pscsi_get_device_rev():
*
*
@@ -1273,7 +1267,6 @@
.check_configfs_dev_params = pscsi_check_configfs_dev_params,
.set_configfs_dev_params = pscsi_set_configfs_dev_params,
.show_configfs_dev_params = pscsi_show_configfs_dev_params,
- .get_sense_buffer = pscsi_get_sense_buffer,
.get_device_rev = pscsi_get_device_rev,
.get_device_type = pscsi_get_device_type,
.get_blocks = pscsi_get_blocks,
diff --git a/drivers/target/target_core_spc.c b/drivers/target/target_core_spc.c
index 4c861de..388a922 100644
--- a/drivers/target/target_core_spc.c
+++ b/drivers/target/target_core_spc.c
@@ -877,9 +877,11 @@
static int spc_emulate_request_sense(struct se_cmd *cmd)
{
unsigned char *cdb = cmd->t_task_cdb;
- unsigned char *buf;
+ unsigned char *rbuf;
u8 ua_asc = 0, ua_ascq = 0;
- int err = 0;
+ unsigned char buf[SE_SENSE_BUF];
+
+ memset(buf, 0, SE_SENSE_BUF);
if (cdb[1] & 0x01) {
pr_err("REQUEST_SENSE description emulation not"
@@ -888,20 +890,21 @@
return -ENOSYS;
}
- buf = transport_kmap_data_sg(cmd);
-
- if (!core_scsi3_ua_clear_for_request_sense(cmd, &ua_asc, &ua_ascq)) {
+ rbuf = transport_kmap_data_sg(cmd);
+ if (cmd->scsi_sense_reason != 0) {
+ /*
+ * Out of memory. We will fail with CHECK CONDITION, so
+ * we must not clear the unit attention condition.
+ */
+ target_complete_cmd(cmd, CHECK_CONDITION);
+ return 0;
+ } else if (!core_scsi3_ua_clear_for_request_sense(cmd, &ua_asc, &ua_ascq)) {
/*
* CURRENT ERROR, UNIT ATTENTION
*/
buf[0] = 0x70;
buf[SPC_SENSE_KEY_OFFSET] = UNIT_ATTENTION;
- if (cmd->data_length < 18) {
- buf[7] = 0x00;
- err = -EINVAL;
- goto end;
- }
/*
* The Additional Sense Code (ASC) from the UNIT ATTENTION
*/
@@ -915,11 +918,6 @@
buf[0] = 0x70;
buf[SPC_SENSE_KEY_OFFSET] = NO_SENSE;
- if (cmd->data_length < 18) {
- buf[7] = 0x00;
- err = -EINVAL;
- goto end;
- }
/*
* NO ADDITIONAL SENSE INFORMATION
*/
@@ -927,8 +925,11 @@
buf[7] = 0x0A;
}
-end:
- transport_kunmap_data_sg(cmd);
+ if (rbuf) {
+ memcpy(rbuf, buf, min_t(u32, sizeof(buf), cmd->data_length));
+ transport_kunmap_data_sg(cmd);
+ }
+
target_complete_cmd(cmd, GOOD);
return 0;
}
diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index 4de3186dc..269f544 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -567,6 +567,34 @@
transport_generic_request_failure(cmd);
}
+/*
+ * Used when asking transport to copy Sense Data from the underlying
+ * Linux/SCSI struct scsi_cmnd
+ */
+static unsigned char *transport_get_sense_buffer(struct se_cmd *cmd)
+{
+ unsigned char *buffer = cmd->sense_buffer;
+ struct se_device *dev = cmd->se_dev;
+ u32 offset = 0;
+
+ WARN_ON(!cmd->se_lun);
+
+ if (!dev)
+ return NULL;
+
+ if (cmd->se_cmd_flags & SCF_SENT_CHECK_CONDITION)
+ return NULL;
+
+ offset = cmd->se_tfo->set_fabric_sense_len(cmd, TRANSPORT_SENSE_BUFFER);
+
+ /* Automatically padded */
+ cmd->scsi_sense_length = TRANSPORT_SENSE_BUFFER + offset;
+
+ pr_debug("HBA_[%u]_PLUG[%s]: Requesting sense for SAM STATUS: 0x%02x\n",
+ dev->se_hba->hba_id, dev->transport->name, cmd->scsi_status);
+ return &buffer[offset];
+}
+
void target_complete_cmd(struct se_cmd *cmd, u8 scsi_status)
{
struct se_device *dev = cmd->se_dev;
@@ -580,11 +608,11 @@
cmd->transport_state &= ~CMD_T_BUSY;
if (dev && dev->transport->transport_complete) {
- if (dev->transport->transport_complete(cmd,
- cmd->t_data_sg) != 0) {
- cmd->se_cmd_flags |= SCF_TRANSPORT_TASK_SENSE;
+ dev->transport->transport_complete(cmd,
+ cmd->t_data_sg,
+ transport_get_sense_buffer(cmd));
+ if (cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE)
success = 1;
- }
}
/*
@@ -1181,15 +1209,20 @@
/* Returns CHECK_CONDITION + INVALID_CDB_FIELD */
goto out_invalid_cdb_field;
}
-
+ /*
+ * For the overflow case keep the existing fabric provided
+ * ->data_length. Otherwise for the underflow case, reset
+ * ->data_length to the smaller SCSI expected data transfer
+ * length.
+ */
if (size > cmd->data_length) {
cmd->se_cmd_flags |= SCF_OVERFLOW_BIT;
cmd->residual_count = (size - cmd->data_length);
} else {
cmd->se_cmd_flags |= SCF_UNDERFLOW_BIT;
cmd->residual_count = (cmd->data_length - size);
+ cmd->data_length = size;
}
- cmd->data_length = size;
}
return 0;
@@ -1816,61 +1849,6 @@
EXPORT_SYMBOL(target_execute_cmd);
/*
- * Used to obtain Sense Data from underlying Linux/SCSI struct scsi_cmnd
- */
-static int transport_get_sense_data(struct se_cmd *cmd)
-{
- unsigned char *buffer = cmd->sense_buffer, *sense_buffer = NULL;
- struct se_device *dev = cmd->se_dev;
- unsigned long flags;
- u32 offset = 0;
-
- WARN_ON(!cmd->se_lun);
-
- if (!dev)
- return 0;
-
- spin_lock_irqsave(&cmd->t_state_lock, flags);
- if (cmd->se_cmd_flags & SCF_SENT_CHECK_CONDITION) {
- spin_unlock_irqrestore(&cmd->t_state_lock, flags);
- return 0;
- }
-
- if (!(cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE))
- goto out;
-
- if (!dev->transport->get_sense_buffer) {
- pr_err("dev->transport->get_sense_buffer is NULL\n");
- goto out;
- }
-
- sense_buffer = dev->transport->get_sense_buffer(cmd);
- if (!sense_buffer) {
- pr_err("ITT 0x%08x cmd %p: Unable to locate"
- " sense buffer for task with sense\n",
- cmd->se_tfo->get_task_tag(cmd), cmd);
- goto out;
- }
-
- spin_unlock_irqrestore(&cmd->t_state_lock, flags);
-
- offset = cmd->se_tfo->set_fabric_sense_len(cmd, TRANSPORT_SENSE_BUFFER);
-
- memcpy(&buffer[offset], sense_buffer, TRANSPORT_SENSE_BUFFER);
-
- /* Automatically padded */
- cmd->scsi_sense_length = TRANSPORT_SENSE_BUFFER + offset;
-
- pr_debug("HBA_[%u]_PLUG[%s]: Set SAM STATUS: 0x%02x and sense\n",
- dev->se_hba->hba_id, dev->transport->name, cmd->scsi_status);
- return 0;
-
-out:
- spin_unlock_irqrestore(&cmd->t_state_lock, flags);
- return -1;
-}
-
-/*
* Process all commands up to the last received ORDERED task attribute which
* requires another blocking boundary
*/
@@ -1985,7 +1963,7 @@
static void target_complete_ok_work(struct work_struct *work)
{
struct se_cmd *cmd = container_of(work, struct se_cmd, work);
- int reason = 0, ret;
+ int ret;
/*
* Check if we need to move delayed/dormant tasks from cmds on the
@@ -2002,23 +1980,19 @@
schedule_work(&cmd->se_dev->qf_work_queue);
/*
- * Check if we need to retrieve a sense buffer from
+ * Check if we need to send a sense buffer from
* the struct se_cmd in question.
*/
if (cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE) {
- if (transport_get_sense_data(cmd) < 0)
- reason = TCM_NON_EXISTENT_LUN;
+ WARN_ON(!cmd->scsi_status);
+ ret = transport_send_check_condition_and_sense(
+ cmd, 0, 1);
+ if (ret == -EAGAIN || ret == -ENOMEM)
+ goto queue_full;
- if (cmd->scsi_status) {
- ret = transport_send_check_condition_and_sense(
- cmd, reason, 1);
- if (ret == -EAGAIN || ret == -ENOMEM)
- goto queue_full;
-
- transport_lun_remove_cmd(cmd);
- transport_cmd_check_stop_to_fabric(cmd);
- return;
- }
+ transport_lun_remove_cmd(cmd);
+ transport_cmd_check_stop_to_fabric(cmd);
+ return;
}
/*
* Check for a callback, used by amongst other things
@@ -2216,7 +2190,6 @@
struct page **pages;
int i;
- BUG_ON(!sg);
/*
* We need to take into account a possible offset here for fabrics like
* tcm_loop who may be using a contig buffer from the SCSI midlayer for
@@ -2224,13 +2197,17 @@
*/
if (!cmd->t_data_nents)
return NULL;
- else if (cmd->t_data_nents == 1)
+
+ BUG_ON(!sg);
+ if (cmd->t_data_nents == 1)
return kmap(sg_page(sg)) + sg->offset;
/* >1 page. use vmap */
pages = kmalloc(sizeof(*pages) * cmd->t_data_nents, GFP_KERNEL);
- if (!pages)
+ if (!pages) {
+ cmd->scsi_sense_reason = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
return NULL;
+ }
/* convert sg[] to pages[] */
for_each_sg(cmd->t_data_sg, sg, cmd->t_data_nents, i) {
@@ -2239,8 +2216,10 @@
cmd->t_data_vmap = vmap(pages, cmd->t_data_nents, VM_MAP, PAGE_KERNEL);
kfree(pages);
- if (!cmd->t_data_vmap)
+ if (!cmd->t_data_vmap) {
+ cmd->scsi_sense_reason = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
return NULL;
+ }
return cmd->t_data_vmap + cmd->t_data_sg[0].offset;
}
@@ -2326,19 +2305,14 @@
* into the fabric for data transfers, go ahead and complete it right
* away.
*/
- if (!cmd->data_length) {
+ if (!cmd->data_length &&
+ cmd->t_task_cdb[0] != REQUEST_SENSE &&
+ cmd->se_dev->transport->transport_type != TRANSPORT_PLUGIN_PHBA_PDEV) {
spin_lock_irq(&cmd->t_state_lock);
cmd->t_state = TRANSPORT_COMPLETE;
cmd->transport_state |= CMD_T_ACTIVE;
spin_unlock_irq(&cmd->t_state_lock);
- if (cmd->t_task_cdb[0] == REQUEST_SENSE) {
- u8 ua_asc = 0, ua_ascq = 0;
-
- core_scsi3_ua_clear_for_request_sense(cmd,
- &ua_asc, &ua_ascq);
- }
-
INIT_WORK(&cmd->work, target_complete_ok_work);
queue_work(target_completion_wq, &cmd->work);
return 0;
diff --git a/drivers/tty/serial/8250/8250_pci.c b/drivers/tty/serial/8250/8250_pci.c
index 28e7c7c..452278e 100644
--- a/drivers/tty/serial/8250/8250_pci.c
+++ b/drivers/tty/serial/8250/8250_pci.c
@@ -4236,7 +4236,7 @@
pciserial_resume_ports(priv);
}
-static struct pci_error_handlers serial8250_err_handler = {
+static const struct pci_error_handlers serial8250_err_handler = {
.error_detected = serial8250_io_error_detected,
.slot_reset = serial8250_io_slot_reset,
.resume = serial8250_io_resume,
diff --git a/drivers/tty/serial/imx.c b/drivers/tty/serial/imx.c
index d5c689d6..e309e8b 100644
--- a/drivers/tty/serial/imx.c
+++ b/drivers/tty/serial/imx.c
@@ -132,6 +132,7 @@
#define UCR4_OREN (1<<1) /* Receiver overrun interrupt enable */
#define UCR4_DREN (1<<0) /* Recv data ready interrupt enable */
#define UFCR_RXTL_SHF 0 /* Receiver trigger level shift */
+#define UFCR_DCEDTE (1<<6) /* DCE/DTE mode select */
#define UFCR_RFDIV (7<<7) /* Reference freq divider mask */
#define UFCR_RFDIV_REG(x) (((x) < 7 ? 6 - (x) : 6) << 7)
#define UFCR_TXTL_SHF 10 /* Transmitter trigger level shift */
@@ -667,22 +668,11 @@
static int imx_setup_ufcr(struct imx_port *sport, unsigned int mode)
{
unsigned int val;
- unsigned int ufcr_rfdiv;
- /* set receiver / transmitter trigger level.
- * RFDIV is set such way to satisfy requested uartclk value
- */
- val = TXTL << 10 | RXTL;
- ufcr_rfdiv = (clk_get_rate(sport->clk_per) + sport->port.uartclk / 2)
- / sport->port.uartclk;
-
- if(!ufcr_rfdiv)
- ufcr_rfdiv = 1;
-
- val |= UFCR_RFDIV_REG(ufcr_rfdiv);
-
+ /* set receiver / transmitter trigger level */
+ val = readl(sport->port.membase + UFCR) & (UFCR_RFDIV | UFCR_DCEDTE);
+ val |= TXTL << UFCR_TXTL_SHF | RXTL;
writel(val, sport->port.membase + UFCR);
-
return 0;
}
@@ -754,6 +744,7 @@
}
}
+ spin_lock_irqsave(&sport->port.lock, flags);
/*
* Finally, clear and enable interrupts
*/
@@ -807,7 +798,6 @@
/*
* Enable modem status interrupts
*/
- spin_lock_irqsave(&sport->port.lock,flags);
imx_enable_ms(&sport->port);
spin_unlock_irqrestore(&sport->port.lock,flags);
@@ -837,10 +827,13 @@
{
struct imx_port *sport = (struct imx_port *)port;
unsigned long temp;
+ unsigned long flags;
+ spin_lock_irqsave(&sport->port.lock, flags);
temp = readl(sport->port.membase + UCR2);
temp &= ~(UCR2_TXEN);
writel(temp, sport->port.membase + UCR2);
+ spin_unlock_irqrestore(&sport->port.lock, flags);
if (USE_IRDA(sport)) {
struct imxuart_platform_data *pdata;
@@ -869,12 +862,14 @@
* Disable all interrupts, port and break condition.
*/
+ spin_lock_irqsave(&sport->port.lock, flags);
temp = readl(sport->port.membase + UCR1);
temp &= ~(UCR1_TXMPTYEN | UCR1_RRDYEN | UCR1_RTSDEN | UCR1_UARTEN);
if (USE_IRDA(sport))
temp &= ~(UCR1_IREN);
writel(temp, sport->port.membase + UCR1);
+ spin_unlock_irqrestore(&sport->port.lock, flags);
}
static void
@@ -1217,6 +1212,9 @@
struct imx_port *sport = imx_ports[co->index];
struct imx_port_ucrs old_ucr;
unsigned int ucr1;
+ unsigned long flags;
+
+ spin_lock_irqsave(&sport->port.lock, flags);
/*
* First, save UCR1/2/3 and then disable interrupts
@@ -1242,6 +1240,8 @@
while (!(readl(sport->port.membase + USR2) & USR2_TXDC));
imx_port_ucrs_restore(&sport->port, &old_ucr);
+
+ spin_unlock_irqrestore(&sport->port.lock, flags);
}
/*
diff --git a/drivers/tty/serial/jsm/jsm_driver.c b/drivers/tty/serial/jsm/jsm_driver.c
index 7545fe1..5ab3c3b 100644
--- a/drivers/tty/serial/jsm/jsm_driver.c
+++ b/drivers/tty/serial/jsm/jsm_driver.c
@@ -54,7 +54,7 @@
static pci_ers_result_t jsm_io_slot_reset(struct pci_dev *pdev);
static void jsm_io_resume(struct pci_dev *pdev);
-static struct pci_error_handlers jsm_err_handler = {
+static const struct pci_error_handlers jsm_err_handler = {
.error_detected = jsm_io_error_detected,
.slot_reset = jsm_io_slot_reset,
.resume = jsm_io_resume,
diff --git a/drivers/usb/chipidea/udc.c b/drivers/usb/chipidea/udc.c
index c7a032a..d214448 100644
--- a/drivers/usb/chipidea/udc.c
+++ b/drivers/usb/chipidea/udc.c
@@ -78,8 +78,7 @@
}
/**
- * hw_device_state: enables/disables interrupts & starts/stops device (execute
- * without interruption)
+ * hw_device_state: enables/disables interrupts (execute without interruption)
* @dma: 0 => disable, !0 => enable and set dma engine
*
* This function returns an error code
@@ -91,9 +90,7 @@
/* interrupt, error, port change, reset, sleep/suspend */
hw_write(ci, OP_USBINTR, ~0,
USBi_UI|USBi_UEI|USBi_PCI|USBi_URI|USBi_SLI);
- hw_write(ci, OP_USBCMD, USBCMD_RS, USBCMD_RS);
} else {
- hw_write(ci, OP_USBCMD, USBCMD_RS, 0);
hw_write(ci, OP_USBINTR, ~0, 0);
}
return 0;
@@ -774,10 +771,7 @@
{
struct ci13xxx_req *mReq, *mReqTemp;
struct ci13xxx_ep *mEpTemp = mEp;
- int uninitialized_var(retval);
-
- if (list_empty(&mEp->qh.queue))
- return -EINVAL;
+ int retval = 0;
list_for_each_entry_safe(mReq, mReqTemp, &mEp->qh.queue,
queue) {
@@ -1420,6 +1414,21 @@
return -ENOTSUPP;
}
+/* Change Data+ pullup status
+ * this func is used by usb_gadget_connect/disconnet
+ */
+static int ci13xxx_pullup(struct usb_gadget *_gadget, int is_on)
+{
+ struct ci13xxx *ci = container_of(_gadget, struct ci13xxx, gadget);
+
+ if (is_on)
+ hw_write(ci, OP_USBCMD, USBCMD_RS, USBCMD_RS);
+ else
+ hw_write(ci, OP_USBCMD, USBCMD_RS, 0);
+
+ return 0;
+}
+
static int ci13xxx_start(struct usb_gadget *gadget,
struct usb_gadget_driver *driver);
static int ci13xxx_stop(struct usb_gadget *gadget,
@@ -1432,6 +1441,7 @@
static const struct usb_gadget_ops usb_gadget_ops = {
.vbus_session = ci13xxx_vbus_session,
.wakeup = ci13xxx_wakeup,
+ .pullup = ci13xxx_pullup,
.vbus_draw = ci13xxx_vbus_draw,
.udc_start = ci13xxx_start,
.udc_stop = ci13xxx_stop,
@@ -1455,7 +1465,12 @@
mEp->ep.name = mEp->name;
mEp->ep.ops = &usb_ep_ops;
- mEp->ep.maxpacket = CTRL_PAYLOAD_MAX;
+ /*
+ * for ep0: maxP defined in desc, for other
+ * eps, maxP is set by epautoconfig() called
+ * by gadget layer
+ */
+ mEp->ep.maxpacket = (unsigned short)~0;
INIT_LIST_HEAD(&mEp->qh.queue);
mEp->qh.ptr = dma_pool_alloc(ci->qh_pool, GFP_KERNEL,
@@ -1475,6 +1490,7 @@
else
ci->ep0in = mEp;
+ mEp->ep.maxpacket = CTRL_PAYLOAD_MAX;
continue;
}
@@ -1484,6 +1500,17 @@
return retval;
}
+static void destroy_eps(struct ci13xxx *ci)
+{
+ int i;
+
+ for (i = 0; i < ci->hw_ep_max; i++) {
+ struct ci13xxx_ep *mEp = &ci->ci13xxx_ep[i];
+
+ dma_pool_free(ci->qh_pool, mEp->qh.ptr, mEp->qh.dma);
+ }
+}
+
/**
* ci13xxx_start: register a gadget driver
* @gadget: our gadget
@@ -1691,7 +1718,7 @@
if (ci->platdata->flags & CI13XXX_REQUIRE_TRANSCEIVER) {
if (ci->transceiver == NULL) {
retval = -ENODEV;
- goto free_pools;
+ goto destroy_eps;
}
}
@@ -1729,7 +1756,7 @@
remove_trans:
if (!IS_ERR_OR_NULL(ci->transceiver)) {
- otg_set_peripheral(ci->transceiver->otg, &ci->gadget);
+ otg_set_peripheral(ci->transceiver->otg, NULL);
if (ci->global_phy)
usb_put_phy(ci->transceiver);
}
@@ -1742,6 +1769,8 @@
put_transceiver:
if (!IS_ERR_OR_NULL(ci->transceiver) && ci->global_phy)
usb_put_phy(ci->transceiver);
+destroy_eps:
+ destroy_eps(ci);
free_pools:
dma_pool_destroy(ci->td_pool);
free_qh_pool:
@@ -1756,18 +1785,12 @@
*/
static void udc_stop(struct ci13xxx *ci)
{
- int i;
-
if (ci == NULL)
return;
usb_del_gadget_udc(&ci->gadget);
- for (i = 0; i < ci->hw_ep_max; i++) {
- struct ci13xxx_ep *mEp = &ci->ci13xxx_ep[i];
-
- dma_pool_free(ci->qh_pool, mEp->qh.ptr, mEp->qh.dma);
- }
+ destroy_eps(ci);
dma_pool_destroy(ci->td_pool);
dma_pool_destroy(ci->qh_pool);
diff --git a/drivers/usb/class/cdc-wdm.c b/drivers/usb/class/cdc-wdm.c
index 65a55ab..5f0cb41 100644
--- a/drivers/usb/class/cdc-wdm.c
+++ b/drivers/usb/class/cdc-wdm.c
@@ -109,12 +109,14 @@
/* return intfdata if we own the interface, else look up intf in the list */
static struct wdm_device *wdm_find_device(struct usb_interface *intf)
{
- struct wdm_device *desc = NULL;
+ struct wdm_device *desc;
spin_lock(&wdm_device_list_lock);
list_for_each_entry(desc, &wdm_device_list, device_list)
if (desc->intf == intf)
- break;
+ goto found;
+ desc = NULL;
+found:
spin_unlock(&wdm_device_list_lock);
return desc;
@@ -122,12 +124,14 @@
static struct wdm_device *wdm_find_device_by_minor(int minor)
{
- struct wdm_device *desc = NULL;
+ struct wdm_device *desc;
spin_lock(&wdm_device_list_lock);
list_for_each_entry(desc, &wdm_device_list, device_list)
if (desc->intf->minor == minor)
- break;
+ goto found;
+ desc = NULL;
+found:
spin_unlock(&wdm_device_list_lock);
return desc;
diff --git a/drivers/usb/core/devices.c b/drivers/usb/core/devices.c
index d956965..3440812 100644
--- a/drivers/usb/core/devices.c
+++ b/drivers/usb/core/devices.c
@@ -624,7 +624,7 @@
/* print devices for all busses */
list_for_each_entry(bus, &usb_bus_list, bus_list) {
/* recurse through all children of the root hub */
- if (!bus->root_hub)
+ if (!bus_to_hcd(bus)->rh_registered)
continue;
usb_lock_device(bus->root_hub);
ret = usb_device_dump(&buf, &nbytes, &skip_bytes, ppos,
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
index bc84106..75ba209 100644
--- a/drivers/usb/core/hcd.c
+++ b/drivers/usb/core/hcd.c
@@ -1011,10 +1011,7 @@
if (retval) {
dev_err (parent_dev, "can't register root hub for %s, %d\n",
dev_name(&usb_dev->dev), retval);
- }
- mutex_unlock(&usb_bus_list_lock);
-
- if (retval == 0) {
+ } else {
spin_lock_irq (&hcd_root_hub_lock);
hcd->rh_registered = 1;
spin_unlock_irq (&hcd_root_hub_lock);
@@ -1023,6 +1020,7 @@
if (HCD_DEAD(hcd))
usb_hc_died (hcd); /* This time clean up */
}
+ mutex_unlock(&usb_bus_list_lock);
return retval;
}
diff --git a/drivers/usb/core/quirks.c b/drivers/usb/core/quirks.c
index f15501f4c..e77a8e8 100644
--- a/drivers/usb/core/quirks.c
+++ b/drivers/usb/core/quirks.c
@@ -71,6 +71,10 @@
{ USB_DEVICE(0x04b4, 0x0526), .driver_info =
USB_QUIRK_CONFIG_INTF_STRINGS },
+ /* Microchip Joss Optical infrared touchboard device */
+ { USB_DEVICE(0x04d8, 0x000c), .driver_info =
+ USB_QUIRK_CONFIG_INTF_STRINGS },
+
/* Samsung Android phone modem - ID conflict with SPH-I500 */
{ USB_DEVICE(0x04e8, 0x6601), .driver_info =
USB_QUIRK_CONFIG_INTF_STRINGS },
diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index c34452a..a68ff53 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -436,16 +436,21 @@
dev_err(dev, "missing IRQ\n");
return -ENODEV;
}
- dwc->xhci_resources[1] = *res;
+ dwc->xhci_resources[1].start = res->start;
+ dwc->xhci_resources[1].end = res->end;
+ dwc->xhci_resources[1].flags = res->flags;
+ dwc->xhci_resources[1].name = res->name;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!res) {
dev_err(dev, "missing memory resource\n");
return -ENODEV;
}
- dwc->xhci_resources[0] = *res;
+ dwc->xhci_resources[0].start = res->start;
dwc->xhci_resources[0].end = dwc->xhci_resources[0].start +
DWC3_XHCI_REGS_END;
+ dwc->xhci_resources[0].flags = res->flags;
+ dwc->xhci_resources[0].name = res->name;
/*
* Request memory region but exclude xHCI regs,
diff --git a/drivers/usb/dwc3/ep0.c b/drivers/usb/dwc3/ep0.c
index 9b94886..e4d5ca8 100644
--- a/drivers/usb/dwc3/ep0.c
+++ b/drivers/usb/dwc3/ep0.c
@@ -720,7 +720,6 @@
transferred = min_t(u32, ur->length,
transfer_size - length);
memcpy(ur->buf, dwc->ep0_bounce, transferred);
- dwc->ep0_bounced = false;
} else {
transferred = ur->length - length;
}
diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
index 58fdfad..c2813c2b 100644
--- a/drivers/usb/dwc3/gadget.c
+++ b/drivers/usb/dwc3/gadget.c
@@ -263,8 +263,11 @@
if (req->request.status == -EINPROGRESS)
req->request.status = status;
- usb_gadget_unmap_request(&dwc->gadget, &req->request,
- req->direction);
+ if (dwc->ep0_bounced && dep->number == 0)
+ dwc->ep0_bounced = false;
+ else
+ usb_gadget_unmap_request(&dwc->gadget, &req->request,
+ req->direction);
dev_dbg(dwc->dev, "request %p from %s completed %d/%d ===> %d\n",
req, dep->name, req->request.actual,
@@ -1026,6 +1029,7 @@
if (list_empty(&dep->request_list)) {
dev_vdbg(dwc->dev, "ISOC ep %s run out for requests.\n",
dep->name);
+ dep->flags |= DWC3_EP_PENDING_REQUEST;
return;
}
@@ -1089,6 +1093,17 @@
if (dep->flags & DWC3_EP_PENDING_REQUEST) {
int ret;
+ /*
+ * If xfernotready is already elapsed and it is a case
+ * of isoc transfer, then issue END TRANSFER, so that
+ * you can receive xfernotready again and can have
+ * notion of current microframe.
+ */
+ if (usb_endpoint_xfer_isoc(dep->endpoint.desc)) {
+ dwc3_stop_active_transfer(dwc, dep->number);
+ return 0;
+ }
+
ret = __dwc3_gadget_kick_transfer(dep, 0, true);
if (ret && ret != -EBUSY) {
struct dwc3 *dwc = dep->dwc;
diff --git a/drivers/usb/gadget/at91_udc.c b/drivers/usb/gadget/at91_udc.c
index c9e66df..1e35963 100644
--- a/drivers/usb/gadget/at91_udc.c
+++ b/drivers/usb/gadget/at91_udc.c
@@ -475,8 +475,7 @@
unsigned long flags;
if (!_ep || !ep
- || !desc || ep->ep.desc
- || _ep->name == ep0name
+ || !desc || _ep->name == ep0name
|| desc->bDescriptorType != USB_DT_ENDPOINT
|| (maxpacket = usb_endpoint_maxp(desc)) == 0
|| maxpacket > ep->maxpacket) {
@@ -530,7 +529,6 @@
tmp |= AT91_UDP_EPEDS;
__raw_writel(tmp, ep->creg);
- ep->ep.desc = desc;
ep->ep.maxpacket = maxpacket;
/*
@@ -1635,7 +1633,6 @@
udc->driver = driver;
udc->gadget.dev.driver = &driver->driver;
udc->gadget.dev.of_node = udc->pdev->dev.of_node;
- dev_set_drvdata(&udc->gadget.dev, &driver->driver);
udc->enabled = 1;
udc->selfpowered = 1;
@@ -1656,7 +1653,6 @@
spin_unlock_irqrestore(&udc->lock, flags);
udc->gadget.dev.driver = NULL;
- dev_set_drvdata(&udc->gadget.dev, NULL);
udc->driver = NULL;
DBG("unbound from %s\n", driver->driver.name);
diff --git a/drivers/usb/gadget/dummy_hcd.c b/drivers/usb/gadget/dummy_hcd.c
index b799106..afdbb1c 100644
--- a/drivers/usb/gadget/dummy_hcd.c
+++ b/drivers/usb/gadget/dummy_hcd.c
@@ -1916,6 +1916,27 @@
return retval;
}
+/* usb 3.0 root hub device descriptor */
+struct {
+ struct usb_bos_descriptor bos;
+ struct usb_ss_cap_descriptor ss_cap;
+} __packed usb3_bos_desc = {
+
+ .bos = {
+ .bLength = USB_DT_BOS_SIZE,
+ .bDescriptorType = USB_DT_BOS,
+ .wTotalLength = cpu_to_le16(sizeof(usb3_bos_desc)),
+ .bNumDeviceCaps = 1,
+ },
+ .ss_cap = {
+ .bLength = USB_DT_USB_SS_CAP_SIZE,
+ .bDescriptorType = USB_DT_DEVICE_CAPABILITY,
+ .bDevCapabilityType = USB_SS_CAP_TYPE,
+ .wSpeedSupported = cpu_to_le16(USB_5GBPS_OPERATION),
+ .bFunctionalitySupport = ilog2(USB_5GBPS_OPERATION),
+ },
+};
+
static inline void
ss_hub_descriptor(struct usb_hub_descriptor *desc)
{
@@ -2006,6 +2027,18 @@
else
hub_descriptor((struct usb_hub_descriptor *) buf);
break;
+
+ case DeviceRequest | USB_REQ_GET_DESCRIPTOR:
+ if (hcd->speed != HCD_USB3)
+ goto error;
+
+ if ((wValue >> 8) != USB_DT_BOS)
+ goto error;
+
+ memcpy(buf, &usb3_bos_desc, sizeof(usb3_bos_desc));
+ retval = sizeof(usb3_bos_desc);
+ break;
+
case GetHubStatus:
*(__le32 *) buf = cpu_to_le32(0);
break;
@@ -2503,10 +2536,8 @@
hs_hcd->has_tt = 1;
retval = usb_add_hcd(hs_hcd, 0, 0);
- if (retval != 0) {
- usb_put_hcd(hs_hcd);
- return retval;
- }
+ if (retval)
+ goto put_usb2_hcd;
if (mod_data.is_super_speed) {
ss_hcd = usb_create_shared_hcd(&dummy_hcd, &pdev->dev,
@@ -2525,6 +2556,8 @@
put_usb3_hcd:
usb_put_hcd(ss_hcd);
dealloc_usb2_hcd:
+ usb_remove_hcd(hs_hcd);
+put_usb2_hcd:
usb_put_hcd(hs_hcd);
the_controller.hs_hcd = the_controller.ss_hcd = NULL;
return retval;
diff --git a/drivers/usb/gadget/f_fs.c b/drivers/usb/gadget/f_fs.c
index 8adc79d..829aba7 100644
--- a/drivers/usb/gadget/f_fs.c
+++ b/drivers/usb/gadget/f_fs.c
@@ -34,11 +34,15 @@
/* Debugging ****************************************************************/
#ifdef VERBOSE_DEBUG
+#ifndef pr_vdebug
# define pr_vdebug pr_debug
+#endif /* pr_vdebug */
# define ffs_dump_mem(prefix, ptr, len) \
print_hex_dump_bytes(pr_fmt(prefix ": "), DUMP_PREFIX_NONE, ptr, len)
#else
+#ifndef pr_vdebug
# define pr_vdebug(...) do { } while (0)
+#endif /* pr_vdebug */
# define ffs_dump_mem(prefix, ptr, len) do { } while (0)
#endif /* VERBOSE_DEBUG */
diff --git a/drivers/usb/gadget/s3c-hsotg.c b/drivers/usb/gadget/s3c-hsotg.c
index b13e0bb..0bb617e 100644
--- a/drivers/usb/gadget/s3c-hsotg.c
+++ b/drivers/usb/gadget/s3c-hsotg.c
@@ -3599,6 +3599,7 @@
if (hsotg->num_of_eps == 0) {
dev_err(dev, "wrong number of EPs (zero)\n");
+ ret = -EINVAL;
goto err_supplies;
}
@@ -3606,6 +3607,7 @@
GFP_KERNEL);
if (!eps) {
dev_err(dev, "cannot get memory\n");
+ ret = -ENOMEM;
goto err_supplies;
}
@@ -3622,6 +3624,7 @@
GFP_KERNEL);
if (!hsotg->ctrl_req) {
dev_err(dev, "failed to allocate ctrl req\n");
+ ret = -ENOMEM;
goto err_ep_mem;
}
diff --git a/drivers/usb/gadget/u_serial.c b/drivers/usb/gadget/u_serial.c
index 5b3f5ff..da6d479 100644
--- a/drivers/usb/gadget/u_serial.c
+++ b/drivers/usb/gadget/u_serial.c
@@ -132,11 +132,15 @@
#ifdef VERBOSE_DEBUG
+#ifndef pr_vdebug
#define pr_vdebug(fmt, arg...) \
pr_debug(fmt, ##arg)
+#endif /* pr_vdebug */
#else
+#ifndef pr_vdebig
#define pr_vdebug(fmt, arg...) \
({ if (0) pr_debug(fmt, ##arg); })
+#endif /* pr_vdebug */
#endif
/*-------------------------------------------------------------------------*/
diff --git a/drivers/usb/host/ehci-q.c b/drivers/usb/host/ehci-q.c
index 9bc39ca..4b66374 100644
--- a/drivers/usb/host/ehci-q.c
+++ b/drivers/usb/host/ehci-q.c
@@ -128,9 +128,17 @@
else {
qtd = list_entry (qh->qtd_list.next,
struct ehci_qtd, qtd_list);
- /* first qtd may already be partially processed */
- if (cpu_to_hc32(ehci, qtd->qtd_dma) == qh->hw->hw_current)
+ /*
+ * first qtd may already be partially processed.
+ * If we come here during unlink, the QH overlay region
+ * might have reference to the just unlinked qtd. The
+ * qtd is updated in qh_completions(). Update the QH
+ * overlay here.
+ */
+ if (cpu_to_hc32(ehci, qtd->qtd_dma) == qh->hw->hw_current) {
+ qh->hw->hw_qtd_next = qtd->hw_next;
qtd = NULL;
+ }
}
if (qtd)
diff --git a/drivers/usb/host/ohci-at91.c b/drivers/usb/host/ohci-at91.c
index a665b3e..0bf72f9 100644
--- a/drivers/usb/host/ohci-at91.c
+++ b/drivers/usb/host/ohci-at91.c
@@ -467,7 +467,8 @@
/* From the GPIO notifying the over-current situation, find
* out the corresponding port */
at91_for_each_port(port) {
- if (gpio_to_irq(pdata->overcurrent_pin[port]) == irq) {
+ if (gpio_is_valid(pdata->overcurrent_pin[port]) &&
+ gpio_to_irq(pdata->overcurrent_pin[port]) == irq) {
gpio = pdata->overcurrent_pin[port];
break;
}
@@ -570,6 +571,16 @@
if (pdata) {
at91_for_each_port(i) {
+ /*
+ * do not configure PIO if not in relation with
+ * real USB port on board
+ */
+ if (i >= pdata->ports) {
+ pdata->vbus_pin[i] = -EINVAL;
+ pdata->overcurrent_pin[i] = -EINVAL;
+ break;
+ }
+
if (!gpio_is_valid(pdata->vbus_pin[i]))
continue;
gpio = pdata->vbus_pin[i];
diff --git a/drivers/usb/host/pci-quirks.c b/drivers/usb/host/pci-quirks.c
index c5e9e4a..966d148 100644
--- a/drivers/usb/host/pci-quirks.c
+++ b/drivers/usb/host/pci-quirks.c
@@ -75,7 +75,9 @@
#define NB_PIF0_PWRDOWN_1 0x01100013
#define USB_INTEL_XUSB2PR 0xD0
+#define USB_INTEL_USB2PRM 0xD4
#define USB_INTEL_USB3_PSSEN 0xD8
+#define USB_INTEL_USB3PRM 0xDC
static struct amd_chipset_info {
struct pci_dev *nb_dev;
@@ -772,10 +774,18 @@
return;
}
- ports_available = 0xffffffff;
+ /* Read USB3PRM, the USB 3.0 Port Routing Mask Register
+ * Indicate the ports that can be changed from OS.
+ */
+ pci_read_config_dword(xhci_pdev, USB_INTEL_USB3PRM,
+ &ports_available);
+
+ dev_dbg(&xhci_pdev->dev, "Configurable ports to enable SuperSpeed: 0x%x\n",
+ ports_available);
+
/* Write USB3_PSSEN, the USB 3.0 Port SuperSpeed Enable
- * Register, to turn on SuperSpeed terminations for all
- * available ports.
+ * Register, to turn on SuperSpeed terminations for the
+ * switchable ports.
*/
pci_write_config_dword(xhci_pdev, USB_INTEL_USB3_PSSEN,
cpu_to_le32(ports_available));
@@ -785,7 +795,16 @@
dev_dbg(&xhci_pdev->dev, "USB 3.0 ports that are now enabled "
"under xHCI: 0x%x\n", ports_available);
- ports_available = 0xffffffff;
+ /* Read XUSB2PRM, xHCI USB 2.0 Port Routing Mask Register
+ * Indicate the USB 2.0 ports to be controlled by the xHCI host.
+ */
+
+ pci_read_config_dword(xhci_pdev, USB_INTEL_USB2PRM,
+ &ports_available);
+
+ dev_dbg(&xhci_pdev->dev, "Configurable USB 2.0 ports to hand over to xCHI: 0x%x\n",
+ ports_available);
+
/* Write XUSB2PR, the xHC USB 2.0 Port Routing Register, to
* switch the USB 2.0 power and data lines over to the xHCI
* host.
@@ -822,12 +841,12 @@
void __iomem *op_reg_base;
u32 val;
int timeout;
+ int len = pci_resource_len(pdev, 0);
if (!mmio_resource_enabled(pdev, 0))
return;
- base = ioremap_nocache(pci_resource_start(pdev, 0),
- pci_resource_len(pdev, 0));
+ base = ioremap_nocache(pci_resource_start(pdev, 0), len);
if (base == NULL)
return;
@@ -837,9 +856,17 @@
*/
ext_cap_offset = xhci_find_next_cap_offset(base, XHCI_HCC_PARAMS_OFFSET);
do {
+ if ((ext_cap_offset + sizeof(val)) > len) {
+ /* We're reading garbage from the controller */
+ dev_warn(&pdev->dev,
+ "xHCI controller failing to respond");
+ return;
+ }
+
if (!ext_cap_offset)
/* We've reached the end of the extended capabilities */
goto hc_init;
+
val = readl(base + ext_cap_offset);
if (XHCI_EXT_CAPS_ID(val) == XHCI_EXT_CAPS_LEGACY)
break;
@@ -870,9 +897,10 @@
/* Disable any BIOS SMIs and clear all SMI events*/
writel(val, base + ext_cap_offset + XHCI_LEGACY_CONTROL_OFFSET);
+hc_init:
if (usb_is_intel_switchable_xhci(pdev))
usb_enable_xhci_ports(pdev);
-hc_init:
+
op_reg_base = base + XHCI_HC_LENGTH(readl(base));
/* Wait for the host controller to be ready before writing any
diff --git a/drivers/usb/host/pci-quirks.h b/drivers/usb/host/pci-quirks.h
index ef004a5..7f69a39 100644
--- a/drivers/usb/host/pci-quirks.h
+++ b/drivers/usb/host/pci-quirks.h
@@ -15,6 +15,7 @@
static inline void usb_amd_quirk_pll_disable(void) {}
static inline void usb_amd_quirk_pll_enable(void) {}
static inline void usb_amd_dev_put(void) {}
+static inline void usb_disable_xhci_ports(struct pci_dev *xhci_pdev) {}
#endif /* CONFIG_PCI */
#endif /* __LINUX_USB_PCI_QUIRKS_H */
diff --git a/drivers/usb/host/xhci-hub.c b/drivers/usb/host/xhci-hub.c
index 74bfc86..d5eb357 100644
--- a/drivers/usb/host/xhci-hub.c
+++ b/drivers/usb/host/xhci-hub.c
@@ -493,11 +493,48 @@
* when this bit is set.
*/
pls |= USB_PORT_STAT_CONNECTION;
+ } else {
+ /*
+ * If CAS bit isn't set but the Port is already at
+ * Compliance Mode, fake a connection so the USB core
+ * notices the Compliance state and resets the port.
+ * This resolves an issue generated by the SN65LVPE502CP
+ * in which sometimes the port enters compliance mode
+ * caused by a delay on the host-device negotiation.
+ */
+ if (pls == USB_SS_PORT_LS_COMP_MOD)
+ pls |= USB_PORT_STAT_CONNECTION;
}
+
/* update status field */
*status |= pls;
}
+/*
+ * Function for Compliance Mode Quirk.
+ *
+ * This Function verifies if all xhc USB3 ports have entered U0, if so,
+ * the compliance mode timer is deleted. A port won't enter
+ * compliance mode if it has previously entered U0.
+ */
+void xhci_del_comp_mod_timer(struct xhci_hcd *xhci, u32 status, u16 wIndex)
+{
+ u32 all_ports_seen_u0 = ((1 << xhci->num_usb3_ports)-1);
+ bool port_in_u0 = ((status & PORT_PLS_MASK) == XDEV_U0);
+
+ if (!(xhci->quirks & XHCI_COMP_MODE_QUIRK))
+ return;
+
+ if ((xhci->port_status_u0 != all_ports_seen_u0) && port_in_u0) {
+ xhci->port_status_u0 |= 1 << wIndex;
+ if (xhci->port_status_u0 == all_ports_seen_u0) {
+ del_timer_sync(&xhci->comp_mode_recovery_timer);
+ xhci_dbg(xhci, "All USB3 ports have entered U0 already!\n");
+ xhci_dbg(xhci, "Compliance Mode Recovery Timer Deleted.\n");
+ }
+ }
+}
+
int xhci_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue,
u16 wIndex, char *buf, u16 wLength)
{
@@ -651,6 +688,11 @@
/* Update Port Link State for super speed ports*/
if (hcd->speed == HCD_USB3) {
xhci_hub_report_link_state(&status, temp);
+ /*
+ * Verify if all USB3 Ports Have entered U0 already.
+ * Delete Compliance Mode Timer if so.
+ */
+ xhci_del_comp_mod_timer(xhci, temp, wIndex);
}
if (bus_state->port_c_suspend & (1 << wIndex))
status |= 1 << USB_PORT_FEAT_C_SUSPEND;
diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c
index 689bc18..df90fe5 100644
--- a/drivers/usb/host/xhci-plat.c
+++ b/drivers/usb/host/xhci-plat.c
@@ -118,7 +118,7 @@
goto put_hcd;
}
- hcd->regs = ioremap(hcd->rsrc_start, hcd->rsrc_len);
+ hcd->regs = ioremap_nocache(hcd->rsrc_start, hcd->rsrc_len);
if (!hcd->regs) {
dev_dbg(&pdev->dev, "error mapping memory\n");
ret = -EFAULT;
diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index c59d5b5..6ece0ed 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -26,6 +26,7 @@
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/slab.h>
+#include <linux/dmi.h>
#include "xhci.h"
@@ -398,6 +399,95 @@
#endif
+static void compliance_mode_recovery(unsigned long arg)
+{
+ struct xhci_hcd *xhci;
+ struct usb_hcd *hcd;
+ u32 temp;
+ int i;
+
+ xhci = (struct xhci_hcd *)arg;
+
+ for (i = 0; i < xhci->num_usb3_ports; i++) {
+ temp = xhci_readl(xhci, xhci->usb3_ports[i]);
+ if ((temp & PORT_PLS_MASK) == USB_SS_PORT_LS_COMP_MOD) {
+ /*
+ * Compliance Mode Detected. Letting USB Core
+ * handle the Warm Reset
+ */
+ xhci_dbg(xhci, "Compliance Mode Detected->Port %d!\n",
+ i + 1);
+ xhci_dbg(xhci, "Attempting Recovery routine!\n");
+ hcd = xhci->shared_hcd;
+
+ if (hcd->state == HC_STATE_SUSPENDED)
+ usb_hcd_resume_root_hub(hcd);
+
+ usb_hcd_poll_rh_status(hcd);
+ }
+ }
+
+ if (xhci->port_status_u0 != ((1 << xhci->num_usb3_ports)-1))
+ mod_timer(&xhci->comp_mode_recovery_timer,
+ jiffies + msecs_to_jiffies(COMP_MODE_RCVRY_MSECS));
+}
+
+/*
+ * Quirk to work around issue generated by the SN65LVPE502CP USB3.0 re-driver
+ * that causes ports behind that hardware to enter compliance mode sometimes.
+ * The quirk creates a timer that polls every 2 seconds the link state of
+ * each host controller's port and recovers it by issuing a Warm reset
+ * if Compliance mode is detected, otherwise the port will become "dead" (no
+ * device connections or disconnections will be detected anymore). Becasue no
+ * status event is generated when entering compliance mode (per xhci spec),
+ * this quirk is needed on systems that have the failing hardware installed.
+ */
+static void compliance_mode_recovery_timer_init(struct xhci_hcd *xhci)
+{
+ xhci->port_status_u0 = 0;
+ init_timer(&xhci->comp_mode_recovery_timer);
+
+ xhci->comp_mode_recovery_timer.data = (unsigned long) xhci;
+ xhci->comp_mode_recovery_timer.function = compliance_mode_recovery;
+ xhci->comp_mode_recovery_timer.expires = jiffies +
+ msecs_to_jiffies(COMP_MODE_RCVRY_MSECS);
+
+ set_timer_slack(&xhci->comp_mode_recovery_timer,
+ msecs_to_jiffies(COMP_MODE_RCVRY_MSECS));
+ add_timer(&xhci->comp_mode_recovery_timer);
+ xhci_dbg(xhci, "Compliance Mode Recovery Timer Initialized.\n");
+}
+
+/*
+ * This function identifies the systems that have installed the SN65LVPE502CP
+ * USB3.0 re-driver and that need the Compliance Mode Quirk.
+ * Systems:
+ * Vendor: Hewlett-Packard -> System Models: Z420, Z620 and Z820
+ */
+static bool compliance_mode_recovery_timer_quirk_check(void)
+{
+ const char *dmi_product_name, *dmi_sys_vendor;
+
+ dmi_product_name = dmi_get_system_info(DMI_PRODUCT_NAME);
+ dmi_sys_vendor = dmi_get_system_info(DMI_SYS_VENDOR);
+
+ if (!(strstr(dmi_sys_vendor, "Hewlett-Packard")))
+ return false;
+
+ if (strstr(dmi_product_name, "Z420") ||
+ strstr(dmi_product_name, "Z620") ||
+ strstr(dmi_product_name, "Z820"))
+ return true;
+
+ return false;
+}
+
+static int xhci_all_ports_seen_u0(struct xhci_hcd *xhci)
+{
+ return (xhci->port_status_u0 == ((1 << xhci->num_usb3_ports)-1));
+}
+
+
/*
* Initialize memory for HCD and xHC (one-time init).
*
@@ -421,6 +511,12 @@
retval = xhci_mem_init(xhci, GFP_KERNEL);
xhci_dbg(xhci, "Finished xhci_init\n");
+ /* Initializing Compliance Mode Recovery Data If Needed */
+ if (compliance_mode_recovery_timer_quirk_check()) {
+ xhci->quirks |= XHCI_COMP_MODE_QUIRK;
+ compliance_mode_recovery_timer_init(xhci);
+ }
+
return retval;
}
@@ -629,6 +725,11 @@
del_timer_sync(&xhci->event_ring_timer);
#endif
+ /* Deleting Compliance Mode Recovery Timer */
+ if ((xhci->quirks & XHCI_COMP_MODE_QUIRK) &&
+ (!(xhci_all_ports_seen_u0(xhci))))
+ del_timer_sync(&xhci->comp_mode_recovery_timer);
+
if (xhci->quirks & XHCI_AMD_PLL_FIX)
usb_amd_dev_put();
@@ -659,7 +760,7 @@
{
struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- if (xhci->quirks && XHCI_SPURIOUS_REBOOT)
+ if (xhci->quirks & XHCI_SPURIOUS_REBOOT)
usb_disable_xhci_ports(to_pci_dev(hcd->self.controller));
spin_lock_irq(&xhci->lock);
@@ -806,6 +907,16 @@
}
spin_unlock_irq(&xhci->lock);
+ /*
+ * Deleting Compliance Mode Recovery Timer because the xHCI Host
+ * is about to be suspended.
+ */
+ if ((xhci->quirks & XHCI_COMP_MODE_QUIRK) &&
+ (!(xhci_all_ports_seen_u0(xhci)))) {
+ del_timer_sync(&xhci->comp_mode_recovery_timer);
+ xhci_dbg(xhci, "Compliance Mode Recovery Timer Deleted!\n");
+ }
+
/* step 5: remove core well power */
/* synchronize irq when using MSI-X */
xhci_msix_sync_irqs(xhci);
@@ -938,6 +1049,16 @@
usb_hcd_resume_root_hub(hcd);
usb_hcd_resume_root_hub(xhci->shared_hcd);
}
+
+ /*
+ * If system is subject to the Quirk, Compliance Mode Timer needs to
+ * be re-initialized Always after a system resume. Ports are subject
+ * to suffer the Compliance Mode issue again. It doesn't matter if
+ * ports have entered previously to U0 before system's suspension.
+ */
+ if (xhci->quirks & XHCI_COMP_MODE_QUIRK)
+ compliance_mode_recovery_timer_init(xhci);
+
return retval;
}
#endif /* CONFIG_PM */
diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h
index c713256..1a05908 100644
--- a/drivers/usb/host/xhci.h
+++ b/drivers/usb/host/xhci.h
@@ -1495,6 +1495,7 @@
#define XHCI_LPM_SUPPORT (1 << 11)
#define XHCI_INTEL_HOST (1 << 12)
#define XHCI_SPURIOUS_REBOOT (1 << 13)
+#define XHCI_COMP_MODE_QUIRK (1 << 14)
unsigned int num_active_eps;
unsigned int limit_active_eps;
/* There are two roothubs to keep track of bus suspend info for */
@@ -1511,6 +1512,11 @@
unsigned sw_lpm_support:1;
/* support xHCI 1.0 spec USB2 hardware LPM */
unsigned hw_lpm_support:1;
+ /* Compliance Mode Recovery Data */
+ struct timer_list comp_mode_recovery_timer;
+ u32 port_status_u0;
+/* Compliance Mode Timer Triggered every 2 seconds */
+#define COMP_MODE_RCVRY_MSECS 2000
};
/* convert between an HCD pointer and the corresponding EHCI_HCD */
diff --git a/drivers/usb/musb/musb_host.c b/drivers/usb/musb/musb_host.c
index 4bb717d..1ae378d 100644
--- a/drivers/usb/musb/musb_host.c
+++ b/drivers/usb/musb/musb_host.c
@@ -2049,7 +2049,7 @@
* we only have work to do in the former case.
*/
spin_lock_irqsave(&musb->lock, flags);
- if (hep->hcpriv) {
+ if (hep->hcpriv || !next_urb(qh)) {
/* some concurrent activity submitted another urb to hep...
* odd, rare, error prone, but legal.
*/
diff --git a/drivers/usb/musb/musbhsdma.c b/drivers/usb/musb/musbhsdma.c
index 57a6085..c1be687 100644
--- a/drivers/usb/musb/musbhsdma.c
+++ b/drivers/usb/musb/musbhsdma.c
@@ -388,7 +388,7 @@
struct platform_device *pdev = to_platform_device(dev);
int irq = platform_get_irq_byname(pdev, "dma");
- if (irq == 0) {
+ if (irq <= 0) {
dev_err(dev, "No DMA interrupt line!\n");
return NULL;
}
diff --git a/drivers/usb/musb/tusb6010.c b/drivers/usb/musb/tusb6010.c
index 1a1bd9c..3416254 100644
--- a/drivers/usb/musb/tusb6010.c
+++ b/drivers/usb/musb/tusb6010.c
@@ -1215,7 +1215,7 @@
ret = platform_device_add(musb);
if (ret) {
dev_err(&pdev->dev, "failed to register musb device\n");
- goto err1;
+ goto err2;
}
return 0;
diff --git a/drivers/usb/renesas_usbhs/fifo.c b/drivers/usb/renesas_usbhs/fifo.c
index ecd1730..143c4e9 100644
--- a/drivers/usb/renesas_usbhs/fifo.c
+++ b/drivers/usb/renesas_usbhs/fifo.c
@@ -818,7 +818,7 @@
usbhs_pipe_is_dcp(pipe))
goto usbhsf_pio_prepare_push;
- if (len % 4) /* 32bit alignment */
+ if (len & 0x7) /* 8byte alignment */
goto usbhsf_pio_prepare_push;
if ((uintptr_t)(pkt->buf + pkt->actual) & 0x7) /* 8byte alignment */
@@ -905,7 +905,7 @@
/* use PIO if packet is less than pio_dma_border */
len = usbhsf_fifo_rcv_len(priv, fifo);
len = min(pkt->length - pkt->actual, len);
- if (len % 4) /* 32bit alignment */
+ if (len & 0x7) /* 8byte alignment */
goto usbhsf_pio_prepare_pop_unselect;
if (len < usbhs_get_dparam(priv, pio_dma_border))
diff --git a/drivers/usb/serial/ftdi_sio.c b/drivers/usb/serial/ftdi_sio.c
index 5620db6..f906b3a 100644
--- a/drivers/usb/serial/ftdi_sio.c
+++ b/drivers/usb/serial/ftdi_sio.c
@@ -704,6 +704,7 @@
{ USB_DEVICE(FTDI_VID, FTDI_PCDJ_DAC2_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_RRCIRKITS_LOCOBUFFER_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_ASK_RDR400_PID) },
+ { USB_DEVICE(FTDI_VID, FTDI_NZR_SEM_USB_PID) },
{ USB_DEVICE(ICOM_VID, ICOM_ID_1_PID) },
{ USB_DEVICE(ICOM_VID, ICOM_OPC_U_UC_PID) },
{ USB_DEVICE(ICOM_VID, ICOM_ID_RP2C1_PID) },
@@ -804,13 +805,32 @@
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
{ USB_DEVICE(ADI_VID, ADI_GNICEPLUS_PID),
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
- { USB_DEVICE(MICROCHIP_VID, MICROCHIP_USB_BOARD_PID) },
+ { USB_DEVICE_AND_INTERFACE_INFO(MICROCHIP_VID, MICROCHIP_USB_BOARD_PID,
+ USB_CLASS_VENDOR_SPEC,
+ USB_SUBCLASS_VENDOR_SPEC, 0x00) },
{ USB_DEVICE(JETI_VID, JETI_SPC1201_PID) },
{ USB_DEVICE(MARVELL_VID, MARVELL_SHEEVAPLUG_PID),
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
{ USB_DEVICE(LARSENBRUSGAARD_VID, LB_ALTITRACK_PID) },
{ USB_DEVICE(GN_OTOMETRICS_VID, AURICAL_USB_PID) },
+ { USB_DEVICE(FTDI_VID, PI_C865_PID) },
+ { USB_DEVICE(FTDI_VID, PI_C857_PID) },
+ { USB_DEVICE(PI_VID, PI_C866_PID) },
+ { USB_DEVICE(PI_VID, PI_C663_PID) },
+ { USB_DEVICE(PI_VID, PI_C725_PID) },
+ { USB_DEVICE(PI_VID, PI_E517_PID) },
+ { USB_DEVICE(PI_VID, PI_C863_PID) },
{ USB_DEVICE(PI_VID, PI_E861_PID) },
+ { USB_DEVICE(PI_VID, PI_C867_PID) },
+ { USB_DEVICE(PI_VID, PI_E609_PID) },
+ { USB_DEVICE(PI_VID, PI_E709_PID) },
+ { USB_DEVICE(PI_VID, PI_100F_PID) },
+ { USB_DEVICE(PI_VID, PI_1011_PID) },
+ { USB_DEVICE(PI_VID, PI_1012_PID) },
+ { USB_DEVICE(PI_VID, PI_1013_PID) },
+ { USB_DEVICE(PI_VID, PI_1014_PID) },
+ { USB_DEVICE(PI_VID, PI_1015_PID) },
+ { USB_DEVICE(PI_VID, PI_1016_PID) },
{ USB_DEVICE(KONDO_VID, KONDO_USB_SERIAL_PID) },
{ USB_DEVICE(BAYER_VID, BAYER_CONTOUR_CABLE_PID) },
{ USB_DEVICE(FTDI_VID, MARVELL_OPENRD_PID),
diff --git a/drivers/usb/serial/ftdi_sio_ids.h b/drivers/usb/serial/ftdi_sio_ids.h
index 5dd96ca..41fe582 100644
--- a/drivers/usb/serial/ftdi_sio_ids.h
+++ b/drivers/usb/serial/ftdi_sio_ids.h
@@ -75,6 +75,9 @@
#define FTDI_OPENDCC_GATEWAY_PID 0xBFDB
#define FTDI_OPENDCC_GBM_PID 0xBFDC
+/* NZR SEM 16+ USB (http://www.nzr.de) */
+#define FTDI_NZR_SEM_USB_PID 0xC1E0 /* NZR SEM-LOG16+ */
+
/*
* RR-CirKits LocoBuffer USB (http://www.rr-cirkits.com)
*/
@@ -539,7 +542,10 @@
/*
* Microchip Technology, Inc.
*
- * MICROCHIP_VID (0x04D8) and MICROCHIP_USB_BOARD_PID (0x000A) are also used by:
+ * MICROCHIP_VID (0x04D8) and MICROCHIP_USB_BOARD_PID (0x000A) are
+ * used by single function CDC ACM class based firmware demo
+ * applications. The VID/PID has also been used in firmware
+ * emulating FTDI serial chips by:
* Hornby Elite - Digital Command Control Console
* http://www.hornby.com/hornby-dcc/controllers/
*/
@@ -791,8 +797,27 @@
* Physik Instrumente
* http://www.physikinstrumente.com/en/products/
*/
+/* These two devices use the VID of FTDI */
+#define PI_C865_PID 0xe0a0 /* PI C-865 Piezomotor Controller */
+#define PI_C857_PID 0xe0a1 /* PI Encoder Trigger Box */
+
#define PI_VID 0x1a72 /* Vendor ID */
-#define PI_E861_PID 0x1008 /* E-861 piezo controller USB connection */
+#define PI_C866_PID 0x1000 /* PI C-866 Piezomotor Controller */
+#define PI_C663_PID 0x1001 /* PI C-663 Mercury-Step */
+#define PI_C725_PID 0x1002 /* PI C-725 Piezomotor Controller */
+#define PI_E517_PID 0x1005 /* PI E-517 Digital Piezo Controller Operation Module */
+#define PI_C863_PID 0x1007 /* PI C-863 */
+#define PI_E861_PID 0x1008 /* PI E-861 Piezomotor Controller */
+#define PI_C867_PID 0x1009 /* PI C-867 Piezomotor Controller */
+#define PI_E609_PID 0x100D /* PI E-609 Digital Piezo Controller */
+#define PI_E709_PID 0x100E /* PI E-709 Digital Piezo Controller */
+#define PI_100F_PID 0x100F /* PI Digital Piezo Controller */
+#define PI_1011_PID 0x1011 /* PI Digital Piezo Controller */
+#define PI_1012_PID 0x1012 /* PI Motion Controller */
+#define PI_1013_PID 0x1013 /* PI Motion Controller */
+#define PI_1014_PID 0x1014 /* PI Device */
+#define PI_1015_PID 0x1015 /* PI Device */
+#define PI_1016_PID 0x1016 /* PI Digital Servo Module */
/*
* Kondo Kagaku Co.Ltd.
diff --git a/drivers/usb/serial/option.c b/drivers/usb/serial/option.c
index cc40f47..5ce88d1 100644
--- a/drivers/usb/serial/option.c
+++ b/drivers/usb/serial/option.c
@@ -886,8 +886,6 @@
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x1010, 0xff, 0xff, 0xff),
.driver_info = (kernel_ulong_t)&net_intf4_blacklist },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x1012, 0xff, 0xff, 0xff) },
- { USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x1018, 0xff, 0xff, 0xff),
- .driver_info = (kernel_ulong_t)&net_intf3_blacklist },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x1057, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x1058, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0x1059, 0xff, 0xff, 0xff) },
@@ -1092,6 +1090,10 @@
.driver_info = (kernel_ulong_t)&zte_ad3812_z_blacklist },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, ZTE_PRODUCT_MC2716, 0xff, 0xff, 0xff),
.driver_info = (kernel_ulong_t)&zte_mc2716_z_blacklist },
+ { USB_VENDOR_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff, 0x02, 0x01) },
+ { USB_VENDOR_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff, 0x02, 0x05) },
+ { USB_VENDOR_AND_INTERFACE_INFO(ZTE_VENDOR_ID, 0xff, 0x86, 0x10) },
+
{ USB_DEVICE(BENQ_VENDOR_ID, BENQ_PRODUCT_H10) },
{ USB_DEVICE(DLINK_VENDOR_ID, DLINK_PRODUCT_DWM_652) },
{ USB_DEVICE(ALINK_VENDOR_ID, DLINK_PRODUCT_DWM_652_U5) }, /* Yes, ALINK_VENDOR_ID */
diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c
index 211a492..d8dedc7 100644
--- a/drivers/vfio/pci/vfio_pci_intrs.c
+++ b/drivers/vfio/pci/vfio_pci_intrs.c
@@ -76,9 +76,24 @@
schedule_work(&virqfd->inject);
}
- if (flags & POLLHUP)
- /* The eventfd is closing, detach from VFIO */
- virqfd_deactivate(virqfd);
+ if (flags & POLLHUP) {
+ unsigned long flags;
+ spin_lock_irqsave(&virqfd->vdev->irqlock, flags);
+
+ /*
+ * The eventfd is closing, if the virqfd has not yet been
+ * queued for release, as determined by testing whether the
+ * vdev pointer to it is still valid, queue it now. As
+ * with kvm irqfds, we know we won't race against the virqfd
+ * going away because we hold wqh->lock to get here.
+ */
+ if (*(virqfd->pvirqfd) == virqfd) {
+ *(virqfd->pvirqfd) = NULL;
+ virqfd_deactivate(virqfd);
+ }
+
+ spin_unlock_irqrestore(&virqfd->vdev->irqlock, flags);
+ }
return 0;
}
@@ -93,7 +108,6 @@
static void virqfd_shutdown(struct work_struct *work)
{
struct virqfd *virqfd = container_of(work, struct virqfd, shutdown);
- struct virqfd **pvirqfd = virqfd->pvirqfd;
u64 cnt;
eventfd_ctx_remove_wait_queue(virqfd->eventfd, &virqfd->wait, &cnt);
@@ -101,7 +115,6 @@
eventfd_ctx_put(virqfd->eventfd);
kfree(virqfd);
- *pvirqfd = NULL;
}
static void virqfd_inject(struct work_struct *work)
@@ -122,15 +135,11 @@
int ret = 0;
unsigned int events;
- if (*pvirqfd)
- return -EBUSY;
-
virqfd = kzalloc(sizeof(*virqfd), GFP_KERNEL);
if (!virqfd)
return -ENOMEM;
virqfd->pvirqfd = pvirqfd;
- *pvirqfd = virqfd;
virqfd->vdev = vdev;
virqfd->handler = handler;
virqfd->thread = thread;
@@ -154,6 +163,23 @@
virqfd->eventfd = ctx;
/*
+ * virqfds can be released by closing the eventfd or directly
+ * through ioctl. These are both done through a workqueue, so
+ * we update the pointer to the virqfd under lock to avoid
+ * pushing multiple jobs to release the same virqfd.
+ */
+ spin_lock_irq(&vdev->irqlock);
+
+ if (*pvirqfd) {
+ spin_unlock_irq(&vdev->irqlock);
+ ret = -EBUSY;
+ goto fail;
+ }
+ *pvirqfd = virqfd;
+
+ spin_unlock_irq(&vdev->irqlock);
+
+ /*
* Install our own custom wake-up handling so we are notified via
* a callback whenever someone signals the underlying eventfd.
*/
@@ -187,19 +213,29 @@
fput(file);
kfree(virqfd);
- *pvirqfd = NULL;
return ret;
}
-static void virqfd_disable(struct virqfd *virqfd)
+static void virqfd_disable(struct vfio_pci_device *vdev,
+ struct virqfd **pvirqfd)
{
- if (!virqfd)
- return;
+ unsigned long flags;
- virqfd_deactivate(virqfd);
+ spin_lock_irqsave(&vdev->irqlock, flags);
- /* Block until we know all outstanding shutdown jobs have completed. */
+ if (*pvirqfd) {
+ virqfd_deactivate(*pvirqfd);
+ *pvirqfd = NULL;
+ }
+
+ spin_unlock_irqrestore(&vdev->irqlock, flags);
+
+ /*
+ * Block until we know all outstanding shutdown jobs have completed.
+ * Even if we don't queue the job, flush the wq to be sure it's
+ * been released.
+ */
flush_workqueue(vfio_irqfd_cleanup_wq);
}
@@ -392,8 +428,8 @@
static void vfio_intx_disable(struct vfio_pci_device *vdev)
{
vfio_intx_set_signal(vdev, -1);
- virqfd_disable(vdev->ctx[0].unmask);
- virqfd_disable(vdev->ctx[0].mask);
+ virqfd_disable(vdev, &vdev->ctx[0].unmask);
+ virqfd_disable(vdev, &vdev->ctx[0].mask);
vdev->irq_type = VFIO_PCI_NUM_IRQS;
vdev->num_ctx = 0;
kfree(vdev->ctx);
@@ -539,8 +575,8 @@
vfio_msi_set_block(vdev, 0, vdev->num_ctx, NULL, msix);
for (i = 0; i < vdev->num_ctx; i++) {
- virqfd_disable(vdev->ctx[i].unmask);
- virqfd_disable(vdev->ctx[i].mask);
+ virqfd_disable(vdev, &vdev->ctx[i].unmask);
+ virqfd_disable(vdev, &vdev->ctx[i].mask);
}
if (msix) {
@@ -577,7 +613,7 @@
vfio_send_intx_eventfd, NULL,
&vdev->ctx[0].unmask, fd);
- virqfd_disable(vdev->ctx[0].unmask);
+ virqfd_disable(vdev, &vdev->ctx[0].unmask);
}
return 0;
diff --git a/drivers/video/auo_k190x.c b/drivers/video/auo_k190x.c
index 77da6a2..c03ecdd 100644
--- a/drivers/video/auo_k190x.c
+++ b/drivers/video/auo_k190x.c
@@ -987,7 +987,6 @@
fb_dealloc_cmap(&info->cmap);
err_cmap:
fb_deferred_io_cleanup(info);
- kfree(info->fbdefio);
err_defio:
vfree((void *)info->screen_base);
err_irq:
@@ -1022,7 +1021,6 @@
fb_dealloc_cmap(&info->cmap);
fb_deferred_io_cleanup(info);
- kfree(info->fbdefio);
vfree((void *)info->screen_base);
diff --git a/drivers/video/backlight/88pm860x_bl.c b/drivers/video/backlight/88pm860x_bl.c
index f75da87..f49181c 100644
--- a/drivers/video/backlight/88pm860x_bl.c
+++ b/drivers/video/backlight/88pm860x_bl.c
@@ -228,7 +228,6 @@
data->port = pdata->flags;
if (data->port < 0) {
dev_err(&pdev->dev, "wrong platform data is assigned");
- kfree(data);
return -EINVAL;
}
diff --git a/drivers/video/console/bitblit.c b/drivers/video/console/bitblit.c
index 28b1a83..61b182b 100644
--- a/drivers/video/console/bitblit.c
+++ b/drivers/video/console/bitblit.c
@@ -162,7 +162,7 @@
image.depth = 1;
if (attribute) {
- buf = kmalloc(cellsize, GFP_KERNEL);
+ buf = kmalloc(cellsize, GFP_ATOMIC);
if (!buf)
return;
}
diff --git a/drivers/video/console/fbcon.c b/drivers/video/console/fbcon.c
index 88e9204..fdefa8f 100644
--- a/drivers/video/console/fbcon.c
+++ b/drivers/video/console/fbcon.c
@@ -449,7 +449,7 @@
while ((options = strsep(&this_opt, ",")) != NULL) {
if (!strncmp(options, "font:", 5))
- strcpy(fontname, options + 5);
+ strlcpy(fontname, options + 5, sizeof(fontname));
if (!strncmp(options, "scrollback:", 11)) {
options += 11;
diff --git a/drivers/video/efifb.c b/drivers/video/efifb.c
index b4a632a..932abaa 100644
--- a/drivers/video/efifb.c
+++ b/drivers/video/efifb.c
@@ -553,7 +553,9 @@
int ret;
char *option = NULL;
- dmi_check_system(dmi_system_table);
+ if (screen_info.orig_video_isVGA != VIDEO_TYPE_EFI ||
+ !(screen_info.capabilities & VIDEO_CAPABILITY_SKIP_QUIRKS))
+ dmi_check_system(dmi_system_table);
if (screen_info.orig_video_isVGA != VIDEO_TYPE_EFI)
return -ENODEV;
diff --git a/drivers/video/exynos/exynos_mipi_dsi.c b/drivers/video/exynos/exynos_mipi_dsi.c
index 4bc2b8a..663c308 100644
--- a/drivers/video/exynos/exynos_mipi_dsi.c
+++ b/drivers/video/exynos/exynos_mipi_dsi.c
@@ -461,7 +461,7 @@
done:
platform_set_drvdata(pdev, dsim);
- dev_dbg(&pdev->dev, "%s() completed sucessfuly (%s mode)\n", __func__,
+ dev_dbg(&pdev->dev, "%s() completed successfully (%s mode)\n", __func__,
dsim_config->e_interface == DSIM_COMMAND ? "CPU" : "RGB");
return 0;
diff --git a/drivers/video/mb862xx/mb862xxfbdrv.c b/drivers/video/mb862xx/mb862xxfbdrv.c
index 00ce1f3..57d940b 100644
--- a/drivers/video/mb862xx/mb862xxfbdrv.c
+++ b/drivers/video/mb862xx/mb862xxfbdrv.c
@@ -328,6 +328,8 @@
case MB862XX_L1_SET_CFG:
if (copy_from_user(l1_cfg, argp, sizeof(*l1_cfg)))
return -EFAULT;
+ if (l1_cfg->dh == 0 || l1_cfg->dw == 0)
+ return -EINVAL;
if ((l1_cfg->sw >= l1_cfg->dw) && (l1_cfg->sh >= l1_cfg->dh)) {
/* downscaling */
outreg(cap, GC_CAP_CSC,
diff --git a/drivers/video/omap2/dss/sdi.c b/drivers/video/omap2/dss/sdi.c
index 5d31699..f43bfe1 100644
--- a/drivers/video/omap2/dss/sdi.c
+++ b/drivers/video/omap2/dss/sdi.c
@@ -105,6 +105,20 @@
sdi_config_lcd_manager(dssdev);
+ /*
+ * LCLK and PCLK divisors are located in shadow registers, and we
+ * normally write them to DISPC registers when enabling the output.
+ * However, SDI uses pck-free as source clock for its PLL, and pck-free
+ * is affected by the divisors. And as we need the PLL before enabling
+ * the output, we need to write the divisors early.
+ *
+ * It seems just writing to the DISPC register is enough, and we don't
+ * need to care about the shadow register mechanism for pck-free. The
+ * exact reason for this is unknown.
+ */
+ dispc_mgr_set_clock_div(dssdev->manager->id,
+ &sdi.mgr_config.clock_info);
+
dss_sdi_init(dssdev->phy.sdi.datapairs);
r = dss_sdi_enable();
if (r)
diff --git a/drivers/video/omap2/omapfb/omapfb-main.c b/drivers/video/omap2/omapfb/omapfb-main.c
index 08ec1a7..fc671d3 100644
--- a/drivers/video/omap2/omapfb/omapfb-main.c
+++ b/drivers/video/omap2/omapfb/omapfb-main.c
@@ -1192,7 +1192,7 @@
break;
if (regno < 16) {
- u16 pal;
+ u32 pal;
pal = ((red >> (16 - var->red.length)) <<
var->red.offset) |
((green >> (16 - var->green.length)) <<
diff --git a/drivers/video/tmiofb.c b/drivers/video/tmiofb.c
index 8e4a446..b244f06 100644
--- a/drivers/video/tmiofb.c
+++ b/drivers/video/tmiofb.c
@@ -694,6 +694,10 @@
dev_err(&dev->dev, "NULL platform data!\n");
return -EINVAL;
}
+ if (ccr == NULL || lcr == NULL || vram == NULL || irq < 0) {
+ dev_err(&dev->dev, "missing resources\n");
+ return -EINVAL;
+ }
info = framebuffer_alloc(sizeof(struct tmiofb_par), &dev->dev);
diff --git a/drivers/w1/masters/ds1wm.c b/drivers/w1/masters/ds1wm.c
index 530a2d3..7c294f4 100644
--- a/drivers/w1/masters/ds1wm.c
+++ b/drivers/w1/masters/ds1wm.c
@@ -349,7 +349,7 @@
"pass: %d entering ASM\n", pass);
ds1wm_write_register(ds1wm_data, DS1WM_CMD, DS1WM_CMD_SRA);
dev_dbg(&ds1wm_data->pdev->dev,
- "pass: %d begining nibble loop\n", pass);
+ "pass: %d beginning nibble loop\n", pass);
r_prime = 0;
d = 0;
diff --git a/drivers/watchdog/booke_wdt.c b/drivers/watchdog/booke_wdt.c
index 3fe82d0..5b06d31 100644
--- a/drivers/watchdog/booke_wdt.c
+++ b/drivers/watchdog/booke_wdt.c
@@ -166,18 +166,17 @@
switch (cmd) {
case WDIOC_GETSUPPORT:
- if (copy_to_user((void *)arg, &ident, sizeof(ident)))
- return -EFAULT;
+ return copy_to_user(p, &ident, sizeof(ident)) ? -EFAULT : 0;
case WDIOC_GETSTATUS:
return put_user(0, p);
case WDIOC_GETBOOTSTATUS:
/* XXX: something is clearing TSR */
tmp = mfspr(SPRN_TSR) & TSR_WRS(3);
/* returns CARDRESET if last reset was caused by the WDT */
- return (tmp ? WDIOF_CARDRESET : 0);
+ return put_user((tmp ? WDIOF_CARDRESET : 0), p);
case WDIOC_SETOPTIONS:
if (get_user(tmp, p))
- return -EINVAL;
+ return -EFAULT;
if (tmp == WDIOS_ENABLECARD) {
booke_wdt_ping();
break;
diff --git a/drivers/watchdog/da9052_wdt.c b/drivers/watchdog/da9052_wdt.c
index 3f75129..f7abbae 100644
--- a/drivers/watchdog/da9052_wdt.c
+++ b/drivers/watchdog/da9052_wdt.c
@@ -21,7 +21,6 @@
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/jiffies.h>
-#include <linux/delay.h>
#include <linux/mfd/da9052/reg.h>
#include <linux/mfd/da9052/da9052.h>
diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c
index 1eff743..ae60406 100644
--- a/drivers/watchdog/hpwdt.c
+++ b/drivers/watchdog/hpwdt.c
@@ -814,6 +814,9 @@
hpwdt_timer_reg = pci_mem_addr + 0x70;
hpwdt_timer_con = pci_mem_addr + 0x72;
+ /* Make sure that timer is disabled until /dev/watchdog is opened */
+ hpwdt_stop();
+
/* Make sure that we have a valid soft_margin */
if (hpwdt_change_timer(soft_margin))
hpwdt_change_timer(DEFAULT_MARGIN);
diff --git a/drivers/watchdog/watchdog_core.c b/drivers/watchdog/watchdog_core.c
index 6aa46a9..3796434 100644
--- a/drivers/watchdog/watchdog_core.c
+++ b/drivers/watchdog/watchdog_core.c
@@ -128,11 +128,12 @@
void watchdog_unregister_device(struct watchdog_device *wdd)
{
int ret;
- int devno = wdd->cdev.dev;
+ int devno;
if (wdd == NULL)
return;
+ devno = wdd->cdev.dev;
ret = watchdog_dev_unregister(wdd);
if (ret)
pr_err("error unregistering /dev/watchdog (err=%d)\n", ret);
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 1ffd03b..7f12416 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -314,8 +314,9 @@
}
}
- err = gnttab_unmap_refs(map->unmap_ops + offset, map->pages + offset,
- pages, true);
+ err = gnttab_unmap_refs(map->unmap_ops + offset,
+ use_ptemod ? map->kmap_ops + offset : NULL, map->pages + offset,
+ pages);
if (err)
return err;
diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c
index 0bfc1ef..0067266 100644
--- a/drivers/xen/grant-table.c
+++ b/drivers/xen/grant-table.c
@@ -870,7 +870,8 @@
EXPORT_SYMBOL_GPL(gnttab_map_refs);
int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops,
- struct page **pages, unsigned int count, bool clear_pte)
+ struct gnttab_map_grant_ref *kmap_ops,
+ struct page **pages, unsigned int count)
{
int i, ret;
bool lazy = false;
@@ -888,7 +889,8 @@
}
for (i = 0; i < count; i++) {
- ret = m2p_remove_override(pages[i], clear_pte);
+ ret = m2p_remove_override(pages[i], kmap_ops ?
+ &kmap_ops[i] : NULL);
if (ret)
return ret;
}
diff --git a/drivers/xen/platform-pci.c b/drivers/xen/platform-pci.c
index d4c50d6..97ca359 100644
--- a/drivers/xen/platform-pci.c
+++ b/drivers/xen/platform-pci.c
@@ -101,19 +101,6 @@
return 0;
}
-static void __devinit prepare_shared_info(void)
-{
-#ifdef CONFIG_KEXEC
- unsigned long addr;
- struct shared_info *hvm_shared_info;
-
- addr = alloc_xen_mmio(PAGE_SIZE);
- hvm_shared_info = ioremap(addr, PAGE_SIZE);
- memset(hvm_shared_info, 0, PAGE_SIZE);
- xen_hvm_prepare_kexec(hvm_shared_info, addr >> PAGE_SHIFT);
-#endif
-}
-
static int __devinit platform_pci_init(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
@@ -151,8 +138,6 @@
platform_mmio = mmio_addr;
platform_mmiolen = mmio_len;
- prepare_shared_info();
-
if (!xen_have_vector_callback) {
ret = xen_allocate_irq(pdev);
if (ret) {
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 1afb4fb..4d51948 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -232,7 +232,7 @@
return ret;
if (hwdev && hwdev->coherent_dma_mask)
- dma_mask = hwdev->coherent_dma_mask;
+ dma_mask = dma_alloc_coherent_mask(hwdev, flags);
phys = virt_to_phys(ret);
dev_addr = xen_phys_to_bus(phys);
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 097e536..92ff01d 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -353,16 +353,16 @@
if (err)
goto config_release;
- dev_dbg(&dev->dev, "reseting (FLR, D3, etc) the device\n");
- __pci_reset_function_locked(dev);
-
/* We need the device active to save the state. */
dev_dbg(&dev->dev, "save state of device\n");
pci_save_state(dev);
dev_data->pci_saved_state = pci_store_saved_state(dev);
if (!dev_data->pci_saved_state)
dev_err(&dev->dev, "Could not store PCI conf saved state!\n");
-
+ else {
+ dev_dbg(&dev->dev, "reseting (FLR, D3, etc) the device\n");
+ __pci_reset_function_locked(dev);
+ }
/* Now disable the device (this also ensures some private device
* data is setup before we export)
*/
@@ -871,7 +871,7 @@
}
/*add xen_pcibk AER handling*/
-static struct pci_error_handlers xen_pcibk_error_handler = {
+static const struct pci_error_handlers xen_pcibk_error_handler = {
.error_detected = xen_pcibk_error_detected,
.mmio_enabled = xen_pcibk_mmio_enabled,
.slot_reset = xen_pcibk_slot_reset,
diff --git a/fs/bio.c b/fs/bio.c
index 5eaa70c..71072ab 100644
--- a/fs/bio.c
+++ b/fs/bio.c
@@ -73,7 +73,7 @@
{
unsigned int sz = sizeof(struct bio) + extra_size;
struct kmem_cache *slab = NULL;
- struct bio_slab *bslab;
+ struct bio_slab *bslab, *new_bio_slabs;
unsigned int i, entry = -1;
mutex_lock(&bio_slab_lock);
@@ -97,11 +97,12 @@
if (bio_slab_nr == bio_slab_max && entry == -1) {
bio_slab_max <<= 1;
- bio_slabs = krealloc(bio_slabs,
- bio_slab_max * sizeof(struct bio_slab),
- GFP_KERNEL);
- if (!bio_slabs)
+ new_bio_slabs = krealloc(bio_slabs,
+ bio_slab_max * sizeof(struct bio_slab),
+ GFP_KERNEL);
+ if (!new_bio_slabs)
goto out_unlock;
+ bio_slabs = new_bio_slabs;
}
if (entry == -1)
entry = bio_slab_nr++;
diff --git a/fs/block_dev.c b/fs/block_dev.c
index 1e51919..38e721b 100644
--- a/fs/block_dev.c
+++ b/fs/block_dev.c
@@ -1578,10 +1578,12 @@
unsigned long nr_segs, loff_t pos)
{
struct file *file = iocb->ki_filp;
+ struct blk_plug plug;
ssize_t ret;
BUG_ON(iocb->ki_pos != pos);
+ blk_start_plug(&plug);
ret = __generic_file_aio_write(iocb, iov, nr_segs, &iocb->ki_pos);
if (ret > 0 || ret == -EIOCBQUEUED) {
ssize_t err;
@@ -1590,6 +1592,7 @@
if (err < 0 && ret > 0)
ret = err;
}
+ blk_finish_plug(&plug);
return ret;
}
EXPORT_SYMBOL_GPL(blkdev_aio_write);
diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c
index a256f3b..ff6475f 100644
--- a/fs/btrfs/backref.c
+++ b/fs/btrfs/backref.c
@@ -1438,10 +1438,10 @@
ret = extent_from_logical(fs_info, logical, path,
&found_key);
btrfs_release_path(path);
- if (ret & BTRFS_EXTENT_FLAG_TREE_BLOCK)
- ret = -EINVAL;
if (ret < 0)
return ret;
+ if (ret & BTRFS_EXTENT_FLAG_TREE_BLOCK)
+ return -EINVAL;
extent_item_pos = logical - found_key.objectid;
ret = iterate_extent_inodes(fs_info, found_key.objectid,
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index 86eff48..43d1c5a 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -818,6 +818,7 @@
btrfs_compress_op[idx]->free_workspace(workspace);
atomic_dec(alloc_workspace);
wake:
+ smp_mb();
if (waitqueue_active(workspace_wait))
wake_up(workspace_wait);
}
diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 9d7621f..6d183f6 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
@@ -421,12 +421,6 @@
spin_unlock(&fs_info->tree_mod_seq_lock);
/*
- * we removed the lowest blocker from the blocker list, so there may be
- * more processible delayed refs.
- */
- wake_up(&fs_info->tree_mod_seq_wait);
-
- /*
* anything that's lower than the lowest existing (read: blocked)
* sequence number can be removed from the tree.
*/
@@ -631,6 +625,9 @@
u32 nritems;
int ret;
+ if (btrfs_header_level(eb) == 0)
+ return;
+
nritems = btrfs_header_nritems(eb);
for (i = nritems - 1; i >= 0; i--) {
ret = tree_mod_log_insert_key_locked(fs_info, eb, i,
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 4bab807..9821b67 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -116,7 +116,7 @@
#define BTRFS_FREE_SPACE_OBJECTID -11ULL
/*
- * The inode number assigned to the special inode for sotring
+ * The inode number assigned to the special inode for storing
* free ino cache
*/
#define BTRFS_FREE_INO_OBJECTID -12ULL
@@ -1252,7 +1252,6 @@
atomic_t tree_mod_seq;
struct list_head tree_mod_seq_list;
struct seq_list tree_mod_seq_elem;
- wait_queue_head_t tree_mod_seq_wait;
/* this protects tree_mod_log */
rwlock_t tree_mod_log_lock;
@@ -3192,7 +3191,7 @@
int btrfs_lookup_bio_sums(struct btrfs_root *root, struct inode *inode,
struct bio *bio, u32 *dst);
int btrfs_lookup_bio_sums_dio(struct btrfs_root *root, struct inode *inode,
- struct bio *bio, u64 logical_offset, u32 *dst);
+ struct bio *bio, u64 logical_offset);
int btrfs_insert_file_extent(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
u64 objectid, u64 pos,
diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c
index 335605c..07d5eeb 100644
--- a/fs/btrfs/delayed-inode.c
+++ b/fs/btrfs/delayed-inode.c
@@ -512,8 +512,8 @@
rb_erase(&delayed_item->rb_node, root);
delayed_item->delayed_node->count--;
- atomic_dec(&delayed_root->items);
- if (atomic_read(&delayed_root->items) < BTRFS_DELAYED_BACKGROUND &&
+ if (atomic_dec_return(&delayed_root->items) <
+ BTRFS_DELAYED_BACKGROUND &&
waitqueue_active(&delayed_root->wait))
wake_up(&delayed_root->wait);
}
@@ -1028,9 +1028,10 @@
btrfs_release_delayed_item(prev);
ret = 0;
btrfs_release_path(path);
- if (curr)
+ if (curr) {
+ mutex_unlock(&node->mutex);
goto do_again;
- else
+ } else
goto delete_fail;
}
@@ -1055,8 +1056,7 @@
delayed_node->count--;
delayed_root = delayed_node->root->fs_info->delayed_root;
- atomic_dec(&delayed_root->items);
- if (atomic_read(&delayed_root->items) <
+ if (atomic_dec_return(&delayed_root->items) <
BTRFS_DELAYED_BACKGROUND &&
waitqueue_active(&delayed_root->wait))
wake_up(&delayed_root->wait);
diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c
index da7419e..ae94117 100644
--- a/fs/btrfs/delayed-ref.c
+++ b/fs/btrfs/delayed-ref.c
@@ -38,17 +38,14 @@
static int comp_tree_refs(struct btrfs_delayed_tree_ref *ref2,
struct btrfs_delayed_tree_ref *ref1)
{
- if (ref1->node.type == BTRFS_TREE_BLOCK_REF_KEY) {
- if (ref1->root < ref2->root)
- return -1;
- if (ref1->root > ref2->root)
- return 1;
- } else {
- if (ref1->parent < ref2->parent)
- return -1;
- if (ref1->parent > ref2->parent)
- return 1;
- }
+ if (ref1->root < ref2->root)
+ return -1;
+ if (ref1->root > ref2->root)
+ return 1;
+ if (ref1->parent < ref2->parent)
+ return -1;
+ if (ref1->parent > ref2->parent)
+ return 1;
return 0;
}
@@ -85,7 +82,8 @@
* type of the delayed backrefs and content of delayed backrefs.
*/
static int comp_entry(struct btrfs_delayed_ref_node *ref2,
- struct btrfs_delayed_ref_node *ref1)
+ struct btrfs_delayed_ref_node *ref1,
+ bool compare_seq)
{
if (ref1->bytenr < ref2->bytenr)
return -1;
@@ -102,10 +100,12 @@
if (ref1->type > ref2->type)
return 1;
/* merging of sequenced refs is not allowed */
- if (ref1->seq < ref2->seq)
- return -1;
- if (ref1->seq > ref2->seq)
- return 1;
+ if (compare_seq) {
+ if (ref1->seq < ref2->seq)
+ return -1;
+ if (ref1->seq > ref2->seq)
+ return 1;
+ }
if (ref1->type == BTRFS_TREE_BLOCK_REF_KEY ||
ref1->type == BTRFS_SHARED_BLOCK_REF_KEY) {
return comp_tree_refs(btrfs_delayed_node_to_tree_ref(ref2),
@@ -139,7 +139,7 @@
entry = rb_entry(parent_node, struct btrfs_delayed_ref_node,
rb_node);
- cmp = comp_entry(entry, ins);
+ cmp = comp_entry(entry, ins, 1);
if (cmp < 0)
p = &(*p)->rb_left;
else if (cmp > 0)
@@ -233,6 +233,114 @@
return 0;
}
+static void inline drop_delayed_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_delayed_ref_root *delayed_refs,
+ struct btrfs_delayed_ref_node *ref)
+{
+ rb_erase(&ref->rb_node, &delayed_refs->root);
+ ref->in_tree = 0;
+ btrfs_put_delayed_ref(ref);
+ delayed_refs->num_entries--;
+ if (trans->delayed_ref_updates)
+ trans->delayed_ref_updates--;
+}
+
+static int merge_ref(struct btrfs_trans_handle *trans,
+ struct btrfs_delayed_ref_root *delayed_refs,
+ struct btrfs_delayed_ref_node *ref, u64 seq)
+{
+ struct rb_node *node;
+ int merged = 0;
+ int mod = 0;
+ int done = 0;
+
+ node = rb_prev(&ref->rb_node);
+ while (node) {
+ struct btrfs_delayed_ref_node *next;
+
+ next = rb_entry(node, struct btrfs_delayed_ref_node, rb_node);
+ node = rb_prev(node);
+ if (next->bytenr != ref->bytenr)
+ break;
+ if (seq && next->seq >= seq)
+ break;
+ if (comp_entry(ref, next, 0))
+ continue;
+
+ if (ref->action == next->action) {
+ mod = next->ref_mod;
+ } else {
+ if (ref->ref_mod < next->ref_mod) {
+ struct btrfs_delayed_ref_node *tmp;
+
+ tmp = ref;
+ ref = next;
+ next = tmp;
+ done = 1;
+ }
+ mod = -next->ref_mod;
+ }
+
+ merged++;
+ drop_delayed_ref(trans, delayed_refs, next);
+ ref->ref_mod += mod;
+ if (ref->ref_mod == 0) {
+ drop_delayed_ref(trans, delayed_refs, ref);
+ break;
+ } else {
+ /*
+ * You can't have multiples of the same ref on a tree
+ * block.
+ */
+ WARN_ON(ref->type == BTRFS_TREE_BLOCK_REF_KEY ||
+ ref->type == BTRFS_SHARED_BLOCK_REF_KEY);
+ }
+
+ if (done)
+ break;
+ node = rb_prev(&ref->rb_node);
+ }
+
+ return merged;
+}
+
+void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info,
+ struct btrfs_delayed_ref_root *delayed_refs,
+ struct btrfs_delayed_ref_head *head)
+{
+ struct rb_node *node;
+ u64 seq = 0;
+
+ spin_lock(&fs_info->tree_mod_seq_lock);
+ if (!list_empty(&fs_info->tree_mod_seq_list)) {
+ struct seq_list *elem;
+
+ elem = list_first_entry(&fs_info->tree_mod_seq_list,
+ struct seq_list, list);
+ seq = elem->seq;
+ }
+ spin_unlock(&fs_info->tree_mod_seq_lock);
+
+ node = rb_prev(&head->node.rb_node);
+ while (node) {
+ struct btrfs_delayed_ref_node *ref;
+
+ ref = rb_entry(node, struct btrfs_delayed_ref_node,
+ rb_node);
+ if (ref->bytenr != head->node.bytenr)
+ break;
+
+ /* We can't merge refs that are outside of our seq count */
+ if (seq && ref->seq >= seq)
+ break;
+ if (merge_ref(trans, delayed_refs, ref, seq))
+ node = rb_prev(&head->node.rb_node);
+ else
+ node = rb_prev(node);
+ }
+}
+
int btrfs_check_delayed_seq(struct btrfs_fs_info *fs_info,
struct btrfs_delayed_ref_root *delayed_refs,
u64 seq)
@@ -336,18 +444,11 @@
* every changing the extent allocation tree.
*/
existing->ref_mod--;
- if (existing->ref_mod == 0) {
- rb_erase(&existing->rb_node,
- &delayed_refs->root);
- existing->in_tree = 0;
- btrfs_put_delayed_ref(existing);
- delayed_refs->num_entries--;
- if (trans->delayed_ref_updates)
- trans->delayed_ref_updates--;
- } else {
+ if (existing->ref_mod == 0)
+ drop_delayed_ref(trans, delayed_refs, existing);
+ else
WARN_ON(existing->type == BTRFS_TREE_BLOCK_REF_KEY ||
existing->type == BTRFS_SHARED_BLOCK_REF_KEY);
- }
} else {
WARN_ON(existing->type == BTRFS_TREE_BLOCK_REF_KEY ||
existing->type == BTRFS_SHARED_BLOCK_REF_KEY);
@@ -662,9 +763,6 @@
add_delayed_tree_ref(fs_info, trans, &ref->node, bytenr,
num_bytes, parent, ref_root, level, action,
for_cow);
- if (!need_ref_seq(for_cow, ref_root) &&
- waitqueue_active(&fs_info->tree_mod_seq_wait))
- wake_up(&fs_info->tree_mod_seq_wait);
spin_unlock(&delayed_refs->lock);
if (need_ref_seq(for_cow, ref_root))
btrfs_qgroup_record_ref(trans, &ref->node, extent_op);
@@ -713,9 +811,6 @@
add_delayed_data_ref(fs_info, trans, &ref->node, bytenr,
num_bytes, parent, ref_root, owner, offset,
action, for_cow);
- if (!need_ref_seq(for_cow, ref_root) &&
- waitqueue_active(&fs_info->tree_mod_seq_wait))
- wake_up(&fs_info->tree_mod_seq_wait);
spin_unlock(&delayed_refs->lock);
if (need_ref_seq(for_cow, ref_root))
btrfs_qgroup_record_ref(trans, &ref->node, extent_op);
@@ -744,8 +839,6 @@
num_bytes, BTRFS_UPDATE_DELAYED_HEAD,
extent_op->is_data);
- if (waitqueue_active(&fs_info->tree_mod_seq_wait))
- wake_up(&fs_info->tree_mod_seq_wait);
spin_unlock(&delayed_refs->lock);
return 0;
}
diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h
index 0d7c90c..c9d7036 100644
--- a/fs/btrfs/delayed-ref.h
+++ b/fs/btrfs/delayed-ref.h
@@ -18,7 +18,7 @@
#ifndef __DELAYED_REF__
#define __DELAYED_REF__
-/* these are the possible values of struct btrfs_delayed_ref->action */
+/* these are the possible values of struct btrfs_delayed_ref_node->action */
#define BTRFS_ADD_DELAYED_REF 1 /* add one backref to the tree */
#define BTRFS_DROP_DELAYED_REF 2 /* delete one backref from the tree */
#define BTRFS_ADD_DELAYED_EXTENT 3 /* record a full extent allocation */
@@ -167,6 +167,10 @@
struct btrfs_trans_handle *trans,
u64 bytenr, u64 num_bytes,
struct btrfs_delayed_extent_op *extent_op);
+void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,
+ struct btrfs_fs_info *fs_info,
+ struct btrfs_delayed_ref_root *delayed_refs,
+ struct btrfs_delayed_ref_head *head);
struct btrfs_delayed_ref_head *
btrfs_find_delayed_ref_head(struct btrfs_trans_handle *trans, u64 bytenr);
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 62e0caf..22e98e0 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -377,9 +377,13 @@
ret = read_extent_buffer_pages(io_tree, eb, start,
WAIT_COMPLETE,
btree_get_extent, mirror_num);
- if (!ret && !verify_parent_transid(io_tree, eb,
+ if (!ret) {
+ if (!verify_parent_transid(io_tree, eb,
parent_transid, 0))
- break;
+ break;
+ else
+ ret = -EIO;
+ }
/*
* This buffer's crc is fine, but its contents are corrupted, so
@@ -754,9 +758,7 @@
limit = btrfs_async_submit_limit(fs_info);
limit = limit * 2 / 3;
- atomic_dec(&fs_info->nr_async_submits);
-
- if (atomic_read(&fs_info->nr_async_submits) < limit &&
+ if (atomic_dec_return(&fs_info->nr_async_submits) < limit &&
waitqueue_active(&fs_info->async_submit_wait))
wake_up(&fs_info->async_submit_wait);
@@ -2032,8 +2034,6 @@
fs_info->free_chunk_space = 0;
fs_info->tree_mod_log = RB_ROOT;
- init_waitqueue_head(&fs_info->tree_mod_seq_wait);
-
/* readahead state */
INIT_RADIX_TREE(&fs_info->reada_tree, GFP_NOFS & ~__GFP_WAIT);
spin_lock_init(&fs_info->reada_lock);
@@ -2528,8 +2528,7 @@
goto fail_trans_kthread;
/* do not make disk changes in broken FS */
- if (btrfs_super_log_root(disk_super) != 0 &&
- !(fs_info->fs_state & BTRFS_SUPER_FLAG_ERROR)) {
+ if (btrfs_super_log_root(disk_super) != 0) {
u64 bytenr = btrfs_super_log_root(disk_super);
if (fs_devices->rw_devices == 0) {
@@ -3189,30 +3188,14 @@
/* clear out the rbtree of defraggable inodes */
btrfs_run_defrag_inodes(fs_info);
- /*
- * Here come 2 situations when btrfs is broken to flip readonly:
- *
- * 1. when btrfs flips readonly somewhere else before
- * btrfs_commit_super, sb->s_flags has MS_RDONLY flag,
- * and btrfs will skip to write sb directly to keep
- * ERROR state on disk.
- *
- * 2. when btrfs flips readonly just in btrfs_commit_super,
- * and in such case, btrfs cannot write sb via btrfs_commit_super,
- * and since fs_state has been set BTRFS_SUPER_FLAG_ERROR flag,
- * btrfs will cleanup all FS resources first and write sb then.
- */
if (!(fs_info->sb->s_flags & MS_RDONLY)) {
ret = btrfs_commit_super(root);
if (ret)
printk(KERN_ERR "btrfs: commit super ret %d\n", ret);
}
- if (fs_info->fs_state & BTRFS_SUPER_FLAG_ERROR) {
- ret = btrfs_error_commit_super(root);
- if (ret)
- printk(KERN_ERR "btrfs: commit super ret %d\n", ret);
- }
+ if (fs_info->fs_state & BTRFS_SUPER_FLAG_ERROR)
+ btrfs_error_commit_super(root);
btrfs_put_block_group_cache(fs_info);
@@ -3434,18 +3417,11 @@
if (read_only)
return 0;
- if (fs_info->fs_state & BTRFS_SUPER_FLAG_ERROR) {
- printk(KERN_WARNING "warning: mount fs with errors, "
- "running btrfsck is recommended\n");
- }
-
return 0;
}
-int btrfs_error_commit_super(struct btrfs_root *root)
+void btrfs_error_commit_super(struct btrfs_root *root)
{
- int ret;
-
mutex_lock(&root->fs_info->cleaner_mutex);
btrfs_run_delayed_iputs(root);
mutex_unlock(&root->fs_info->cleaner_mutex);
@@ -3455,10 +3431,6 @@
/* cleanup FS via transaction */
btrfs_cleanup_transaction(root);
-
- ret = write_ctree_super(NULL, root, 0);
-
- return ret;
}
static void btrfs_destroy_ordered_operations(struct btrfs_root *root)
@@ -3782,14 +3754,17 @@
/* FIXME: cleanup wait for commit */
t->in_commit = 1;
t->blocked = 1;
+ smp_mb();
if (waitqueue_active(&root->fs_info->transaction_blocked_wait))
wake_up(&root->fs_info->transaction_blocked_wait);
t->blocked = 0;
+ smp_mb();
if (waitqueue_active(&root->fs_info->transaction_wait))
wake_up(&root->fs_info->transaction_wait);
t->commit_done = 1;
+ smp_mb();
if (waitqueue_active(&t->commit_wait))
wake_up(&t->commit_wait);
diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h
index 95e147e..c5b00a7 100644
--- a/fs/btrfs/disk-io.h
+++ b/fs/btrfs/disk-io.h
@@ -54,7 +54,7 @@
struct btrfs_root *root, int max_mirrors);
struct buffer_head *btrfs_read_dev_super(struct block_device *bdev);
int btrfs_commit_super(struct btrfs_root *root);
-int btrfs_error_commit_super(struct btrfs_root *root);
+void btrfs_error_commit_super(struct btrfs_root *root);
struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
u64 bytenr, u32 blocksize);
struct btrfs_root *btrfs_read_fs_root_no_radix(struct btrfs_root *tree_root,
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 4e1b153..ba58024 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -2252,6 +2252,16 @@
}
/*
+ * We need to try and merge add/drops of the same ref since we
+ * can run into issues with relocate dropping the implicit ref
+ * and then it being added back again before the drop can
+ * finish. If we merged anything we need to re-loop so we can
+ * get a good ref.
+ */
+ btrfs_merge_delayed_refs(trans, fs_info, delayed_refs,
+ locked_ref);
+
+ /*
* locked_ref is the head node, so we have to go one
* node back for any delayed ref updates
*/
@@ -2318,12 +2328,23 @@
ref->in_tree = 0;
rb_erase(&ref->rb_node, &delayed_refs->root);
delayed_refs->num_entries--;
- /*
- * we modified num_entries, but as we're currently running
- * delayed refs, skip
- * wake_up(&delayed_refs->seq_wait);
- * here.
- */
+ if (locked_ref) {
+ /*
+ * when we play the delayed ref, also correct the
+ * ref_mod on head
+ */
+ switch (ref->action) {
+ case BTRFS_ADD_DELAYED_REF:
+ case BTRFS_ADD_DELAYED_EXTENT:
+ locked_ref->node.ref_mod -= ref->ref_mod;
+ break;
+ case BTRFS_DROP_DELAYED_REF:
+ locked_ref->node.ref_mod += ref->ref_mod;
+ break;
+ default:
+ WARN_ON(1);
+ }
+ }
spin_unlock(&delayed_refs->lock);
ret = run_one_delayed_ref(trans, root, ref, extent_op,
@@ -2350,22 +2371,6 @@
return count;
}
-static void wait_for_more_refs(struct btrfs_fs_info *fs_info,
- struct btrfs_delayed_ref_root *delayed_refs,
- unsigned long num_refs,
- struct list_head *first_seq)
-{
- spin_unlock(&delayed_refs->lock);
- pr_debug("waiting for more refs (num %ld, first %p)\n",
- num_refs, first_seq);
- wait_event(fs_info->tree_mod_seq_wait,
- num_refs != delayed_refs->num_entries ||
- fs_info->tree_mod_seq_list.next != first_seq);
- pr_debug("done waiting for more refs (num %ld, first %p)\n",
- delayed_refs->num_entries, fs_info->tree_mod_seq_list.next);
- spin_lock(&delayed_refs->lock);
-}
-
#ifdef SCRAMBLE_DELAYED_REFS
/*
* Normally delayed refs get processed in ascending bytenr order. This
@@ -2460,13 +2465,11 @@
struct btrfs_delayed_ref_root *delayed_refs;
struct btrfs_delayed_ref_node *ref;
struct list_head cluster;
- struct list_head *first_seq = NULL;
int ret;
u64 delayed_start;
int run_all = count == (unsigned long)-1;
int run_most = 0;
- unsigned long num_refs = 0;
- int consider_waiting;
+ int loops;
/* We'll clean this up in btrfs_cleanup_transaction */
if (trans->aborted)
@@ -2484,7 +2487,7 @@
delayed_refs = &trans->transaction->delayed_refs;
INIT_LIST_HEAD(&cluster);
again:
- consider_waiting = 0;
+ loops = 0;
spin_lock(&delayed_refs->lock);
#ifdef SCRAMBLE_DELAYED_REFS
@@ -2512,31 +2515,6 @@
if (ret)
break;
- if (delayed_start >= delayed_refs->run_delayed_start) {
- if (consider_waiting == 0) {
- /*
- * btrfs_find_ref_cluster looped. let's do one
- * more cycle. if we don't run any delayed ref
- * during that cycle (because we can't because
- * all of them are blocked) and if the number of
- * refs doesn't change, we avoid busy waiting.
- */
- consider_waiting = 1;
- num_refs = delayed_refs->num_entries;
- first_seq = root->fs_info->tree_mod_seq_list.next;
- } else {
- wait_for_more_refs(root->fs_info, delayed_refs,
- num_refs, first_seq);
- /*
- * after waiting, things have changed. we
- * dropped the lock and someone else might have
- * run some refs, built new clusters and so on.
- * therefore, we restart staleness detection.
- */
- consider_waiting = 0;
- }
- }
-
ret = run_clustered_refs(trans, root, &cluster);
if (ret < 0) {
spin_unlock(&delayed_refs->lock);
@@ -2549,9 +2527,26 @@
if (count == 0)
break;
- if (ret || delayed_refs->run_delayed_start == 0) {
+ if (delayed_start >= delayed_refs->run_delayed_start) {
+ if (loops == 0) {
+ /*
+ * btrfs_find_ref_cluster looped. let's do one
+ * more cycle. if we don't run any delayed ref
+ * during that cycle (because we can't because
+ * all of them are blocked), bail out.
+ */
+ loops = 1;
+ } else {
+ /*
+ * no runnable refs left, stop trying
+ */
+ BUG_ON(run_all);
+ break;
+ }
+ }
+ if (ret) {
/* refs were run, let's reset staleness detection */
- consider_waiting = 0;
+ loops = 0;
}
}
@@ -3007,17 +3002,16 @@
}
spin_unlock(&block_group->lock);
- num_pages = (int)div64_u64(block_group->key.offset, 1024 * 1024 * 1024);
+ /*
+ * Try to preallocate enough space based on how big the block group is.
+ * Keep in mind this has to include any pinned space which could end up
+ * taking up quite a bit since it's not folded into the other space
+ * cache.
+ */
+ num_pages = (int)div64_u64(block_group->key.offset, 256 * 1024 * 1024);
if (!num_pages)
num_pages = 1;
- /*
- * Just to make absolutely sure we have enough space, we're going to
- * preallocate 12 pages worth of space for each block group. In
- * practice we ought to use at most 8, but we need extra space so we can
- * add our header and have a terminator between the extents and the
- * bitmaps.
- */
num_pages *= 16;
num_pages *= PAGE_CACHE_SIZE;
@@ -4571,8 +4565,10 @@
if (root->fs_info->quota_enabled) {
ret = btrfs_qgroup_reserve(root, num_bytes +
nr_extents * root->leafsize);
- if (ret)
+ if (ret) {
+ mutex_unlock(&BTRFS_I(inode)->delalloc_mutex);
return ret;
+ }
}
ret = reserve_metadata_bytes(root, block_rsv, to_reserve, flush);
@@ -5294,9 +5290,6 @@
rb_erase(&head->node.rb_node, &delayed_refs->root);
delayed_refs->num_entries--;
- smp_mb();
- if (waitqueue_active(&root->fs_info->tree_mod_seq_wait))
- wake_up(&root->fs_info->tree_mod_seq_wait);
/*
* we don't take a ref on the node because we're removing it from the
diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c
index 45c81bb..4c87847 100644
--- a/fs/btrfs/extent_io.c
+++ b/fs/btrfs/extent_io.c
@@ -2330,23 +2330,10 @@
if (uptodate && tree->ops && tree->ops->readpage_end_io_hook) {
ret = tree->ops->readpage_end_io_hook(page, start, end,
state, mirror);
- if (ret) {
- /* no IO indicated but software detected errors
- * in the block, either checksum errors or
- * issues with the contents */
- struct btrfs_root *root =
- BTRFS_I(page->mapping->host)->root;
- struct btrfs_device *device;
-
+ if (ret)
uptodate = 0;
- device = btrfs_find_device_for_logical(
- root, start, mirror);
- if (device)
- btrfs_dev_stat_inc_and_print(device,
- BTRFS_DEV_STAT_CORRUPTION_ERRS);
- } else {
+ else
clean_io_failure(start, page);
- }
}
if (!uptodate && tree->ops && tree->ops->readpage_io_failed_hook) {
diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c
index b45b9de..857d93c 100644
--- a/fs/btrfs/file-item.c
+++ b/fs/btrfs/file-item.c
@@ -272,9 +272,9 @@
}
int btrfs_lookup_bio_sums_dio(struct btrfs_root *root, struct inode *inode,
- struct bio *bio, u64 offset, u32 *dst)
+ struct bio *bio, u64 offset)
{
- return __btrfs_lookup_bio_sums(root, inode, bio, offset, dst, 1);
+ return __btrfs_lookup_bio_sums(root, inode, bio, offset, NULL, 1);
}
int btrfs_lookup_csums_range(struct btrfs_root *root, u64 start, u64 end,
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 6e8f416..316b07a 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -1008,9 +1008,7 @@
nr_pages = (async_cow->end - async_cow->start + PAGE_CACHE_SIZE) >>
PAGE_CACHE_SHIFT;
- atomic_sub(nr_pages, &root->fs_info->async_delalloc_pages);
-
- if (atomic_read(&root->fs_info->async_delalloc_pages) <
+ if (atomic_sub_return(nr_pages, &root->fs_info->async_delalloc_pages) <
5 * 1024 * 1024 &&
waitqueue_active(&root->fs_info->async_submit_wait))
wake_up(&root->fs_info->async_submit_wait);
@@ -1885,8 +1883,11 @@
trans = btrfs_join_transaction_nolock(root);
else
trans = btrfs_join_transaction(root);
- if (IS_ERR(trans))
- return PTR_ERR(trans);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ trans = NULL;
+ goto out;
+ }
trans->block_rsv = &root->fs_info->delalloc_block_rsv;
ret = btrfs_update_inode_fallback(trans, root, inode);
if (ret) /* -ENOMEM or corruption */
@@ -1970,8 +1971,8 @@
ordered_extent->len - 1, NULL, GFP_NOFS);
/*
- * This needs to be dont to make sure anybody waiting knows we are done
- * upating everything for this ordered extent.
+ * This needs to be done to make sure anybody waiting knows we are done
+ * updating everything for this ordered extent.
*/
btrfs_remove_ordered_extent(inode, ordered_extent);
@@ -3174,7 +3175,7 @@
btrfs_i_size_write(dir, dir->i_size - name_len * 2);
inode_inc_iversion(dir);
dir->i_mtime = dir->i_ctime = CURRENT_TIME;
- ret = btrfs_update_inode(trans, root, dir);
+ ret = btrfs_update_inode_fallback(trans, root, dir);
if (ret)
btrfs_abort_transaction(trans, root, ret);
out:
@@ -5774,18 +5775,112 @@
return ret;
}
+static int lock_extent_direct(struct inode *inode, u64 lockstart, u64 lockend,
+ struct extent_state **cached_state, int writing)
+{
+ struct btrfs_ordered_extent *ordered;
+ int ret = 0;
+
+ while (1) {
+ lock_extent_bits(&BTRFS_I(inode)->io_tree, lockstart, lockend,
+ 0, cached_state);
+ /*
+ * We're concerned with the entire range that we're going to be
+ * doing DIO to, so we need to make sure theres no ordered
+ * extents in this range.
+ */
+ ordered = btrfs_lookup_ordered_range(inode, lockstart,
+ lockend - lockstart + 1);
+
+ /*
+ * We need to make sure there are no buffered pages in this
+ * range either, we could have raced between the invalidate in
+ * generic_file_direct_write and locking the extent. The
+ * invalidate needs to happen so that reads after a write do not
+ * get stale data.
+ */
+ if (!ordered && (!writing ||
+ !test_range_bit(&BTRFS_I(inode)->io_tree,
+ lockstart, lockend, EXTENT_UPTODATE, 0,
+ *cached_state)))
+ break;
+
+ unlock_extent_cached(&BTRFS_I(inode)->io_tree, lockstart, lockend,
+ cached_state, GFP_NOFS);
+
+ if (ordered) {
+ btrfs_start_ordered_extent(inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ } else {
+ /* Screw you mmap */
+ ret = filemap_write_and_wait_range(inode->i_mapping,
+ lockstart,
+ lockend);
+ if (ret)
+ break;
+
+ /*
+ * If we found a page that couldn't be invalidated just
+ * fall back to buffered.
+ */
+ ret = invalidate_inode_pages2_range(inode->i_mapping,
+ lockstart >> PAGE_CACHE_SHIFT,
+ lockend >> PAGE_CACHE_SHIFT);
+ if (ret)
+ break;
+ }
+
+ cond_resched();
+ }
+
+ return ret;
+}
+
static int btrfs_get_blocks_direct(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create)
{
struct extent_map *em;
struct btrfs_root *root = BTRFS_I(inode)->root;
+ struct extent_state *cached_state = NULL;
u64 start = iblock << inode->i_blkbits;
+ u64 lockstart, lockend;
u64 len = bh_result->b_size;
struct btrfs_trans_handle *trans;
+ int unlock_bits = EXTENT_LOCKED;
+ int ret;
+
+ if (create) {
+ ret = btrfs_delalloc_reserve_space(inode, len);
+ if (ret)
+ return ret;
+ unlock_bits |= EXTENT_DELALLOC | EXTENT_DIRTY;
+ } else {
+ len = min_t(u64, len, root->sectorsize);
+ }
+
+ lockstart = start;
+ lockend = start + len - 1;
+
+ /*
+ * If this errors out it's because we couldn't invalidate pagecache for
+ * this range and we need to fallback to buffered.
+ */
+ if (lock_extent_direct(inode, lockstart, lockend, &cached_state, create))
+ return -ENOTBLK;
+
+ if (create) {
+ ret = set_extent_bit(&BTRFS_I(inode)->io_tree, lockstart,
+ lockend, EXTENT_DELALLOC, NULL,
+ &cached_state, GFP_NOFS);
+ if (ret)
+ goto unlock_err;
+ }
em = btrfs_get_extent(inode, NULL, 0, start, len, 0);
- if (IS_ERR(em))
- return PTR_ERR(em);
+ if (IS_ERR(em)) {
+ ret = PTR_ERR(em);
+ goto unlock_err;
+ }
/*
* Ok for INLINE and COMPRESSED extents we need to fallback on buffered
@@ -5804,17 +5899,16 @@
if (test_bit(EXTENT_FLAG_COMPRESSED, &em->flags) ||
em->block_start == EXTENT_MAP_INLINE) {
free_extent_map(em);
- return -ENOTBLK;
+ ret = -ENOTBLK;
+ goto unlock_err;
}
/* Just a good old fashioned hole, return */
if (!create && (em->block_start == EXTENT_MAP_HOLE ||
test_bit(EXTENT_FLAG_PREALLOC, &em->flags))) {
free_extent_map(em);
- /* DIO will do one hole at a time, so just unlock a sector */
- unlock_extent(&BTRFS_I(inode)->io_tree, start,
- start + root->sectorsize - 1);
- return 0;
+ ret = 0;
+ goto unlock_err;
}
/*
@@ -5827,8 +5921,9 @@
*
*/
if (!create) {
- len = em->len - (start - em->start);
- goto map;
+ len = min(len, em->len - (start - em->start));
+ lockstart = start + len;
+ goto unlock;
}
if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags) ||
@@ -5860,7 +5955,7 @@
btrfs_end_transaction(trans, root);
if (ret) {
free_extent_map(em);
- return ret;
+ goto unlock_err;
}
goto unlock;
}
@@ -5873,14 +5968,12 @@
*/
len = bh_result->b_size;
em = btrfs_new_extent_direct(inode, em, start, len);
- if (IS_ERR(em))
- return PTR_ERR(em);
+ if (IS_ERR(em)) {
+ ret = PTR_ERR(em);
+ goto unlock_err;
+ }
len = min(len, em->len - (start - em->start));
unlock:
- clear_extent_bit(&BTRFS_I(inode)->io_tree, start, start + len - 1,
- EXTENT_LOCKED | EXTENT_DELALLOC | EXTENT_DIRTY, 1,
- 0, NULL, GFP_NOFS);
-map:
bh_result->b_blocknr = (em->block_start + (start - em->start)) >>
inode->i_blkbits;
bh_result->b_size = len;
@@ -5898,9 +5991,44 @@
i_size_write(inode, start + len);
}
+ /*
+ * In the case of write we need to clear and unlock the entire range,
+ * in the case of read we need to unlock only the end area that we
+ * aren't using if there is any left over space.
+ */
+ if (lockstart < lockend) {
+ if (create && len < lockend - lockstart) {
+ clear_extent_bit(&BTRFS_I(inode)->io_tree, lockstart,
+ lockstart + len - 1, unlock_bits, 1, 0,
+ &cached_state, GFP_NOFS);
+ /*
+ * Beside unlock, we also need to cleanup reserved space
+ * for the left range by attaching EXTENT_DO_ACCOUNTING.
+ */
+ clear_extent_bit(&BTRFS_I(inode)->io_tree,
+ lockstart + len, lockend,
+ unlock_bits | EXTENT_DO_ACCOUNTING,
+ 1, 0, NULL, GFP_NOFS);
+ } else {
+ clear_extent_bit(&BTRFS_I(inode)->io_tree, lockstart,
+ lockend, unlock_bits, 1, 0,
+ &cached_state, GFP_NOFS);
+ }
+ } else {
+ free_extent_state(cached_state);
+ }
+
free_extent_map(em);
return 0;
+
+unlock_err:
+ if (create)
+ unlock_bits |= EXTENT_DO_ACCOUNTING;
+
+ clear_extent_bit(&BTRFS_I(inode)->io_tree, lockstart, lockend,
+ unlock_bits, 1, 0, &cached_state, GFP_NOFS);
+ return ret;
}
struct btrfs_dio_private {
@@ -5908,7 +6036,6 @@
u64 logical_offset;
u64 disk_bytenr;
u64 bytes;
- u32 *csums;
void *private;
/* number of bios pending for this dio */
@@ -5928,7 +6055,6 @@
struct inode *inode = dip->inode;
struct btrfs_root *root = BTRFS_I(inode)->root;
u64 start;
- u32 *private = dip->csums;
start = dip->logical_offset;
do {
@@ -5936,8 +6062,12 @@
struct page *page = bvec->bv_page;
char *kaddr;
u32 csum = ~(u32)0;
+ u64 private = ~(u32)0;
unsigned long flags;
+ if (get_state_private(&BTRFS_I(inode)->io_tree,
+ start, &private))
+ goto failed;
local_irq_save(flags);
kaddr = kmap_atomic(page);
csum = btrfs_csum_data(root, kaddr + bvec->bv_offset,
@@ -5947,18 +6077,18 @@
local_irq_restore(flags);
flush_dcache_page(bvec->bv_page);
- if (csum != *private) {
+ if (csum != private) {
+failed:
printk(KERN_ERR "btrfs csum failed ino %llu off"
" %llu csum %u private %u\n",
(unsigned long long)btrfs_ino(inode),
(unsigned long long)start,
- csum, *private);
+ csum, (unsigned)private);
err = -EIO;
}
}
start += bvec->bv_len;
- private++;
bvec++;
} while (bvec <= bvec_end);
@@ -5966,7 +6096,6 @@
dip->logical_offset + dip->bytes - 1);
bio->bi_private = dip->private;
- kfree(dip->csums);
kfree(dip);
/* If we had a csum failure make sure to clear the uptodate flag */
@@ -6072,7 +6201,7 @@
static inline int __btrfs_submit_dio_bio(struct bio *bio, struct inode *inode,
int rw, u64 file_offset, int skip_sum,
- u32 *csums, int async_submit)
+ int async_submit)
{
int write = rw & REQ_WRITE;
struct btrfs_root *root = BTRFS_I(inode)->root;
@@ -6105,8 +6234,7 @@
if (ret)
goto err;
} else if (!skip_sum) {
- ret = btrfs_lookup_bio_sums_dio(root, inode, bio,
- file_offset, csums);
+ ret = btrfs_lookup_bio_sums_dio(root, inode, bio, file_offset);
if (ret)
goto err;
}
@@ -6132,10 +6260,8 @@
u64 submit_len = 0;
u64 map_length;
int nr_pages = 0;
- u32 *csums = dip->csums;
int ret = 0;
int async_submit = 0;
- int write = rw & REQ_WRITE;
map_length = orig_bio->bi_size;
ret = btrfs_map_block(map_tree, READ, start_sector << 9,
@@ -6171,16 +6297,13 @@
atomic_inc(&dip->pending_bios);
ret = __btrfs_submit_dio_bio(bio, inode, rw,
file_offset, skip_sum,
- csums, async_submit);
+ async_submit);
if (ret) {
bio_put(bio);
atomic_dec(&dip->pending_bios);
goto out_err;
}
- /* Write's use the ordered csums */
- if (!write && !skip_sum)
- csums = csums + nr_pages;
start_sector += submit_len >> 9;
file_offset += submit_len;
@@ -6210,7 +6333,7 @@
submit:
ret = __btrfs_submit_dio_bio(bio, inode, rw, file_offset, skip_sum,
- csums, async_submit);
+ async_submit);
if (!ret)
return 0;
@@ -6246,17 +6369,6 @@
ret = -ENOMEM;
goto free_ordered;
}
- dip->csums = NULL;
-
- /* Write's use the ordered csum stuff, so we don't need dip->csums */
- if (!write && !skip_sum) {
- dip->csums = kmalloc(sizeof(u32) * bio->bi_vcnt, GFP_NOFS);
- if (!dip->csums) {
- kfree(dip);
- ret = -ENOMEM;
- goto free_ordered;
- }
- }
dip->private = bio->bi_private;
dip->inode = inode;
@@ -6341,132 +6453,22 @@
out:
return retval;
}
+
static ssize_t btrfs_direct_IO(int rw, struct kiocb *iocb,
const struct iovec *iov, loff_t offset,
unsigned long nr_segs)
{
struct file *file = iocb->ki_filp;
struct inode *inode = file->f_mapping->host;
- struct btrfs_ordered_extent *ordered;
- struct extent_state *cached_state = NULL;
- u64 lockstart, lockend;
- ssize_t ret;
- int writing = rw & WRITE;
- int write_bits = 0;
- size_t count = iov_length(iov, nr_segs);
if (check_direct_IO(BTRFS_I(inode)->root, rw, iocb, iov,
- offset, nr_segs)) {
+ offset, nr_segs))
return 0;
- }
- lockstart = offset;
- lockend = offset + count - 1;
-
- if (writing) {
- ret = btrfs_delalloc_reserve_space(inode, count);
- if (ret)
- goto out;
- }
-
- while (1) {
- lock_extent_bits(&BTRFS_I(inode)->io_tree, lockstart, lockend,
- 0, &cached_state);
- /*
- * We're concerned with the entire range that we're going to be
- * doing DIO to, so we need to make sure theres no ordered
- * extents in this range.
- */
- ordered = btrfs_lookup_ordered_range(inode, lockstart,
- lockend - lockstart + 1);
-
- /*
- * We need to make sure there are no buffered pages in this
- * range either, we could have raced between the invalidate in
- * generic_file_direct_write and locking the extent. The
- * invalidate needs to happen so that reads after a write do not
- * get stale data.
- */
- if (!ordered && (!writing ||
- !test_range_bit(&BTRFS_I(inode)->io_tree,
- lockstart, lockend, EXTENT_UPTODATE, 0,
- cached_state)))
- break;
-
- unlock_extent_cached(&BTRFS_I(inode)->io_tree, lockstart, lockend,
- &cached_state, GFP_NOFS);
-
- if (ordered) {
- btrfs_start_ordered_extent(inode, ordered, 1);
- btrfs_put_ordered_extent(ordered);
- } else {
- /* Screw you mmap */
- ret = filemap_write_and_wait_range(file->f_mapping,
- lockstart,
- lockend);
- if (ret)
- goto out;
-
- /*
- * If we found a page that couldn't be invalidated just
- * fall back to buffered.
- */
- ret = invalidate_inode_pages2_range(file->f_mapping,
- lockstart >> PAGE_CACHE_SHIFT,
- lockend >> PAGE_CACHE_SHIFT);
- if (ret) {
- if (ret == -EBUSY)
- ret = 0;
- goto out;
- }
- }
-
- cond_resched();
- }
-
- /*
- * we don't use btrfs_set_extent_delalloc because we don't want
- * the dirty or uptodate bits
- */
- if (writing) {
- write_bits = EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING;
- ret = set_extent_bit(&BTRFS_I(inode)->io_tree, lockstart, lockend,
- EXTENT_DELALLOC, NULL, &cached_state,
- GFP_NOFS);
- if (ret) {
- clear_extent_bit(&BTRFS_I(inode)->io_tree, lockstart,
- lockend, EXTENT_LOCKED | write_bits,
- 1, 0, &cached_state, GFP_NOFS);
- goto out;
- }
- }
-
- free_extent_state(cached_state);
- cached_state = NULL;
-
- ret = __blockdev_direct_IO(rw, iocb, inode,
+ return __blockdev_direct_IO(rw, iocb, inode,
BTRFS_I(inode)->root->fs_info->fs_devices->latest_bdev,
iov, offset, nr_segs, btrfs_get_blocks_direct, NULL,
btrfs_submit_direct, 0);
-
- if (ret < 0 && ret != -EIOCBQUEUED) {
- clear_extent_bit(&BTRFS_I(inode)->io_tree, offset,
- offset + iov_length(iov, nr_segs) - 1,
- EXTENT_LOCKED | write_bits, 1, 0,
- &cached_state, GFP_NOFS);
- } else if (ret >= 0 && ret < iov_length(iov, nr_segs)) {
- /*
- * We're falling back to buffered, unlock the section we didn't
- * do IO on.
- */
- clear_extent_bit(&BTRFS_I(inode)->io_tree, offset + ret,
- offset + iov_length(iov, nr_segs) - 1,
- EXTENT_LOCKED | write_bits, 1, 0,
- &cached_state, GFP_NOFS);
- }
-out:
- free_extent_state(cached_state);
- return ret;
}
static int btrfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 7bb7556..9df50fa 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -424,7 +424,7 @@
uuid_le_gen(&new_uuid);
memcpy(root_item.uuid, new_uuid.b, BTRFS_UUID_SIZE);
root_item.otime.sec = cpu_to_le64(cur_time.tv_sec);
- root_item.otime.nsec = cpu_to_le64(cur_time.tv_nsec);
+ root_item.otime.nsec = cpu_to_le32(cur_time.tv_nsec);
root_item.ctime = root_item.otime;
btrfs_set_root_ctransid(&root_item, trans->transid);
btrfs_set_root_otransid(&root_item, trans->transid);
diff --git a/fs/btrfs/locking.c b/fs/btrfs/locking.c
index a44eff074..2a1762c 100644
--- a/fs/btrfs/locking.c
+++ b/fs/btrfs/locking.c
@@ -67,7 +67,7 @@
{
if (eb->lock_nested) {
read_lock(&eb->lock);
- if (&eb->lock_nested && current->pid == eb->lock_owner) {
+ if (eb->lock_nested && current->pid == eb->lock_owner) {
read_unlock(&eb->lock);
return;
}
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index bc424ae..b650155 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -1364,8 +1364,10 @@
spin_lock(&fs_info->qgroup_lock);
dstgroup = add_qgroup_rb(fs_info, objectid);
- if (!dstgroup)
+ if (IS_ERR(dstgroup)) {
+ ret = PTR_ERR(dstgroup);
goto unlock;
+ }
if (srcid) {
srcgroup = find_qgroup_rb(fs_info, srcid);
diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c
index 6bb465c..10d8e4d 100644
--- a/fs/btrfs/root-tree.c
+++ b/fs/btrfs/root-tree.c
@@ -544,8 +544,8 @@
struct timespec ct = CURRENT_TIME;
spin_lock(&root->root_times_lock);
- item->ctransid = trans->transid;
+ item->ctransid = cpu_to_le64(trans->transid);
item->ctime.sec = cpu_to_le64(ct.tv_sec);
- item->ctime.nsec = cpu_to_le64(ct.tv_nsec);
+ item->ctime.nsec = cpu_to_le32(ct.tv_nsec);
spin_unlock(&root->root_times_lock);
}
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index f2eb24c..83d6f9f 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -838,7 +838,6 @@
struct btrfs_trans_handle *trans;
struct btrfs_fs_info *fs_info = btrfs_sb(sb);
struct btrfs_root *root = fs_info->tree_root;
- int ret;
trace_btrfs_sync_fs(wait);
@@ -849,11 +848,17 @@
btrfs_wait_ordered_extents(root, 0, 0);
- trans = btrfs_start_transaction(root, 0);
+ spin_lock(&fs_info->trans_lock);
+ if (!fs_info->running_transaction) {
+ spin_unlock(&fs_info->trans_lock);
+ return 0;
+ }
+ spin_unlock(&fs_info->trans_lock);
+
+ trans = btrfs_join_transaction(root);
if (IS_ERR(trans))
return PTR_ERR(trans);
- ret = btrfs_commit_transaction(trans, root);
- return ret;
+ return btrfs_commit_transaction(trans, root);
}
static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
@@ -1530,6 +1535,8 @@
while (cur_devices) {
head = &cur_devices->devices;
list_for_each_entry(dev, head, dev_list) {
+ if (dev->missing)
+ continue;
if (!first_dev || dev->devid < first_dev->devid)
first_dev = dev;
}
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 17be3de..27c2600 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -1031,6 +1031,7 @@
btrfs_i_size_write(parent_inode, parent_inode->i_size +
dentry->d_name.len * 2);
+ parent_inode->i_mtime = parent_inode->i_ctime = CURRENT_TIME;
ret = btrfs_update_inode(trans, parent_root, parent_inode);
if (ret)
goto abort_trans_dput;
@@ -1066,7 +1067,7 @@
memcpy(new_root_item->parent_uuid, root->root_item.uuid,
BTRFS_UUID_SIZE);
new_root_item->otime.sec = cpu_to_le64(cur_time.tv_sec);
- new_root_item->otime.nsec = cpu_to_le64(cur_time.tv_nsec);
+ new_root_item->otime.nsec = cpu_to_le32(cur_time.tv_nsec);
btrfs_set_root_otransid(new_root_item, trans->transid);
memset(&new_root_item->stime, 0, sizeof(new_root_item->stime));
memset(&new_root_item->rtime, 0, sizeof(new_root_item->rtime));
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index e86ae04..88b969a 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -227,9 +227,8 @@
cur = pending;
pending = pending->bi_next;
cur->bi_next = NULL;
- atomic_dec(&fs_info->nr_async_bios);
- if (atomic_read(&fs_info->nr_async_bios) < limit &&
+ if (atomic_dec_return(&fs_info->nr_async_bios) < limit &&
waitqueue_active(&fs_info->async_submit_wait))
wake_up(&fs_info->async_submit_wait);
@@ -569,9 +568,11 @@
memcpy(new_device, device, sizeof(*new_device));
/* Safe because we are under uuid_mutex */
- name = rcu_string_strdup(device->name->str, GFP_NOFS);
- BUG_ON(device->name && !name); /* -ENOMEM */
- rcu_assign_pointer(new_device->name, name);
+ if (device->name) {
+ name = rcu_string_strdup(device->name->str, GFP_NOFS);
+ BUG_ON(device->name && !name); /* -ENOMEM */
+ rcu_assign_pointer(new_device->name, name);
+ }
new_device->bdev = NULL;
new_device->writeable = 0;
new_device->in_fs_metadata = 0;
@@ -4605,28 +4606,6 @@
return ret;
}
-struct btrfs_device *btrfs_find_device_for_logical(struct btrfs_root *root,
- u64 logical, int mirror_num)
-{
- struct btrfs_mapping_tree *map_tree = &root->fs_info->mapping_tree;
- int ret;
- u64 map_length = 0;
- struct btrfs_bio *bbio = NULL;
- struct btrfs_device *device;
-
- BUG_ON(mirror_num == 0);
- ret = btrfs_map_block(map_tree, WRITE, logical, &map_length, &bbio,
- mirror_num);
- if (ret) {
- BUG_ON(bbio != NULL);
- return NULL;
- }
- BUG_ON(mirror_num != bbio->mirror_num);
- device = bbio->stripes[mirror_num - 1].dev;
- kfree(bbio);
- return device;
-}
-
int btrfs_read_chunk_tree(struct btrfs_root *root)
{
struct btrfs_path *path;
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 5479325..53c06af 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -289,8 +289,6 @@
int btrfs_chunk_readonly(struct btrfs_root *root, u64 chunk_offset);
int find_free_dev_extent(struct btrfs_device *device, u64 num_bytes,
u64 *start, u64 *max_avail);
-struct btrfs_device *btrfs_find_device_for_logical(struct btrfs_root *root,
- u64 logical, int mirror_num);
void btrfs_dev_stat_print_on_error(struct btrfs_device *device);
void btrfs_dev_stat_inc_and_print(struct btrfs_device *dev, int index);
int btrfs_get_dev_stats(struct btrfs_root *root,
diff --git a/fs/buffer.c b/fs/buffer.c
index 9f6d2e4..58e2e7b 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -914,7 +914,7 @@
/*
* Initialise the state of a blockdev page's buffers.
*/
-static void
+static sector_t
init_page_buffers(struct page *page, struct block_device *bdev,
sector_t block, int size)
{
@@ -936,33 +936,41 @@
block++;
bh = bh->b_this_page;
} while (bh != head);
+
+ /*
+ * Caller needs to validate requested block against end of device.
+ */
+ return end_block;
}
/*
* Create the page-cache page that contains the requested block.
*
- * This is user purely for blockdev mappings.
+ * This is used purely for blockdev mappings.
*/
-static struct page *
+static int
grow_dev_page(struct block_device *bdev, sector_t block,
- pgoff_t index, int size)
+ pgoff_t index, int size, int sizebits)
{
struct inode *inode = bdev->bd_inode;
struct page *page;
struct buffer_head *bh;
+ sector_t end_block;
+ int ret = 0; /* Will call free_more_memory() */
page = find_or_create_page(inode->i_mapping, index,
(mapping_gfp_mask(inode->i_mapping) & ~__GFP_FS)|__GFP_MOVABLE);
if (!page)
- return NULL;
+ return ret;
BUG_ON(!PageLocked(page));
if (page_has_buffers(page)) {
bh = page_buffers(page);
if (bh->b_size == size) {
- init_page_buffers(page, bdev, block, size);
- return page;
+ end_block = init_page_buffers(page, bdev,
+ index << sizebits, size);
+ goto done;
}
if (!try_to_free_buffers(page))
goto failed;
@@ -982,14 +990,14 @@
*/
spin_lock(&inode->i_mapping->private_lock);
link_dev_buffers(page, bh);
- init_page_buffers(page, bdev, block, size);
+ end_block = init_page_buffers(page, bdev, index << sizebits, size);
spin_unlock(&inode->i_mapping->private_lock);
- return page;
-
+done:
+ ret = (block < end_block) ? 1 : -ENXIO;
failed:
unlock_page(page);
page_cache_release(page);
- return NULL;
+ return ret;
}
/*
@@ -999,7 +1007,6 @@
static int
grow_buffers(struct block_device *bdev, sector_t block, int size)
{
- struct page *page;
pgoff_t index;
int sizebits;
@@ -1023,22 +1030,14 @@
bdevname(bdev, b));
return -EIO;
}
- block = index << sizebits;
+
/* Create a page with the proper size buffers.. */
- page = grow_dev_page(bdev, block, index, size);
- if (!page)
- return 0;
- unlock_page(page);
- page_cache_release(page);
- return 1;
+ return grow_dev_page(bdev, block, index, size, sizebits);
}
static struct buffer_head *
__getblk_slow(struct block_device *bdev, sector_t block, int size)
{
- int ret;
- struct buffer_head *bh;
-
/* Size must be multiple of hard sectorsize */
if (unlikely(size & (bdev_logical_block_size(bdev)-1) ||
(size < 512 || size > PAGE_SIZE))) {
@@ -1051,21 +1050,20 @@
return NULL;
}
-retry:
- bh = __find_get_block(bdev, block, size);
- if (bh)
- return bh;
+ for (;;) {
+ struct buffer_head *bh;
+ int ret;
- ret = grow_buffers(bdev, block, size);
- if (ret == 0) {
- free_more_memory();
- goto retry;
- } else if (ret > 0) {
bh = __find_get_block(bdev, block, size);
if (bh)
return bh;
+
+ ret = grow_buffers(bdev, block, size);
+ if (ret < 0)
+ return NULL;
+ if (ret == 0)
+ free_more_memory();
}
- return NULL;
}
/*
@@ -1321,10 +1319,6 @@
* which corresponds to the passed block_device, block and size. The
* returned buffer has its reference count incremented.
*
- * __getblk() cannot fail - it just keeps trying. If you pass it an
- * illegal block number, __getblk() will happily return a buffer_head
- * which represents the non-existent block. Very weird.
- *
* __getblk() will lock up the machine if grow_dev_page's try_to_free_buffers()
* attempt is failing. FIXME, perhaps?
*/
diff --git a/fs/cifs/cifs_unicode.c b/fs/cifs/cifs_unicode.c
index 7dab9c0..53cf2aa 100644
--- a/fs/cifs/cifs_unicode.c
+++ b/fs/cifs/cifs_unicode.c
@@ -328,7 +328,7 @@
}
ctoUTF16_out:
- return i;
+ return j;
}
#ifdef CONFIG_CIFS_SMB2
diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c
index 074923c..f0cf934 100644
--- a/fs/cifs/cifssmb.c
+++ b/fs/cifs/cifssmb.c
@@ -1576,9 +1576,14 @@
/* result already set, check signature */
if (server->sec_mode &
(SECMODE_SIGN_REQUIRED | SECMODE_SIGN_ENABLED)) {
- if (cifs_verify_signature(rdata->iov, rdata->nr_iov,
- server, mid->sequence_number + 1))
- cERROR(1, "Unexpected SMB signature");
+ int rc = 0;
+
+ rc = cifs_verify_signature(rdata->iov, rdata->nr_iov,
+ server,
+ mid->sequence_number + 1);
+ if (rc)
+ cERROR(1, "SMB signature verification returned "
+ "error = %d", rc);
}
/* FIXME: should this be counted toward the initiating task? */
task_io_account_read(rdata->bytes);
diff --git a/fs/cifs/dir.c b/fs/cifs/dir.c
index cbe709a..781025b 100644
--- a/fs/cifs/dir.c
+++ b/fs/cifs/dir.c
@@ -356,19 +356,12 @@
cifs_create_set_dentry:
if (rc != 0) {
cFYI(1, "Create worked, get_inode_info failed rc = %d", rc);
+ CIFSSMBClose(xid, tcon, *fileHandle);
goto out;
}
d_drop(direntry);
d_add(direntry, newinode);
- /* ENOENT for create? How weird... */
- rc = -ENOENT;
- if (!newinode) {
- CIFSSMBClose(xid, tcon, *fileHandle);
- goto out;
- }
- rc = 0;
-
out:
kfree(buf);
kfree(full_path);
diff --git a/fs/cifs/file.c b/fs/cifs/file.c
index 9154192..71e9ad9 100644
--- a/fs/cifs/file.c
+++ b/fs/cifs/file.c
@@ -917,7 +917,7 @@
if (!buf) {
mutex_unlock(&cinode->lock_mutex);
free_xid(xid);
- return rc;
+ return -ENOMEM;
}
for (i = 0; i < 2; i++) {
diff --git a/fs/cifs/inode.c b/fs/cifs/inode.c
index 7354877..cb79c7e 100644
--- a/fs/cifs/inode.c
+++ b/fs/cifs/inode.c
@@ -124,10 +124,10 @@
{
struct cifsInodeInfo *cifs_i = CIFS_I(inode);
struct cifs_sb_info *cifs_sb = CIFS_SB(inode->i_sb);
- unsigned long oldtime = cifs_i->time;
cifs_revalidate_cache(inode, fattr);
+ spin_lock(&inode->i_lock);
inode->i_atime = fattr->cf_atime;
inode->i_mtime = fattr->cf_mtime;
inode->i_ctime = fattr->cf_ctime;
@@ -148,9 +148,6 @@
else
cifs_i->time = jiffies;
- cFYI(1, "inode 0x%p old_time=%ld new_time=%ld", inode,
- oldtime, cifs_i->time);
-
cifs_i->delete_pending = fattr->cf_flags & CIFS_FATTR_DELETE_PENDING;
cifs_i->server_eof = fattr->cf_eof;
@@ -158,7 +155,6 @@
* Can't safely change the file size here if the client is writing to
* it due to potential races.
*/
- spin_lock(&inode->i_lock);
if (is_size_safe_to_change(cifs_i, fattr->cf_eof)) {
i_size_write(inode, fattr->cf_eof);
@@ -859,12 +855,14 @@
if (rc && tcon->ipc) {
cFYI(1, "ipc connection - fake read inode");
+ spin_lock(&inode->i_lock);
inode->i_mode |= S_IFDIR;
set_nlink(inode, 2);
inode->i_op = &cifs_ipc_inode_ops;
inode->i_fop = &simple_dir_operations;
inode->i_uid = cifs_sb->mnt_uid;
inode->i_gid = cifs_sb->mnt_gid;
+ spin_unlock(&inode->i_lock);
} else if (rc) {
iget_failed(inode);
inode = ERR_PTR(rc);
@@ -1110,6 +1108,15 @@
goto out_close;
}
+/* copied from fs/nfs/dir.c with small changes */
+static void
+cifs_drop_nlink(struct inode *inode)
+{
+ spin_lock(&inode->i_lock);
+ if (inode->i_nlink > 0)
+ drop_nlink(inode);
+ spin_unlock(&inode->i_lock);
+}
/*
* If dentry->d_inode is null (usually meaning the cached dentry
@@ -1166,13 +1173,13 @@
psx_del_no_retry:
if (!rc) {
if (inode)
- drop_nlink(inode);
+ cifs_drop_nlink(inode);
} else if (rc == -ENOENT) {
d_drop(dentry);
} else if (rc == -ETXTBSY) {
rc = cifs_rename_pending_delete(full_path, dentry, xid);
if (rc == 0)
- drop_nlink(inode);
+ cifs_drop_nlink(inode);
} else if ((rc == -EACCES) && (dosattr == 0) && inode) {
attrs = kzalloc(sizeof(*attrs), GFP_KERNEL);
if (attrs == NULL) {
@@ -1241,9 +1248,10 @@
* setting nlink not necessary except in cases where we failed to get it
* from the server or was set bogus
*/
+ spin_lock(&dentry->d_inode->i_lock);
if ((dentry->d_inode) && (dentry->d_inode->i_nlink < 2))
set_nlink(dentry->d_inode, 2);
-
+ spin_unlock(&dentry->d_inode->i_lock);
mode &= ~current_umask();
/* must turn on setgid bit if parent dir has it */
if (inode->i_mode & S_ISGID)
diff --git a/fs/cifs/link.c b/fs/cifs/link.c
index 09e4b3a..e6ce3b1 100644
--- a/fs/cifs/link.c
+++ b/fs/cifs/link.c
@@ -433,7 +433,9 @@
if (old_file->d_inode) {
cifsInode = CIFS_I(old_file->d_inode);
if (rc == 0) {
+ spin_lock(&old_file->d_inode->i_lock);
inc_nlink(old_file->d_inode);
+ spin_unlock(&old_file->d_inode->i_lock);
/* BB should we make this contingent on superblock flag NOATIME? */
/* old_file->d_inode->i_ctime = CURRENT_TIME;*/
/* parent dir timestamps will update from srv
diff --git a/fs/cifs/smb2misc.c b/fs/cifs/smb2misc.c
index a4ff5d5..e4d3b99 100644
--- a/fs/cifs/smb2misc.c
+++ b/fs/cifs/smb2misc.c
@@ -52,7 +52,8 @@
cERROR(1, "Bad protocol string signature header %x",
*(unsigned int *) hdr->ProtocolId);
if (mid != hdr->MessageId)
- cERROR(1, "Mids do not match");
+ cERROR(1, "Mids do not match: %llu and %llu", mid,
+ hdr->MessageId);
}
cERROR(1, "Bad SMB detected. The Mid=%llu", hdr->MessageId);
return 1;
@@ -107,7 +108,7 @@
* ie Validate the wct via smb2_struct_sizes table above
*/
- if (length < 2 + sizeof(struct smb2_hdr)) {
+ if (length < sizeof(struct smb2_pdu)) {
if ((length >= sizeof(struct smb2_hdr)) && (hdr->Status != 0)) {
pdu->StructureSize2 = 0;
/*
@@ -121,15 +122,15 @@
return 1;
}
if (len > CIFSMaxBufSize + MAX_SMB2_HDR_SIZE - 4) {
- cERROR(1, "SMB length greater than maximum, mid=%lld", mid);
+ cERROR(1, "SMB length greater than maximum, mid=%llu", mid);
return 1;
}
if (check_smb2_hdr(hdr, mid))
return 1;
- if (hdr->StructureSize != SMB2_HEADER_SIZE) {
- cERROR(1, "Illegal structure size %d",
+ if (hdr->StructureSize != SMB2_HEADER_STRUCTURE_SIZE) {
+ cERROR(1, "Illegal structure size %u",
le16_to_cpu(hdr->StructureSize));
return 1;
}
@@ -161,8 +162,9 @@
if (4 + len != clc_len) {
cFYI(1, "Calculated size %u length %u mismatch mid %llu",
clc_len, 4 + len, mid);
- if (clc_len == 4 + len + 1) /* BB FIXME (fix samba) */
- return 0; /* BB workaround Samba 3 bug SessSetup rsp */
+ /* server can return one byte more */
+ if (clc_len == 4 + len + 1)
+ return 0;
return 1;
}
return 0;
diff --git a/fs/cifs/smb2pdu.h b/fs/cifs/smb2pdu.h
index f37a1b4..15dc8ee 100644
--- a/fs/cifs/smb2pdu.h
+++ b/fs/cifs/smb2pdu.h
@@ -87,10 +87,6 @@
#define SMB2_PROTO_NUMBER __constant_cpu_to_le32(0x424d53fe)
-#define SMB2_HEADER_SIZE __constant_le16_to_cpu(64)
-
-#define SMB2_ERROR_STRUCTURE_SIZE2 __constant_le16_to_cpu(9)
-
/*
* SMB2 Header Definition
*
@@ -99,6 +95,9 @@
* "PDU" : "Protocol Data Unit" (ie a network "frame")
*
*/
+
+#define SMB2_HEADER_STRUCTURE_SIZE __constant_cpu_to_le16(64)
+
struct smb2_hdr {
__be32 smb2_buf_length; /* big endian on wire */
/* length is only two or three bytes - with
@@ -140,6 +139,9 @@
* command code name for the struct. Note that structures must be packed.
*
*/
+
+#define SMB2_ERROR_STRUCTURE_SIZE2 __constant_cpu_to_le16(9)
+
struct smb2_err_rsp {
struct smb2_hdr hdr;
__le16 StructureSize;
diff --git a/fs/cifs/transport.c b/fs/cifs/transport.c
index 83867ef..d9b639b 100644
--- a/fs/cifs/transport.c
+++ b/fs/cifs/transport.c
@@ -503,13 +503,16 @@
/* convert the length into a more usable form */
if (server->sec_mode & (SECMODE_SIGN_REQUIRED | SECMODE_SIGN_ENABLED)) {
struct kvec iov;
+ int rc = 0;
iov.iov_base = mid->resp_buf;
iov.iov_len = len;
/* FIXME: add code to kill session */
- if (cifs_verify_signature(&iov, 1, server,
- mid->sequence_number + 1) != 0)
- cERROR(1, "Unexpected SMB signature");
+ rc = cifs_verify_signature(&iov, 1, server,
+ mid->sequence_number + 1);
+ if (rc)
+ cERROR(1, "SMB signature verification returned error = "
+ "%d", rc);
}
/* BB special case reconnect tid and uid here? */
diff --git a/fs/dcache.c b/fs/dcache.c
index 8086636..693f95b 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -389,7 +389,7 @@
* Inform try_to_ascend() that we are no longer attached to the
* dentry tree
*/
- dentry->d_flags |= DCACHE_DISCONNECTED;
+ dentry->d_flags |= DCACHE_DENTRY_KILLED;
if (parent)
spin_unlock(&parent->d_lock);
dentry_iput(dentry);
@@ -1048,7 +1048,7 @@
* or deletion
*/
if (new != old->d_parent ||
- (old->d_flags & DCACHE_DISCONNECTED) ||
+ (old->d_flags & DCACHE_DENTRY_KILLED) ||
(!locked && read_seqretry(&rename_lock, seq))) {
spin_unlock(&new->d_lock);
new = NULL;
@@ -1134,6 +1134,8 @@
return 1;
rename_retry:
+ if (locked)
+ goto again;
locked = 1;
write_seqlock(&rename_lock);
goto again;
@@ -1141,7 +1143,7 @@
EXPORT_SYMBOL(have_submounts);
/*
- * Search the dentry child list for the specified parent,
+ * Search the dentry child list of the specified parent,
* and move any unused dentries to the end of the unused
* list for prune_dcache(). We descend to the next level
* whenever the d_subdirs list is non-empty and continue
@@ -1236,6 +1238,8 @@
rename_retry:
if (found)
return found;
+ if (locked)
+ goto again;
locked = 1;
write_seqlock(&rename_lock);
goto again;
@@ -3035,6 +3039,8 @@
return;
rename_retry:
+ if (locked)
+ goto again;
locked = 1;
write_seqlock(&rename_lock);
goto again;
diff --git a/fs/debugfs/file.c b/fs/debugfs/file.c
index 2340f69..c5ca6ae 100644
--- a/fs/debugfs/file.c
+++ b/fs/debugfs/file.c
@@ -526,73 +526,51 @@
u32 elements;
};
-static int u32_array_open(struct inode *inode, struct file *file)
-{
- file->private_data = NULL;
- return nonseekable_open(inode, file);
-}
-
-static size_t format_array(char *buf, size_t bufsize, const char *fmt,
- u32 *array, u32 array_size)
+static size_t u32_format_array(char *buf, size_t bufsize,
+ u32 *array, int array_size)
{
size_t ret = 0;
- u32 i;
- for (i = 0; i < array_size; i++) {
+ while (--array_size >= 0) {
size_t len;
+ char term = array_size ? ' ' : '\n';
- len = snprintf(buf, bufsize, fmt, array[i]);
- len++; /* ' ' or '\n' */
+ len = snprintf(buf, bufsize, "%u%c", *array++, term);
ret += len;
- if (buf) {
- buf += len;
- bufsize -= len;
- buf[-1] = (i == array_size-1) ? '\n' : ' ';
- }
+ buf += len;
+ bufsize -= len;
}
-
- ret++; /* \0 */
- if (buf)
- *buf = '\0';
-
return ret;
}
-static char *format_array_alloc(const char *fmt, u32 *array,
- u32 array_size)
+static int u32_array_open(struct inode *inode, struct file *file)
{
- size_t len = format_array(NULL, 0, fmt, array, array_size);
- char *ret;
+ struct array_data *data = inode->i_private;
+ int size, elements = data->elements;
+ char *buf;
- ret = kmalloc(len, GFP_KERNEL);
- if (ret == NULL)
- return NULL;
+ /*
+ * Max size:
+ * - 10 digits + ' '/'\n' = 11 bytes per number
+ * - terminating NUL character
+ */
+ size = elements*11;
+ buf = kmalloc(size+1, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+ buf[size] = 0;
- format_array(ret, len, fmt, array, array_size);
- return ret;
+ file->private_data = buf;
+ u32_format_array(buf, size, data->array, data->elements);
+
+ return nonseekable_open(inode, file);
}
static ssize_t u32_array_read(struct file *file, char __user *buf, size_t len,
loff_t *ppos)
{
- struct inode *inode = file->f_path.dentry->d_inode;
- struct array_data *data = inode->i_private;
- size_t size;
-
- if (*ppos == 0) {
- if (file->private_data) {
- kfree(file->private_data);
- file->private_data = NULL;
- }
-
- file->private_data = format_array_alloc("%u", data->array,
- data->elements);
- }
-
- size = 0;
- if (file->private_data)
- size = strlen(file->private_data);
+ size_t size = strlen(file->private_data);
return simple_read_from_buffer(buf, len, ppos,
file->private_data, size);
diff --git a/fs/direct-io.c b/fs/direct-io.c
index 1faf4cb..f86c720 100644
--- a/fs/direct-io.c
+++ b/fs/direct-io.c
@@ -1062,6 +1062,7 @@
unsigned long user_addr;
size_t bytes;
struct buffer_head map_bh = { 0, };
+ struct blk_plug plug;
if (rw & WRITE)
rw = WRITE_ODIRECT;
@@ -1177,6 +1178,8 @@
PAGE_SIZE - user_addr / PAGE_SIZE);
}
+ blk_start_plug(&plug);
+
for (seg = 0; seg < nr_segs; seg++) {
user_addr = (unsigned long)iov[seg].iov_base;
sdio.size += bytes = iov[seg].iov_len;
@@ -1235,6 +1238,8 @@
if (sdio.bio)
dio_bio_submit(dio, &sdio);
+ blk_finish_plug(&plug);
+
/*
* It is possible that, we return short IO due to end of file.
* In that case, we need to release all the pages we got hold on.
diff --git a/fs/ecryptfs/file.c b/fs/ecryptfs/file.c
index 44ce5c6..d45ba45 100644
--- a/fs/ecryptfs/file.c
+++ b/fs/ecryptfs/file.c
@@ -275,8 +275,14 @@
static int ecryptfs_flush(struct file *file, fl_owner_t td)
{
- return file->f_mode & FMODE_WRITE
- ? filemap_write_and_wait(file->f_mapping) : 0;
+ struct file *lower_file = ecryptfs_file_to_lower(file);
+
+ if (lower_file->f_op && lower_file->f_op->flush) {
+ filemap_write_and_wait(file->f_mapping);
+ return lower_file->f_op->flush(lower_file, td);
+ }
+
+ return 0;
}
static int ecryptfs_release(struct inode *inode, struct file *file)
diff --git a/fs/ecryptfs/inode.c b/fs/ecryptfs/inode.c
index 534b129..cc7709e7 100644
--- a/fs/ecryptfs/inode.c
+++ b/fs/ecryptfs/inode.c
@@ -619,6 +619,7 @@
struct dentry *lower_old_dir_dentry;
struct dentry *lower_new_dir_dentry;
struct dentry *trap = NULL;
+ struct inode *target_inode;
lower_old_dentry = ecryptfs_dentry_to_lower(old_dentry);
lower_new_dentry = ecryptfs_dentry_to_lower(new_dentry);
@@ -626,6 +627,7 @@
dget(lower_new_dentry);
lower_old_dir_dentry = dget_parent(lower_old_dentry);
lower_new_dir_dentry = dget_parent(lower_new_dentry);
+ target_inode = new_dentry->d_inode;
trap = lock_rename(lower_old_dir_dentry, lower_new_dir_dentry);
/* source should not be ancestor of target */
if (trap == lower_old_dentry) {
@@ -641,6 +643,9 @@
lower_new_dir_dentry->d_inode, lower_new_dentry);
if (rc)
goto out_lock;
+ if (target_inode)
+ fsstack_copy_attr_all(target_inode,
+ ecryptfs_inode_to_lower(target_inode));
fsstack_copy_attr_all(new_dir, lower_new_dir_dentry->d_inode);
if (new_dir != old_dir)
fsstack_copy_attr_all(old_dir, lower_old_dir_dentry->d_inode);
diff --git a/fs/ecryptfs/main.c b/fs/ecryptfs/main.c
index 2768138..9b627c1 100644
--- a/fs/ecryptfs/main.c
+++ b/fs/ecryptfs/main.c
@@ -162,6 +162,7 @@
inode_info = ecryptfs_inode_to_private(inode);
if (atomic_dec_and_mutex_lock(&inode_info->lower_file_count,
&inode_info->lower_file_mutex)) {
+ filemap_write_and_wait(inode->i_mapping);
fput(inode_info->lower_file);
inode_info->lower_file = NULL;
mutex_unlock(&inode_info->lower_file_mutex);
diff --git a/fs/ext2/balloc.c b/fs/ext2/balloc.c
index 376aa77..2616d0e 100644
--- a/fs/ext2/balloc.c
+++ b/fs/ext2/balloc.c
@@ -479,7 +479,7 @@
/**
* ext2_free_blocks() -- Free given blocks and update quota and i_blocks
* @inode: inode
- * @block: start physcial block to free
+ * @block: start physical block to free
* @count: number of blocks to free
*/
void ext2_free_blocks (struct inode * inode, unsigned long block,
diff --git a/fs/ext3/balloc.c b/fs/ext3/balloc.c
index 90d901f..7320a66 100644
--- a/fs/ext3/balloc.c
+++ b/fs/ext3/balloc.c
@@ -483,7 +483,7 @@
* ext3_free_blocks_sb() -- Free given blocks and update quota
* @handle: handle to this transaction
* @sb: super block
- * @block: start physcial block to free
+ * @block: start physical block to free
* @count: number of blocks to free
* @pdquot_freed_blocks: pointer to quota
*/
diff --git a/fs/ext3/inode.c b/fs/ext3/inode.c
index a075973..7e87e37 100644
--- a/fs/ext3/inode.c
+++ b/fs/ext3/inode.c
@@ -3072,6 +3072,8 @@
struct ext3_inode_info *ei = EXT3_I(inode);
struct buffer_head *bh = iloc->bh;
int err = 0, rc, block;
+ int need_datasync = 0;
+ __le32 disksize;
uid_t i_uid;
gid_t i_gid;
@@ -3113,7 +3115,11 @@
raw_inode->i_gid_high = 0;
}
raw_inode->i_links_count = cpu_to_le16(inode->i_nlink);
- raw_inode->i_size = cpu_to_le32(ei->i_disksize);
+ disksize = cpu_to_le32(ei->i_disksize);
+ if (disksize != raw_inode->i_size) {
+ need_datasync = 1;
+ raw_inode->i_size = disksize;
+ }
raw_inode->i_atime = cpu_to_le32(inode->i_atime.tv_sec);
raw_inode->i_ctime = cpu_to_le32(inode->i_ctime.tv_sec);
raw_inode->i_mtime = cpu_to_le32(inode->i_mtime.tv_sec);
@@ -3129,8 +3135,11 @@
if (!S_ISREG(inode->i_mode)) {
raw_inode->i_dir_acl = cpu_to_le32(ei->i_dir_acl);
} else {
- raw_inode->i_size_high =
- cpu_to_le32(ei->i_disksize >> 32);
+ disksize = cpu_to_le32(ei->i_disksize >> 32);
+ if (disksize != raw_inode->i_size_high) {
+ raw_inode->i_size_high = disksize;
+ need_datasync = 1;
+ }
if (ei->i_disksize > 0x7fffffffULL) {
struct super_block *sb = inode->i_sb;
if (!EXT3_HAS_RO_COMPAT_FEATURE(sb,
@@ -3183,6 +3192,8 @@
ext3_clear_inode_state(inode, EXT3_STATE_NEW);
atomic_set(&ei->i_sync_tid, handle->h_transaction->t_tid);
+ if (need_datasync)
+ atomic_set(&ei->i_datasync_tid, handle->h_transaction->t_tid);
out_brelse:
brelse (bh);
ext3_std_error(inode->i_sb, err);
@@ -3196,7 +3207,7 @@
*
* - Within generic_file_write() for O_SYNC files.
* Here, there will be no transaction running. We wait for any running
- * trasnaction to commit.
+ * transaction to commit.
*
* - Within sys_sync(), kupdate and such.
* We wait on commit, if tol to.
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index dff171c..c862ee5 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3313,7 +3313,7 @@
* handle: The journal handle
* inode: The files inode
* page: A locked page that contains the offset "from"
- * from: The starting byte offset (from the begining of the file)
+ * from: The starting byte offset (from the beginning of the file)
* to begin discarding
* len: The length of bytes to discard
* flags: Optional flags that may be used:
@@ -3321,11 +3321,11 @@
* EXT4_DISCARD_PARTIAL_PG_ZERO_UNMAPPED
* Only zero the regions of the page whose buffer heads
* have already been unmapped. This flag is appropriate
- * for updateing the contents of a page whose blocks may
+ * for updating the contents of a page whose blocks may
* have already been released, and we only want to zero
* out the regions that correspond to those released blocks.
*
- * Returns zero on sucess or negative on failure.
+ * Returns zero on success or negative on failure.
*/
static int ext4_discard_partial_page_buffers_no_lock(handle_t *handle,
struct inode *inode, struct page *page, loff_t from,
@@ -3486,7 +3486,7 @@
* @offset: The offset where the hole will begin
* @len: The length of the hole
*
- * Returns: 0 on sucess or negative on failure
+ * Returns: 0 on success or negative on failure
*/
int ext4_punch_hole(struct file *file, loff_t offset, loff_t length)
@@ -4008,7 +4008,7 @@
if (i_blocks <= ~0U) {
/*
- * i_blocks can be represnted in a 32 bit variable
+ * i_blocks can be represented in a 32 bit variable
* as multiple of 512 bytes
*/
raw_inode->i_blocks_lo = cpu_to_le32(i_blocks);
@@ -4169,7 +4169,7 @@
*
* - Within generic_file_write() for O_SYNC files.
* Here, there will be no transaction running. We wait for any running
- * trasnaction to commit.
+ * transaction to commit.
*
* - Within sys_sync(), kupdate and such.
* We wait on commit, if tol to.
@@ -4413,7 +4413,7 @@
* worse case, the indexs blocks spread over different block groups
*
* If datablocks are discontiguous, they are possible to spread over
- * different block groups too. If they are contiuguous, with flexbg,
+ * different block groups too. If they are contiguous, with flexbg,
* they could still across block group boundary.
*
* Also account for superblock, inode, quota and xattr blocks
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 8eae947..08778f6 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -4709,7 +4709,7 @@
* ext4_group_add_blocks() -- Add given blocks to an existing group
* @handle: handle to this transaction
* @sb: super block
- * @block: start physcial block to add to the block group
+ * @block: start physical block to add to the block group
* @count: number of blocks to free
*
* This marks the blocks as free in the bitmap and buddy.
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index be3efc4..6d46c0d 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -577,10 +577,6 @@
/*
* Write a portion of b_io inodes which belong to @sb.
*
- * If @only_this_sb is true, then find and write all such
- * inodes. Otherwise write only ones which go sequentially
- * in reverse order.
- *
* Return the number of pages and/or inodes written.
*/
static long writeback_sb_inodes(struct super_block *sb,
diff --git a/fs/fuse/control.c b/fs/fuse/control.c
index 03ff5b1..75a20c0 100644
--- a/fs/fuse/control.c
+++ b/fs/fuse/control.c
@@ -117,7 +117,7 @@
const char __user *buf,
size_t count, loff_t *ppos)
{
- unsigned val;
+ unsigned uninitialized_var(val);
ssize_t ret;
ret = fuse_conn_limit_write(file, buf, count, ppos, &val,
@@ -154,7 +154,7 @@
const char __user *buf,
size_t count, loff_t *ppos)
{
- unsigned val;
+ unsigned uninitialized_var(val);
ssize_t ret;
ret = fuse_conn_limit_write(file, buf, count, ppos, &val,
diff --git a/fs/fuse/cuse.c b/fs/fuse/cuse.c
index 3426521..ee8d550 100644
--- a/fs/fuse/cuse.c
+++ b/fs/fuse/cuse.c
@@ -396,7 +396,7 @@
err_region:
unregister_chrdev_region(devt, 1);
err:
- fc->conn_error = 1;
+ fuse_conn_kill(fc);
goto out;
}
@@ -532,8 +532,6 @@
cdev_del(cc->cdev);
}
- /* kill connection and shutdown channel */
- fuse_conn_kill(&cc->fc);
rc = fuse_dev_release(inode, file); /* puts the base reference */
return rc;
diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c
index 7df2b5e..f4246cf 100644
--- a/fs/fuse/dev.c
+++ b/fs/fuse/dev.c
@@ -1576,6 +1576,7 @@
req->pages[req->num_pages] = page;
req->num_pages++;
+ offset = 0;
num -= this_num;
total_len += this_num;
index++;
diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c
index ce0a283..fca222d 100644
--- a/fs/fuse/inode.c
+++ b/fs/fuse/inode.c
@@ -367,11 +367,6 @@
wake_up_all(&fc->waitq);
wake_up_all(&fc->blocked_waitq);
wake_up_all(&fc->reserved_req_waitq);
- mutex_lock(&fuse_mutex);
- list_del(&fc->entry);
- fuse_ctl_remove_conn(fc);
- mutex_unlock(&fuse_mutex);
- fuse_bdi_destroy(fc);
}
EXPORT_SYMBOL_GPL(fuse_conn_kill);
@@ -380,7 +375,14 @@
struct fuse_conn *fc = get_fuse_conn_super(sb);
fuse_send_destroy(fc);
+
fuse_conn_kill(fc);
+ mutex_lock(&fuse_mutex);
+ list_del(&fc->entry);
+ fuse_ctl_remove_conn(fc);
+ mutex_unlock(&fuse_mutex);
+ fuse_bdi_destroy(fc);
+
fuse_conn_put(fc);
}
diff --git a/fs/gfs2/aops.c b/fs/gfs2/aops.c
index d652634..01c4975 100644
--- a/fs/gfs2/aops.c
+++ b/fs/gfs2/aops.c
@@ -612,6 +612,7 @@
struct gfs2_sbd *sdp = GFS2_SB(mapping->host);
struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode);
unsigned int data_blocks = 0, ind_blocks = 0, rblocks;
+ unsigned requested = 0;
int alloc_required;
int error = 0;
pgoff_t index = pos >> PAGE_CACHE_SHIFT;
@@ -641,7 +642,8 @@
if (error)
goto out_unlock;
- error = gfs2_inplace_reserve(ip, data_blocks + ind_blocks);
+ requested = data_blocks + ind_blocks;
+ error = gfs2_inplace_reserve(ip, requested);
if (error)
goto out_qunlock;
}
@@ -654,7 +656,7 @@
if (&ip->i_inode == sdp->sd_rindex)
rblocks += 2 * RES_STATFS;
if (alloc_required)
- rblocks += gfs2_rg_blocks(ip);
+ rblocks += gfs2_rg_blocks(ip, requested);
error = gfs2_trans_begin(sdp, rblocks,
PAGE_CACHE_SIZE/sdp->sd_sb.sb_bsize);
@@ -868,8 +870,7 @@
brelse(dibh);
failed:
gfs2_trans_end(sdp);
- if (gfs2_mb_reserved(ip))
- gfs2_inplace_release(ip);
+ gfs2_inplace_release(ip);
if (ip->i_res->rs_qa_qd_num)
gfs2_quota_unlock(ip);
if (inode == sdp->sd_rindex) {
@@ -1023,7 +1024,7 @@
offset, nr_segs, gfs2_get_block_direct,
NULL, NULL, 0);
out:
- gfs2_glock_dq_m(1, &gh);
+ gfs2_glock_dq(&gh);
gfs2_holder_uninit(&gh);
return rv;
}
diff --git a/fs/gfs2/bmap.c b/fs/gfs2/bmap.c
index 49cd7dd..1fd3ae2 100644
--- a/fs/gfs2/bmap.c
+++ b/fs/gfs2/bmap.c
@@ -786,7 +786,7 @@
goto out_rlist;
if (gfs2_rs_active(ip->i_res)) /* needs to be done with the rgrp glock held */
- gfs2_rs_deltree(ip->i_res);
+ gfs2_rs_deltree(ip, ip->i_res);
error = gfs2_trans_begin(sdp, rg_blocks + RES_DINODE +
RES_INDIRECT + RES_STATFS + RES_QUOTA,
diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c
index d1d791e..30e2199 100644
--- a/fs/gfs2/file.c
+++ b/fs/gfs2/file.c
@@ -323,6 +323,29 @@
}
/**
+ * gfs2_size_hint - Give a hint to the size of a write request
+ * @file: The struct file
+ * @offset: The file offset of the write
+ * @size: The length of the write
+ *
+ * When we are about to do a write, this function records the total
+ * write size in order to provide a suitable hint to the lower layers
+ * about how many blocks will be required.
+ *
+ */
+
+static void gfs2_size_hint(struct file *filep, loff_t offset, size_t size)
+{
+ struct inode *inode = filep->f_dentry->d_inode;
+ struct gfs2_sbd *sdp = GFS2_SB(inode);
+ struct gfs2_inode *ip = GFS2_I(inode);
+ size_t blks = (size + sdp->sd_sb.sb_bsize - 1) >> sdp->sd_sb.sb_bsize_shift;
+ int hint = min_t(size_t, INT_MAX, blks);
+
+ atomic_set(&ip->i_res->rs_sizehint, hint);
+}
+
+/**
* gfs2_allocate_page_backing - Use bmap to allocate blocks
* @page: The (locked) page to allocate backing for
*
@@ -382,8 +405,7 @@
if (ret)
return ret;
- atomic_set(&ip->i_res->rs_sizehint,
- PAGE_CACHE_SIZE >> sdp->sd_sb.sb_bsize_shift);
+ gfs2_size_hint(vma->vm_file, pos, PAGE_CACHE_SIZE);
gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &gh);
ret = gfs2_glock_nq(&gh);
@@ -419,7 +441,7 @@
rblocks += data_blocks ? data_blocks : 1;
if (ind_blocks || data_blocks) {
rblocks += RES_STATFS + RES_QUOTA;
- rblocks += gfs2_rg_blocks(ip);
+ rblocks += gfs2_rg_blocks(ip, data_blocks + ind_blocks);
}
ret = gfs2_trans_begin(sdp, rblocks, 0);
if (ret)
@@ -663,7 +685,8 @@
if (ret)
return ret;
- atomic_set(&ip->i_res->rs_sizehint, writesize >> sdp->sd_sb.sb_bsize_shift);
+ gfs2_size_hint(file, pos, writesize);
+
if (file->f_flags & O_APPEND) {
struct gfs2_holder gh;
@@ -789,7 +812,7 @@
if (unlikely(error))
goto out_uninit;
- atomic_set(&ip->i_res->rs_sizehint, len >> sdp->sd_sb.sb_bsize_shift);
+ gfs2_size_hint(file, offset, len);
while (len > 0) {
if (len < bytes)
@@ -822,7 +845,7 @@
&max_bytes, &data_blocks, &ind_blocks);
rblocks = RES_DINODE + ind_blocks + RES_STATFS + RES_QUOTA +
- RES_RG_HDR + gfs2_rg_blocks(ip);
+ RES_RG_HDR + gfs2_rg_blocks(ip, data_blocks + ind_blocks);
if (gfs2_is_jdata(ip))
rblocks += data_blocks ? data_blocks : 1;
diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c
index 1ed81f4..e6c2fd5 100644
--- a/fs/gfs2/glock.c
+++ b/fs/gfs2/glock.c
@@ -186,20 +186,6 @@
}
/**
- * __gfs2_glock_schedule_for_reclaim - Add a glock to the reclaim list
- * @gl: the glock
- *
- * If the glock is demotable, then we add it (or move it) to the end
- * of the glock LRU list.
- */
-
-static void __gfs2_glock_schedule_for_reclaim(struct gfs2_glock *gl)
-{
- if (demote_ok(gl))
- gfs2_glock_add_to_lru(gl);
-}
-
-/**
* gfs2_glock_put_nolock() - Decrement reference count on glock
* @gl: The glock to put
*
@@ -883,7 +869,14 @@
return 0;
}
-static void wait_on_holder(struct gfs2_holder *gh)
+/**
+ * gfs2_glock_wait - wait on a glock acquisition
+ * @gh: the glock holder
+ *
+ * Returns: 0 on success
+ */
+
+int gfs2_glock_wait(struct gfs2_holder *gh)
{
unsigned long time1 = jiffies;
@@ -894,12 +887,7 @@
gh->gh_gl->gl_hold_time = min(gh->gh_gl->gl_hold_time +
GL_GLOCK_HOLD_INCR,
GL_GLOCK_MAX_HOLD);
-}
-
-static void wait_on_demote(struct gfs2_glock *gl)
-{
- might_sleep();
- wait_on_bit(&gl->gl_flags, GLF_DEMOTE, gfs2_glock_demote_wait, TASK_UNINTERRUPTIBLE);
+ return gh->gh_error;
}
/**
@@ -929,19 +917,6 @@
trace_gfs2_demote_rq(gl);
}
-/**
- * gfs2_glock_wait - wait on a glock acquisition
- * @gh: the glock holder
- *
- * Returns: 0 on success
- */
-
-int gfs2_glock_wait(struct gfs2_holder *gh)
-{
- wait_on_holder(gh);
- return gh->gh_error;
-}
-
void gfs2_print_dbg(struct seq_file *seq, const char *fmt, ...)
{
struct va_format vaf;
@@ -979,7 +954,7 @@
struct gfs2_sbd *sdp = gl->gl_sbd;
struct list_head *insert_pt = NULL;
struct gfs2_holder *gh2;
- int try_lock = 0;
+ int try_futile = 0;
BUG_ON(gh->gh_owner_pid == NULL);
if (test_and_set_bit(HIF_WAIT, &gh->gh_iflags))
@@ -987,7 +962,7 @@
if (gh->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB)) {
if (test_bit(GLF_LOCK, &gl->gl_flags))
- try_lock = 1;
+ try_futile = !may_grant(gl, gh);
if (test_bit(GLF_INVALIDATE_IN_PROGRESS, &gl->gl_flags))
goto fail;
}
@@ -996,9 +971,8 @@
if (unlikely(gh2->gh_owner_pid == gh->gh_owner_pid &&
(gh->gh_gl->gl_ops->go_type != LM_TYPE_FLOCK)))
goto trap_recursive;
- if (try_lock &&
- !(gh2->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB)) &&
- !may_grant(gl, gh)) {
+ if (try_futile &&
+ !(gh2->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB))) {
fail:
gh->gh_error = GLR_TRYFAILED;
gfs2_holder_wake(gh);
@@ -1121,8 +1095,9 @@
!test_bit(GLF_DEMOTE, &gl->gl_flags))
fast_path = 1;
}
- if (!test_bit(GLF_LFLUSH, &gl->gl_flags))
- __gfs2_glock_schedule_for_reclaim(gl);
+ if (!test_bit(GLF_LFLUSH, &gl->gl_flags) && demote_ok(gl))
+ gfs2_glock_add_to_lru(gl);
+
trace_gfs2_glock_queue(gh, 0);
spin_unlock(&gl->gl_spin);
if (likely(fast_path))
@@ -1141,7 +1116,8 @@
{
struct gfs2_glock *gl = gh->gh_gl;
gfs2_glock_dq(gh);
- wait_on_demote(gl);
+ might_sleep();
+ wait_on_bit(&gl->gl_flags, GLF_DEMOTE, gfs2_glock_demote_wait, TASK_UNINTERRUPTIBLE);
}
/**
diff --git a/fs/gfs2/glops.c b/fs/gfs2/glops.c
index 4bdcf37..32cc4fd 100644
--- a/fs/gfs2/glops.c
+++ b/fs/gfs2/glops.c
@@ -94,6 +94,7 @@
/* A shortened, inline version of gfs2_trans_begin() */
tr.tr_reserved = 1 + gfs2_struct2blk(sdp, tr.tr_revokes, sizeof(u64));
tr.tr_ip = (unsigned long)__builtin_return_address(0);
+ sb_start_intwrite(sdp->sd_vfs);
gfs2_log_reserve(sdp, tr.tr_reserved);
BUG_ON(current->journal_info);
current->journal_info = &tr;
diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h
index aaecc80..3d469d3 100644
--- a/fs/gfs2/incore.h
+++ b/fs/gfs2/incore.h
@@ -99,9 +99,26 @@
#define GFS2_RDF_MASK 0xf0000000 /* mask for internal flags */
spinlock_t rd_rsspin; /* protects reservation related vars */
struct rb_root rd_rstree; /* multi-block reservation tree */
- u32 rd_rs_cnt; /* count of current reservations */
};
+struct gfs2_rbm {
+ struct gfs2_rgrpd *rgd;
+ struct gfs2_bitmap *bi; /* Bitmap must belong to the rgd */
+ u32 offset; /* The offset is bitmap relative */
+};
+
+static inline u64 gfs2_rbm_to_block(const struct gfs2_rbm *rbm)
+{
+ return rbm->rgd->rd_data0 + (rbm->bi->bi_start * GFS2_NBBY) + rbm->offset;
+}
+
+static inline bool gfs2_rbm_eq(const struct gfs2_rbm *rbm1,
+ const struct gfs2_rbm *rbm2)
+{
+ return (rbm1->rgd == rbm2->rgd) && (rbm1->bi == rbm2->bi) &&
+ (rbm1->offset == rbm2->offset);
+}
+
enum gfs2_state_bits {
BH_Pinned = BH_PrivateStart,
BH_Escaped = BH_PrivateStart + 1,
@@ -250,18 +267,11 @@
/* components used during write (step 1): */
atomic_t rs_sizehint; /* hint of the write size */
- /* components used during inplace_reserve (step 2): */
- u32 rs_requested; /* Filled in by caller of gfs2_inplace_reserve() */
-
- /* components used during get_local_rgrp (step 3): */
- struct gfs2_rgrpd *rs_rgd; /* pointer to the gfs2_rgrpd */
struct gfs2_holder rs_rgd_gh; /* Filled in by get_local_rgrp */
struct rb_node rs_node; /* link to other block reservations */
-
- /* components used during block searches and assignments (step 4): */
- struct gfs2_bitmap *rs_bi; /* bitmap for the current allocation */
- u32 rs_biblk; /* start block relative to the bi */
+ struct gfs2_rbm rs_rbm; /* Start of reservation */
u32 rs_free; /* how many blocks are still free */
+ u64 rs_inum; /* Inode number for reservation */
/* ancillary quota stuff */
struct gfs2_quota_data *rs_qa_qd[2 * MAXQUOTAS];
diff --git a/fs/gfs2/inode.c b/fs/gfs2/inode.c
index 4ce22e5..381893c 100644
--- a/fs/gfs2/inode.c
+++ b/fs/gfs2/inode.c
@@ -712,14 +712,9 @@
if (error)
goto fail_gunlock2;
- /* The newly created inode needs a reservation so it can allocate
- xattrs. At the same time, we want new blocks allocated to the new
- dinode to be as contiguous as possible. Since we allocated the
- dinode block under the directory's reservation, we transfer
- ownership of that reservation to the new inode. The directory
- doesn't need a reservation unless it needs a new allocation. */
- ip->i_res = dip->i_res;
- dip->i_res = NULL;
+ error = gfs2_rs_alloc(ip);
+ if (error)
+ goto fail_gunlock2;
error = gfs2_acl_create(dip, inode);
if (error)
@@ -737,10 +732,7 @@
brelse(bh);
gfs2_trans_end(sdp);
- /* Check if we reserved space in the rgrp. Function link_dinode may
- not, depending on whether alloc is required. */
- if (gfs2_mb_reserved(dip))
- gfs2_inplace_release(dip);
+ gfs2_inplace_release(dip);
gfs2_quota_unlock(dip);
mark_inode_dirty(inode);
gfs2_glock_dq_uninit_m(2, ghs);
@@ -897,7 +889,7 @@
goto out_gunlock_q;
error = gfs2_trans_begin(sdp, sdp->sd_max_dirres +
- gfs2_rg_blocks(dip) +
+ gfs2_rg_blocks(dip, sdp->sd_max_dirres) +
2 * RES_DINODE + RES_STATFS +
RES_QUOTA, 0);
if (error)
@@ -1378,7 +1370,7 @@
goto out_gunlock_q;
error = gfs2_trans_begin(sdp, sdp->sd_max_dirres +
- gfs2_rg_blocks(ndip) +
+ gfs2_rg_blocks(ndip, sdp->sd_max_dirres) +
4 * RES_DINODE + 4 * RES_LEAF +
RES_STATFS + RES_QUOTA + 4, 0);
if (error)
@@ -1722,7 +1714,9 @@
gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &gh);
ret = gfs2_glock_nq(&gh);
if (ret == 0) {
- ret = generic_setxattr(dentry, name, data, size, flags);
+ ret = gfs2_rs_alloc(ip);
+ if (ret == 0)
+ ret = generic_setxattr(dentry, name, data, size, flags);
gfs2_glock_dq(&gh);
}
gfs2_holder_uninit(&gh);
@@ -1757,7 +1751,9 @@
gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &gh);
ret = gfs2_glock_nq(&gh);
if (ret == 0) {
- ret = generic_removexattr(dentry, name);
+ ret = gfs2_rs_alloc(ip);
+ if (ret == 0)
+ ret = generic_removexattr(dentry, name);
gfs2_glock_dq(&gh);
}
gfs2_holder_uninit(&gh);
diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c
index e5af9dc..e443966 100644
--- a/fs/gfs2/ops_fstype.c
+++ b/fs/gfs2/ops_fstype.c
@@ -19,6 +19,7 @@
#include <linux/mount.h>
#include <linux/gfs2_ondisk.h>
#include <linux/quotaops.h>
+#include <linux/lockdep.h>
#include "gfs2.h"
#include "incore.h"
@@ -766,6 +767,7 @@
return error;
}
+static struct lock_class_key gfs2_quota_imutex_key;
static int init_inodes(struct gfs2_sbd *sdp, int undo)
{
@@ -803,6 +805,12 @@
fs_err(sdp, "can't get quota file inode: %d\n", error);
goto fail_rindex;
}
+ /*
+ * i_mutex on quota files is special. Since this inode is hidden system
+ * file, we are safe to define locking ourselves.
+ */
+ lockdep_set_class(&sdp->sd_quota_inode->i_mutex,
+ &gfs2_quota_imutex_key);
error = gfs2_rindex_update(sdp);
if (error)
diff --git a/fs/gfs2/quota.c b/fs/gfs2/quota.c
index a3bde91..4021dec 100644
--- a/fs/gfs2/quota.c
+++ b/fs/gfs2/quota.c
@@ -765,6 +765,7 @@
struct gfs2_holder *ghs, i_gh;
unsigned int qx, x;
struct gfs2_quota_data *qd;
+ unsigned reserved;
loff_t offset;
unsigned int nalloc = 0, blocks;
int error;
@@ -781,7 +782,7 @@
return -ENOMEM;
sort(qda, num_qd, sizeof(struct gfs2_quota_data *), sort_qd, NULL);
- mutex_lock_nested(&ip->i_inode.i_mutex, I_MUTEX_QUOTA);
+ mutex_lock(&ip->i_inode.i_mutex);
for (qx = 0; qx < num_qd; qx++) {
error = gfs2_glock_nq_init(qda[qx]->qd_gl, LM_ST_EXCLUSIVE,
GL_NOCACHE, &ghs[qx]);
@@ -811,13 +812,13 @@
* two blocks need to be updated instead of 1 */
blocks = num_qd * data_blocks + RES_DINODE + num_qd + 3;
- error = gfs2_inplace_reserve(ip, 1 +
- (nalloc * (data_blocks + ind_blocks)));
+ reserved = 1 + (nalloc * (data_blocks + ind_blocks));
+ error = gfs2_inplace_reserve(ip, reserved);
if (error)
goto out_alloc;
if (nalloc)
- blocks += gfs2_rg_blocks(ip) + nalloc * ind_blocks + RES_STATFS;
+ blocks += gfs2_rg_blocks(ip, reserved) + nalloc * ind_blocks + RES_STATFS;
error = gfs2_trans_begin(sdp, blocks, 0);
if (error)
@@ -1598,7 +1599,7 @@
error = gfs2_inplace_reserve(ip, blocks);
if (error)
goto out_i;
- blocks += gfs2_rg_blocks(ip);
+ blocks += gfs2_rg_blocks(ip, blocks);
}
/* Some quotas span block boundaries and can update two blocks,
diff --git a/fs/gfs2/rgrp.c b/fs/gfs2/rgrp.c
index 4d34887..3cc402c 100644
--- a/fs/gfs2/rgrp.c
+++ b/fs/gfs2/rgrp.c
@@ -35,9 +35,6 @@
#define BFITNOENT ((u32)~0)
#define NO_BLOCK ((u64)~0)
-#define RSRV_CONTENTION_FACTOR 4
-#define RGRP_RSRV_MAX_CONTENDERS 2
-
#if BITS_PER_LONG == 32
#define LBITMASK (0x55555555UL)
#define LBITSKIP55 (0x55555555UL)
@@ -67,53 +64,48 @@
1, 0, 0, 0
};
-static u32 rgblk_search(struct gfs2_rgrpd *rgd, u32 goal,
- unsigned char old_state,
- struct gfs2_bitmap **rbi);
+static int gfs2_rbm_find(struct gfs2_rbm *rbm, u8 state, u32 minext,
+ const struct gfs2_inode *ip, bool nowrap);
+
/**
* gfs2_setbit - Set a bit in the bitmaps
- * @rgd: the resource group descriptor
- * @buf2: the clone buffer that holds the bitmaps
- * @bi: the bitmap structure
- * @block: the block to set
+ * @rbm: The position of the bit to set
+ * @do_clone: Also set the clone bitmap, if it exists
* @new_state: the new state of the block
*
*/
-static inline void gfs2_setbit(struct gfs2_rgrpd *rgd, unsigned char *buf2,
- struct gfs2_bitmap *bi, u32 block,
+static inline void gfs2_setbit(const struct gfs2_rbm *rbm, bool do_clone,
unsigned char new_state)
{
unsigned char *byte1, *byte2, *end, cur_state;
- unsigned int buflen = bi->bi_len;
- const unsigned int bit = (block % GFS2_NBBY) * GFS2_BIT_SIZE;
+ unsigned int buflen = rbm->bi->bi_len;
+ const unsigned int bit = (rbm->offset % GFS2_NBBY) * GFS2_BIT_SIZE;
- byte1 = bi->bi_bh->b_data + bi->bi_offset + (block / GFS2_NBBY);
- end = bi->bi_bh->b_data + bi->bi_offset + buflen;
+ byte1 = rbm->bi->bi_bh->b_data + rbm->bi->bi_offset + (rbm->offset / GFS2_NBBY);
+ end = rbm->bi->bi_bh->b_data + rbm->bi->bi_offset + buflen;
BUG_ON(byte1 >= end);
cur_state = (*byte1 >> bit) & GFS2_BIT_MASK;
if (unlikely(!valid_change[new_state * 4 + cur_state])) {
- printk(KERN_WARNING "GFS2: buf_blk = 0x%llx old_state=%d, "
- "new_state=%d\n",
- (unsigned long long)block, cur_state, new_state);
- printk(KERN_WARNING "GFS2: rgrp=0x%llx bi_start=0x%lx\n",
- (unsigned long long)rgd->rd_addr,
- (unsigned long)bi->bi_start);
- printk(KERN_WARNING "GFS2: bi_offset=0x%lx bi_len=0x%lx\n",
- (unsigned long)bi->bi_offset,
- (unsigned long)bi->bi_len);
+ printk(KERN_WARNING "GFS2: buf_blk = 0x%x old_state=%d, "
+ "new_state=%d\n", rbm->offset, cur_state, new_state);
+ printk(KERN_WARNING "GFS2: rgrp=0x%llx bi_start=0x%x\n",
+ (unsigned long long)rbm->rgd->rd_addr,
+ rbm->bi->bi_start);
+ printk(KERN_WARNING "GFS2: bi_offset=0x%x bi_len=0x%x\n",
+ rbm->bi->bi_offset, rbm->bi->bi_len);
dump_stack();
- gfs2_consist_rgrpd(rgd);
+ gfs2_consist_rgrpd(rbm->rgd);
return;
}
*byte1 ^= (cur_state ^ new_state) << bit;
- if (buf2) {
- byte2 = buf2 + bi->bi_offset + (block / GFS2_NBBY);
+ if (do_clone && rbm->bi->bi_clone) {
+ byte2 = rbm->bi->bi_clone + rbm->bi->bi_offset + (rbm->offset / GFS2_NBBY);
cur_state = (*byte2 >> bit) & GFS2_BIT_MASK;
*byte2 ^= (cur_state ^ new_state) << bit;
}
@@ -121,30 +113,21 @@
/**
* gfs2_testbit - test a bit in the bitmaps
- * @rgd: the resource group descriptor
- * @buffer: the buffer that holds the bitmaps
- * @buflen: the length (in bytes) of the buffer
- * @block: the block to read
+ * @rbm: The bit to test
*
+ * Returns: The two bit block state of the requested bit
*/
-static inline unsigned char gfs2_testbit(struct gfs2_rgrpd *rgd,
- const unsigned char *buffer,
- unsigned int buflen, u32 block)
+static inline u8 gfs2_testbit(const struct gfs2_rbm *rbm)
{
- const unsigned char *byte, *end;
- unsigned char cur_state;
+ const u8 *buffer = rbm->bi->bi_bh->b_data + rbm->bi->bi_offset;
+ const u8 *byte;
unsigned int bit;
- byte = buffer + (block / GFS2_NBBY);
- bit = (block % GFS2_NBBY) * GFS2_BIT_SIZE;
- end = buffer + buflen;
+ byte = buffer + (rbm->offset / GFS2_NBBY);
+ bit = (rbm->offset % GFS2_NBBY) * GFS2_BIT_SIZE;
- gfs2_assert(rgd->rd_sbd, byte < end);
-
- cur_state = (*byte >> bit) & GFS2_BIT_MASK;
-
- return cur_state;
+ return (*byte >> bit) & GFS2_BIT_MASK;
}
/**
@@ -192,7 +175,7 @@
*/
static inline int rs_cmp(u64 blk, u32 len, struct gfs2_blkreserv *rs)
{
- u64 startblk = gfs2_rs_startblk(rs);
+ u64 startblk = gfs2_rbm_to_block(&rs->rs_rbm);
if (blk >= startblk + rs->rs_free)
return 1;
@@ -202,36 +185,6 @@
}
/**
- * rs_find - Find a rgrp multi-block reservation that contains a given block
- * @rgd: The rgrp
- * @rgblk: The block we're looking for, relative to the rgrp
- */
-static struct gfs2_blkreserv *rs_find(struct gfs2_rgrpd *rgd, u32 rgblk)
-{
- struct rb_node **newn;
- int rc;
- u64 fsblk = rgblk + rgd->rd_data0;
-
- spin_lock(&rgd->rd_rsspin);
- newn = &rgd->rd_rstree.rb_node;
- while (*newn) {
- struct gfs2_blkreserv *cur =
- rb_entry(*newn, struct gfs2_blkreserv, rs_node);
- rc = rs_cmp(fsblk, 1, cur);
- if (rc < 0)
- newn = &((*newn)->rb_left);
- else if (rc > 0)
- newn = &((*newn)->rb_right);
- else {
- spin_unlock(&rgd->rd_rsspin);
- return cur;
- }
- }
- spin_unlock(&rgd->rd_rsspin);
- return NULL;
-}
-
-/**
* gfs2_bitfit - Search an rgrp's bitmap buffer to find a bit-pair representing
* a block in a given allocation state.
* @buf: the buffer that holds the bitmaps
@@ -262,8 +215,6 @@
u64 mask = 0x5555555555555555ULL;
u32 bit;
- BUG_ON(state > 3);
-
/* Mask off bits we don't care about at the start of the search */
mask <<= spoint;
tmp = gfs2_bit_search(ptr, mask, state);
@@ -285,6 +236,131 @@
}
/**
+ * gfs2_rbm_from_block - Set the rbm based upon rgd and block number
+ * @rbm: The rbm with rgd already set correctly
+ * @block: The block number (filesystem relative)
+ *
+ * This sets the bi and offset members of an rbm based on a
+ * resource group and a filesystem relative block number. The
+ * resource group must be set in the rbm on entry, the bi and
+ * offset members will be set by this function.
+ *
+ * Returns: 0 on success, or an error code
+ */
+
+static int gfs2_rbm_from_block(struct gfs2_rbm *rbm, u64 block)
+{
+ u64 rblock = block - rbm->rgd->rd_data0;
+ u32 goal = (u32)rblock;
+ int x;
+
+ if (WARN_ON_ONCE(rblock > UINT_MAX))
+ return -EINVAL;
+ if (block >= rbm->rgd->rd_data0 + rbm->rgd->rd_data)
+ return -E2BIG;
+
+ for (x = 0; x < rbm->rgd->rd_length; x++) {
+ rbm->bi = rbm->rgd->rd_bits + x;
+ if (goal < (rbm->bi->bi_start + rbm->bi->bi_len) * GFS2_NBBY) {
+ rbm->offset = goal - (rbm->bi->bi_start * GFS2_NBBY);
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * gfs2_unaligned_extlen - Look for free blocks which are not byte aligned
+ * @rbm: Position to search (value/result)
+ * @n_unaligned: Number of unaligned blocks to check
+ * @len: Decremented for each block found (terminate on zero)
+ *
+ * Returns: true if a non-free block is encountered
+ */
+
+static bool gfs2_unaligned_extlen(struct gfs2_rbm *rbm, u32 n_unaligned, u32 *len)
+{
+ u64 block;
+ u32 n;
+ u8 res;
+
+ for (n = 0; n < n_unaligned; n++) {
+ res = gfs2_testbit(rbm);
+ if (res != GFS2_BLKST_FREE)
+ return true;
+ (*len)--;
+ if (*len == 0)
+ return true;
+ block = gfs2_rbm_to_block(rbm);
+ if (gfs2_rbm_from_block(rbm, block + 1))
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * gfs2_free_extlen - Return extent length of free blocks
+ * @rbm: Starting position
+ * @len: Max length to check
+ *
+ * Starting at the block specified by the rbm, see how many free blocks
+ * there are, not reading more than len blocks ahead. This can be done
+ * using memchr_inv when the blocks are byte aligned, but has to be done
+ * on a block by block basis in case of unaligned blocks. Also this
+ * function can cope with bitmap boundaries (although it must stop on
+ * a resource group boundary)
+ *
+ * Returns: Number of free blocks in the extent
+ */
+
+static u32 gfs2_free_extlen(const struct gfs2_rbm *rrbm, u32 len)
+{
+ struct gfs2_rbm rbm = *rrbm;
+ u32 n_unaligned = rbm.offset & 3;
+ u32 size = len;
+ u32 bytes;
+ u32 chunk_size;
+ u8 *ptr, *start, *end;
+ u64 block;
+
+ if (n_unaligned &&
+ gfs2_unaligned_extlen(&rbm, 4 - n_unaligned, &len))
+ goto out;
+
+ n_unaligned = len & 3;
+ /* Start is now byte aligned */
+ while (len > 3) {
+ start = rbm.bi->bi_bh->b_data;
+ if (rbm.bi->bi_clone)
+ start = rbm.bi->bi_clone;
+ end = start + rbm.bi->bi_bh->b_size;
+ start += rbm.bi->bi_offset;
+ BUG_ON(rbm.offset & 3);
+ start += (rbm.offset / GFS2_NBBY);
+ bytes = min_t(u32, len / GFS2_NBBY, (end - start));
+ ptr = memchr_inv(start, 0, bytes);
+ chunk_size = ((ptr == NULL) ? bytes : (ptr - start));
+ chunk_size *= GFS2_NBBY;
+ BUG_ON(len < chunk_size);
+ len -= chunk_size;
+ block = gfs2_rbm_to_block(&rbm);
+ gfs2_rbm_from_block(&rbm, block + chunk_size);
+ n_unaligned = 3;
+ if (ptr)
+ break;
+ n_unaligned = len & 3;
+ }
+
+ /* Deal with any bits left over at the end */
+ if (n_unaligned)
+ gfs2_unaligned_extlen(&rbm, n_unaligned, &len);
+out:
+ return size - len;
+}
+
+/**
* gfs2_bitcount - count the number of bits in a certain state
* @rgd: the resource group descriptor
* @buffer: the buffer that holds the bitmaps
@@ -487,6 +563,8 @@
if (!res)
error = -ENOMEM;
+ RB_CLEAR_NODE(&res->rs_node);
+
down_write(&ip->i_rw_mutex);
if (ip->i_res)
kmem_cache_free(gfs2_rsrv_cachep, res);
@@ -496,11 +574,12 @@
return error;
}
-static void dump_rs(struct seq_file *seq, struct gfs2_blkreserv *rs)
+static void dump_rs(struct seq_file *seq, const struct gfs2_blkreserv *rs)
{
- gfs2_print_dbg(seq, " r: %llu s:%llu b:%u f:%u\n",
- rs->rs_rgd->rd_addr, gfs2_rs_startblk(rs), rs->rs_biblk,
- rs->rs_free);
+ gfs2_print_dbg(seq, " B: n:%llu s:%llu b:%u f:%u\n",
+ (unsigned long long)rs->rs_inum,
+ (unsigned long long)gfs2_rbm_to_block(&rs->rs_rbm),
+ rs->rs_rbm.offset, rs->rs_free);
}
/**
@@ -508,41 +587,26 @@
* @rs: The reservation to remove
*
*/
-static void __rs_deltree(struct gfs2_blkreserv *rs)
+static void __rs_deltree(struct gfs2_inode *ip, struct gfs2_blkreserv *rs)
{
struct gfs2_rgrpd *rgd;
if (!gfs2_rs_active(rs))
return;
- rgd = rs->rs_rgd;
- /* We can't do this: The reason is that when the rgrp is invalidated,
- it's in the "middle" of acquiring the glock, but the HOLDER bit
- isn't set yet:
- BUG_ON(!gfs2_glock_is_locked_by_me(rs->rs_rgd->rd_gl));*/
- trace_gfs2_rs(NULL, rs, TRACE_RS_TREEDEL);
-
- if (!RB_EMPTY_ROOT(&rgd->rd_rstree))
- rb_erase(&rs->rs_node, &rgd->rd_rstree);
- BUG_ON(!rgd->rd_rs_cnt);
- rgd->rd_rs_cnt--;
+ rgd = rs->rs_rbm.rgd;
+ trace_gfs2_rs(rs, TRACE_RS_TREEDEL);
+ rb_erase(&rs->rs_node, &rgd->rd_rstree);
+ RB_CLEAR_NODE(&rs->rs_node);
if (rs->rs_free) {
/* return reserved blocks to the rgrp and the ip */
- BUG_ON(rs->rs_rgd->rd_reserved < rs->rs_free);
- rs->rs_rgd->rd_reserved -= rs->rs_free;
+ BUG_ON(rs->rs_rbm.rgd->rd_reserved < rs->rs_free);
+ rs->rs_rbm.rgd->rd_reserved -= rs->rs_free;
rs->rs_free = 0;
- clear_bit(GBF_FULL, &rs->rs_bi->bi_flags);
+ clear_bit(GBF_FULL, &rs->rs_rbm.bi->bi_flags);
smp_mb__after_clear_bit();
}
- /* We can't change any of the step 1 or step 2 components of the rs.
- E.g. We can't set rs_rgd to NULL because the rgd glock is held and
- dequeued through this pointer.
- Can't: atomic_set(&rs->rs_sizehint, 0);
- Can't: rs->rs_requested = 0;
- Can't: rs->rs_rgd = NULL;*/
- rs->rs_bi = NULL;
- rs->rs_biblk = 0;
}
/**
@@ -550,17 +614,16 @@
* @rs: The reservation to remove
*
*/
-void gfs2_rs_deltree(struct gfs2_blkreserv *rs)
+void gfs2_rs_deltree(struct gfs2_inode *ip, struct gfs2_blkreserv *rs)
{
struct gfs2_rgrpd *rgd;
- if (!gfs2_rs_active(rs))
- return;
-
- rgd = rs->rs_rgd;
- spin_lock(&rgd->rd_rsspin);
- __rs_deltree(rs);
- spin_unlock(&rgd->rd_rsspin);
+ rgd = rs->rs_rbm.rgd;
+ if (rgd) {
+ spin_lock(&rgd->rd_rsspin);
+ __rs_deltree(ip, rs);
+ spin_unlock(&rgd->rd_rsspin);
+ }
}
/**
@@ -572,8 +635,7 @@
{
down_write(&ip->i_rw_mutex);
if (ip->i_res) {
- gfs2_rs_deltree(ip->i_res);
- trace_gfs2_rs(ip, ip->i_res, TRACE_RS_DELETE);
+ gfs2_rs_deltree(ip, ip->i_res);
BUG_ON(ip->i_res->rs_free);
kmem_cache_free(gfs2_rsrv_cachep, ip->i_res);
ip->i_res = NULL;
@@ -597,7 +659,7 @@
spin_lock(&rgd->rd_rsspin);
while ((n = rb_first(&rgd->rd_rstree))) {
rs = rb_entry(n, struct gfs2_blkreserv, rs_node);
- __rs_deltree(rs);
+ __rs_deltree(NULL, rs);
}
spin_unlock(&rgd->rd_rsspin);
}
@@ -1270,211 +1332,276 @@
/**
* rs_insert - insert a new multi-block reservation into the rgrp's rb_tree
- * @bi: the bitmap with the blocks
* @ip: the inode structure
- * @biblk: the 32-bit block number relative to the start of the bitmap
- * @amount: the number of blocks to reserve
*
- * Returns: NULL - reservation was already taken, so not inserted
- * pointer to the inserted reservation
*/
-static struct gfs2_blkreserv *rs_insert(struct gfs2_bitmap *bi,
- struct gfs2_inode *ip, u32 biblk,
- int amount)
+static void rs_insert(struct gfs2_inode *ip)
{
struct rb_node **newn, *parent = NULL;
int rc;
struct gfs2_blkreserv *rs = ip->i_res;
- struct gfs2_rgrpd *rgd = rs->rs_rgd;
- u64 fsblock = gfs2_bi2rgd_blk(bi, biblk) + rgd->rd_data0;
+ struct gfs2_rgrpd *rgd = rs->rs_rbm.rgd;
+ u64 fsblock = gfs2_rbm_to_block(&rs->rs_rbm);
+
+ BUG_ON(gfs2_rs_active(rs));
spin_lock(&rgd->rd_rsspin);
newn = &rgd->rd_rstree.rb_node;
- BUG_ON(!ip->i_res);
- BUG_ON(gfs2_rs_active(rs));
- /* Figure out where to put new node */
- /*BUG_ON(!gfs2_glock_is_locked_by_me(rgd->rd_gl));*/
while (*newn) {
struct gfs2_blkreserv *cur =
rb_entry(*newn, struct gfs2_blkreserv, rs_node);
parent = *newn;
- rc = rs_cmp(fsblock, amount, cur);
+ rc = rs_cmp(fsblock, rs->rs_free, cur);
if (rc > 0)
newn = &((*newn)->rb_right);
else if (rc < 0)
newn = &((*newn)->rb_left);
else {
spin_unlock(&rgd->rd_rsspin);
- return NULL; /* reservation already in use */
+ WARN_ON(1);
+ return;
}
}
- /* Do our reservation work */
- rs = ip->i_res;
- rs->rs_free = amount;
- rs->rs_biblk = biblk;
- rs->rs_bi = bi;
rb_link_node(&rs->rs_node, parent, newn);
rb_insert_color(&rs->rs_node, &rgd->rd_rstree);
- /* Do our inode accounting for the reservation */
- /*BUG_ON(!gfs2_glock_is_locked_by_me(ip->i_gl));*/
-
/* Do our rgrp accounting for the reservation */
- rgd->rd_reserved += amount; /* blocks reserved */
- rgd->rd_rs_cnt++; /* number of in-tree reservations */
+ rgd->rd_reserved += rs->rs_free; /* blocks reserved */
spin_unlock(&rgd->rd_rsspin);
- trace_gfs2_rs(ip, rs, TRACE_RS_INSERT);
- return rs;
+ trace_gfs2_rs(rs, TRACE_RS_INSERT);
}
/**
- * unclaimed_blocks - return number of blocks that aren't spoken for
- */
-static u32 unclaimed_blocks(struct gfs2_rgrpd *rgd)
-{
- return rgd->rd_free_clone - rgd->rd_reserved;
-}
-
-/**
- * rg_mblk_search - find a group of multiple free blocks
+ * rg_mblk_search - find a group of multiple free blocks to form a reservation
* @rgd: the resource group descriptor
- * @rs: the block reservation
* @ip: pointer to the inode for which we're reserving blocks
+ * @requested: number of blocks required for this allocation
*
- * This is very similar to rgblk_search, except we're looking for whole
- * 64-bit words that represent a chunk of 32 free blocks. I'm only focusing
- * on aligned dwords for speed's sake.
- *
- * Returns: 0 if successful or BFITNOENT if there isn't enough free space
*/
-static int rg_mblk_search(struct gfs2_rgrpd *rgd, struct gfs2_inode *ip)
+static void rg_mblk_search(struct gfs2_rgrpd *rgd, struct gfs2_inode *ip,
+ unsigned requested)
{
- struct gfs2_bitmap *bi = rgd->rd_bits;
- const u32 length = rgd->rd_length;
- u32 blk;
- unsigned int buf, x, search_bytes;
- u8 *buffer = NULL;
- u8 *ptr, *end, *nonzero;
- u32 goal, rsv_bytes;
- struct gfs2_blkreserv *rs;
- u32 best_rs_bytes, unclaimed;
- int best_rs_blocks;
+ struct gfs2_rbm rbm = { .rgd = rgd, };
+ u64 goal;
+ struct gfs2_blkreserv *rs = ip->i_res;
+ u32 extlen;
+ u32 free_blocks = rgd->rd_free_clone - rgd->rd_reserved;
+ int ret;
+
+ extlen = max_t(u32, atomic_read(&rs->rs_sizehint), requested);
+ extlen = clamp(extlen, RGRP_RSRV_MINBLKS, free_blocks);
+ if ((rgd->rd_free_clone < rgd->rd_reserved) || (free_blocks < extlen))
+ return;
/* Find bitmap block that contains bits for goal block */
if (rgrp_contains_block(rgd, ip->i_goal))
- goal = ip->i_goal - rgd->rd_data0;
+ goal = ip->i_goal;
else
- goal = rgd->rd_last_alloc;
- for (buf = 0; buf < length; buf++) {
- bi = rgd->rd_bits + buf;
- /* Convert scope of "goal" from rgrp-wide to within
- found bit block */
- if (goal < (bi->bi_start + bi->bi_len) * GFS2_NBBY) {
- goal -= bi->bi_start * GFS2_NBBY;
- goto do_search;
- }
+ goal = rgd->rd_last_alloc + rgd->rd_data0;
+
+ if (WARN_ON(gfs2_rbm_from_block(&rbm, goal)))
+ return;
+
+ ret = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, extlen, ip, true);
+ if (ret == 0) {
+ rs->rs_rbm = rbm;
+ rs->rs_free = extlen;
+ rs->rs_inum = ip->i_no_addr;
+ rs_insert(ip);
}
- buf = 0;
- goal = 0;
-
-do_search:
- best_rs_blocks = max_t(int, atomic_read(&ip->i_res->rs_sizehint),
- (RGRP_RSRV_MINBLKS * rgd->rd_length));
- best_rs_bytes = (best_rs_blocks *
- (1 + (RSRV_CONTENTION_FACTOR * rgd->rd_rs_cnt))) /
- GFS2_NBBY; /* 1 + is for our not-yet-created reservation */
- best_rs_bytes = ALIGN(best_rs_bytes, sizeof(u64));
- unclaimed = unclaimed_blocks(rgd);
- if (best_rs_bytes * GFS2_NBBY > unclaimed)
- best_rs_bytes = unclaimed >> GFS2_BIT_SIZE;
-
- for (x = 0; x <= length; x++) {
- bi = rgd->rd_bits + buf;
-
- if (test_bit(GBF_FULL, &bi->bi_flags))
- goto skip;
-
- WARN_ON(!buffer_uptodate(bi->bi_bh));
- if (bi->bi_clone)
- buffer = bi->bi_clone + bi->bi_offset;
- else
- buffer = bi->bi_bh->b_data + bi->bi_offset;
-
- /* We have to keep the reservations aligned on u64 boundaries
- otherwise we could get situations where a byte can't be
- used because it's after a reservation, but a free bit still
- is within the reservation's area. */
- ptr = buffer + ALIGN(goal >> GFS2_BIT_SIZE, sizeof(u64));
- end = (buffer + bi->bi_len);
- while (ptr < end) {
- rsv_bytes = 0;
- if ((ptr + best_rs_bytes) <= end)
- search_bytes = best_rs_bytes;
- else
- search_bytes = end - ptr;
- BUG_ON(!search_bytes);
- nonzero = memchr_inv(ptr, 0, search_bytes);
- /* If the lot is all zeroes, reserve the whole size. If
- there's enough zeroes to satisfy the request, use
- what we can. If there's not enough, keep looking. */
- if (nonzero == NULL)
- rsv_bytes = search_bytes;
- else if ((nonzero - ptr) * GFS2_NBBY >=
- ip->i_res->rs_requested)
- rsv_bytes = (nonzero - ptr);
-
- if (rsv_bytes) {
- blk = ((ptr - buffer) * GFS2_NBBY);
- BUG_ON(blk >= bi->bi_len * GFS2_NBBY);
- rs = rs_insert(bi, ip, blk,
- rsv_bytes * GFS2_NBBY);
- if (IS_ERR(rs))
- return PTR_ERR(rs);
- if (rs)
- return 0;
- }
- ptr += ALIGN(search_bytes, sizeof(u64));
- }
-skip:
- /* Try next bitmap block (wrap back to rgrp header
- if at end) */
- buf++;
- buf %= length;
- goal = 0;
- }
-
- return BFITNOENT;
}
/**
- * try_rgrp_fit - See if a given reservation will fit in a given RG
- * @rgd: the RG data
- * @ip: the inode
+ * gfs2_next_unreserved_block - Return next block that is not reserved
+ * @rgd: The resource group
+ * @block: The starting block
+ * @length: The required length
+ * @ip: Ignore any reservations for this inode
*
- * If there's room for the requested blocks to be allocated from the RG:
- * This will try to get a multi-block reservation first, and if that doesn't
- * fit, it will take what it can.
- *
- * Returns: 1 on success (it fits), 0 on failure (it doesn't fit)
+ * If the block does not appear in any reservation, then return the
+ * block number unchanged. If it does appear in the reservation, then
+ * keep looking through the tree of reservations in order to find the
+ * first block number which is not reserved.
*/
-static int try_rgrp_fit(struct gfs2_rgrpd *rgd, struct gfs2_inode *ip)
+static u64 gfs2_next_unreserved_block(struct gfs2_rgrpd *rgd, u64 block,
+ u32 length,
+ const struct gfs2_inode *ip)
{
- struct gfs2_blkreserv *rs = ip->i_res;
+ struct gfs2_blkreserv *rs;
+ struct rb_node *n;
+ int rc;
- if (rgd->rd_flags & (GFS2_RGF_NOALLOC | GFS2_RDF_ERROR))
+ spin_lock(&rgd->rd_rsspin);
+ n = rgd->rd_rstree.rb_node;
+ while (n) {
+ rs = rb_entry(n, struct gfs2_blkreserv, rs_node);
+ rc = rs_cmp(block, length, rs);
+ if (rc < 0)
+ n = n->rb_left;
+ else if (rc > 0)
+ n = n->rb_right;
+ else
+ break;
+ }
+
+ if (n) {
+ while ((rs_cmp(block, length, rs) == 0) && (ip->i_res != rs)) {
+ block = gfs2_rbm_to_block(&rs->rs_rbm) + rs->rs_free;
+ n = n->rb_right;
+ if (n == NULL)
+ break;
+ rs = rb_entry(n, struct gfs2_blkreserv, rs_node);
+ }
+ }
+
+ spin_unlock(&rgd->rd_rsspin);
+ return block;
+}
+
+/**
+ * gfs2_reservation_check_and_update - Check for reservations during block alloc
+ * @rbm: The current position in the resource group
+ * @ip: The inode for which we are searching for blocks
+ * @minext: The minimum extent length
+ *
+ * This checks the current position in the rgrp to see whether there is
+ * a reservation covering this block. If not then this function is a
+ * no-op. If there is, then the position is moved to the end of the
+ * contiguous reservation(s) so that we are pointing at the first
+ * non-reserved block.
+ *
+ * Returns: 0 if no reservation, 1 if @rbm has changed, otherwise an error
+ */
+
+static int gfs2_reservation_check_and_update(struct gfs2_rbm *rbm,
+ const struct gfs2_inode *ip,
+ u32 minext)
+{
+ u64 block = gfs2_rbm_to_block(rbm);
+ u32 extlen = 1;
+ u64 nblock;
+ int ret;
+
+ /*
+ * If we have a minimum extent length, then skip over any extent
+ * which is less than the min extent length in size.
+ */
+ if (minext) {
+ extlen = gfs2_free_extlen(rbm, minext);
+ nblock = block + extlen;
+ if (extlen < minext)
+ goto fail;
+ }
+
+ /*
+ * Check the extent which has been found against the reservations
+ * and skip if parts of it are already reserved
+ */
+ nblock = gfs2_next_unreserved_block(rbm->rgd, block, extlen, ip);
+ if (nblock == block)
return 0;
- /* Look for a multi-block reservation. */
- if (unclaimed_blocks(rgd) >= RGRP_RSRV_MINBLKS &&
- rg_mblk_search(rgd, ip) != BFITNOENT)
- return 1;
- if (unclaimed_blocks(rgd) >= rs->rs_requested)
- return 1;
+fail:
+ ret = gfs2_rbm_from_block(rbm, nblock);
+ if (ret < 0)
+ return ret;
+ return 1;
+}
- return 0;
+/**
+ * gfs2_rbm_find - Look for blocks of a particular state
+ * @rbm: Value/result starting position and final position
+ * @state: The state which we want to find
+ * @minext: The requested extent length (0 for a single block)
+ * @ip: If set, check for reservations
+ * @nowrap: Stop looking at the end of the rgrp, rather than wrapping
+ * around until we've reached the starting point.
+ *
+ * Side effects:
+ * - If looking for free blocks, we set GBF_FULL on each bitmap which
+ * has no free blocks in it.
+ *
+ * Returns: 0 on success, -ENOSPC if there is no block of the requested state
+ */
+
+static int gfs2_rbm_find(struct gfs2_rbm *rbm, u8 state, u32 minext,
+ const struct gfs2_inode *ip, bool nowrap)
+{
+ struct buffer_head *bh;
+ struct gfs2_bitmap *initial_bi;
+ u32 initial_offset;
+ u32 offset;
+ u8 *buffer;
+ int index;
+ int n = 0;
+ int iters = rbm->rgd->rd_length;
+ int ret;
+
+ /* If we are not starting at the beginning of a bitmap, then we
+ * need to add one to the bitmap count to ensure that we search
+ * the starting bitmap twice.
+ */
+ if (rbm->offset != 0)
+ iters++;
+
+ while(1) {
+ if (test_bit(GBF_FULL, &rbm->bi->bi_flags) &&
+ (state == GFS2_BLKST_FREE))
+ goto next_bitmap;
+
+ bh = rbm->bi->bi_bh;
+ buffer = bh->b_data + rbm->bi->bi_offset;
+ WARN_ON(!buffer_uptodate(bh));
+ if (state != GFS2_BLKST_UNLINKED && rbm->bi->bi_clone)
+ buffer = rbm->bi->bi_clone + rbm->bi->bi_offset;
+ initial_offset = rbm->offset;
+ offset = gfs2_bitfit(buffer, rbm->bi->bi_len, rbm->offset, state);
+ if (offset == BFITNOENT)
+ goto bitmap_full;
+ rbm->offset = offset;
+ if (ip == NULL)
+ return 0;
+
+ initial_bi = rbm->bi;
+ ret = gfs2_reservation_check_and_update(rbm, ip, minext);
+ if (ret == 0)
+ return 0;
+ if (ret > 0) {
+ n += (rbm->bi - initial_bi);
+ goto next_iter;
+ }
+ if (ret == -E2BIG) {
+ index = 0;
+ rbm->offset = 0;
+ n += (rbm->bi - initial_bi);
+ goto res_covered_end_of_rgrp;
+ }
+ return ret;
+
+bitmap_full: /* Mark bitmap as full and fall through */
+ if ((state == GFS2_BLKST_FREE) && initial_offset == 0)
+ set_bit(GBF_FULL, &rbm->bi->bi_flags);
+
+next_bitmap: /* Find next bitmap in the rgrp */
+ rbm->offset = 0;
+ index = rbm->bi - rbm->rgd->rd_bits;
+ index++;
+ if (index == rbm->rgd->rd_length)
+ index = 0;
+res_covered_end_of_rgrp:
+ rbm->bi = &rbm->rgd->rd_bits[index];
+ if ((index == 0) && nowrap)
+ break;
+ n++;
+next_iter:
+ if (n >= iters)
+ break;
+ }
+
+ return -ENOSPC;
}
/**
@@ -1489,34 +1616,33 @@
static void try_rgrp_unlink(struct gfs2_rgrpd *rgd, u64 *last_unlinked, u64 skip)
{
- u32 goal = 0, block;
- u64 no_addr;
+ u64 block;
struct gfs2_sbd *sdp = rgd->rd_sbd;
struct gfs2_glock *gl;
struct gfs2_inode *ip;
int error;
int found = 0;
- struct gfs2_bitmap *bi;
+ struct gfs2_rbm rbm = { .rgd = rgd, .bi = rgd->rd_bits, .offset = 0 };
- while (goal < rgd->rd_data) {
+ while (1) {
down_write(&sdp->sd_log_flush_lock);
- block = rgblk_search(rgd, goal, GFS2_BLKST_UNLINKED, &bi);
+ error = gfs2_rbm_find(&rbm, GFS2_BLKST_UNLINKED, 0, NULL, true);
up_write(&sdp->sd_log_flush_lock);
- if (block == BFITNOENT)
+ if (error == -ENOSPC)
+ break;
+ if (WARN_ON_ONCE(error))
break;
- block = gfs2_bi2rgd_blk(bi, block);
- /* rgblk_search can return a block < goal, so we need to
- keep it marching forward. */
- no_addr = block + rgd->rd_data0;
- goal = max(block + 1, goal + 1);
- if (*last_unlinked != NO_BLOCK && no_addr <= *last_unlinked)
+ block = gfs2_rbm_to_block(&rbm);
+ if (gfs2_rbm_from_block(&rbm, block + 1))
+ break;
+ if (*last_unlinked != NO_BLOCK && block <= *last_unlinked)
continue;
- if (no_addr == skip)
+ if (block == skip)
continue;
- *last_unlinked = no_addr;
+ *last_unlinked = block;
- error = gfs2_glock_get(sdp, no_addr, &gfs2_inode_glops, CREATE, &gl);
+ error = gfs2_glock_get(sdp, block, &gfs2_inode_glops, CREATE, &gl);
if (error)
continue;
@@ -1543,6 +1669,19 @@
return;
}
+static bool gfs2_select_rgrp(struct gfs2_rgrpd **pos, const struct gfs2_rgrpd *begin)
+{
+ struct gfs2_rgrpd *rgd = *pos;
+
+ rgd = gfs2_rgrpd_get_next(rgd);
+ if (rgd == NULL)
+ rgd = gfs2_rgrpd_get_next(NULL);
+ *pos = rgd;
+ if (rgd != begin) /* If we didn't wrap */
+ return true;
+ return false;
+}
+
/**
* gfs2_inplace_reserve - Reserve space in the filesystem
* @ip: the inode to reserve space for
@@ -1562,103 +1701,96 @@
if (sdp->sd_args.ar_rgrplvb)
flags |= GL_SKIP;
- rs->rs_requested = requested;
- if (gfs2_assert_warn(sdp, requested)) {
- error = -EINVAL;
- goto out;
- }
+ if (gfs2_assert_warn(sdp, requested))
+ return -EINVAL;
if (gfs2_rs_active(rs)) {
- begin = rs->rs_rgd;
+ begin = rs->rs_rbm.rgd;
flags = 0; /* Yoda: Do or do not. There is no try */
} else if (ip->i_rgd && rgrp_contains_block(ip->i_rgd, ip->i_goal)) {
- rs->rs_rgd = begin = ip->i_rgd;
+ rs->rs_rbm.rgd = begin = ip->i_rgd;
} else {
- rs->rs_rgd = begin = gfs2_blk2rgrpd(sdp, ip->i_goal, 1);
+ rs->rs_rbm.rgd = begin = gfs2_blk2rgrpd(sdp, ip->i_goal, 1);
}
- if (rs->rs_rgd == NULL)
+ if (rs->rs_rbm.rgd == NULL)
return -EBADSLT;
while (loops < 3) {
- rg_locked = 0;
+ rg_locked = 1;
- if (gfs2_glock_is_locked_by_me(rs->rs_rgd->rd_gl)) {
- rg_locked = 1;
- error = 0;
- } else if (!loops && !gfs2_rs_active(rs) &&
- rs->rs_rgd->rd_rs_cnt > RGRP_RSRV_MAX_CONTENDERS) {
- /* If the rgrp already is maxed out for contenders,
- we can eliminate it as a "first pass" without even
- requesting the rgrp glock. */
- error = GLR_TRYFAILED;
- } else {
- error = gfs2_glock_nq_init(rs->rs_rgd->rd_gl,
+ if (!gfs2_glock_is_locked_by_me(rs->rs_rbm.rgd->rd_gl)) {
+ rg_locked = 0;
+ error = gfs2_glock_nq_init(rs->rs_rbm.rgd->rd_gl,
LM_ST_EXCLUSIVE, flags,
&rs->rs_rgd_gh);
- if (!error && sdp->sd_args.ar_rgrplvb) {
- error = update_rgrp_lvb(rs->rs_rgd);
- if (error) {
+ if (error == GLR_TRYFAILED)
+ goto next_rgrp;
+ if (unlikely(error))
+ return error;
+ if (sdp->sd_args.ar_rgrplvb) {
+ error = update_rgrp_lvb(rs->rs_rbm.rgd);
+ if (unlikely(error)) {
gfs2_glock_dq_uninit(&rs->rs_rgd_gh);
return error;
}
}
}
- switch (error) {
- case 0:
- if (gfs2_rs_active(rs)) {
- if (unclaimed_blocks(rs->rs_rgd) +
- rs->rs_free >= rs->rs_requested) {
- ip->i_rgd = rs->rs_rgd;
- return 0;
- }
- /* We have a multi-block reservation, but the
- rgrp doesn't have enough free blocks to
- satisfy the request. Free the reservation
- and look for a suitable rgrp. */
- gfs2_rs_deltree(rs);
- }
- if (try_rgrp_fit(rs->rs_rgd, ip)) {
- if (sdp->sd_args.ar_rgrplvb)
- gfs2_rgrp_bh_get(rs->rs_rgd);
- ip->i_rgd = rs->rs_rgd;
- return 0;
- }
- if (rs->rs_rgd->rd_flags & GFS2_RDF_CHECK) {
- if (sdp->sd_args.ar_rgrplvb)
- gfs2_rgrp_bh_get(rs->rs_rgd);
- try_rgrp_unlink(rs->rs_rgd, &last_unlinked,
- ip->i_no_addr);
- }
- if (!rg_locked)
- gfs2_glock_dq_uninit(&rs->rs_rgd_gh);
- /* fall through */
- case GLR_TRYFAILED:
- rs->rs_rgd = gfs2_rgrpd_get_next(rs->rs_rgd);
- rs->rs_rgd = rs->rs_rgd ? : begin; /* if NULL, wrap */
- if (rs->rs_rgd != begin) /* If we didn't wrap */
- break;
- flags &= ~LM_FLAG_TRY;
- loops++;
- /* Check that fs hasn't grown if writing to rindex */
- if (ip == GFS2_I(sdp->sd_rindex) &&
- !sdp->sd_rindex_uptodate) {
- error = gfs2_ri_update(ip);
- if (error)
- goto out;
- } else if (loops == 2)
- /* Flushing the log may release space */
- gfs2_log_flush(sdp, NULL);
- break;
- default:
- goto out;
+ /* Skip unuseable resource groups */
+ if (rs->rs_rbm.rgd->rd_flags & (GFS2_RGF_NOALLOC | GFS2_RDF_ERROR))
+ goto skip_rgrp;
+
+ if (sdp->sd_args.ar_rgrplvb)
+ gfs2_rgrp_bh_get(rs->rs_rbm.rgd);
+
+ /* Get a reservation if we don't already have one */
+ if (!gfs2_rs_active(rs))
+ rg_mblk_search(rs->rs_rbm.rgd, ip, requested);
+
+ /* Skip rgrps when we can't get a reservation on first pass */
+ if (!gfs2_rs_active(rs) && (loops < 1))
+ goto check_rgrp;
+
+ /* If rgrp has enough free space, use it */
+ if (rs->rs_rbm.rgd->rd_free_clone >= requested) {
+ ip->i_rgd = rs->rs_rbm.rgd;
+ return 0;
}
- }
- error = -ENOSPC;
-out:
- if (error)
- rs->rs_requested = 0;
- return error;
+ /* Drop reservation, if we couldn't use reserved rgrp */
+ if (gfs2_rs_active(rs))
+ gfs2_rs_deltree(ip, rs);
+check_rgrp:
+ /* Check for unlinked inodes which can be reclaimed */
+ if (rs->rs_rbm.rgd->rd_flags & GFS2_RDF_CHECK)
+ try_rgrp_unlink(rs->rs_rbm.rgd, &last_unlinked,
+ ip->i_no_addr);
+skip_rgrp:
+ /* Unlock rgrp if required */
+ if (!rg_locked)
+ gfs2_glock_dq_uninit(&rs->rs_rgd_gh);
+next_rgrp:
+ /* Find the next rgrp, and continue looking */
+ if (gfs2_select_rgrp(&rs->rs_rbm.rgd, begin))
+ continue;
+
+ /* If we've scanned all the rgrps, but found no free blocks
+ * then this checks for some less likely conditions before
+ * trying again.
+ */
+ flags &= ~LM_FLAG_TRY;
+ loops++;
+ /* Check that fs hasn't grown if writing to rindex */
+ if (ip == GFS2_I(sdp->sd_rindex) && !sdp->sd_rindex_uptodate) {
+ error = gfs2_ri_update(ip);
+ if (error)
+ return error;
+ }
+ /* Flushing the log may release space */
+ if (loops == 2)
+ gfs2_log_flush(sdp, NULL);
+ }
+
+ return -ENOSPC;
}
/**
@@ -1672,15 +1804,8 @@
{
struct gfs2_blkreserv *rs = ip->i_res;
- if (!rs)
- return;
-
- if (!rs->rs_free)
- gfs2_rs_deltree(rs);
-
if (rs->rs_rgd_gh.gh_gl)
gfs2_glock_dq_uninit(&rs->rs_rgd_gh);
- rs->rs_requested = 0;
}
/**
@@ -1693,173 +1818,47 @@
static unsigned char gfs2_get_block_type(struct gfs2_rgrpd *rgd, u64 block)
{
- struct gfs2_bitmap *bi = NULL;
- u32 length, rgrp_block, buf_block;
- unsigned int buf;
- unsigned char type;
+ struct gfs2_rbm rbm = { .rgd = rgd, };
+ int ret;
- length = rgd->rd_length;
- rgrp_block = block - rgd->rd_data0;
+ ret = gfs2_rbm_from_block(&rbm, block);
+ WARN_ON_ONCE(ret != 0);
- for (buf = 0; buf < length; buf++) {
- bi = rgd->rd_bits + buf;
- if (rgrp_block < (bi->bi_start + bi->bi_len) * GFS2_NBBY)
- break;
- }
-
- gfs2_assert(rgd->rd_sbd, buf < length);
- buf_block = rgrp_block - bi->bi_start * GFS2_NBBY;
-
- type = gfs2_testbit(rgd, bi->bi_bh->b_data + bi->bi_offset,
- bi->bi_len, buf_block);
-
- return type;
+ return gfs2_testbit(&rbm);
}
-/**
- * rgblk_search - find a block in @state
- * @rgd: the resource group descriptor
- * @goal: the goal block within the RG (start here to search for avail block)
- * @state: GFS2_BLKST_XXX the before-allocation state to find
- * @rbi: address of the pointer to the bitmap containing the block found
- *
- * Walk rgrp's bitmap to find bits that represent a block in @state.
- *
- * This function never fails, because we wouldn't call it unless we
- * know (from reservation results, etc.) that a block is available.
- *
- * Scope of @goal is just within rgrp, not the whole filesystem.
- * Scope of @returned block is just within bitmap, not the whole filesystem.
- *
- * Returns: the block number found relative to the bitmap rbi
- */
-
-static u32 rgblk_search(struct gfs2_rgrpd *rgd, u32 goal, unsigned char state,
- struct gfs2_bitmap **rbi)
-{
- struct gfs2_bitmap *bi = NULL;
- const u32 length = rgd->rd_length;
- u32 biblk = BFITNOENT;
- unsigned int buf, x;
- const u8 *buffer = NULL;
-
- *rbi = NULL;
- /* Find bitmap block that contains bits for goal block */
- for (buf = 0; buf < length; buf++) {
- bi = rgd->rd_bits + buf;
- /* Convert scope of "goal" from rgrp-wide to within found bit block */
- if (goal < (bi->bi_start + bi->bi_len) * GFS2_NBBY) {
- goal -= bi->bi_start * GFS2_NBBY;
- goto do_search;
- }
- }
- buf = 0;
- goal = 0;
-
-do_search:
- /* Search (up to entire) bitmap in this rgrp for allocatable block.
- "x <= length", instead of "x < length", because we typically start
- the search in the middle of a bit block, but if we can't find an
- allocatable block anywhere else, we want to be able wrap around and
- search in the first part of our first-searched bit block. */
- for (x = 0; x <= length; x++) {
- bi = rgd->rd_bits + buf;
-
- if (test_bit(GBF_FULL, &bi->bi_flags) &&
- (state == GFS2_BLKST_FREE))
- goto skip;
-
- /* The GFS2_BLKST_UNLINKED state doesn't apply to the clone
- bitmaps, so we must search the originals for that. */
- buffer = bi->bi_bh->b_data + bi->bi_offset;
- WARN_ON(!buffer_uptodate(bi->bi_bh));
- if (state != GFS2_BLKST_UNLINKED && bi->bi_clone)
- buffer = bi->bi_clone + bi->bi_offset;
-
- while (1) {
- struct gfs2_blkreserv *rs;
- u32 rgblk;
-
- biblk = gfs2_bitfit(buffer, bi->bi_len, goal, state);
- if (biblk == BFITNOENT)
- break;
- /* Check if this block is reserved() */
- rgblk = gfs2_bi2rgd_blk(bi, biblk);
- rs = rs_find(rgd, rgblk);
- if (rs == NULL)
- break;
-
- BUG_ON(rs->rs_bi != bi);
- biblk = BFITNOENT;
- /* This should jump to the first block after the
- reservation. */
- goal = rs->rs_biblk + rs->rs_free;
- if (goal >= bi->bi_len * GFS2_NBBY)
- break;
- }
- if (biblk != BFITNOENT)
- break;
-
- if ((goal == 0) && (state == GFS2_BLKST_FREE))
- set_bit(GBF_FULL, &bi->bi_flags);
-
- /* Try next bitmap block (wrap back to rgrp header if at end) */
-skip:
- buf++;
- buf %= length;
- goal = 0;
- }
-
- if (biblk != BFITNOENT)
- *rbi = bi;
-
- return biblk;
-}
/**
* gfs2_alloc_extent - allocate an extent from a given bitmap
- * @rgd: the resource group descriptor
- * @bi: the bitmap within the rgrp
- * @blk: the block within the bitmap
+ * @rbm: the resource group information
* @dinode: TRUE if the first block we allocate is for a dinode
- * @n: The extent length
+ * @n: The extent length (value/result)
*
- * Add the found bitmap buffer to the transaction.
+ * Add the bitmap buffer to the transaction.
* Set the found bits to @new_state to change block's allocation state.
- * Returns: starting block number of the extent (fs scope)
*/
-static u64 gfs2_alloc_extent(struct gfs2_rgrpd *rgd, struct gfs2_bitmap *bi,
- u32 blk, bool dinode, unsigned int *n)
+static void gfs2_alloc_extent(const struct gfs2_rbm *rbm, bool dinode,
+ unsigned int *n)
{
+ struct gfs2_rbm pos = { .rgd = rbm->rgd, };
const unsigned int elen = *n;
- u32 goal, rgblk;
- const u8 *buffer = NULL;
- struct gfs2_blkreserv *rs;
+ u64 block;
+ int ret;
- *n = 0;
- buffer = bi->bi_bh->b_data + bi->bi_offset;
- gfs2_trans_add_bh(rgd->rd_gl, bi->bi_bh, 1);
- gfs2_setbit(rgd, bi->bi_clone, bi, blk,
- dinode ? GFS2_BLKST_DINODE : GFS2_BLKST_USED);
- (*n)++;
- goal = blk;
+ *n = 1;
+ block = gfs2_rbm_to_block(rbm);
+ gfs2_trans_add_bh(rbm->rgd->rd_gl, rbm->bi->bi_bh, 1);
+ gfs2_setbit(rbm, true, dinode ? GFS2_BLKST_DINODE : GFS2_BLKST_USED);
+ block++;
while (*n < elen) {
- goal++;
- if (goal >= (bi->bi_len * GFS2_NBBY))
+ ret = gfs2_rbm_from_block(&pos, block);
+ if (ret || gfs2_testbit(&pos) != GFS2_BLKST_FREE)
break;
- rgblk = gfs2_bi2rgd_blk(bi, goal);
- rs = rs_find(rgd, rgblk);
- if (rs) /* Oops, we bumped into someone's reservation */
- break;
- if (gfs2_testbit(rgd, buffer, bi->bi_len, goal) !=
- GFS2_BLKST_FREE)
- break;
- gfs2_setbit(rgd, bi->bi_clone, bi, goal, GFS2_BLKST_USED);
+ gfs2_trans_add_bh(pos.rgd->rd_gl, pos.bi->bi_bh, 1);
+ gfs2_setbit(&pos, true, GFS2_BLKST_USED);
(*n)++;
+ block++;
}
- blk = gfs2_bi2rgd_blk(bi, blk);
- rgd->rd_last_alloc = blk + *n - 1;
- return rgd->rd_data0 + blk;
}
/**
@@ -1875,46 +1874,30 @@
static struct gfs2_rgrpd *rgblk_free(struct gfs2_sbd *sdp, u64 bstart,
u32 blen, unsigned char new_state)
{
- struct gfs2_rgrpd *rgd;
- struct gfs2_bitmap *bi = NULL;
- u32 length, rgrp_blk, buf_blk;
- unsigned int buf;
+ struct gfs2_rbm rbm;
- rgd = gfs2_blk2rgrpd(sdp, bstart, 1);
- if (!rgd) {
+ rbm.rgd = gfs2_blk2rgrpd(sdp, bstart, 1);
+ if (!rbm.rgd) {
if (gfs2_consist(sdp))
fs_err(sdp, "block = %llu\n", (unsigned long long)bstart);
return NULL;
}
- length = rgd->rd_length;
-
- rgrp_blk = bstart - rgd->rd_data0;
-
while (blen--) {
- for (buf = 0; buf < length; buf++) {
- bi = rgd->rd_bits + buf;
- if (rgrp_blk < (bi->bi_start + bi->bi_len) * GFS2_NBBY)
- break;
+ gfs2_rbm_from_block(&rbm, bstart);
+ bstart++;
+ if (!rbm.bi->bi_clone) {
+ rbm.bi->bi_clone = kmalloc(rbm.bi->bi_bh->b_size,
+ GFP_NOFS | __GFP_NOFAIL);
+ memcpy(rbm.bi->bi_clone + rbm.bi->bi_offset,
+ rbm.bi->bi_bh->b_data + rbm.bi->bi_offset,
+ rbm.bi->bi_len);
}
-
- gfs2_assert(rgd->rd_sbd, buf < length);
-
- buf_blk = rgrp_blk - bi->bi_start * GFS2_NBBY;
- rgrp_blk++;
-
- if (!bi->bi_clone) {
- bi->bi_clone = kmalloc(bi->bi_bh->b_size,
- GFP_NOFS | __GFP_NOFAIL);
- memcpy(bi->bi_clone + bi->bi_offset,
- bi->bi_bh->b_data + bi->bi_offset,
- bi->bi_len);
- }
- gfs2_trans_add_bh(rgd->rd_gl, bi->bi_bh, 1);
- gfs2_setbit(rgd, NULL, bi, buf_blk, new_state);
+ gfs2_trans_add_bh(rbm.rgd->rd_gl, rbm.bi->bi_bh, 1);
+ gfs2_setbit(&rbm, false, new_state);
}
- return rgd;
+ return rbm.rgd;
}
/**
@@ -1956,56 +1939,41 @@
}
/**
- * claim_reserved_blks - Claim previously reserved blocks
- * @ip: the inode that's claiming the reservation
- * @dinode: 1 if this block is a dinode block, otherwise data block
- * @nblocks: desired extent length
+ * gfs2_adjust_reservation - Adjust (or remove) a reservation after allocation
+ * @ip: The inode we have just allocated blocks for
+ * @rbm: The start of the allocated blocks
+ * @len: The extent length
*
- * Lay claim to previously allocated block reservation blocks.
- * Returns: Starting block number of the blocks claimed.
- * Sets *nblocks to the actual extent length allocated.
+ * Adjusts a reservation after an allocation has taken place. If the
+ * reservation does not match the allocation, or if it is now empty
+ * then it is removed.
*/
-static u64 claim_reserved_blks(struct gfs2_inode *ip, bool dinode,
- unsigned int *nblocks)
+
+static void gfs2_adjust_reservation(struct gfs2_inode *ip,
+ const struct gfs2_rbm *rbm, unsigned len)
{
struct gfs2_blkreserv *rs = ip->i_res;
- struct gfs2_rgrpd *rgd = rs->rs_rgd;
- struct gfs2_sbd *sdp = GFS2_SB(&ip->i_inode);
- struct gfs2_bitmap *bi;
- u64 start_block = gfs2_rs_startblk(rs);
- const unsigned int elen = *nblocks;
+ struct gfs2_rgrpd *rgd = rbm->rgd;
+ unsigned rlen;
+ u64 block;
+ int ret;
- /*BUG_ON(!gfs2_glock_is_locked_by_me(ip->i_gl));*/
- gfs2_assert_withdraw(sdp, rgd);
- /*BUG_ON(!gfs2_glock_is_locked_by_me(rgd->rd_gl));*/
- bi = rs->rs_bi;
- gfs2_trans_add_bh(rgd->rd_gl, bi->bi_bh, 1);
-
- for (*nblocks = 0; *nblocks < elen && rs->rs_free; (*nblocks)++) {
- /* Make sure the bitmap hasn't changed */
- gfs2_setbit(rgd, bi->bi_clone, bi, rs->rs_biblk,
- dinode ? GFS2_BLKST_DINODE : GFS2_BLKST_USED);
- rs->rs_biblk++;
- rs->rs_free--;
-
- BUG_ON(!rgd->rd_reserved);
- rgd->rd_reserved--;
- dinode = false;
- trace_gfs2_rs(ip, rs, TRACE_RS_CLAIM);
+ spin_lock(&rgd->rd_rsspin);
+ if (gfs2_rs_active(rs)) {
+ if (gfs2_rbm_eq(&rs->rs_rbm, rbm)) {
+ block = gfs2_rbm_to_block(rbm);
+ ret = gfs2_rbm_from_block(&rs->rs_rbm, block + len);
+ rlen = min(rs->rs_free, len);
+ rs->rs_free -= rlen;
+ rgd->rd_reserved -= rlen;
+ trace_gfs2_rs(rs, TRACE_RS_CLAIM);
+ if (rs->rs_free && !ret)
+ goto out;
+ }
+ __rs_deltree(ip, rs);
}
-
- if (!rs->rs_free) {
- struct gfs2_rgrpd *rgd = ip->i_res->rs_rgd;
-
- gfs2_rs_deltree(rs);
- /* -nblocks because we haven't returned to do the math yet.
- I'm doing the math backwards to prevent negative numbers,
- but think of it as:
- if (unclaimed_blocks(rgd) - *nblocks >= RGRP_RSRV_MINBLKS */
- if (unclaimed_blocks(rgd) >= RGRP_RSRV_MINBLKS + *nblocks)
- rg_mblk_search(rgd, ip);
- }
- return start_block;
+out:
+ spin_unlock(&rgd->rd_rsspin);
}
/**
@@ -2024,47 +1992,40 @@
{
struct gfs2_sbd *sdp = GFS2_SB(&ip->i_inode);
struct buffer_head *dibh;
- struct gfs2_rgrpd *rgd;
+ struct gfs2_rbm rbm = { .rgd = ip->i_rgd, };
unsigned int ndata;
- u32 goal, blk; /* block, within the rgrp scope */
+ u64 goal;
u64 block; /* block, within the file system scope */
int error;
- struct gfs2_bitmap *bi;
- /* Only happens if there is a bug in gfs2, return something distinctive
- * to ensure that it is noticed.
- */
- if (ip->i_res->rs_requested == 0)
- return -ECANCELED;
+ if (gfs2_rs_active(ip->i_res))
+ goal = gfs2_rbm_to_block(&ip->i_res->rs_rbm);
+ else if (!dinode && rgrp_contains_block(rbm.rgd, ip->i_goal))
+ goal = ip->i_goal;
+ else
+ goal = rbm.rgd->rd_last_alloc + rbm.rgd->rd_data0;
- /* Check if we have a multi-block reservation, and if so, claim the
- next free block from it. */
- if (gfs2_rs_active(ip->i_res)) {
- BUG_ON(!ip->i_res->rs_free);
- rgd = ip->i_res->rs_rgd;
- block = claim_reserved_blks(ip, dinode, nblocks);
- } else {
- rgd = ip->i_rgd;
+ gfs2_rbm_from_block(&rbm, goal);
+ error = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, 0, ip, false);
- if (!dinode && rgrp_contains_block(rgd, ip->i_goal))
- goal = ip->i_goal - rgd->rd_data0;
- else
- goal = rgd->rd_last_alloc;
-
- blk = rgblk_search(rgd, goal, GFS2_BLKST_FREE, &bi);
-
- /* Since all blocks are reserved in advance, this shouldn't
- happen */
- if (blk == BFITNOENT) {
- printk(KERN_WARNING "BFITNOENT, nblocks=%u\n",
- *nblocks);
- printk(KERN_WARNING "FULL=%d\n",
- test_bit(GBF_FULL, &rgd->rd_bits->bi_flags));
- goto rgrp_error;
- }
-
- block = gfs2_alloc_extent(rgd, bi, blk, dinode, nblocks);
+ if (error == -ENOSPC) {
+ gfs2_rbm_from_block(&rbm, goal);
+ error = gfs2_rbm_find(&rbm, GFS2_BLKST_FREE, 0, NULL, false);
}
+
+ /* Since all blocks are reserved in advance, this shouldn't happen */
+ if (error) {
+ fs_warn(sdp, "inum=%llu error=%d, nblocks=%u, full=%d\n",
+ (unsigned long long)ip->i_no_addr, error, *nblocks,
+ test_bit(GBF_FULL, &rbm.rgd->rd_bits->bi_flags));
+ goto rgrp_error;
+ }
+
+ gfs2_alloc_extent(&rbm, dinode, nblocks);
+ block = gfs2_rbm_to_block(&rbm);
+ rbm.rgd->rd_last_alloc = block - rbm.rgd->rd_data0;
+ if (gfs2_rs_active(ip->i_res))
+ gfs2_adjust_reservation(ip, &rbm, *nblocks);
ndata = *nblocks;
if (dinode)
ndata--;
@@ -2081,22 +2042,22 @@
brelse(dibh);
}
}
- if (rgd->rd_free < *nblocks) {
+ if (rbm.rgd->rd_free < *nblocks) {
printk(KERN_WARNING "nblocks=%u\n", *nblocks);
goto rgrp_error;
}
- rgd->rd_free -= *nblocks;
+ rbm.rgd->rd_free -= *nblocks;
if (dinode) {
- rgd->rd_dinodes++;
- *generation = rgd->rd_igeneration++;
+ rbm.rgd->rd_dinodes++;
+ *generation = rbm.rgd->rd_igeneration++;
if (*generation == 0)
- *generation = rgd->rd_igeneration++;
+ *generation = rbm.rgd->rd_igeneration++;
}
- gfs2_trans_add_bh(rgd->rd_gl, rgd->rd_bits[0].bi_bh, 1);
- gfs2_rgrp_out(rgd, rgd->rd_bits[0].bi_bh->b_data);
- gfs2_rgrp_ondisk2lvb(rgd->rd_rgl, rgd->rd_bits[0].bi_bh->b_data);
+ gfs2_trans_add_bh(rbm.rgd->rd_gl, rbm.rgd->rd_bits[0].bi_bh, 1);
+ gfs2_rgrp_out(rbm.rgd, rbm.rgd->rd_bits[0].bi_bh->b_data);
+ gfs2_rgrp_ondisk2lvb(rbm.rgd->rd_rgl, rbm.rgd->rd_bits[0].bi_bh->b_data);
gfs2_statfs_change(sdp, 0, -(s64)*nblocks, dinode ? 1 : 0);
if (dinode)
@@ -2110,14 +2071,14 @@
gfs2_quota_change(ip, ndata, ip->i_inode.i_uid,
ip->i_inode.i_gid);
- rgd->rd_free_clone -= *nblocks;
- trace_gfs2_block_alloc(ip, rgd, block, *nblocks,
+ rbm.rgd->rd_free_clone -= *nblocks;
+ trace_gfs2_block_alloc(ip, rbm.rgd, block, *nblocks,
dinode ? GFS2_BLKST_DINODE : GFS2_BLKST_USED);
*bn = block;
return 0;
rgrp_error:
- gfs2_rgrp_error(rgd);
+ gfs2_rgrp_error(rbm.rgd);
return -EIO;
}
diff --git a/fs/gfs2/rgrp.h b/fs/gfs2/rgrp.h
index ca6e267..2407795 100644
--- a/fs/gfs2/rgrp.h
+++ b/fs/gfs2/rgrp.h
@@ -46,7 +46,7 @@
bool dinode, u64 *generation);
extern int gfs2_rs_alloc(struct gfs2_inode *ip);
-extern void gfs2_rs_deltree(struct gfs2_blkreserv *rs);
+extern void gfs2_rs_deltree(struct gfs2_inode *ip, struct gfs2_blkreserv *rs);
extern void gfs2_rs_delete(struct gfs2_inode *ip);
extern void __gfs2_free_blocks(struct gfs2_inode *ip, u64 bstart, u32 blen, int meta);
extern void gfs2_free_meta(struct gfs2_inode *ip, u64 bstart, u32 blen);
@@ -73,30 +73,10 @@
const struct gfs2_bitmap *bi, unsigned minlen, u64 *ptrimmed);
extern int gfs2_fitrim(struct file *filp, void __user *argp);
-/* This is how to tell if a multi-block reservation is "inplace" reserved: */
-static inline int gfs2_mb_reserved(struct gfs2_inode *ip)
+/* This is how to tell if a reservation is in the rgrp tree: */
+static inline bool gfs2_rs_active(struct gfs2_blkreserv *rs)
{
- if (ip->i_res && ip->i_res->rs_requested)
- return 1;
- return 0;
-}
-
-/* This is how to tell if a multi-block reservation is in the rgrp tree: */
-static inline int gfs2_rs_active(struct gfs2_blkreserv *rs)
-{
- if (rs && rs->rs_bi)
- return 1;
- return 0;
-}
-
-static inline u32 gfs2_bi2rgd_blk(const struct gfs2_bitmap *bi, u32 blk)
-{
- return (bi->bi_start * GFS2_NBBY) + blk;
-}
-
-static inline u64 gfs2_rs_startblk(const struct gfs2_blkreserv *rs)
-{
- return gfs2_bi2rgd_blk(rs->rs_bi, rs->rs_biblk) + rs->rs_rgd->rd_data0;
+ return rs && !RB_EMPTY_NODE(&rs->rs_node);
}
#endif /* __RGRP_DOT_H__ */
diff --git a/fs/gfs2/super.c b/fs/gfs2/super.c
index fc3168f..a8d90f2 100644
--- a/fs/gfs2/super.c
+++ b/fs/gfs2/super.c
@@ -1366,6 +1366,8 @@
val = sdp->sd_tune.gt_statfs_quantum;
if (val != 30)
seq_printf(s, ",statfs_quantum=%d", val);
+ else if (sdp->sd_tune.gt_statfs_slow)
+ seq_puts(s, ",statfs_quantum=0");
val = sdp->sd_tune.gt_quota_quantum;
if (val != 60)
seq_printf(s, ",quota_quantum=%d", val);
@@ -1543,6 +1545,11 @@
out_truncate:
gfs2_log_flush(sdp, ip->i_gl);
+ if (test_bit(GLF_DIRTY, &ip->i_gl->gl_flags)) {
+ struct address_space *metamapping = gfs2_glock2aspace(ip->i_gl);
+ filemap_fdatawrite(metamapping);
+ filemap_fdatawait(metamapping);
+ }
write_inode_now(inode, 1);
gfs2_ail_flush(ip->i_gl, 0);
@@ -1557,7 +1564,7 @@
out_unlock:
/* Error path for case 1 */
if (gfs2_rs_active(ip->i_res))
- gfs2_rs_deltree(ip->i_res);
+ gfs2_rs_deltree(ip, ip->i_res);
if (test_bit(HIF_HOLDER, &ip->i_iopen_gh.gh_iflags))
gfs2_glock_dq(&ip->i_iopen_gh);
diff --git a/fs/gfs2/trace_gfs2.h b/fs/gfs2/trace_gfs2.h
index a25c252..bbdc78a 100644
--- a/fs/gfs2/trace_gfs2.h
+++ b/fs/gfs2/trace_gfs2.h
@@ -509,10 +509,9 @@
/* Keep track of multi-block reservations as they are allocated/freed */
TRACE_EVENT(gfs2_rs,
- TP_PROTO(const struct gfs2_inode *ip, const struct gfs2_blkreserv *rs,
- u8 func),
+ TP_PROTO(const struct gfs2_blkreserv *rs, u8 func),
- TP_ARGS(ip, rs, func),
+ TP_ARGS(rs, func),
TP_STRUCT__entry(
__field( dev_t, dev )
@@ -526,18 +525,17 @@
),
TP_fast_assign(
- __entry->dev = rs->rs_rgd ? rs->rs_rgd->rd_sbd->sd_vfs->s_dev : 0;
- __entry->rd_addr = rs->rs_rgd ? rs->rs_rgd->rd_addr : 0;
- __entry->rd_free_clone = rs->rs_rgd ? rs->rs_rgd->rd_free_clone : 0;
- __entry->rd_reserved = rs->rs_rgd ? rs->rs_rgd->rd_reserved : 0;
- __entry->inum = ip ? ip->i_no_addr : 0;
- __entry->start = gfs2_rs_startblk(rs);
+ __entry->dev = rs->rs_rbm.rgd->rd_sbd->sd_vfs->s_dev;
+ __entry->rd_addr = rs->rs_rbm.rgd->rd_addr;
+ __entry->rd_free_clone = rs->rs_rbm.rgd->rd_free_clone;
+ __entry->rd_reserved = rs->rs_rbm.rgd->rd_reserved;
+ __entry->inum = rs->rs_inum;
+ __entry->start = gfs2_rbm_to_block(&rs->rs_rbm);
__entry->free = rs->rs_free;
__entry->func = func;
),
- TP_printk("%u,%u bmap %llu resrv %llu rg:%llu rf:%lu rr:%lu %s "
- "f:%lu",
+ TP_printk("%u,%u bmap %llu resrv %llu rg:%llu rf:%lu rr:%lu %s f:%lu",
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->inum,
(unsigned long long)__entry->start,
diff --git a/fs/gfs2/trans.h b/fs/gfs2/trans.h
index 41f42cd..bf2ae9a 100644
--- a/fs/gfs2/trans.h
+++ b/fs/gfs2/trans.h
@@ -28,11 +28,10 @@
/* reserve either the number of blocks to be allocated plus the rg header
* block, or all of the blocks in the rg, whichever is smaller */
-static inline unsigned int gfs2_rg_blocks(const struct gfs2_inode *ip)
+static inline unsigned int gfs2_rg_blocks(const struct gfs2_inode *ip, unsigned requested)
{
- const struct gfs2_blkreserv *rs = ip->i_res;
- if (rs && rs->rs_requested < ip->i_rgd->rd_length)
- return rs->rs_requested + 1;
+ if (requested < ip->i_rgd->rd_length)
+ return requested + 1;
return ip->i_rgd->rd_length;
}
diff --git a/fs/gfs2/xattr.c b/fs/gfs2/xattr.c
index 27a0b4a..db330e5 100644
--- a/fs/gfs2/xattr.c
+++ b/fs/gfs2/xattr.c
@@ -448,17 +448,18 @@
}
/**
- * ea_get_unstuffed - actually copies the unstuffed data into the
- * request buffer
+ * ea_iter_unstuffed - copies the unstuffed xattr data to/from the
+ * request buffer
* @ip: The GFS2 inode
* @ea: The extended attribute header structure
- * @data: The data to be copied
+ * @din: The data to be copied in
+ * @dout: The data to be copied out (one of din,dout will be NULL)
*
* Returns: errno
*/
-static int ea_get_unstuffed(struct gfs2_inode *ip, struct gfs2_ea_header *ea,
- char *data)
+static int gfs2_iter_unstuffed(struct gfs2_inode *ip, struct gfs2_ea_header *ea,
+ const char *din, char *dout)
{
struct gfs2_sbd *sdp = GFS2_SB(&ip->i_inode);
struct buffer_head **bh;
@@ -467,6 +468,8 @@
__be64 *dataptrs = GFS2_EA2DATAPTRS(ea);
unsigned int x;
int error = 0;
+ unsigned char *pos;
+ unsigned cp_size;
bh = kcalloc(nptrs, sizeof(struct buffer_head *), GFP_NOFS);
if (!bh)
@@ -497,12 +500,21 @@
goto out;
}
- memcpy(data, bh[x]->b_data + sizeof(struct gfs2_meta_header),
- (sdp->sd_jbsize > amount) ? amount : sdp->sd_jbsize);
+ pos = bh[x]->b_data + sizeof(struct gfs2_meta_header);
+ cp_size = (sdp->sd_jbsize > amount) ? amount : sdp->sd_jbsize;
+
+ if (dout) {
+ memcpy(dout, pos, cp_size);
+ dout += sdp->sd_jbsize;
+ }
+
+ if (din) {
+ gfs2_trans_add_bh(ip->i_gl, bh[x], 1);
+ memcpy(pos, din, cp_size);
+ din += sdp->sd_jbsize;
+ }
amount -= sdp->sd_jbsize;
- data += sdp->sd_jbsize;
-
brelse(bh[x]);
}
@@ -523,7 +535,7 @@
memcpy(data, GFS2_EA2DATA(el->el_ea), len);
return len;
}
- ret = ea_get_unstuffed(ip, el->el_ea, data);
+ ret = gfs2_iter_unstuffed(ip, el->el_ea, NULL, data);
if (ret < 0)
return ret;
return len;
@@ -727,7 +739,7 @@
goto out_gunlock_q;
error = gfs2_trans_begin(GFS2_SB(&ip->i_inode),
- blks + gfs2_rg_blocks(ip) +
+ blks + gfs2_rg_blocks(ip, blks) +
RES_DINODE + RES_STATFS + RES_QUOTA, 0);
if (error)
goto out_ipres;
@@ -1220,69 +1232,23 @@
size, flags, type);
}
+
static int ea_acl_chmod_unstuffed(struct gfs2_inode *ip,
struct gfs2_ea_header *ea, char *data)
{
struct gfs2_sbd *sdp = GFS2_SB(&ip->i_inode);
- struct buffer_head **bh;
unsigned int amount = GFS2_EA_DATA_LEN(ea);
unsigned int nptrs = DIV_ROUND_UP(amount, sdp->sd_jbsize);
- __be64 *dataptrs = GFS2_EA2DATAPTRS(ea);
- unsigned int x;
- int error;
+ int ret;
- bh = kcalloc(nptrs, sizeof(struct buffer_head *), GFP_NOFS);
- if (!bh)
- return -ENOMEM;
+ ret = gfs2_trans_begin(sdp, nptrs + RES_DINODE, 0);
+ if (ret)
+ return ret;
- error = gfs2_trans_begin(sdp, nptrs + RES_DINODE, 0);
- if (error)
- goto out;
-
- for (x = 0; x < nptrs; x++) {
- error = gfs2_meta_read(ip->i_gl, be64_to_cpu(*dataptrs), 0,
- bh + x);
- if (error) {
- while (x--)
- brelse(bh[x]);
- goto fail;
- }
- dataptrs++;
- }
-
- for (x = 0; x < nptrs; x++) {
- error = gfs2_meta_wait(sdp, bh[x]);
- if (error) {
- for (; x < nptrs; x++)
- brelse(bh[x]);
- goto fail;
- }
- if (gfs2_metatype_check(sdp, bh[x], GFS2_METATYPE_ED)) {
- for (; x < nptrs; x++)
- brelse(bh[x]);
- error = -EIO;
- goto fail;
- }
-
- gfs2_trans_add_bh(ip->i_gl, bh[x], 1);
-
- memcpy(bh[x]->b_data + sizeof(struct gfs2_meta_header), data,
- (sdp->sd_jbsize > amount) ? amount : sdp->sd_jbsize);
-
- amount -= sdp->sd_jbsize;
- data += sdp->sd_jbsize;
-
- brelse(bh[x]);
- }
-
-out:
- kfree(bh);
- return error;
-
-fail:
+ ret = gfs2_iter_unstuffed(ip, ea, data, NULL);
gfs2_trans_end(sdp);
- kfree(bh);
- return error;
+
+ return ret;
}
int gfs2_xattr_acl_chmod(struct gfs2_inode *ip, struct iattr *attr, char *data)
diff --git a/fs/jbd/journal.c b/fs/jbd/journal.c
index 0935750..a286233 100644
--- a/fs/jbd/journal.c
+++ b/fs/jbd/journal.c
@@ -1113,6 +1113,11 @@
BUG_ON(!mutex_is_locked(&journal->j_checkpoint_mutex));
spin_lock(&journal->j_state_lock);
+ /* Is it already empty? */
+ if (sb->s_start == 0) {
+ spin_unlock(&journal->j_state_lock);
+ return;
+ }
jbd_debug(1, "JBD: Marking journal as empty (seq %d)\n",
journal->j_tail_sequence);
diff --git a/fs/libfs.c b/fs/libfs.c
index a74cb17..7cc37ca 100644
--- a/fs/libfs.c
+++ b/fs/libfs.c
@@ -874,7 +874,7 @@
EXPORT_SYMBOL_GPL(generic_fh_to_dentry);
/**
- * generic_fh_to_dentry - generic helper for the fh_to_parent export operation
+ * generic_fh_to_parent - generic helper for the fh_to_parent export operation
* @sb: filesystem to do the file handle conversion on
* @fid: file handle to convert
* @fh_len: length of the file handle in bytes
diff --git a/fs/lockd/svclock.c b/fs/lockd/svclock.c
index fb1a2be..8d80c99 100644
--- a/fs/lockd/svclock.c
+++ b/fs/lockd/svclock.c
@@ -289,7 +289,6 @@
dprintk("lockd: freeing block %p...\n", block);
/* Remove block from file's list of blocks */
- mutex_lock(&file->f_mutex);
list_del_init(&block->b_flist);
mutex_unlock(&file->f_mutex);
@@ -303,7 +302,7 @@
static void nlmsvc_release_block(struct nlm_block *block)
{
if (block != NULL)
- kref_put(&block->b_count, nlmsvc_free_block);
+ kref_put_mutex(&block->b_count, nlmsvc_free_block, &block->b_file->f_mutex);
}
/*
diff --git a/fs/logfs/dev_bdev.c b/fs/logfs/dev_bdev.c
index df0de27..e784a21 100644
--- a/fs/logfs/dev_bdev.c
+++ b/fs/logfs/dev_bdev.c
@@ -26,6 +26,7 @@
struct completion complete;
bio_init(&bio);
+ bio.bi_max_vecs = 1;
bio.bi_io_vec = &bio_vec;
bio_vec.bv_page = page;
bio_vec.bv_len = PAGE_SIZE;
@@ -95,12 +96,11 @@
struct address_space *mapping = super->s_mapping_inode->i_mapping;
struct bio *bio;
struct page *page;
- struct request_queue *q = bdev_get_queue(sb->s_bdev);
- unsigned int max_pages = queue_max_hw_sectors(q) >> (PAGE_SHIFT - 9);
+ unsigned int max_pages;
int i;
- if (max_pages > BIO_MAX_PAGES)
- max_pages = BIO_MAX_PAGES;
+ max_pages = min(nr_pages, (size_t) bio_get_nr_vecs(super->s_bdev));
+
bio = bio_alloc(GFP_NOFS, max_pages);
BUG_ON(!bio);
@@ -190,12 +190,11 @@
{
struct logfs_super *super = logfs_super(sb);
struct bio *bio;
- struct request_queue *q = bdev_get_queue(sb->s_bdev);
- unsigned int max_pages = queue_max_hw_sectors(q) >> (PAGE_SHIFT - 9);
+ unsigned int max_pages;
int i;
- if (max_pages > BIO_MAX_PAGES)
- max_pages = BIO_MAX_PAGES;
+ max_pages = min(nr_pages, (size_t) bio_get_nr_vecs(super->s_bdev));
+
bio = bio_alloc(GFP_NOFS, max_pages);
BUG_ON(!bio);
diff --git a/fs/logfs/inode.c b/fs/logfs/inode.c
index a422f42..6984562 100644
--- a/fs/logfs/inode.c
+++ b/fs/logfs/inode.c
@@ -156,10 +156,26 @@
call_rcu(&inode->i_rcu, logfs_i_callback);
}
+static void __logfs_destroy_meta_inode(struct inode *inode)
+{
+ struct logfs_inode *li = logfs_inode(inode);
+ BUG_ON(li->li_block);
+ call_rcu(&inode->i_rcu, logfs_i_callback);
+}
+
static void logfs_destroy_inode(struct inode *inode)
{
struct logfs_inode *li = logfs_inode(inode);
+ if (inode->i_ino < LOGFS_RESERVED_INOS) {
+ /*
+ * The reserved inodes are never destroyed unless we are in
+ * unmont path.
+ */
+ __logfs_destroy_meta_inode(inode);
+ return;
+ }
+
BUG_ON(list_empty(&li->li_freeing_list));
spin_lock(&logfs_inode_lock);
li->li_refcount--;
@@ -373,8 +389,8 @@
{
struct logfs_super *super = logfs_super(sb);
/* kill the meta-inodes */
- iput(super->s_master_inode);
iput(super->s_segfile_inode);
+ iput(super->s_master_inode);
iput(super->s_mapping_inode);
}
diff --git a/fs/logfs/journal.c b/fs/logfs/journal.c
index 1e1c369..2a09b8d 100644
--- a/fs/logfs/journal.c
+++ b/fs/logfs/journal.c
@@ -565,7 +565,7 @@
index = ofs >> PAGE_SHIFT;
page_ofs = ofs & (PAGE_SIZE - 1);
- page = find_lock_page(mapping, index);
+ page = find_or_create_page(mapping, index, GFP_NOFS);
BUG_ON(!page);
memcpy(wbuf, page_address(page) + page_ofs, super->s_writesize);
unlock_page(page);
diff --git a/fs/logfs/readwrite.c b/fs/logfs/readwrite.c
index f1cb512c50..5be0abe 100644
--- a/fs/logfs/readwrite.c
+++ b/fs/logfs/readwrite.c
@@ -2189,7 +2189,6 @@
return;
}
- BUG_ON(inode->i_ino < LOGFS_RESERVED_INOS);
page = inode_to_page(inode);
BUG_ON(!page); /* FIXME: Use emergency page */
logfs_put_write_page(page);
diff --git a/fs/logfs/segment.c b/fs/logfs/segment.c
index e28d090..038da09 100644
--- a/fs/logfs/segment.c
+++ b/fs/logfs/segment.c
@@ -886,7 +886,7 @@
static void map_invalidatepage(struct page *page, unsigned long l)
{
- BUG();
+ return;
}
static int map_releasepage(struct page *page, gfp_t g)
diff --git a/fs/namespace.c b/fs/namespace.c
index 4d31f73..7bdf790 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1886,8 +1886,14 @@
return err;
err = -EINVAL;
- if (!(mnt_flags & MNT_SHRINKABLE) && !check_mnt(real_mount(path->mnt)))
- goto unlock;
+ if (unlikely(!check_mnt(real_mount(path->mnt)))) {
+ /* that's acceptable only for automounts done in private ns */
+ if (!(mnt_flags & MNT_SHRINKABLE))
+ goto unlock;
+ /* ... and for those we'd better have mountpoint still alive */
+ if (!real_mount(path->mnt)->mnt_ns)
+ goto unlock;
+ }
/* Refuse the same filesystem on the same mount point */
err = -EBUSY;
diff --git a/fs/nfs/file.c b/fs/nfs/file.c
index 75d6d0a..6a7fcab 100644
--- a/fs/nfs/file.c
+++ b/fs/nfs/file.c
@@ -287,10 +287,12 @@
struct inode *inode = file->f_path.dentry->d_inode;
ret = filemap_write_and_wait_range(inode->i_mapping, start, end);
+ if (ret != 0)
+ goto out;
mutex_lock(&inode->i_mutex);
ret = nfs_file_fsync_commit(file, start, end, datasync);
mutex_unlock(&inode->i_mutex);
-
+out:
return ret;
}
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index c6e895f..9b47610 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -154,7 +154,7 @@
nfsi->attrtimeo = NFS_MINATTRTIMEO(inode);
nfsi->attrtimeo_timestamp = jiffies;
- memset(NFS_COOKIEVERF(inode), 0, sizeof(NFS_COOKIEVERF(inode)));
+ memset(NFS_I(inode)->cookieverf, 0, sizeof(NFS_I(inode)->cookieverf));
if (S_ISREG(mode) || S_ISDIR(mode) || S_ISLNK(mode))
nfsi->cache_validity |= NFS_INO_INVALID_ATTR|NFS_INO_INVALID_DATA|NFS_INO_INVALID_ACCESS|NFS_INO_INVALID_ACL|NFS_INO_REVAL_PAGECACHE;
else
diff --git a/fs/nfs/nfs3proc.c b/fs/nfs/nfs3proc.c
index d6b3b5f..6932209 100644
--- a/fs/nfs/nfs3proc.c
+++ b/fs/nfs/nfs3proc.c
@@ -643,7 +643,7 @@
u64 cookie, struct page **pages, unsigned int count, int plus)
{
struct inode *dir = dentry->d_inode;
- __be32 *verf = NFS_COOKIEVERF(dir);
+ __be32 *verf = NFS_I(dir)->cookieverf;
struct nfs3_readdirargs arg = {
.fh = NFS_FH(dir),
.cookie = cookie,
diff --git a/fs/nfs/nfs4file.c b/fs/nfs/nfs4file.c
index acb65e7..eb5eb8e 100644
--- a/fs/nfs/nfs4file.c
+++ b/fs/nfs/nfs4file.c
@@ -96,13 +96,15 @@
struct inode *inode = file->f_path.dentry->d_inode;
ret = filemap_write_and_wait_range(inode->i_mapping, start, end);
+ if (ret != 0)
+ goto out;
mutex_lock(&inode->i_mutex);
ret = nfs_file_fsync_commit(file, start, end, datasync);
if (!ret && !datasync)
/* application has asked for meta-data sync */
ret = pnfs_layoutcommit_inode(inode, true);
mutex_unlock(&inode->i_mutex);
-
+out:
return ret;
}
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 6352741..1e50326 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3215,11 +3215,11 @@
dentry->d_parent->d_name.name,
dentry->d_name.name,
(unsigned long long)cookie);
- nfs4_setup_readdir(cookie, NFS_COOKIEVERF(dir), dentry, &args);
+ nfs4_setup_readdir(cookie, NFS_I(dir)->cookieverf, dentry, &args);
res.pgbase = args.pgbase;
status = nfs4_call_sync(NFS_SERVER(dir)->client, NFS_SERVER(dir), &msg, &args.seq_args, &res.seq_res, 0);
if (status >= 0) {
- memcpy(NFS_COOKIEVERF(dir), res.verifier.data, NFS4_VERIFIER_SIZE);
+ memcpy(NFS_I(dir)->cookieverf, res.verifier.data, NFS4_VERIFIER_SIZE);
status += args.pgbase;
}
@@ -3653,11 +3653,11 @@
&& (server->acl_bitmask & ACL4_SUPPORT_DENY_ACL);
}
-/* Assuming that XATTR_SIZE_MAX is a multiple of PAGE_CACHE_SIZE, and that
- * it's OK to put sizeof(void) * (XATTR_SIZE_MAX/PAGE_CACHE_SIZE) bytes on
+/* Assuming that XATTR_SIZE_MAX is a multiple of PAGE_SIZE, and that
+ * it's OK to put sizeof(void) * (XATTR_SIZE_MAX/PAGE_SIZE) bytes on
* the stack.
*/
-#define NFS4ACL_MAXPAGES (XATTR_SIZE_MAX >> PAGE_CACHE_SHIFT)
+#define NFS4ACL_MAXPAGES DIV_ROUND_UP(XATTR_SIZE_MAX, PAGE_SIZE)
static int buf_to_pages_noslab(const void *buf, size_t buflen,
struct page **pages, unsigned int *pgbase)
@@ -3668,7 +3668,7 @@
spages = pages;
do {
- len = min_t(size_t, PAGE_CACHE_SIZE, buflen);
+ len = min_t(size_t, PAGE_SIZE, buflen);
newpage = alloc_page(GFP_KERNEL);
if (newpage == NULL)
@@ -3739,7 +3739,7 @@
struct nfs4_cached_acl *acl;
size_t buflen = sizeof(*acl) + acl_len;
- if (pages && buflen <= PAGE_SIZE) {
+ if (buflen <= PAGE_SIZE) {
acl = kmalloc(buflen, GFP_KERNEL);
if (acl == NULL)
goto out;
@@ -3782,17 +3782,15 @@
.rpc_argp = &args,
.rpc_resp = &res,
};
- int ret = -ENOMEM, npages, i;
- size_t acl_len = 0;
+ unsigned int npages = DIV_ROUND_UP(buflen, PAGE_SIZE);
+ int ret = -ENOMEM, i;
- npages = (buflen + PAGE_SIZE - 1) >> PAGE_SHIFT;
/* As long as we're doing a round trip to the server anyway,
* let's be prepared for a page of acl data. */
if (npages == 0)
npages = 1;
-
- /* Add an extra page to handle the bitmap returned */
- npages++;
+ if (npages > ARRAY_SIZE(pages))
+ return -ERANGE;
for (i = 0; i < npages; i++) {
pages[i] = alloc_page(GFP_KERNEL);
@@ -3808,11 +3806,6 @@
args.acl_len = npages * PAGE_SIZE;
args.acl_pgbase = 0;
- /* Let decode_getfacl know not to fail if the ACL data is larger than
- * the page we send as a guess */
- if (buf == NULL)
- res.acl_flags |= NFS4_ACL_LEN_REQUEST;
-
dprintk("%s buf %p buflen %zu npages %d args.acl_len %zu\n",
__func__, buf, buflen, npages, args.acl_len);
ret = nfs4_call_sync(NFS_SERVER(inode)->client, NFS_SERVER(inode),
@@ -3820,20 +3813,19 @@
if (ret)
goto out_free;
- acl_len = res.acl_len;
- if (acl_len > args.acl_len)
- nfs4_write_cached_acl(inode, NULL, 0, acl_len);
- else
- nfs4_write_cached_acl(inode, pages, res.acl_data_offset,
- acl_len);
- if (buf) {
+ /* Handle the case where the passed-in buffer is too short */
+ if (res.acl_flags & NFS4_ACL_TRUNC) {
+ /* Did the user only issue a request for the acl length? */
+ if (buf == NULL)
+ goto out_ok;
ret = -ERANGE;
- if (acl_len > buflen)
- goto out_free;
- _copy_from_pages(buf, pages, res.acl_data_offset,
- acl_len);
+ goto out_free;
}
- ret = acl_len;
+ nfs4_write_cached_acl(inode, pages, res.acl_data_offset, res.acl_len);
+ if (buf)
+ _copy_from_pages(buf, pages, res.acl_data_offset, res.acl_len);
+out_ok:
+ ret = res.acl_len;
out_free:
for (i = 0; i < npages; i++)
if (pages[i])
@@ -3891,10 +3883,13 @@
.rpc_argp = &arg,
.rpc_resp = &res,
};
+ unsigned int npages = DIV_ROUND_UP(buflen, PAGE_SIZE);
int ret, i;
if (!nfs4_server_supports_acls(server))
return -EOPNOTSUPP;
+ if (npages > ARRAY_SIZE(pages))
+ return -ERANGE;
i = buf_to_pages_noslab(buf, buflen, arg.acl_pages, &arg.acl_pgbase);
if (i < 0)
return i;
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 1bfbd67..8dba6bd 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -5072,18 +5072,14 @@
* are stored with the acl data to handle the problem of
* variable length bitmaps.*/
res->acl_data_offset = xdr_stream_pos(xdr) - pg_offset;
-
- /* We ignore &savep and don't do consistency checks on
- * the attr length. Let userspace figure it out.... */
res->acl_len = attrlen;
- if (attrlen > (xdr->nwords << 2)) {
- if (res->acl_flags & NFS4_ACL_LEN_REQUEST) {
- /* getxattr interface called with a NULL buf */
- goto out;
- }
+
+ /* Check for receive buffer overflow */
+ if (res->acl_len > (xdr->nwords << 2) ||
+ res->acl_len + res->acl_data_offset > xdr->buf->page_len) {
+ res->acl_flags |= NFS4_ACL_TRUNC;
dprintk("NFS: acl reply: attrlen %u > page_len %u\n",
attrlen, xdr->nwords << 2);
- return -EINVAL;
}
} else
status = -EOPNOTSUPP;
@@ -6229,7 +6225,8 @@
status = decode_open(xdr, res);
if (status)
goto out;
- if (decode_getfh(xdr, &res->fh) != 0)
+ status = decode_getfh(xdr, &res->fh);
+ if (status)
goto out;
decode_getfattr(xdr, res->f_attr, res->server);
out:
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index 239aff7..d2c7f5d 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1537,7 +1537,7 @@
/*
* verify that any proto=/mountproto= options match the address
- * familiies in the addr=/mountaddr= options.
+ * families in the addr=/mountaddr= options.
*/
if (protofamily != AF_UNSPEC &&
protofamily != mnt->nfs_server.address.ss_family)
@@ -1867,6 +1867,7 @@
memcpy(sap, &data->addr, sizeof(data->addr));
args->nfs_server.addrlen = sizeof(data->addr);
+ args->nfs_server.port = ntohs(data->addr.sin_port);
if (!nfs_verify_server_address(sap))
goto out_no_address;
@@ -2564,6 +2565,7 @@
return -EFAULT;
if (!nfs_verify_server_address(sap))
goto out_no_address;
+ args->nfs_server.port = ntohs(((struct sockaddr_in *)sap)->sin_port);
if (data->auth_flavourlen) {
if (data->auth_flavourlen > 1)
diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index cbaf4f8..4c7bd35 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -651,12 +651,12 @@
if (clp->cl_minorversion == 0) {
if (!clp->cl_cred.cr_principal &&
- (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
+ (clp->cl_cred.cr_flavor >= RPC_AUTH_GSS_KRB5))
return -EINVAL;
args.client_name = clp->cl_cred.cr_principal;
args.prognumber = conn->cb_prog,
args.protocol = XPRT_TRANSPORT_TCP;
- args.authflavor = clp->cl_flavor;
+ args.authflavor = clp->cl_cred.cr_flavor;
clp->cl_cb_ident = conn->cb_ident;
} else {
if (!conn->cb_xprt)
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index e617314..22bd0a6 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -231,7 +231,6 @@
nfs4_verifier cl_verifier; /* generated by client */
time_t cl_time; /* time of last lease renewal */
struct sockaddr_storage cl_addr; /* client ipaddress */
- u32 cl_flavor; /* setclientid pseudoflavor */
struct svc_cred cl_cred; /* setclientid principal */
clientid_t cl_clientid; /* generated by server */
nfs4_verifier cl_confirm; /* generated by server */
diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c
index dfafeb2..eb7cc91 100644
--- a/fs/proc/proc_sysctl.c
+++ b/fs/proc/proc_sysctl.c
@@ -462,9 +462,6 @@
err = ERR_PTR(-ENOMEM);
inode = proc_sys_make_inode(dir->i_sb, h ? h : head, p);
- if (h)
- sysctl_head_finish(h);
-
if (!inode)
goto out;
@@ -473,6 +470,8 @@
d_add(dentry, inode);
out:
+ if (h)
+ sysctl_head_finish(h);
sysctl_head_finish(head);
return err;
}
diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
index 36a29b7..c495a30 100644
--- a/fs/quota/dquot.c
+++ b/fs/quota/dquot.c
@@ -1589,10 +1589,10 @@
goto out;
}
- down_read(&sb_dqopt(inode->i_sb)->dqptr_sem);
for (cnt = 0; cnt < MAXQUOTAS; cnt++)
warn[cnt].w_type = QUOTA_NL_NOWARN;
+ down_read(&sb_dqopt(inode->i_sb)->dqptr_sem);
spin_lock(&dq_data_lock);
for (cnt = 0; cnt < MAXQUOTAS; cnt++) {
if (!dquots[cnt])
diff --git a/fs/reiserfs/bitmap.c b/fs/reiserfs/bitmap.c
index 4c0c7d1..a98b774 100644
--- a/fs/reiserfs/bitmap.c
+++ b/fs/reiserfs/bitmap.c
@@ -1334,9 +1334,7 @@
else if (bitmap == 0)
block = (REISERFS_DISK_OFFSET_IN_BYTES >> sb->s_blocksize_bits) + 1;
- reiserfs_write_unlock(sb);
bh = sb_bread(sb, block);
- reiserfs_write_lock(sb);
if (bh == NULL)
reiserfs_warning(sb, "sh-2029: %s: bitmap block (#%u) "
"reading failed", __func__, block);
diff --git a/fs/reiserfs/inode.c b/fs/reiserfs/inode.c
index a6d4268..855da58 100644
--- a/fs/reiserfs/inode.c
+++ b/fs/reiserfs/inode.c
@@ -76,10 +76,10 @@
;
}
out:
+ reiserfs_write_unlock_once(inode->i_sb, depth);
clear_inode(inode); /* note this must go after the journal_end to prevent deadlock */
dquot_drop(inode);
inode->i_blocks = 0;
- reiserfs_write_unlock_once(inode->i_sb, depth);
return;
no_delete:
diff --git a/fs/stat.c b/fs/stat.c
index b6ff118..208039e 100644
--- a/fs/stat.c
+++ b/fs/stat.c
@@ -58,7 +58,7 @@
int vfs_fstat(unsigned int fd, struct kstat *stat)
{
int fput_needed;
- struct file *f = fget_light(fd, &fput_needed);
+ struct file *f = fget_raw_light(fd, &fput_needed);
int error = -EBADF;
if (f) {
@@ -326,7 +326,7 @@
/* ---------- LFS-64 ----------- */
-#ifdef __ARCH_WANT_STAT64
+#if defined(__ARCH_WANT_STAT64) || defined(__ARCH_WANT_COMPAT_STAT64)
#ifndef INIT_STRUCT_STAT64_PADDING
# define INIT_STRUCT_STAT64_PADDING(st) memset(&st, 0, sizeof(st))
@@ -415,7 +415,7 @@
return error;
return cp_new_stat64(&stat, statbuf);
}
-#endif /* __ARCH_WANT_STAT64 */
+#endif /* __ARCH_WANT_STAT64 || __ARCH_WANT_COMPAT_STAT64 */
/* Caller is here responsible for sufficient locking (ie. inode->i_lock) */
void __inode_add_bytes(struct inode *inode, loff_t bytes)
diff --git a/fs/ubifs/debug.h b/fs/ubifs/debug.h
index 8b8cc4e..760de72 100644
--- a/fs/ubifs/debug.h
+++ b/fs/ubifs/debug.h
@@ -167,7 +167,7 @@
#define ubifs_dbg_msg(type, fmt, ...) \
pr_debug("UBIFS DBG " type ": " fmt "\n", ##__VA_ARGS__)
-#define DBG_KEY_BUF_LEN 32
+#define DBG_KEY_BUF_LEN 48
#define ubifs_dbg_msg_key(type, key, fmt, ...) do { \
char __tmp_key_buf[DBG_KEY_BUF_LEN]; \
pr_debug("UBIFS DBG " type ": " fmt "%s\n", ##__VA_ARGS__, \
diff --git a/fs/ubifs/lpt.c b/fs/ubifs/lpt.c
index ce33b2b..8640920 100644
--- a/fs/ubifs/lpt.c
+++ b/fs/ubifs/lpt.c
@@ -1749,7 +1749,10 @@
return 0;
out_err:
- ubifs_lpt_free(c, 0);
+ if (wr)
+ ubifs_lpt_free(c, 1);
+ if (rd)
+ ubifs_lpt_free(c, 0);
return err;
}
diff --git a/fs/ubifs/recovery.c b/fs/ubifs/recovery.c
index c30d976b..edeec49 100644
--- a/fs/ubifs/recovery.c
+++ b/fs/ubifs/recovery.c
@@ -788,7 +788,7 @@
corrupted_rescan:
/* Re-scan the corrupted data with verbose messages */
- ubifs_err("corruptio %d", ret);
+ ubifs_err("corruption %d", ret);
ubifs_scan_a_node(c, buf, len, lnum, offs, 1);
corrupted:
ubifs_scanned_corruption(c, lnum, offs, buf);
diff --git a/fs/ubifs/replay.c b/fs/ubifs/replay.c
index eba46d4..94d78fc 100644
--- a/fs/ubifs/replay.c
+++ b/fs/ubifs/replay.c
@@ -1026,7 +1026,6 @@
c->replaying = 1;
lnum = c->ltail_lnum = c->lhead_lnum;
- lnum = UBIFS_LOG_LNUM;
do {
err = replay_log_leb(c, lnum, 0, c->sbuf);
if (err == 1)
@@ -1035,7 +1034,7 @@
if (err)
goto out;
lnum = ubifs_next_log_lnum(c, lnum);
- } while (lnum != UBIFS_LOG_LNUM);
+ } while (lnum != c->ltail_lnum);
err = replay_buds(c);
if (err)
diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c
index c3fa6c5..71a197f 100644
--- a/fs/ubifs/super.c
+++ b/fs/ubifs/super.c
@@ -1157,9 +1157,6 @@
*
* This function mounts UBIFS file system. Returns zero in case of success and
* a negative error code in case of failure.
- *
- * Note, the function does not de-allocate resources it it fails half way
- * through, and the caller has to do this instead.
*/
static int mount_ubifs(struct ubifs_info *c)
{
diff --git a/fs/udf/file.c b/fs/udf/file.c
index 7f3f7ba..d1c6093 100644
--- a/fs/udf/file.c
+++ b/fs/udf/file.c
@@ -39,20 +39,24 @@
#include "udf_i.h"
#include "udf_sb.h"
-static int udf_adinicb_readpage(struct file *file, struct page *page)
+static void __udf_adinicb_readpage(struct page *page)
{
struct inode *inode = page->mapping->host;
char *kaddr;
struct udf_inode_info *iinfo = UDF_I(inode);
- BUG_ON(!PageLocked(page));
-
kaddr = kmap(page);
- memset(kaddr, 0, PAGE_CACHE_SIZE);
memcpy(kaddr, iinfo->i_ext.i_data + iinfo->i_lenEAttr, inode->i_size);
+ memset(kaddr + inode->i_size, 0, PAGE_CACHE_SIZE - inode->i_size);
flush_dcache_page(page);
SetPageUptodate(page);
kunmap(page);
+}
+
+static int udf_adinicb_readpage(struct file *file, struct page *page)
+{
+ BUG_ON(!PageLocked(page));
+ __udf_adinicb_readpage(page);
unlock_page(page);
return 0;
@@ -77,6 +81,25 @@
return 0;
}
+static int udf_adinicb_write_begin(struct file *file,
+ struct address_space *mapping, loff_t pos,
+ unsigned len, unsigned flags, struct page **pagep,
+ void **fsdata)
+{
+ struct page *page;
+
+ if (WARN_ON_ONCE(pos >= PAGE_CACHE_SIZE))
+ return -EIO;
+ page = grab_cache_page_write_begin(mapping, 0, flags);
+ if (!page)
+ return -ENOMEM;
+ *pagep = page;
+
+ if (!PageUptodate(page) && len != PAGE_CACHE_SIZE)
+ __udf_adinicb_readpage(page);
+ return 0;
+}
+
static int udf_adinicb_write_end(struct file *file,
struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied,
@@ -98,8 +121,8 @@
const struct address_space_operations udf_adinicb_aops = {
.readpage = udf_adinicb_readpage,
.writepage = udf_adinicb_writepage,
- .write_begin = simple_write_begin,
- .write_end = udf_adinicb_write_end,
+ .write_begin = udf_adinicb_write_begin,
+ .write_end = udf_adinicb_write_end,
};
static ssize_t udf_file_aio_write(struct kiocb *iocb, const struct iovec *iov,
diff --git a/fs/udf/inode.c b/fs/udf/inode.c
index fafaad7..aa23346 100644
--- a/fs/udf/inode.c
+++ b/fs/udf/inode.c
@@ -1124,14 +1124,17 @@
if (err)
return err;
down_write(&iinfo->i_data_sem);
- } else
+ } else {
iinfo->i_lenAlloc = newsize;
+ goto set_size;
+ }
}
err = udf_extend_file(inode, newsize);
if (err) {
up_write(&iinfo->i_data_sem);
return err;
}
+set_size:
truncate_setsize(inode, newsize);
up_write(&iinfo->i_data_sem);
} else {
diff --git a/fs/udf/super.c b/fs/udf/super.c
index dcbf987..18fc038 100644
--- a/fs/udf/super.c
+++ b/fs/udf/super.c
@@ -1344,6 +1344,7 @@
udf_err(sb, "error loading logical volume descriptor: "
"Partition table too long (%u > %lu)\n", table_len,
sb->s_blocksize - sizeof(*lvd));
+ ret = 1;
goto out_bh;
}
@@ -1388,8 +1389,10 @@
UDF_ID_SPARABLE,
strlen(UDF_ID_SPARABLE))) {
if (udf_load_sparable_map(sb, map,
- (struct sparablePartitionMap *)gpm) < 0)
+ (struct sparablePartitionMap *)gpm) < 0) {
+ ret = 1;
goto out_bh;
+ }
} else if (!strncmp(upm2->partIdent.ident,
UDF_ID_METADATA,
strlen(UDF_ID_METADATA))) {
@@ -2000,6 +2003,8 @@
if (!silent)
pr_notice("Rescanning with blocksize %d\n",
UDF_DEFAULT_BLOCKSIZE);
+ brelse(sbi->s_lvid_bh);
+ sbi->s_lvid_bh = NULL;
uopt.blocksize = UDF_DEFAULT_BLOCKSIZE;
ret = udf_load_vrs(sb, &uopt, silent, &fileset);
}
diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c
index d7a9dd7..933b793 100644
--- a/fs/xfs/xfs_buf.c
+++ b/fs/xfs/xfs_buf.c
@@ -96,6 +96,7 @@
atomic_inc(&bp->b_hold);
list_add_tail(&bp->b_lru, &btp->bt_lru);
btp->bt_lru_nr++;
+ bp->b_lru_flags &= ~_XBF_LRU_DISPOSE;
}
spin_unlock(&btp->bt_lru_lock);
}
@@ -154,7 +155,8 @@
struct xfs_buftarg *btp = bp->b_target;
spin_lock(&btp->bt_lru_lock);
- if (!list_empty(&bp->b_lru)) {
+ if (!list_empty(&bp->b_lru) &&
+ !(bp->b_lru_flags & _XBF_LRU_DISPOSE)) {
list_del_init(&bp->b_lru);
btp->bt_lru_nr--;
atomic_dec(&bp->b_hold);
@@ -1501,6 +1503,7 @@
*/
list_move(&bp->b_lru, &dispose);
btp->bt_lru_nr--;
+ bp->b_lru_flags |= _XBF_LRU_DISPOSE;
}
spin_unlock(&btp->bt_lru_lock);
diff --git a/fs/xfs/xfs_buf.h b/fs/xfs/xfs_buf.h
index d03b73b..7c0b6a0 100644
--- a/fs/xfs/xfs_buf.h
+++ b/fs/xfs/xfs_buf.h
@@ -38,27 +38,28 @@
XBRW_ZERO = 3, /* Zero target memory */
} xfs_buf_rw_t;
-#define XBF_READ (1 << 0) /* buffer intended for reading from device */
-#define XBF_WRITE (1 << 1) /* buffer intended for writing to device */
-#define XBF_READ_AHEAD (1 << 2) /* asynchronous read-ahead */
-#define XBF_ASYNC (1 << 4) /* initiator will not wait for completion */
-#define XBF_DONE (1 << 5) /* all pages in the buffer uptodate */
-#define XBF_STALE (1 << 6) /* buffer has been staled, do not find it */
+#define XBF_READ (1 << 0) /* buffer intended for reading from device */
+#define XBF_WRITE (1 << 1) /* buffer intended for writing to device */
+#define XBF_READ_AHEAD (1 << 2) /* asynchronous read-ahead */
+#define XBF_ASYNC (1 << 4) /* initiator will not wait for completion */
+#define XBF_DONE (1 << 5) /* all pages in the buffer uptodate */
+#define XBF_STALE (1 << 6) /* buffer has been staled, do not find it */
/* I/O hints for the BIO layer */
-#define XBF_SYNCIO (1 << 10)/* treat this buffer as synchronous I/O */
-#define XBF_FUA (1 << 11)/* force cache write through mode */
-#define XBF_FLUSH (1 << 12)/* flush the disk cache before a write */
+#define XBF_SYNCIO (1 << 10)/* treat this buffer as synchronous I/O */
+#define XBF_FUA (1 << 11)/* force cache write through mode */
+#define XBF_FLUSH (1 << 12)/* flush the disk cache before a write */
/* flags used only as arguments to access routines */
-#define XBF_TRYLOCK (1 << 16)/* lock requested, but do not wait */
-#define XBF_UNMAPPED (1 << 17)/* do not map the buffer */
+#define XBF_TRYLOCK (1 << 16)/* lock requested, but do not wait */
+#define XBF_UNMAPPED (1 << 17)/* do not map the buffer */
/* flags used only internally */
-#define _XBF_PAGES (1 << 20)/* backed by refcounted pages */
-#define _XBF_KMEM (1 << 21)/* backed by heap memory */
-#define _XBF_DELWRI_Q (1 << 22)/* buffer on a delwri queue */
-#define _XBF_COMPOUND (1 << 23)/* compound buffer */
+#define _XBF_PAGES (1 << 20)/* backed by refcounted pages */
+#define _XBF_KMEM (1 << 21)/* backed by heap memory */
+#define _XBF_DELWRI_Q (1 << 22)/* buffer on a delwri queue */
+#define _XBF_COMPOUND (1 << 23)/* compound buffer */
+#define _XBF_LRU_DISPOSE (1 << 24)/* buffer being discarded */
typedef unsigned int xfs_buf_flags_t;
@@ -72,12 +73,13 @@
{ XBF_SYNCIO, "SYNCIO" }, \
{ XBF_FUA, "FUA" }, \
{ XBF_FLUSH, "FLUSH" }, \
- { XBF_TRYLOCK, "TRYLOCK" }, /* should never be set */\
+ { XBF_TRYLOCK, "TRYLOCK" }, /* should never be set */\
{ XBF_UNMAPPED, "UNMAPPED" }, /* ditto */\
{ _XBF_PAGES, "PAGES" }, \
{ _XBF_KMEM, "KMEM" }, \
{ _XBF_DELWRI_Q, "DELWRI_Q" }, \
- { _XBF_COMPOUND, "COMPOUND" }
+ { _XBF_COMPOUND, "COMPOUND" }, \
+ { _XBF_LRU_DISPOSE, "LRU_DISPOSE" }
typedef struct xfs_buftarg {
dev_t bt_dev;
@@ -124,7 +126,12 @@
xfs_buf_flags_t b_flags; /* status flags */
struct semaphore b_sema; /* semaphore for lockables */
+ /*
+ * concurrent access to b_lru and b_lru_flags are protected by
+ * bt_lru_lock and not by b_sema
+ */
struct list_head b_lru; /* lru list */
+ xfs_buf_flags_t b_lru_flags; /* internal lru status flags */
wait_queue_head_t b_waiters; /* unpin waiters */
struct list_head b_list;
struct xfs_perag *b_pag; /* contains rbtree root */
diff --git a/fs/xfs/xfs_discard.c b/fs/xfs/xfs_discard.c
index f9c3fe3..69cf4fc 100644
--- a/fs/xfs/xfs_discard.c
+++ b/fs/xfs/xfs_discard.c
@@ -179,12 +179,14 @@
* used by the fstrim application. In the end it really doesn't
* matter as trimming blocks is an advisory interface.
*/
+ if (range.start >= XFS_FSB_TO_B(mp, mp->m_sb.sb_dblocks) ||
+ range.minlen > XFS_FSB_TO_B(mp, XFS_ALLOC_AG_MAX_USABLE(mp)))
+ return -XFS_ERROR(EINVAL);
+
start = BTOBB(range.start);
end = start + BTOBBT(range.len) - 1;
minlen = BTOBB(max_t(u64, granularity, range.minlen));
- if (XFS_BB_TO_FSB(mp, start) >= mp->m_sb.sb_dblocks)
- return -XFS_ERROR(EINVAL);
if (end > XFS_FSB_TO_BB(mp, mp->m_sb.sb_dblocks) - 1)
end = XFS_FSB_TO_BB(mp, mp->m_sb.sb_dblocks)- 1;
diff --git a/fs/xfs/xfs_ialloc.c b/fs/xfs/xfs_ialloc.c
index 21e37b5..5aceb3f 100644
--- a/fs/xfs/xfs_ialloc.c
+++ b/fs/xfs/xfs_ialloc.c
@@ -962,23 +962,22 @@
if (!pag->pagi_freecount && !okalloc)
goto nextag;
+ /*
+ * Then read in the AGI buffer and recheck with the AGI buffer
+ * lock held.
+ */
error = xfs_ialloc_read_agi(mp, tp, agno, &agbp);
if (error)
goto out_error;
- /*
- * Once the AGI has been read in we have to recheck
- * pagi_freecount with the AGI buffer lock held.
- */
if (pag->pagi_freecount) {
xfs_perag_put(pag);
goto out_alloc;
}
- if (!okalloc) {
- xfs_trans_brelse(tp, agbp);
- goto nextag;
- }
+ if (!okalloc)
+ goto nextag_relse_buffer;
+
error = xfs_ialloc_ag_alloc(tp, agbp, &ialloced);
if (error) {
@@ -1007,6 +1006,8 @@
return 0;
}
+nextag_relse_buffer:
+ xfs_trans_brelse(tp, agbp);
nextag:
xfs_perag_put(pag);
if (++agno == mp->m_sb.sb_agcount)
diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c
index 92d4331..ca28a4b 100644
--- a/fs/xfs/xfs_rtalloc.c
+++ b/fs/xfs/xfs_rtalloc.c
@@ -857,7 +857,7 @@
xfs_buf_t *bp; /* block buffer, result */
xfs_inode_t *ip; /* bitmap or summary inode */
xfs_bmbt_irec_t map;
- int nmap;
+ int nmap = 1;
int error; /* error value */
ip = issum ? mp->m_rsumip : mp->m_rbmip;
diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
index bdaf4cb..19e2380 100644
--- a/fs/xfs/xfs_super.c
+++ b/fs/xfs/xfs_super.c
@@ -919,6 +919,7 @@
struct xfs_mount *mp = XFS_M(sb);
xfs_filestream_unmount(mp);
+ cancel_delayed_work_sync(&mp->m_sync_work);
xfs_unmountfs(mp);
xfs_syncd_stop(mp);
xfs_freesb(mp);
diff --git a/include/asm-generic/bitops/builtin-__ffs.h b/include/asm-generic/bitops/builtin-__ffs.h
new file mode 100644
index 0000000..90041e3
--- /dev/null
+++ b/include/asm-generic/bitops/builtin-__ffs.h
@@ -0,0 +1,15 @@
+#ifndef _ASM_GENERIC_BITOPS_BUILTIN___FFS_H_
+#define _ASM_GENERIC_BITOPS_BUILTIN___FFS_H_
+
+/**
+ * __ffs - find first bit in word.
+ * @word: The word to search
+ *
+ * Undefined if no bit exists, so code should check against 0 first.
+ */
+static __always_inline unsigned long __ffs(unsigned long word)
+{
+ return __builtin_ctzl(word);
+}
+
+#endif
diff --git a/include/asm-generic/bitops/builtin-__fls.h b/include/asm-generic/bitops/builtin-__fls.h
new file mode 100644
index 0000000..0248f38
--- /dev/null
+++ b/include/asm-generic/bitops/builtin-__fls.h
@@ -0,0 +1,15 @@
+#ifndef _ASM_GENERIC_BITOPS_BUILTIN___FLS_H_
+#define _ASM_GENERIC_BITOPS_BUILTIN___FLS_H_
+
+/**
+ * __fls - find last (most-significant) set bit in a long word
+ * @word: the word to search
+ *
+ * Undefined if no set bit exists, so code should check against 0 first.
+ */
+static __always_inline unsigned long __fls(unsigned long word)
+{
+ return (sizeof(word) * 8) - 1 - __builtin_clzl(word);
+}
+
+#endif
diff --git a/include/asm-generic/bitops/builtin-ffs.h b/include/asm-generic/bitops/builtin-ffs.h
new file mode 100644
index 0000000..0648258
--- /dev/null
+++ b/include/asm-generic/bitops/builtin-ffs.h
@@ -0,0 +1,17 @@
+#ifndef _ASM_GENERIC_BITOPS_BUILTIN_FFS_H_
+#define _ASM_GENERIC_BITOPS_BUILTIN_FFS_H_
+
+/**
+ * ffs - find first bit set
+ * @x: the word to search
+ *
+ * This is defined the same way as
+ * the libc and compiler builtin ffs routines, therefore
+ * differs in spirit from the above ffz (man ffs).
+ */
+static __always_inline int ffs(int x)
+{
+ return __builtin_ffs(x);
+}
+
+#endif
diff --git a/include/asm-generic/bitops/builtin-fls.h b/include/asm-generic/bitops/builtin-fls.h
new file mode 100644
index 0000000..eda652d
--- /dev/null
+++ b/include/asm-generic/bitops/builtin-fls.h
@@ -0,0 +1,16 @@
+#ifndef _ASM_GENERIC_BITOPS_BUILTIN_FLS_H_
+#define _ASM_GENERIC_BITOPS_BUILTIN_FLS_H_
+
+/**
+ * fls - find last (most-significant) bit set
+ * @x: the word to search
+ *
+ * This is defined the same way as ffs.
+ * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32.
+ */
+static __always_inline int fls(int x)
+{
+ return x ? sizeof(x) * 8 - __builtin_clz(x) : 0;
+}
+
+#endif
diff --git a/include/asm-generic/unistd.h b/include/asm-generic/unistd.h
index 991ef01..3748ec9 100644
--- a/include/asm-generic/unistd.h
+++ b/include/asm-generic/unistd.h
@@ -691,9 +691,11 @@
#define __NR_process_vm_writev 271
__SC_COMP(__NR_process_vm_writev, sys_process_vm_writev, \
compat_sys_process_vm_writev)
+#define __NR_kcmp 272
+__SYSCALL(__NR_kcmp, sys_kcmp)
#undef __NR_syscalls
-#define __NR_syscalls 272
+#define __NR_syscalls 273
/*
* All syscalls below here should go away really,
diff --git a/include/clocksource/arm_generic.h b/include/clocksource/arm_generic.h
new file mode 100644
index 0000000..5b41b0d
--- /dev/null
+++ b/include/clocksource/arm_generic.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (C) 2012 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __CLKSOURCE_ARM_GENERIC_H
+#define __CLKSOURCE_ARM_GENERIC_H
+
+extern int arm_generic_timer_init(void);
+
+#endif
diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h
index ced3625..bfacf0d 100644
--- a/include/drm/drm_crtc.h
+++ b/include/drm/drm_crtc.h
@@ -118,7 +118,8 @@
.hdisplay = (hd), .hsync_start = (hss), .hsync_end = (hse), \
.htotal = (ht), .hskew = (hsk), .vdisplay = (vd), \
.vsync_start = (vss), .vsync_end = (vse), .vtotal = (vt), \
- .vscan = (vs), .flags = (f), .vrefresh = 0
+ .vscan = (vs), .flags = (f), .vrefresh = 0, \
+ .base.type = DRM_MODE_OBJECT_MODE
#define CRTC_INTERLACE_HALVE_V 0x1 /* halve V values for interlacing */
diff --git a/include/drm/drm_fourcc.h b/include/drm/drm_fourcc.h
index bdf0152..f462118 100644
--- a/include/drm/drm_fourcc.h
+++ b/include/drm/drm_fourcc.h
@@ -107,8 +107,7 @@
#define DRM_FORMAT_NV16 fourcc_code('N', 'V', '1', '6') /* 2x1 subsampled Cr:Cb plane */
#define DRM_FORMAT_NV61 fourcc_code('N', 'V', '6', '1') /* 2x1 subsampled Cb:Cr plane */
-/* 2 non contiguous plane YCbCr */
-#define DRM_FORMAT_NV12M fourcc_code('N', 'M', '1', '2') /* 2x2 subsampled Cr:Cb plane */
+/* special NV12 tiled format */
#define DRM_FORMAT_NV12MT fourcc_code('T', 'M', '1', '2') /* 2x2 subsampled Cr:Cb plane 64x32 macroblocks */
/*
@@ -131,7 +130,4 @@
#define DRM_FORMAT_YUV444 fourcc_code('Y', 'U', '2', '4') /* non-subsampled Cb (1) and Cr (2) planes */
#define DRM_FORMAT_YVU444 fourcc_code('Y', 'V', '2', '4') /* non-subsampled Cr (1) and Cb (2) planes */
-/* 3 non contiguous plane YCbCr */
-#define DRM_FORMAT_YUV420M fourcc_code('Y', 'M', '1', '2') /* 2x2 subsampled Cb (1) and Cr (2) planes */
-
#endif /* DRM_FOURCC_H */
diff --git a/include/drm/drm_mode.h b/include/drm/drm_mode.h
index 5581980..3d6301b 100644
--- a/include/drm/drm_mode.h
+++ b/include/drm/drm_mode.h
@@ -359,8 +359,9 @@
struct drm_mode_modeinfo mode;
};
-#define DRM_MODE_CURSOR_BO (1<<0)
-#define DRM_MODE_CURSOR_MOVE (1<<1)
+#define DRM_MODE_CURSOR_BO 0x01
+#define DRM_MODE_CURSOR_MOVE 0x02
+#define DRM_MODE_CURSOR_FLAGS 0x03
/*
* depending on the value in flags different members are used.
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index 4f2a762..90be989 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -138,9 +138,9 @@
void acpi_pci_irq_disable (struct pci_dev *dev);
struct acpi_pci_driver {
- struct acpi_pci_driver *next;
- int (*add)(acpi_handle handle);
- void (*remove)(acpi_handle handle);
+ struct list_head node;
+ int (*add)(struct acpi_pci_root *root);
+ void (*remove)(struct acpi_pci_root *root);
};
int acpi_pci_register_driver(struct acpi_pci_driver *driver);
diff --git a/include/linux/atmel-ssc.h b/include/linux/atmel-ssc.h
index 0602339..4eb3175 100644
--- a/include/linux/atmel-ssc.h
+++ b/include/linux/atmel-ssc.h
@@ -3,6 +3,7 @@
#include <linux/platform_device.h>
#include <linux/list.h>
+#include <linux/io.h>
struct ssc_device {
struct list_head list;
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 4e72a9d..4a2ab7c 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -601,7 +601,7 @@
* it already be started by driver.
*/
#define RQ_NOMERGE_FLAGS \
- (REQ_NOMERGE | REQ_STARTED | REQ_SOFTBARRIER | REQ_FLUSH | REQ_FUA)
+ (REQ_NOMERGE | REQ_STARTED | REQ_SOFTBARRIER | REQ_FLUSH | REQ_FUA | REQ_DISCARD)
#define rq_mergeable(rq) \
(!((rq)->cmd_flags & RQ_NOMERGE_FLAGS) && \
(((rq)->cmd_flags & REQ_DISCARD) || \
@@ -894,6 +894,8 @@
extern struct backing_dev_info *blk_get_backing_dev_info(struct block_device *bdev);
extern int blk_rq_map_sg(struct request_queue *, struct request *, struct scatterlist *);
+extern int blk_bio_map_sg(struct request_queue *q, struct bio *bio,
+ struct scatterlist *sglist);
extern void blk_dump_rq_flags(struct request *, char *);
extern long nr_blockdev_pages(void);
@@ -1139,6 +1141,16 @@
& (lim->discard_granularity - 1);
}
+static inline int bdev_discard_alignment(struct block_device *bdev)
+{
+ struct request_queue *q = bdev_get_queue(bdev);
+
+ if (bdev != bdev->bd_contains)
+ return bdev->bd_part->discard_alignment;
+
+ return q->limits.discard_alignment;
+}
+
static inline unsigned int queue_discard_zeroes_data(struct request_queue *q)
{
if (q->limits.max_discard_sectors && q->limits.discard_zeroes_data == 1)
diff --git a/include/linux/clk-provider.h b/include/linux/clk-provider.h
index 77335fa..c127315 100644
--- a/include/linux/clk-provider.h
+++ b/include/linux/clk-provider.h
@@ -26,6 +26,7 @@
#define CLK_IGNORE_UNUSED BIT(3) /* do not gate even if unused */
#define CLK_IS_ROOT BIT(4) /* root clk, has no parent */
#define CLK_IS_BASIC BIT(5) /* Basic clk, can't do a to_clk_foo() */
+#define CLK_GET_RATE_NOCACHE BIT(6) /* do not use the cached clk rate */
struct clk_hw;
@@ -360,6 +361,11 @@
void of_clk_del_provider(struct device_node *np);
struct clk *of_clk_src_simple_get(struct of_phandle_args *clkspec,
void *data);
+struct clk_onecell_data {
+ struct clk **clks;
+ unsigned int clk_num;
+};
+struct clk *of_clk_src_onecell_get(struct of_phandle_args *clkspec, void *data);
const char *of_clk_get_parent_name(struct device_node *np, int index);
void of_clk_init(const struct of_device_id *matches);
diff --git a/include/linux/compiler-gcc4.h b/include/linux/compiler-gcc4.h
index 2f40791..934bc34 100644
--- a/include/linux/compiler-gcc4.h
+++ b/include/linux/compiler-gcc4.h
@@ -49,6 +49,13 @@
#endif
#endif
+#if __GNUC_MINOR__ >= 6
+/*
+ * Tell the optimizer that something else uses this function or variable.
+ */
+#define __visible __attribute__((externally_visible))
+#endif
+
#if __GNUC_MINOR__ > 0
#define __compiletime_object_size(obj) __builtin_object_size(obj, 0)
#endif
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 923d093..f430e41 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -278,6 +278,10 @@
# define __section(S) __attribute__ ((__section__(#S)))
#endif
+#ifndef __visible
+#define __visible
+#endif
+
/* Are two types/vars the same type (ignoring qualifiers)? */
#ifndef __same_type
# define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h
index 040b13b..279b1ea 100644
--- a/include/linux/cpuidle.h
+++ b/include/linux/cpuidle.h
@@ -194,6 +194,10 @@
#ifdef CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED
void cpuidle_coupled_parallel_barrier(struct cpuidle_device *dev, atomic_t *a);
+#else
+static inline void cpuidle_coupled_parallel_barrier(struct cpuidle_device *dev, atomic_t *a)
+{
+}
#endif
/******************************
diff --git a/include/linux/dcache.h b/include/linux/dcache.h
index caa34e5..5920079 100644
--- a/include/linux/dcache.h
+++ b/include/linux/dcache.h
@@ -206,6 +206,8 @@
#define DCACHE_MANAGED_DENTRY \
(DCACHE_MOUNTED|DCACHE_NEED_AUTOMOUNT|DCACHE_MANAGE_TRANSIT)
+#define DCACHE_DENTRY_KILLED 0x100000
+
extern seqlock_t rename_lock;
static inline int dname_external(struct dentry *dentry)
diff --git a/include/linux/efi-bgrt.h b/include/linux/efi-bgrt.h
new file mode 100644
index 0000000..051b21f
--- /dev/null
+++ b/include/linux/efi-bgrt.h
@@ -0,0 +1,21 @@
+#ifndef _LINUX_EFI_BGRT_H
+#define _LINUX_EFI_BGRT_H
+
+#ifdef CONFIG_ACPI_BGRT
+
+#include <linux/acpi.h>
+
+void efi_bgrt_init(void);
+
+/* The BGRT data itself; only valid if bgrt_image != NULL. */
+extern void *bgrt_image;
+extern size_t bgrt_image_size;
+extern struct acpi_table_bgrt *bgrt_tab;
+
+#else /* !CONFIG_ACPI_BGRT */
+
+static inline void efi_bgrt_init(void) {}
+
+#endif /* !CONFIG_ACPI_BGRT */
+
+#endif /* _LINUX_EFI_BGRT_H */
diff --git a/include/linux/efi.h b/include/linux/efi.h
index ec45ccd..8670eb1 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -496,6 +496,14 @@
extern void efi_memmap_walk (efi_freemem_callback_t callback, void *arg);
extern void efi_gettimeofday (struct timespec *ts);
extern void efi_enter_virtual_mode (void); /* switch EFI to virtual mode, if possible */
+#ifdef CONFIG_X86
+extern void efi_late_init(void);
+extern void efi_free_boot_services(void);
+#else
+static inline void efi_late_init(void) {}
+static inline void efi_free_boot_services(void) {}
+#endif
+extern void __iomem *efi_lookup_mapped_addr(u64 phys_addr);
extern u64 efi_get_iobase (void);
extern u32 efi_mem_type (unsigned long phys_addr);
extern u64 efi_mem_attributes (unsigned long phys_addr);
diff --git a/include/linux/elf.h b/include/linux/elf.h
index 999b4f5..0a05051 100644
--- a/include/linux/elf.h
+++ b/include/linux/elf.h
@@ -387,7 +387,11 @@
#define NT_S390_PREFIX 0x305 /* s390 prefix register */
#define NT_S390_LAST_BREAK 0x306 /* s390 breaking event address */
#define NT_S390_SYSTEM_CALL 0x307 /* s390 system call restart data */
+#define NT_S390_TDB 0x308 /* s390 transaction diagnostic block */
#define NT_ARM_VFP 0x400 /* ARM VFP/NEON registers */
+#define NT_ARM_TLS 0x401 /* ARM TLS register */
+#define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
+#define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
/* Note header in a PT_NOTE section */
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 55e6d63..a52f2f4 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -10,6 +10,7 @@
#include <linux/kallsyms.h>
#include <linux/linkage.h>
#include <linux/bitops.h>
+#include <linux/ptrace.h>
#include <linux/ktime.h>
#include <linux/sched.h>
#include <linux/types.h>
@@ -18,6 +19,28 @@
#include <asm/ftrace.h>
+/*
+ * If the arch supports passing the variable contents of
+ * function_trace_op as the third parameter back from the
+ * mcount call, then the arch should define this as 1.
+ */
+#ifndef ARCH_SUPPORTS_FTRACE_OPS
+#define ARCH_SUPPORTS_FTRACE_OPS 0
+#endif
+
+/*
+ * If the arch's mcount caller does not support all of ftrace's
+ * features, then it must call an indirect function that
+ * does. Or at least does enough to prevent any unwelcomed side effects.
+ */
+#if !defined(CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST) || \
+ !ARCH_SUPPORTS_FTRACE_OPS
+# define FTRACE_FORCE_LIST_FUNC 1
+#else
+# define FTRACE_FORCE_LIST_FUNC 0
+#endif
+
+
struct module;
struct ftrace_hash;
@@ -29,7 +52,10 @@
void __user *buffer, size_t *lenp,
loff_t *ppos);
-typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip);
+struct ftrace_ops;
+
+typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *regs);
/*
* FTRACE_OPS_FL_* bits denote the state of ftrace_ops struct and are
@@ -45,12 +71,33 @@
* could be controled by following calls:
* ftrace_function_local_enable
* ftrace_function_local_disable
+ * SAVE_REGS - The ftrace_ops wants regs saved at each function called
+ * and passed to the callback. If this flag is set, but the
+ * architecture does not support passing regs
+ * (ARCH_SUPPORTS_FTRACE_SAVE_REGS is not defined), then the
+ * ftrace_ops will fail to register, unless the next flag
+ * is set.
+ * SAVE_REGS_IF_SUPPORTED - This is the same as SAVE_REGS, but if the
+ * handler can handle an arch that does not save regs
+ * (the handler tests if regs == NULL), then it can set
+ * this flag instead. It will not fail registering the ftrace_ops
+ * but, the regs field will be NULL if the arch does not support
+ * passing regs to the handler.
+ * Note, if this flag is set, the SAVE_REGS flag will automatically
+ * get set upon registering the ftrace_ops, if the arch supports it.
+ * RECURSION_SAFE - The ftrace_ops can set this to tell the ftrace infrastructure
+ * that the call back has its own recursion protection. If it does
+ * not set this, then the ftrace infrastructure will add recursion
+ * protection for the caller.
*/
enum {
- FTRACE_OPS_FL_ENABLED = 1 << 0,
- FTRACE_OPS_FL_GLOBAL = 1 << 1,
- FTRACE_OPS_FL_DYNAMIC = 1 << 2,
- FTRACE_OPS_FL_CONTROL = 1 << 3,
+ FTRACE_OPS_FL_ENABLED = 1 << 0,
+ FTRACE_OPS_FL_GLOBAL = 1 << 1,
+ FTRACE_OPS_FL_DYNAMIC = 1 << 2,
+ FTRACE_OPS_FL_CONTROL = 1 << 3,
+ FTRACE_OPS_FL_SAVE_REGS = 1 << 4,
+ FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED = 1 << 5,
+ FTRACE_OPS_FL_RECURSION_SAFE = 1 << 6,
};
struct ftrace_ops {
@@ -163,7 +210,8 @@
return *this_cpu_ptr(ops->disabled);
}
-extern void ftrace_stub(unsigned long a0, unsigned long a1);
+extern void ftrace_stub(unsigned long a0, unsigned long a1,
+ struct ftrace_ops *op, struct pt_regs *regs);
#else /* !CONFIG_FUNCTION_TRACER */
/*
@@ -172,6 +220,10 @@
*/
#define register_ftrace_function(ops) ({ 0; })
#define unregister_ftrace_function(ops) ({ 0; })
+static inline int ftrace_nr_registered_ops(void)
+{
+ return 0;
+}
static inline void clear_ftrace_function(void) { }
static inline void ftrace_kill(void) { }
static inline void ftrace_stop(void) { }
@@ -227,12 +279,33 @@
extern int ftrace_text_reserved(void *start, void *end);
+extern int ftrace_nr_registered_ops(void);
+
+/*
+ * The dyn_ftrace record's flags field is split into two parts.
+ * the first part which is '0-FTRACE_REF_MAX' is a counter of
+ * the number of callbacks that have registered the function that
+ * the dyn_ftrace descriptor represents.
+ *
+ * The second part is a mask:
+ * ENABLED - the function is being traced
+ * REGS - the record wants the function to save regs
+ * REGS_EN - the function is set up to save regs.
+ *
+ * When a new ftrace_ops is registered and wants a function to save
+ * pt_regs, the rec->flag REGS is set. When the function has been
+ * set up to save regs, the REG_EN flag is set. Once a function
+ * starts saving regs it will do so until all ftrace_ops are removed
+ * from tracing that function.
+ */
enum {
- FTRACE_FL_ENABLED = (1 << 30),
+ FTRACE_FL_ENABLED = (1UL << 29),
+ FTRACE_FL_REGS = (1UL << 30),
+ FTRACE_FL_REGS_EN = (1UL << 31)
};
-#define FTRACE_FL_MASK (0x3UL << 30)
-#define FTRACE_REF_MAX ((1 << 30) - 1)
+#define FTRACE_FL_MASK (0x7UL << 29)
+#define FTRACE_REF_MAX ((1UL << 29) - 1)
struct dyn_ftrace {
union {
@@ -244,6 +317,8 @@
};
int ftrace_force_update(void);
+int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
+ int remove, int reset);
int ftrace_set_filter(struct ftrace_ops *ops, unsigned char *buf,
int len, int reset);
int ftrace_set_notrace(struct ftrace_ops *ops, unsigned char *buf,
@@ -263,9 +338,23 @@
FTRACE_STOP_FUNC_RET = (1 << 4),
};
+/*
+ * The FTRACE_UPDATE_* enum is used to pass information back
+ * from the ftrace_update_record() and ftrace_test_record()
+ * functions. These are called by the code update routines
+ * to find out what is to be done for a given function.
+ *
+ * IGNORE - The function is already what we want it to be
+ * MAKE_CALL - Start tracing the function
+ * MODIFY_CALL - Stop saving regs for the function
+ * MODIFY_CALL_REGS - Start saving regs for the function
+ * MAKE_NOP - Stop tracing the function
+ */
enum {
FTRACE_UPDATE_IGNORE,
FTRACE_UPDATE_MAKE_CALL,
+ FTRACE_UPDATE_MODIFY_CALL,
+ FTRACE_UPDATE_MODIFY_CALL_REGS,
FTRACE_UPDATE_MAKE_NOP,
};
@@ -317,7 +406,9 @@
extern void ftrace_replace_code(int enable);
extern int ftrace_update_ftrace_func(ftrace_func_t func);
extern void ftrace_caller(void);
+extern void ftrace_regs_caller(void);
extern void ftrace_call(void);
+extern void ftrace_regs_call(void);
extern void mcount_call(void);
void ftrace_modify_all_code(int command);
@@ -325,6 +416,15 @@
#ifndef FTRACE_ADDR
#define FTRACE_ADDR ((unsigned long)ftrace_caller)
#endif
+
+#ifndef FTRACE_REGS_ADDR
+#ifdef ARCH_SUPPORTS_FTRACE_SAVE_REGS
+# define FTRACE_REGS_ADDR ((unsigned long)ftrace_regs_caller)
+#else
+# define FTRACE_REGS_ADDR FTRACE_ADDR
+#endif
+#endif
+
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
extern void ftrace_graph_caller(void);
extern int ftrace_enable_ftrace_graph_caller(void);
@@ -380,6 +480,39 @@
*/
extern int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr);
+#ifdef ARCH_SUPPORTS_FTRACE_SAVE_REGS
+/**
+ * ftrace_modify_call - convert from one addr to another (no nop)
+ * @rec: the mcount call site record
+ * @old_addr: the address expected to be currently called to
+ * @addr: the address to change to
+ *
+ * This is a very sensitive operation and great care needs
+ * to be taken by the arch. The operation should carefully
+ * read the location, check to see if what is read is indeed
+ * what we expect it to be, and then on success of the compare,
+ * it should write to the location.
+ *
+ * The code segment at @rec->ip should be a caller to @old_addr
+ *
+ * Return must be:
+ * 0 on success
+ * -EFAULT on error reading the location
+ * -EINVAL on a failed compare of the contents
+ * -EPERM on error writing to the location
+ * Any other value will be considered a failure.
+ */
+extern int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+ unsigned long addr);
+#else
+/* Should never be called */
+static inline int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
+ unsigned long addr)
+{
+ return -EINVAL;
+}
+#endif
+
/* May be defined in arch */
extern int ftrace_arch_read_dyn_info(char *buf, int size);
@@ -387,7 +520,7 @@
extern void ftrace_disable_daemon(void);
extern void ftrace_enable_daemon(void);
-#else
+#else /* CONFIG_DYNAMIC_FTRACE */
static inline int skip_trace(unsigned long ip) { return 0; }
static inline int ftrace_force_update(void) { return 0; }
static inline void ftrace_disable_daemon(void) { }
@@ -405,6 +538,10 @@
{
return 0;
}
+static inline unsigned long ftrace_location(unsigned long ip)
+{
+ return 0;
+}
/*
* Again users of functions that have ftrace_ops may not
@@ -413,6 +550,7 @@
*/
#define ftrace_regex_open(ops, flag, inod, file) ({ -ENODEV; })
#define ftrace_set_early_filter(ops, buf, enable) do { } while (0)
+#define ftrace_set_filter_ip(ops, ip, remove, reset) ({ -ENODEV; })
#define ftrace_set_filter(ops, buf, len, reset) ({ -ENODEV; })
#define ftrace_set_notrace(ops, buf, len, reset) ({ -ENODEV; })
#define ftrace_free_filter(ops) do { } while (0)
diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h
index 305f23c..cab3da3 100644
--- a/include/linux/hardirq.h
+++ b/include/linux/hardirq.h
@@ -132,11 +132,11 @@
struct task_struct;
#if !defined(CONFIG_VIRT_CPU_ACCOUNTING) && !defined(CONFIG_IRQ_TIME_ACCOUNTING)
-static inline void account_system_vtime(struct task_struct *tsk)
+static inline void vtime_account(struct task_struct *tsk)
{
}
#else
-extern void account_system_vtime(struct task_struct *tsk);
+extern void vtime_account(struct task_struct *tsk);
#endif
#if defined(CONFIG_TINY_RCU) || defined(CONFIG_TINY_PREEMPT_RCU)
@@ -162,7 +162,7 @@
*/
#define __irq_enter() \
do { \
- account_system_vtime(current); \
+ vtime_account(current); \
add_preempt_count(HARDIRQ_OFFSET); \
trace_hardirq_enter(); \
} while (0)
@@ -178,7 +178,7 @@
#define __irq_exit() \
do { \
trace_hardirq_exit(); \
- account_system_vtime(current); \
+ vtime_account(current); \
sub_preempt_count(HARDIRQ_OFFSET); \
} while (0)
diff --git a/include/linux/hid.h b/include/linux/hid.h
index 42970de..7e1f37d 100644
--- a/include/linux/hid.h
+++ b/include/linux/hid.h
@@ -414,7 +414,7 @@
__u16 dpad; /* dpad input code */
};
-#define HID_MAX_FIELDS 128
+#define HID_MAX_FIELDS 256
struct hid_report {
struct list_head list;
@@ -626,6 +626,7 @@
* @report_fixup: called before report descriptor parsing (NULL means nop)
* @input_mapping: invoked on input registering before mapping an usage
* @input_mapped: invoked on input registering after mapping an usage
+ * @input_configured: invoked just before the device is registered
* @feature_mapping: invoked on feature registering
* @suspend: invoked on suspend (NULL means nop)
* @resume: invoked on resume if device was not reset (NULL means nop)
@@ -670,6 +671,8 @@
int (*input_mapped)(struct hid_device *hdev,
struct hid_input *hidinput, struct hid_field *field,
struct hid_usage *usage, unsigned long **bit, int *max);
+ void (*input_configured)(struct hid_device *hdev,
+ struct hid_input *hidinput);
void (*feature_mapping)(struct hid_device *hdev,
struct hid_field *field,
struct hid_usage *usage);
diff --git a/include/linux/i2c-pnx.h b/include/linux/i2c-pnx.h
index 1bc74af..49ed17f 100644
--- a/include/linux/i2c-pnx.h
+++ b/include/linux/i2c-pnx.h
@@ -22,6 +22,7 @@
struct timer_list timer; /* Timeout */
u8 * buf; /* Data buffer */
int len; /* Length of data buffer */
+ int order; /* RX Bytes to order via TX */
};
struct i2c_pnx_algo_data {
diff --git a/include/linux/input.h b/include/linux/input.h
index 725dcd0..ba48743 100644
--- a/include/linux/input.h
+++ b/include/linux/input.h
@@ -1169,6 +1169,18 @@
#include <linux/mod_devicetable.h>
/**
+ * struct input_value - input value representation
+ * @type: type of value (EV_KEY, EV_ABS, etc)
+ * @code: the value code
+ * @value: the value
+ */
+struct input_value {
+ __u16 type;
+ __u16 code;
+ __s32 value;
+};
+
+/**
* struct input_dev - represents an input device
* @name: name of the device
* @phys: physical path to the device in the system hierarchy
@@ -1203,11 +1215,7 @@
* software autorepeat
* @timer: timer for software autorepeat
* @rep: current values for autorepeat parameters (delay, rate)
- * @mt: pointer to array of struct input_mt_slot holding current values
- * of tracked contacts
- * @mtsize: number of MT slots the device uses
- * @slot: MT slot currently being transmitted
- * @trkid: stores MT tracking ID for the current contact
+ * @mt: pointer to multitouch state
* @absinfo: array of &struct input_absinfo elements holding information
* about absolute axes (current value, min, max, flat, fuzz,
* resolution)
@@ -1244,7 +1252,6 @@
* last user closes the device
* @going_away: marks devices that are in a middle of unregistering and
* causes input_open_device*() fail with -ENODEV.
- * @sync: set to %true when there were no new events since last EV_SYN
* @dev: driver model's view of this device
* @h_list: list of input handles associated with the device. When
* accessing the list dev->mutex must be held
@@ -1287,10 +1294,7 @@
int rep[REP_CNT];
- struct input_mt_slot *mt;
- int mtsize;
- int slot;
- int trkid;
+ struct input_mt *mt;
struct input_absinfo *absinfo;
@@ -1312,12 +1316,14 @@
unsigned int users;
bool going_away;
- bool sync;
-
struct device dev;
struct list_head h_list;
struct list_head node;
+
+ unsigned int num_vals;
+ unsigned int max_vals;
+ struct input_value *vals;
};
#define to_input_dev(d) container_of(d, struct input_dev, dev)
@@ -1378,6 +1384,9 @@
* @event: event handler. This method is being called by input core with
* interrupts disabled and dev->event_lock spinlock held and so
* it may not sleep
+ * @events: event sequence handler. This method is being called by
+ * input core with interrupts disabled and dev->event_lock
+ * spinlock held and so it may not sleep
* @filter: similar to @event; separates normal event handlers from
* "filters".
* @match: called after comparing device's id with handler's id_table
@@ -1414,6 +1423,8 @@
void *private;
void (*event)(struct input_handle *handle, unsigned int type, unsigned int code, int value);
+ void (*events)(struct input_handle *handle,
+ const struct input_value *vals, unsigned int count);
bool (*filter)(struct input_handle *handle, unsigned int type, unsigned int code, int value);
bool (*match)(struct input_handler *handler, struct input_dev *dev);
int (*connect)(struct input_handler *handler, struct input_dev *dev, const struct input_device_id *id);
diff --git a/include/linux/input/mt.h b/include/linux/input/mt.h
index f867375..cc5cca7 100644
--- a/include/linux/input/mt.h
+++ b/include/linux/input/mt.h
@@ -15,12 +15,41 @@
#define TRKID_MAX 0xffff
+#define INPUT_MT_POINTER 0x0001 /* pointer device, e.g. trackpad */
+#define INPUT_MT_DIRECT 0x0002 /* direct device, e.g. touchscreen */
+#define INPUT_MT_DROP_UNUSED 0x0004 /* drop contacts not seen in frame */
+#define INPUT_MT_TRACK 0x0008 /* use in-kernel tracking */
+
/**
* struct input_mt_slot - represents the state of an input MT slot
* @abs: holds current values of ABS_MT axes for this slot
+ * @frame: last frame at which input_mt_report_slot_state() was called
+ * @key: optional driver designation of this slot
*/
struct input_mt_slot {
int abs[ABS_MT_LAST - ABS_MT_FIRST + 1];
+ unsigned int frame;
+ unsigned int key;
+};
+
+/**
+ * struct input_mt - state of tracked contacts
+ * @trkid: stores MT tracking ID for the next contact
+ * @num_slots: number of MT slots the device uses
+ * @slot: MT slot currently being transmitted
+ * @flags: input_mt operation flags
+ * @frame: increases every time input_mt_sync_frame() is called
+ * @red: reduced cost matrix for in-kernel tracking
+ * @slots: array of slots holding current values of tracked contacts
+ */
+struct input_mt {
+ int trkid;
+ int num_slots;
+ int slot;
+ unsigned int flags;
+ unsigned int frame;
+ int *red;
+ struct input_mt_slot slots[];
};
static inline void input_mt_set_value(struct input_mt_slot *slot,
@@ -35,12 +64,18 @@
return slot->abs[code - ABS_MT_FIRST];
}
-int input_mt_init_slots(struct input_dev *dev, unsigned int num_slots);
+static inline bool input_mt_is_active(const struct input_mt_slot *slot)
+{
+ return input_mt_get_value(slot, ABS_MT_TRACKING_ID) >= 0;
+}
+
+int input_mt_init_slots(struct input_dev *dev, unsigned int num_slots,
+ unsigned int flags);
void input_mt_destroy_slots(struct input_dev *dev);
-static inline int input_mt_new_trkid(struct input_dev *dev)
+static inline int input_mt_new_trkid(struct input_mt *mt)
{
- return dev->trkid++ & TRKID_MAX;
+ return mt->trkid++ & TRKID_MAX;
}
static inline void input_mt_slot(struct input_dev *dev, int slot)
@@ -64,4 +99,20 @@
void input_mt_report_finger_count(struct input_dev *dev, int count);
void input_mt_report_pointer_emulation(struct input_dev *dev, bool use_count);
+void input_mt_sync_frame(struct input_dev *dev);
+
+/**
+ * struct input_mt_pos - contact position
+ * @x: horizontal coordinate
+ * @y: vertical coordinate
+ */
+struct input_mt_pos {
+ s16 x, y;
+};
+
+int input_mt_assign_slots(struct input_dev *dev, int *slots,
+ const struct input_mt_pos *pos, int num_pos);
+
+int input_mt_get_slot_by_key(struct input_dev *dev, int key);
+
#endif
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index c5f856a..5e4e617 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -430,6 +430,8 @@
NR_SOFTIRQS
};
+#define SOFTIRQ_STOP_IDLE_MASK (~(1 << RCU_SOFTIRQ))
+
/* map softirq index to softirq name. update 'softirq_to_name' in
* kernel/softirq.c when adding a new softirq.
*/
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 7e83370..f3b99e1 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -256,72 +256,78 @@
{
}
-int iommu_attach_group(struct iommu_domain *domain, struct iommu_group *group)
+static inline int iommu_attach_group(struct iommu_domain *domain,
+ struct iommu_group *group)
{
return -ENODEV;
}
-void iommu_detach_group(struct iommu_domain *domain, struct iommu_group *group)
+static inline void iommu_detach_group(struct iommu_domain *domain,
+ struct iommu_group *group)
{
}
-struct iommu_group *iommu_group_alloc(void)
+static inline struct iommu_group *iommu_group_alloc(void)
{
return ERR_PTR(-ENODEV);
}
-void *iommu_group_get_iommudata(struct iommu_group *group)
+static inline void *iommu_group_get_iommudata(struct iommu_group *group)
{
return NULL;
}
-void iommu_group_set_iommudata(struct iommu_group *group, void *iommu_data,
- void (*release)(void *iommu_data))
+static inline void iommu_group_set_iommudata(struct iommu_group *group,
+ void *iommu_data,
+ void (*release)(void *iommu_data))
{
}
-int iommu_group_set_name(struct iommu_group *group, const char *name)
+static inline int iommu_group_set_name(struct iommu_group *group,
+ const char *name)
{
return -ENODEV;
}
-int iommu_group_add_device(struct iommu_group *group, struct device *dev)
+static inline int iommu_group_add_device(struct iommu_group *group,
+ struct device *dev)
{
return -ENODEV;
}
-void iommu_group_remove_device(struct device *dev)
+static inline void iommu_group_remove_device(struct device *dev)
{
}
-int iommu_group_for_each_dev(struct iommu_group *group, void *data,
- int (*fn)(struct device *, void *))
+static inline int iommu_group_for_each_dev(struct iommu_group *group,
+ void *data,
+ int (*fn)(struct device *, void *))
{
return -ENODEV;
}
-struct iommu_group *iommu_group_get(struct device *dev)
+static inline struct iommu_group *iommu_group_get(struct device *dev)
{
return NULL;
}
-void iommu_group_put(struct iommu_group *group)
+static inline void iommu_group_put(struct iommu_group *group)
{
}
-int iommu_group_register_notifier(struct iommu_group *group,
- struct notifier_block *nb)
+static inline int iommu_group_register_notifier(struct iommu_group *group,
+ struct notifier_block *nb)
{
return -ENODEV;
}
-int iommu_group_unregister_notifier(struct iommu_group *group,
- struct notifier_block *nb)
+static inline int iommu_group_unregister_notifier(struct iommu_group *group,
+ struct notifier_block *nb)
{
return 0;
}
-int iommu_group_id(struct iommu_group *group)
+static inline int iommu_group_id(struct iommu_group *group)
{
return -ENODEV;
}
diff --git a/include/linux/irqdesc.h b/include/linux/irqdesc.h
index 9a323d1..0ba014c 100644
--- a/include/linux/irqdesc.h
+++ b/include/linux/irqdesc.h
@@ -10,12 +10,10 @@
struct irq_affinity_notify;
struct proc_dir_entry;
-struct timer_rand_state;
struct module;
/**
* struct irq_desc - interrupt descriptor
* @irq_data: per irq and chip data passed down to chip functions
- * @timer_rand_state: pointer to timer rand state struct
* @kstat_irqs: irq stats per cpu
* @handle_irq: highlevel irq-events handler
* @preflow_handler: handler called before the flow handler (currently used by sparc)
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 6043821..2451f1f 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -82,10 +82,18 @@
__x - (__x % (y)); \
} \
)
+
+/*
+ * Divide positive or negative dividend by positive divisor and round
+ * to closest integer. Result is undefined for negative divisors.
+ */
#define DIV_ROUND_CLOSEST(x, divisor)( \
{ \
- typeof(divisor) __divisor = divisor; \
- (((x) + ((__divisor) / 2)) / (__divisor)); \
+ typeof(x) __x = x; \
+ typeof(divisor) __d = divisor; \
+ (((typeof(x))-1) > 0 || (__x) > 0) ? \
+ (((__x) + ((__d) / 2)) / (__d)) : \
+ (((__x) - ((__d) / 2)) / (__d)); \
} \
)
diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h
index 2fbd905..36d12f0 100644
--- a/include/linux/kernel_stat.h
+++ b/include/linux/kernel_stat.h
@@ -130,4 +130,12 @@
extern void account_steal_ticks(unsigned long ticks);
extern void account_idle_ticks(unsigned long ticks);
+#ifdef CONFIG_VIRT_CPU_ACCOUNTING
+extern void vtime_task_switch(struct task_struct *prev);
+extern void vtime_account_system(struct task_struct *tsk);
+extern void vtime_account_idle(struct task_struct *tsk);
+#else
+static inline void vtime_task_switch(struct task_struct *prev) { }
+#endif
+
#endif /* _LINUX_KERNEL_STAT_H */
diff --git a/include/linux/kobject.h b/include/linux/kobject.h
index fc615a9..1e57449 100644
--- a/include/linux/kobject.h
+++ b/include/linux/kobject.h
@@ -224,7 +224,7 @@
static inline __printf(2, 3)
int add_uevent_var(struct kobj_uevent_env *env, const char *format, ...)
-{ return 0; }
+{ return -ENOMEM; }
static inline int kobject_action_type(const char *buf, size_t count,
enum kobject_action *type)
diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h
index b6e1f8c..23755ba 100644
--- a/include/linux/kprobes.h
+++ b/include/linux/kprobes.h
@@ -38,6 +38,7 @@
#include <linux/spinlock.h>
#include <linux/rcupdate.h>
#include <linux/mutex.h>
+#include <linux/ftrace.h>
#ifdef CONFIG_KPROBES
#include <asm/kprobes.h>
@@ -48,14 +49,26 @@
#define KPROBE_REENTER 0x00000004
#define KPROBE_HIT_SSDONE 0x00000008
+/*
+ * If function tracer is enabled and the arch supports full
+ * passing of pt_regs to function tracing, then kprobes can
+ * optimize on top of function tracing.
+ */
+#if defined(CONFIG_FUNCTION_TRACER) && defined(ARCH_SUPPORTS_FTRACE_SAVE_REGS) \
+ && defined(ARCH_SUPPORTS_KPROBES_ON_FTRACE)
+# define KPROBES_CAN_USE_FTRACE
+#endif
+
/* Attach to insert probes on any functions which should be ignored*/
#define __kprobes __attribute__((__section__(".kprobes.text")))
+
#else /* CONFIG_KPROBES */
typedef int kprobe_opcode_t;
struct arch_specific_insn {
int dummy;
};
#define __kprobes
+
#endif /* CONFIG_KPROBES */
struct kprobe;
@@ -128,6 +141,7 @@
* NOTE:
* this flag is only for optimized_kprobe.
*/
+#define KPROBE_FLAG_FTRACE 8 /* probe is using ftrace */
/* Has this kprobe gone ? */
static inline int kprobe_gone(struct kprobe *p)
@@ -146,6 +160,13 @@
{
return p->flags & KPROBE_FLAG_OPTIMIZED;
}
+
+/* Is this kprobe uses ftrace ? */
+static inline int kprobe_ftrace(struct kprobe *p)
+{
+ return p->flags & KPROBE_FLAG_FTRACE;
+}
+
/*
* Special probe type that uses setjmp-longjmp type tricks to resume
* execution at a specified entry with a matching prototype corresponding
@@ -295,6 +316,12 @@
#endif
#endif /* CONFIG_OPTPROBES */
+#ifdef KPROBES_CAN_USE_FTRACE
+extern void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *ops, struct pt_regs *regs);
+extern int arch_prepare_kprobe_ftrace(struct kprobe *p);
+#endif
+
/* Get the kprobe at this addr (if any) - called with preemption disabled */
struct kprobe *get_kprobe(void *addr);
diff --git a/include/linux/kthread.h b/include/linux/kthread.h
index 22ccf9d..8d81664 100644
--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -14,6 +14,11 @@
kthread_create_on_node(threadfn, data, -1, namefmt, ##arg)
+struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data),
+ void *data,
+ unsigned int cpu,
+ const char *namefmt);
+
/**
* kthread_run - create and wake a thread.
* @threadfn: the function to run until signal_pending(current).
@@ -34,9 +39,13 @@
void kthread_bind(struct task_struct *k, unsigned int cpu);
int kthread_stop(struct task_struct *k);
-int kthread_should_stop(void);
+bool kthread_should_stop(void);
+bool kthread_should_park(void);
bool kthread_freezable_should_stop(bool *was_frozen);
void *kthread_data(struct task_struct *k);
+int kthread_park(struct task_struct *k);
+void kthread_unpark(struct task_struct *k);
+void kthread_parkme(void);
int kthreadd(void *unused);
extern struct task_struct *kthreadd_task;
diff --git a/include/linux/ktime.h b/include/linux/ktime.h
index 603bec2..06177ba10 100644
--- a/include/linux/ktime.h
+++ b/include/linux/ktime.h
@@ -58,13 +58,6 @@
typedef union ktime ktime_t; /* Kill this */
-#define KTIME_MAX ((s64)~((u64)1 << 63))
-#if (BITS_PER_LONG == 64)
-# define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
-#else
-# define KTIME_SEC_MAX LONG_MAX
-#endif
-
/*
* ktime_t definitions when using the 64-bit scalar representation:
*/
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index b70b48b..8a59e0a 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -685,7 +685,7 @@
static inline void kvm_guest_enter(void)
{
BUG_ON(preemptible());
- account_system_vtime(current);
+ vtime_account(current);
current->flags |= PF_VCPU;
/* KVM does not hold any references to rcu protected data when it
* switches CPU into a guest mode. In fact switching to a guest mode
@@ -699,7 +699,7 @@
static inline void kvm_guest_exit(void)
{
- account_system_vtime(current);
+ vtime_account(current);
current->flags &= ~PF_VCPU;
}
diff --git a/include/linux/mISDNhw.h b/include/linux/mISDNhw.h
index d0752ec..9d96d5d 100644
--- a/include/linux/mISDNhw.h
+++ b/include/linux/mISDNhw.h
@@ -183,7 +183,7 @@
unsigned short);
extern int mISDN_freedchannel(struct dchannel *);
extern void mISDN_clear_bchannel(struct bchannel *);
-extern int mISDN_freebchannel(struct bchannel *);
+extern void mISDN_freebchannel(struct bchannel *);
extern int mISDN_ctrl_bchannel(struct bchannel *, struct mISDN_ctrl_req *);
extern void queue_ch_frame(struct mISDNchannel *, u_int,
int, struct sk_buff *);
diff --git a/include/linux/memory.h b/include/linux/memory.h
index 1ac7f6e..ff9a9f8 100644
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -19,7 +19,7 @@
#include <linux/compiler.h>
#include <linux/mutex.h>
-#define MIN_MEMORY_BLOCK_SIZE (1 << SECTION_SIZE_BITS)
+#define MIN_MEMORY_BLOCK_SIZE (1UL << SECTION_SIZE_BITS)
struct memory_block {
unsigned long start_section_nr;
diff --git a/include/linux/mfd/core.h b/include/linux/mfd/core.h
index 3a8435a..cebe97e 100644
--- a/include/linux/mfd/core.h
+++ b/include/linux/mfd/core.h
@@ -16,6 +16,8 @@
#include <linux/platform_device.h>
+struct irq_domain;
+
/*
* This struct describes the MFD part ("cell").
* After registration the copy of this structure will become the platform data
@@ -98,7 +100,7 @@
extern int mfd_add_devices(struct device *parent, int id,
struct mfd_cell *cells, int n_devs,
struct resource *mem_base,
- int irq_base);
+ int irq_base, struct irq_domain *irq_domain);
extern void mfd_remove_devices(struct device *parent);
diff --git a/include/linux/mfd/dbx500-prcmu.h b/include/linux/mfd/dbx500-prcmu.h
index 5b90e94..c410d99 100644
--- a/include/linux/mfd/dbx500-prcmu.h
+++ b/include/linux/mfd/dbx500-prcmu.h
@@ -136,6 +136,7 @@
PRCMU_TIMCLK,
PRCMU_PLLSOC0,
PRCMU_PLLSOC1,
+ PRCMU_ARMSS,
PRCMU_PLLDDR,
PRCMU_PLLDSI,
PRCMU_DSI0CLK,
diff --git a/include/linux/mfd/max77686.h b/include/linux/mfd/max77686.h
index 3d7ae4d..46c0f32 100644
--- a/include/linux/mfd/max77686.h
+++ b/include/linux/mfd/max77686.h
@@ -74,6 +74,7 @@
struct max77686_regulator_data {
int id;
struct regulator_init_data *initdata;
+ struct device_node *of_node;
};
enum max77686_opmode {
diff --git a/include/linux/mfd/max8998.h b/include/linux/mfd/max8998.h
index f4f0dfa..6823548 100644
--- a/include/linux/mfd/max8998.h
+++ b/include/linux/mfd/max8998.h
@@ -67,7 +67,7 @@
/**
* struct max8998_board - packages regulator init data
* @regulators: array of defined regulators
- * @num_regulators: number of regultors used
+ * @num_regulators: number of regulators used
* @irq_base: base IRQ number for max8998, required for IRQs
* @ono: power onoff IRQ number for max8998
* @buck_voltage_lock: Do NOT change the values of the following six
diff --git a/include/linux/mfd/tps65217.h b/include/linux/mfd/tps65217.h
index 12c0687..7cd83d82 100644
--- a/include/linux/mfd/tps65217.h
+++ b/include/linux/mfd/tps65217.h
@@ -22,6 +22,9 @@
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
+/* TPS chip id list */
+#define TPS65217 0xF0
+
/* I2C ID for TPS65217 part */
#define TPS65217_I2C_ID 0x24
@@ -248,13 +251,11 @@
struct tps65217 {
struct device *dev;
struct tps65217_board *pdata;
+ unsigned int id;
struct regulator_desc desc[TPS65217_NUM_REGULATOR];
struct regulator_dev *rdev[TPS65217_NUM_REGULATOR];
struct tps_info *info[TPS65217_NUM_REGULATOR];
struct regmap *regmap;
-
- /* Client devices */
- struct platform_device *regulator_pdev[TPS65217_NUM_REGULATOR];
};
static inline struct tps65217 *dev_to_tps65217(struct device *dev)
@@ -262,6 +263,11 @@
return dev_get_drvdata(dev);
}
+static inline int tps65217_chip_id(struct tps65217 *tps65217)
+{
+ return tps65217->id;
+}
+
int tps65217_reg_read(struct tps65217 *tps, unsigned int reg,
unsigned int *val);
int tps65217_reg_write(struct tps65217 *tps, unsigned int reg,
diff --git a/include/linux/mfd/tps6586x.h b/include/linux/mfd/tps6586x.h
index f350fd0..9451471 100644
--- a/include/linux/mfd/tps6586x.h
+++ b/include/linux/mfd/tps6586x.h
@@ -14,6 +14,7 @@
#define TPS6586X_SLEW_RATE_MASK 0x07
enum {
+ TPS6586X_ID_SYS,
TPS6586X_ID_SM_0,
TPS6586X_ID_SM_1,
TPS6586X_ID_SM_2,
diff --git a/include/linux/micrel_phy.h b/include/linux/micrel_phy.h
index 61f0905..de20120 100644
--- a/include/linux/micrel_phy.h
+++ b/include/linux/micrel_phy.h
@@ -1,3 +1,15 @@
+/*
+ * include/linux/micrel_phy.h
+ *
+ * Micrel PHY IDs
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ */
+
#ifndef _MICREL_PHY_H
#define _MICREL_PHY_H
@@ -5,10 +17,11 @@
#define PHY_ID_KSZ9021 0x00221610
#define PHY_ID_KS8737 0x00221720
-#define PHY_ID_KS8041 0x00221510
-#define PHY_ID_KS8051 0x00221550
+#define PHY_ID_KSZ8021 0x00221555
+#define PHY_ID_KSZ8041 0x00221510
+#define PHY_ID_KSZ8051 0x00221550
/* both for ks8001 Rev. A/B, and for ks8721 Rev 3. */
-#define PHY_ID_KS8001 0x0022161A
+#define PHY_ID_KSZ8001 0x0022161A
/* struct phy_device dev_flags definitions */
#define MICREL_PHY_50MHZ_CLK 0x00000001
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index bd6c9fc..6e1b0f9 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -796,6 +796,19 @@
MLX4_NET_TRANS_RULE_NUM, /* should be last */
};
+extern const u16 __sw_id_hw[];
+
+static inline int map_hw_to_sw_id(u16 header_id)
+{
+
+ int i;
+ for (i = 0; i < MLX4_NET_TRANS_RULE_NUM; i++) {
+ if (header_id == __sw_id_hw[i])
+ return i;
+ }
+ return -EINVAL;
+}
+
enum mlx4_net_trans_promisc_mode {
MLX4_FS_PROMISC_NONE = 0,
MLX4_FS_PROMISC_UPLINK,
diff --git a/include/linux/mmc/card.h b/include/linux/mmc/card.h
index 111aca5..4b27f9f 100644
--- a/include/linux/mmc/card.h
+++ b/include/linux/mmc/card.h
@@ -239,6 +239,7 @@
#define MMC_QUIRK_BLK_NO_CMD23 (1<<7) /* Avoid CMD23 for regular multiblock */
#define MMC_QUIRK_BROKEN_BYTE_MODE_512 (1<<8) /* Avoid sending 512 bytes in */
#define MMC_QUIRK_LONG_READ_TIME (1<<9) /* Data read time > CSD says */
+#define MMC_QUIRK_SEC_ERASE_TRIM_BROKEN (1<<10) /* Skip secure for erase/trim */
/* byte mode */
unsigned int poweroff_notify_state; /* eMMC4.5 notify feature */
#define MMC_NO_POWER_NOTIFICATION 0
diff --git a/include/linux/mv643xx_eth.h b/include/linux/mv643xx_eth.h
index 51bf8ad..49258e0 100644
--- a/include/linux/mv643xx_eth.h
+++ b/include/linux/mv643xx_eth.h
@@ -15,6 +15,8 @@
#define MV643XX_ETH_SIZE_REG_4 0x2224
#define MV643XX_ETH_BASE_ADDR_ENABLE_REG 0x2290
+#define MV643XX_TX_CSUM_DEFAULT_LIMIT 0
+
struct mv643xx_eth_shared_platform_data {
struct mbus_dram_target_info *dram;
struct platform_device *shared_smi;
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index 1f8fc7f..4b03f56 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
@@ -265,11 +265,6 @@
return NFS_SERVER(inode)->nfs_client->rpc_ops;
}
-static inline __be32 *NFS_COOKIEVERF(const struct inode *inode)
-{
- return NFS_I(inode)->cookieverf;
-}
-
static inline unsigned NFS_MINATTRTIMEO(const struct inode *inode)
{
struct nfs_server *nfss = NFS_SERVER(inode);
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index ac7c8ae..be9cf3c 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -652,7 +652,7 @@
};
/* getxattr ACL interface flags */
-#define NFS4_ACL_LEN_REQUEST 0x0001 /* zero length getxattr buffer */
+#define NFS4_ACL_TRUNC 0x0001 /* ACL was truncated */
struct nfs_getaclres {
size_t acl_len;
size_t acl_data_offset;
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 9490a00..c25ccca 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -35,8 +35,10 @@
__u64 acq; /* Admin CQ Base Address */
};
+#define NVME_CAP_MQES(cap) ((cap) & 0xffff)
#define NVME_CAP_TIMEOUT(cap) (((cap) >> 24) & 0xff)
#define NVME_CAP_STRIDE(cap) (((cap) >> 32) & 0xf)
+#define NVME_CAP_MPSMIN(cap) (((cap) >> 48) & 0xf)
enum {
NVME_CC_ENABLE = 1 << 0,
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 248fba2..9a22b5e 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -22,19 +22,24 @@
static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
{
struct pci_bus *pbus = pdev->bus;
+
/* Find a PCI root bus */
while (!pci_is_root_bus(pbus))
pbus = pbus->parent;
- return acpi_get_pci_rootbridge_handle(pci_domain_nr(pbus),
- pbus->number);
+
+ return DEVICE_ACPI_HANDLE(pbus->bridge);
}
static inline acpi_handle acpi_pci_get_bridge_handle(struct pci_bus *pbus)
{
- if (!pci_is_root_bus(pbus))
- return DEVICE_ACPI_HANDLE(&(pbus->self->dev));
- return acpi_get_pci_rootbridge_handle(pci_domain_nr(pbus),
- pbus->number);
+ struct device *dev;
+
+ if (pci_is_root_bus(pbus))
+ dev = pbus->bridge;
+ else
+ dev = &pbus->self->dev;
+
+ return DEVICE_ACPI_HANDLE(dev);
}
#endif
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 5faa831..be1de01 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -254,10 +254,10 @@
u8 revision; /* PCI revision, low byte of class word */
u8 hdr_type; /* PCI header type (`multi' flag masked out) */
u8 pcie_cap; /* PCI-E capability offset */
- u8 pcie_type:4; /* PCI-E device/port type */
u8 pcie_mpss:3; /* PCI-E Max Payload Size Supported */
u8 rom_base_reg; /* which config register controls the ROM */
u8 pin; /* which interrupt pin this device uses */
+ u16 pcie_flags_reg; /* cached PCI-E Capabilities Register */
struct pci_driver *driver; /* which driver has allocated this device */
u64 dma_mask; /* Mask of the bits of bus address this
@@ -369,7 +369,6 @@
extern struct pci_dev *alloc_pci_dev(void);
-#define pci_dev_b(n) list_entry(n, struct pci_dev, bus_list)
#define to_pci_dev(n) container_of(n, struct pci_dev, dev)
#define for_each_pci_dev(d) while ((d = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, d)) != NULL)
@@ -596,7 +595,7 @@
int (*resume_early) (struct pci_dev *dev);
int (*resume) (struct pci_dev *dev); /* Device woken up */
void (*shutdown) (struct pci_dev *dev);
- struct pci_error_handlers *err_handler;
+ const struct pci_error_handlers *err_handler;
struct device_driver driver;
struct pci_dynids dynids;
};
@@ -734,9 +733,7 @@
extern struct pci_dev *pci_dev_get(struct pci_dev *dev);
extern void pci_dev_put(struct pci_dev *dev);
extern void pci_remove_bus(struct pci_bus *b);
-extern void __pci_remove_bus_device(struct pci_dev *dev);
extern void pci_stop_and_remove_bus_device(struct pci_dev *dev);
-extern void pci_stop_bus_device(struct pci_dev *dev);
void pci_setup_cardbus(struct pci_bus *bus);
extern void pci_sort_breadthfirst(void);
#define dev_is_pci(d) ((d)->bus == &pci_bus_type)
@@ -755,6 +752,7 @@
int pci_find_capability(struct pci_dev *dev, int cap);
int pci_find_next_capability(struct pci_dev *dev, u8 pos, int cap);
int pci_find_ext_capability(struct pci_dev *dev, int cap);
+int pci_find_next_ext_capability(struct pci_dev *dev, int pos, int cap);
int pci_find_ht_capability(struct pci_dev *dev, int ht_cap);
int pci_find_next_ht_capability(struct pci_dev *dev, int pos, int ht_cap);
struct pci_bus *pci_find_next_bus(const struct pci_bus *from);
@@ -816,6 +814,39 @@
return pci_bus_write_config_dword(dev->bus, dev->devfn, where, val);
}
+int pcie_capability_read_word(struct pci_dev *dev, int pos, u16 *val);
+int pcie_capability_read_dword(struct pci_dev *dev, int pos, u32 *val);
+int pcie_capability_write_word(struct pci_dev *dev, int pos, u16 val);
+int pcie_capability_write_dword(struct pci_dev *dev, int pos, u32 val);
+int pcie_capability_clear_and_set_word(struct pci_dev *dev, int pos,
+ u16 clear, u16 set);
+int pcie_capability_clear_and_set_dword(struct pci_dev *dev, int pos,
+ u32 clear, u32 set);
+
+static inline int pcie_capability_set_word(struct pci_dev *dev, int pos,
+ u16 set)
+{
+ return pcie_capability_clear_and_set_word(dev, pos, 0, set);
+}
+
+static inline int pcie_capability_set_dword(struct pci_dev *dev, int pos,
+ u32 set)
+{
+ return pcie_capability_clear_and_set_dword(dev, pos, 0, set);
+}
+
+static inline int pcie_capability_clear_word(struct pci_dev *dev, int pos,
+ u16 clear)
+{
+ return pcie_capability_clear_and_set_word(dev, pos, clear, 0);
+}
+
+static inline int pcie_capability_clear_dword(struct pci_dev *dev, int pos,
+ u32 clear)
+{
+ return pcie_capability_clear_and_set_dword(dev, pos, clear, 0);
+}
+
/* user-space driven config access */
int pci_user_read_config_byte(struct pci_dev *dev, int where, u8 *val);
int pci_user_read_config_word(struct pci_dev *dev, int where, u16 *val);
@@ -1013,7 +1044,6 @@
module_driver(__pci_driver, pci_register_driver, \
pci_unregister_driver)
-void pci_stop_and_remove_behind_bridge(struct pci_dev *dev);
struct pci_driver *pci_dev_driver(const struct pci_dev *dev);
int pci_add_dynid(struct pci_driver *drv,
unsigned int vendor, unsigned int device,
@@ -1031,6 +1061,8 @@
int pci_cfg_space_size(struct pci_dev *dev);
unsigned char pci_bus_max_busnr(struct pci_bus *bus);
void pci_setup_bridge(struct pci_bus *bus);
+resource_size_t pcibios_window_alignment(struct pci_bus *bus,
+ unsigned long type);
#define PCI_VGA_STATE_CHANGE_BRIDGE (1 << 0)
#define PCI_VGA_STATE_CHANGE_DECODES (1 << 1)
@@ -1472,7 +1504,7 @@
/* Anonymous variables would be nice... */
#define DECLARE_PCI_FIXUP_SECTION(section, name, vendor, device, class, \
class_shift, hook) \
- static const struct pci_fixup const __pci_fixup_##name __used \
+ static const struct pci_fixup __pci_fixup_##name __used \
__attribute__((__section__(#section), aligned((sizeof(void *))))) \
= { vendor, device, class, class_shift, hook };
@@ -1650,6 +1682,15 @@
return !!pci_pcie_cap(dev);
}
+/**
+ * pci_pcie_type - get the PCIe device/port type
+ * @dev: PCI device
+ */
+static inline int pci_pcie_type(const struct pci_dev *dev)
+{
+ return (dev->pcie_flags_reg & PCI_EXP_FLAGS_TYPE) >> 4;
+}
+
void pci_request_acs(void);
bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags);
bool pci_acs_path_enabled(struct pci_dev *start,
diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h
index fc35260..6b4565c 100644
--- a/include/linux/pci_ids.h
+++ b/include/linux/pci_ids.h
@@ -2149,7 +2149,7 @@
#define PCI_DEVICE_ID_TIGON3_5704S 0x16a8
#define PCI_DEVICE_ID_NX2_57800_VF 0x16a9
#define PCI_DEVICE_ID_NX2_5706S 0x16aa
-#define PCI_DEVICE_ID_NX2_57840_MF 0x16ab
+#define PCI_DEVICE_ID_NX2_57840_MF 0x16a4
#define PCI_DEVICE_ID_NX2_5708S 0x16ac
#define PCI_DEVICE_ID_NX2_57840_VF 0x16ad
#define PCI_DEVICE_ID_NX2_57810_MF 0x16ae
diff --git a/include/linux/pci_regs.h b/include/linux/pci_regs.h
index 7fb75b1..20ae747 100644
--- a/include/linux/pci_regs.h
+++ b/include/linux/pci_regs.h
@@ -549,6 +549,7 @@
#define PCI_EXP_LNKCAP2_SLS_8_0GB 0x04 /* Current Link Speed 8.0GT/s */
#define PCI_EXP_LNKCAP2_CROSSLINK 0x100 /* Crosslink supported */
#define PCI_EXP_LNKCTL2 48 /* Link Control 2 */
+#define PCI_EXP_LNKSTA2 50 /* Link Status 2 */
#define PCI_EXP_SLTCTL2 56 /* Slot Control 2 */
/* Extended Capabilities (PCI-X 2.0 and Express) */
@@ -677,6 +678,12 @@
#define PCI_PWR_CAP_BUDGET(x) ((x) & 1) /* Included in system budget */
#define PCI_EXT_CAP_PWR_SIZEOF 16
+/* Vendor-Specific (VSEC, PCI_EXT_CAP_ID_VNDR) */
+#define PCI_VNDR_HEADER 4 /* Vendor-Specific Header */
+#define PCI_VNDR_HEADER_ID(x) ((x) & 0xffff)
+#define PCI_VNDR_HEADER_REV(x) (((x) >> 16) & 0xf)
+#define PCI_VNDR_HEADER_LEN(x) (((x) >> 20) & 0xfff)
+
/*
* Hypertransport sub capability types
*
diff --git a/include/linux/pcieport_if.h b/include/linux/pcieport_if.h
index 6775532..e6f91b1 100644
--- a/include/linux/pcieport_if.h
+++ b/include/linux/pcieport_if.h
@@ -49,7 +49,7 @@
int (*resume) (struct pcie_device *dev);
/* Service Error Recovery Handler */
- struct pci_error_handlers *err_handler;
+ const struct pci_error_handlers *err_handler;
/* Link Reset Capability - AER service driver specific */
pci_ers_result_t (*reset_link) (struct pci_dev *dev);
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 7602ccb..599afc4 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -130,8 +130,10 @@
PERF_SAMPLE_STREAM_ID = 1U << 9,
PERF_SAMPLE_RAW = 1U << 10,
PERF_SAMPLE_BRANCH_STACK = 1U << 11,
+ PERF_SAMPLE_REGS_USER = 1U << 12,
+ PERF_SAMPLE_STACK_USER = 1U << 13,
- PERF_SAMPLE_MAX = 1U << 12, /* non-ABI */
+ PERF_SAMPLE_MAX = 1U << 14, /* non-ABI */
};
/*
@@ -163,6 +165,15 @@
PERF_SAMPLE_BRANCH_HV)
/*
+ * Values to determine ABI of the registers dump.
+ */
+enum perf_sample_regs_abi {
+ PERF_SAMPLE_REGS_ABI_NONE = 0,
+ PERF_SAMPLE_REGS_ABI_32 = 1,
+ PERF_SAMPLE_REGS_ABI_64 = 2,
+};
+
+/*
* The format of the data returned by read() on a perf event fd,
* as specified by attr.read_format:
*
@@ -194,6 +205,8 @@
#define PERF_ATTR_SIZE_VER0 64 /* sizeof first published struct */
#define PERF_ATTR_SIZE_VER1 72 /* add: config2 */
#define PERF_ATTR_SIZE_VER2 80 /* add: branch_sample_type */
+#define PERF_ATTR_SIZE_VER3 96 /* add: sample_regs_user */
+ /* add: sample_stack_user */
/*
* Hardware event_id to monitor via a performance monitoring event:
@@ -255,7 +268,10 @@
exclude_host : 1, /* don't count in host */
exclude_guest : 1, /* don't count in guest */
- __reserved_1 : 43;
+ exclude_callchain_kernel : 1, /* exclude kernel callchains */
+ exclude_callchain_user : 1, /* exclude user callchains */
+
+ __reserved_1 : 41;
union {
__u32 wakeup_events; /* wakeup every n events */
@@ -271,9 +287,25 @@
__u64 bp_len;
__u64 config2; /* extension of config1 */
};
- __u64 branch_sample_type; /* enum branch_sample_type */
+ __u64 branch_sample_type; /* enum perf_branch_sample_type */
+
+ /*
+ * Defines set of user regs to dump on samples.
+ * See asm/perf_regs.h for details.
+ */
+ __u64 sample_regs_user;
+
+ /*
+ * Defines size of the user stack to dump on samples.
+ */
+ __u32 sample_stack_user;
+
+ /* Align to u64. */
+ __u32 __reserved_2;
};
+#define perf_flags(attr) (*(&(attr)->read_format + 1))
+
/*
* Ioctls that can be done on a perf event fd:
*/
@@ -548,6 +580,13 @@
* char data[size];}&& PERF_SAMPLE_RAW
*
* { u64 from, to, flags } lbr[nr];} && PERF_SAMPLE_BRANCH_STACK
+ *
+ * { u64 abi; # enum perf_sample_regs_abi
+ * u64 regs[weight(mask)]; } && PERF_SAMPLE_REGS_USER
+ *
+ * { u64 size;
+ * char data[size];
+ * u64 dyn_size; } && PERF_SAMPLE_STACK_USER
* };
*/
PERF_RECORD_SAMPLE = 9,
@@ -609,6 +648,7 @@
#include <linux/static_key.h>
#include <linux/atomic.h>
#include <linux/sysfs.h>
+#include <linux/perf_regs.h>
#include <asm/local.h>
struct perf_callchain_entry {
@@ -654,6 +694,11 @@
struct perf_branch_entry entries[0];
};
+struct perf_regs_user {
+ __u64 abi;
+ struct pt_regs *regs;
+};
+
struct task_struct;
/*
@@ -926,7 +971,7 @@
struct hw_perf_event hw;
struct perf_event_context *ctx;
- struct file *filp;
+ atomic_long_t refcount;
/*
* These accumulate total time (in nanoseconds) that children
@@ -1133,6 +1178,8 @@
struct perf_callchain_entry *callchain;
struct perf_raw_record *raw;
struct perf_branch_stack *br_stack;
+ struct perf_regs_user regs_user;
+ u64 stack_user_size;
};
static inline void perf_sample_data_init(struct perf_sample_data *data,
@@ -1142,7 +1189,10 @@
data->addr = addr;
data->raw = NULL;
data->br_stack = NULL;
- data->period = period;
+ data->period = period;
+ data->regs_user.abi = PERF_SAMPLE_REGS_ABI_NONE;
+ data->regs_user.regs = NULL;
+ data->stack_user_size = 0;
}
extern void perf_output_sample(struct perf_output_handle *handle,
@@ -1290,12 +1340,15 @@
extern int perf_output_begin(struct perf_output_handle *handle,
struct perf_event *event, unsigned int size);
extern void perf_output_end(struct perf_output_handle *handle);
-extern void perf_output_copy(struct perf_output_handle *handle,
+extern unsigned int perf_output_copy(struct perf_output_handle *handle,
const void *buf, unsigned int len);
+extern unsigned int perf_output_skip(struct perf_output_handle *handle,
+ unsigned int len);
extern int perf_swevent_get_recursion_context(void);
extern void perf_swevent_put_recursion_context(int rctx);
extern void perf_event_enable(struct perf_event *event);
extern void perf_event_disable(struct perf_event *event);
+extern int __perf_event_disable(void *info);
extern void perf_event_task_tick(void);
#else
static inline void
@@ -1334,6 +1387,7 @@
static inline void perf_swevent_put_recursion_context(int rctx) { }
static inline void perf_event_enable(struct perf_event *event) { }
static inline void perf_event_disable(struct perf_event *event) { }
+static inline int __perf_event_disable(void *info) { return -1; }
static inline void perf_event_task_tick(void) { }
#endif
diff --git a/include/linux/perf_regs.h b/include/linux/perf_regs.h
new file mode 100644
index 0000000..3c73d5f
--- /dev/null
+++ b/include/linux/perf_regs.h
@@ -0,0 +1,25 @@
+#ifndef _LINUX_PERF_REGS_H
+#define _LINUX_PERF_REGS_H
+
+#ifdef CONFIG_HAVE_PERF_REGS
+#include <asm/perf_regs.h>
+u64 perf_reg_value(struct pt_regs *regs, int idx);
+int perf_reg_validate(u64 mask);
+u64 perf_reg_abi(struct task_struct *task);
+#else
+static inline u64 perf_reg_value(struct pt_regs *regs, int idx)
+{
+ return 0;
+}
+
+static inline int perf_reg_validate(u64 mask)
+{
+ return mask ? -ENOSYS : 0;
+}
+
+static inline u64 perf_reg_abi(struct task_struct *task)
+{
+ return PERF_SAMPLE_REGS_ABI_NONE;
+}
+#endif /* CONFIG_HAVE_PERF_REGS */
+#endif /* _LINUX_PERF_REGS_H */
diff --git a/include/linux/platform_data/clk-realview.h b/include/linux/platform_data/clk-realview.h
new file mode 100644
index 0000000..2e426a7
--- /dev/null
+++ b/include/linux/platform_data/clk-realview.h
@@ -0,0 +1 @@
+void realview_clk_init(void __iomem *sysbase, bool is_pb1176);
diff --git a/include/linux/platform_data/clk-ux500.h b/include/linux/platform_data/clk-ux500.h
new file mode 100644
index 0000000..3af0da1
--- /dev/null
+++ b/include/linux/platform_data/clk-ux500.h
@@ -0,0 +1,17 @@
+/*
+ * Clock definitions for ux500 platforms
+ *
+ * Copyright (C) 2012 ST-Ericsson SA
+ * Author: Ulf Hansson <ulf.hansson@linaro.org>
+ *
+ * License terms: GNU General Public License (GPL) version 2
+ */
+
+#ifndef __CLK_UX500_H
+#define __CLK_UX500_H
+
+void u8500_clk_init(void);
+void u9540_clk_init(void);
+void u8540_clk_init(void);
+
+#endif /* __CLK_UX500_H */
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 115ead2..7c968e4 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -191,6 +191,21 @@
extern void rcu_idle_exit(void);
extern void rcu_irq_enter(void);
extern void rcu_irq_exit(void);
+
+#ifdef CONFIG_RCU_USER_QS
+extern void rcu_user_enter(void);
+extern void rcu_user_exit(void);
+extern void rcu_user_enter_after_irq(void);
+extern void rcu_user_exit_after_irq(void);
+extern void rcu_user_hooks_switch(struct task_struct *prev,
+ struct task_struct *next);
+#else
+static inline void rcu_user_enter(void) { }
+static inline void rcu_user_exit(void) { }
+static inline void rcu_user_enter_after_irq(void) { }
+static inline void rcu_user_exit_after_irq(void) { }
+#endif /* CONFIG_RCU_USER_QS */
+
extern void exit_rcu(void);
/**
@@ -210,14 +225,12 @@
* to nest RCU_NONIDLE() wrappers, but the nesting level is currently
* quite limited. If deeper nesting is required, it will be necessary
* to adjust DYNTICK_TASK_NESTING_VALUE accordingly.
- *
- * This macro may be used from process-level code only.
*/
#define RCU_NONIDLE(a) \
do { \
- rcu_idle_exit(); \
+ rcu_irq_enter(); \
do { a; } while (0); \
- rcu_idle_enter(); \
+ rcu_irq_exit(); \
} while (0)
/*
diff --git a/include/linux/regmap.h b/include/linux/regmap.h
index 7f7e00d..e3bcc3f 100644
--- a/include/linux/regmap.h
+++ b/include/linux/regmap.h
@@ -285,6 +285,7 @@
* @ack_base: Base ack address. If zero then the chip is clear on read.
* @wake_base: Base address for wake enables. If zero unsupported.
* @irq_reg_stride: Stride to use for chips where registers are not contiguous.
+ * @runtime_pm: Hold a runtime PM lock on the device when accessing it.
*
* @num_regs: Number of registers in each control bank.
* @irqs: Descriptors for individual IRQs. Interrupt numbers are
@@ -299,6 +300,8 @@
unsigned int ack_base;
unsigned int wake_base;
unsigned int irq_reg_stride;
+ unsigned int mask_invert;
+ bool runtime_pm;
int num_regs;
diff --git a/include/linux/regulator/consumer.h b/include/linux/regulator/consumer.h
index da339fd..c43cd35 100644
--- a/include/linux/regulator/consumer.h
+++ b/include/linux/regulator/consumer.h
@@ -177,6 +177,8 @@
unsigned int regulator_get_mode(struct regulator *regulator);
int regulator_set_optimum_mode(struct regulator *regulator, int load_uA);
+int regulator_allow_bypass(struct regulator *regulator, bool allow);
+
/* regulator notifier block */
int regulator_register_notifier(struct regulator *regulator,
struct notifier_block *nb);
@@ -328,6 +330,12 @@
return REGULATOR_MODE_NORMAL;
}
+static inline int regulator_allow_bypass(struct regulator *regulator,
+ bool allow)
+{
+ return 0;
+}
+
static inline int regulator_register_notifier(struct regulator *regulator,
struct notifier_block *nb)
{
@@ -352,4 +360,11 @@
#endif
+static inline int regulator_set_voltage_tol(struct regulator *regulator,
+ int new_uV, int tol_uV)
+{
+ return regulator_set_voltage(regulator,
+ new_uV - tol_uV, new_uV + tol_uV);
+}
+
#endif
diff --git a/include/linux/regulator/driver.h b/include/linux/regulator/driver.h
index bac4c87..7932a3b 100644
--- a/include/linux/regulator/driver.h
+++ b/include/linux/regulator/driver.h
@@ -32,6 +32,8 @@
REGULATOR_STATUS_NORMAL,
REGULATOR_STATUS_IDLE,
REGULATOR_STATUS_STANDBY,
+ /* The regulator is enabled but not regulating */
+ REGULATOR_STATUS_BYPASS,
/* in case that any other status doesn't apply */
REGULATOR_STATUS_UNDEFINED,
};
@@ -58,6 +60,7 @@
* regulator_desc.n_voltages. Voltages may be reported in any order.
*
* @set_current_limit: Configure a limit for a current-limited regulator.
+ * The driver should select the current closest to max_uA.
* @get_current_limit: Get the configured limit for a current-limited regulator.
*
* @set_mode: Set the configured operating mode for the regulator.
@@ -67,6 +70,9 @@
* @get_optimum_mode: Get the most efficient operating mode for the regulator
* when running with the specified parameters.
*
+ * @set_bypass: Set the regulator in bypass mode.
+ * @get_bypass: Get the regulator bypass mode state.
+ *
* @enable_time: Time taken for the regulator voltage output voltage to
* stabilise after being enabled, in microseconds.
* @set_ramp_delay: Set the ramp delay for the regulator. The driver should
@@ -133,6 +139,10 @@
unsigned int (*get_optimum_mode) (struct regulator_dev *, int input_uV,
int output_uV, int load_uA);
+ /* control and report on bypass mode */
+ int (*set_bypass)(struct regulator_dev *dev, bool enable);
+ int (*get_bypass)(struct regulator_dev *dev, bool *enable);
+
/* the operations below are for configuration of regulator state when
* its parent PMIC enters a global STANDBY/HIBERNATE state */
@@ -205,6 +215,8 @@
unsigned int vsel_mask;
unsigned int enable_reg;
unsigned int enable_mask;
+ unsigned int bypass_reg;
+ unsigned int bypass_mask;
unsigned int enable_time;
};
@@ -221,7 +233,8 @@
* @driver_data: private regulator data
* @of_node: OpenFirmware node to parse for device tree bindings (may be
* NULL).
- * @regmap: regmap to use for core regmap helpers
+ * @regmap: regmap to use for core regmap helpers if dev_get_regulator() is
+ * insufficient.
* @ena_gpio: GPIO controlling regulator enable.
* @ena_gpio_invert: Sense for GPIO enable control.
* @ena_gpio_flags: Flags to use when calling gpio_request_one()
@@ -253,6 +266,7 @@
int exclusive;
u32 use_count;
u32 open_count;
+ u32 bypass_count;
/* lists we belong to */
struct list_head list; /* list of all regulators */
@@ -310,6 +324,8 @@
int regulator_set_voltage_time_sel(struct regulator_dev *rdev,
unsigned int old_selector,
unsigned int new_selector);
+int regulator_set_bypass_regmap(struct regulator_dev *rdev, bool enable);
+int regulator_get_bypass_regmap(struct regulator_dev *rdev, bool *enable);
void *regulator_get_init_drvdata(struct regulator_init_data *reg_init_data);
diff --git a/include/linux/regulator/fan53555.h b/include/linux/regulator/fan53555.h
new file mode 100644
index 0000000..5c45c85
--- /dev/null
+++ b/include/linux/regulator/fan53555.h
@@ -0,0 +1,60 @@
+/*
+ * fan53555.h - Fairchild Regulator FAN53555 Driver
+ *
+ * Copyright (C) 2012 Marvell Technology Ltd.
+ * Yunfan Zhang <yfzhang@marvell.com>
+ *
+ * This package is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#ifndef __FAN53555_H__
+
+/* VSEL ID */
+enum {
+ FAN53555_VSEL_ID_0 = 0,
+ FAN53555_VSEL_ID_1,
+};
+
+/* Transition slew rate limiting from a low to high voltage.
+ * -----------------------
+ * Bin |Slew Rate(mV/uS)
+ * ------|----------------
+ * 000 | 64.00
+ * ------|----------------
+ * 001 | 32.00
+ * ------|----------------
+ * 010 | 16.00
+ * ------|----------------
+ * 011 | 8.00
+ * ------|----------------
+ * 100 | 4.00
+ * ------|----------------
+ * 101 | 2.00
+ * ------|----------------
+ * 110 | 1.00
+ * ------|----------------
+ * 111 | 0.50
+ * -----------------------
+ */
+enum {
+ FAN53555_SLEW_RATE_64MV = 0,
+ FAN53555_SLEW_RATE_32MV,
+ FAN53555_SLEW_RATE_16MV,
+ FAN53555_SLEW_RATE_8MV,
+ FAN53555_SLEW_RATE_4MV,
+ FAN53555_SLEW_RATE_2MV,
+ FAN53555_SLEW_RATE_1MV,
+ FAN53555_SLEW_RATE_0_5MV,
+};
+
+struct fan53555_platform_data {
+ struct regulator_init_data *regulator;
+ unsigned int slew_rate;
+ /* Sleep VSEL ID */
+ unsigned int sleep_vsel_id;
+};
+
+#endif /* __FAN53555_H__ */
diff --git a/include/linux/regulator/machine.h b/include/linux/regulator/machine.h
index 40dd0a3..36adbc8 100644
--- a/include/linux/regulator/machine.h
+++ b/include/linux/regulator/machine.h
@@ -32,6 +32,7 @@
* board/machine.
* STATUS: Regulator can be enabled and disabled.
* DRMS: Dynamic Regulator Mode Switching is enabled for this regulator.
+ * BYPASS: Regulator can be put into bypass mode
*/
#define REGULATOR_CHANGE_VOLTAGE 0x1
@@ -39,6 +40,7 @@
#define REGULATOR_CHANGE_MODE 0x4
#define REGULATOR_CHANGE_STATUS 0x8
#define REGULATOR_CHANGE_DRMS 0x10
+#define REGULATOR_CHANGE_BYPASS 0x20
/**
* struct regulator_state - regulator state during low power system states
diff --git a/include/linux/sched.h b/include/linux/sched.h
index b8c8664..765dffb 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -273,11 +273,11 @@
extern int runqueue_is_locked(int cpu);
#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ)
-extern void select_nohz_load_balancer(int stop_tick);
+extern void nohz_balance_enter_idle(int cpu);
extern void set_cpu_sd_state_idle(void);
extern int get_nohz_timer_target(void);
#else
-static inline void select_nohz_load_balancer(int stop_tick) { }
+static inline void nohz_balance_enter_idle(int cpu) { }
static inline void set_cpu_sd_state_idle(void) { }
#endif
@@ -446,6 +446,9 @@
#define MMF_VM_HUGEPAGE 17 /* set when VM_HUGEPAGE is set on vma */
#define MMF_EXE_FILE_CHANGED 18 /* see prctl_set_mm_exe_file() */
+#define MMF_HAS_UPROBES 19 /* has uprobes */
+#define MMF_RECALC_UPROBES 20 /* MMF_HAS_UPROBES can be wrong */
+
#define MMF_INIT_MASK (MMF_DUMPABLE_MASK | MMF_DUMP_FILTER_MASK)
struct sighand_struct {
@@ -678,11 +681,6 @@
* (notably. ptrace) */
};
-/* Context switch must be unlocked if interrupts are to be enabled */
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
-# define __ARCH_WANT_UNLOCKED_CTXSW
-#endif
-
/*
* Bits in flags field of signal_struct.
*/
@@ -860,7 +858,6 @@
#define SD_BALANCE_FORK 0x0008 /* Balance on fork, clone */
#define SD_BALANCE_WAKE 0x0010 /* Balance on wakeup */
#define SD_WAKE_AFFINE 0x0020 /* Wake task to waking CPU */
-#define SD_PREFER_LOCAL 0x0040 /* Prefer to keep tasks local to this domain */
#define SD_SHARE_CPUPOWER 0x0080 /* Domain members share cpu power */
#define SD_SHARE_PKG_RESOURCES 0x0200 /* Domain members share cpu pkg resources */
#define SD_SERIALIZE 0x0400 /* Only a single load balancing instance */
@@ -954,7 +951,6 @@
unsigned int smt_gain;
int flags; /* See SD_* */
int level;
- int idle_buddy; /* cpu assigned to select_idle_sibling() */
/* Runtime fields. */
unsigned long last_balance; /* init to jiffies. units in jiffies */
@@ -1886,6 +1882,14 @@
#endif
+static inline void rcu_switch(struct task_struct *prev,
+ struct task_struct *next)
+{
+#ifdef CONFIG_RCU_USER_QS
+ rcu_user_hooks_switch(prev, next);
+#endif
+}
+
static inline void tsk_restore_flags(struct task_struct *task,
unsigned long orig_flags, unsigned long flags)
{
diff --git a/include/linux/screen_info.h b/include/linux/screen_info.h
index 899fbb4..fb3c5a8 100644
--- a/include/linux/screen_info.h
+++ b/include/linux/screen_info.h
@@ -68,6 +68,8 @@
#define VIDEO_FLAGS_NOCURSOR (1 << 0) /* The video mode has no cursor set */
+#define VIDEO_CAPABILITY_SKIP_QUIRKS (1 << 0)
+
#ifdef __KERNEL__
extern struct screen_info screen_info;
diff --git a/include/linux/security.h b/include/linux/security.h
index 3dea6a9..d143b8e 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -118,6 +118,7 @@
extern unsigned long mmap_min_addr;
extern unsigned long dac_mmap_min_addr;
#else
+#define mmap_min_addr 0UL
#define dac_mmap_min_addr 0UL
#endif
diff --git a/include/linux/smpboot.h b/include/linux/smpboot.h
new file mode 100644
index 0000000..e0106d8
--- /dev/null
+++ b/include/linux/smpboot.h
@@ -0,0 +1,43 @@
+#ifndef _LINUX_SMPBOOT_H
+#define _LINUX_SMPBOOT_H
+
+#include <linux/types.h>
+
+struct task_struct;
+/* Cookie handed to the thread_fn*/
+struct smpboot_thread_data;
+
+/**
+ * struct smp_hotplug_thread - CPU hotplug related thread descriptor
+ * @store: Pointer to per cpu storage for the task pointers
+ * @list: List head for core management
+ * @thread_should_run: Check whether the thread should run or not. Called with
+ * preemption disabled.
+ * @thread_fn: The associated thread function
+ * @setup: Optional setup function, called when the thread gets
+ * operational the first time
+ * @cleanup: Optional cleanup function, called when the thread
+ * should stop (module exit)
+ * @park: Optional park function, called when the thread is
+ * parked (cpu offline)
+ * @unpark: Optional unpark function, called when the thread is
+ * unparked (cpu online)
+ * @thread_comm: The base name of the thread
+ */
+struct smp_hotplug_thread {
+ struct task_struct __percpu **store;
+ struct list_head list;
+ int (*thread_should_run)(unsigned int cpu);
+ void (*thread_fn)(unsigned int cpu);
+ void (*setup)(unsigned int cpu);
+ void (*cleanup)(unsigned int cpu, bool online);
+ void (*park)(unsigned int cpu);
+ void (*unpark)(unsigned int cpu);
+ const char *thread_comm;
+};
+
+int smpboot_register_percpu_thread(struct smp_hotplug_thread *plug_thread);
+void smpboot_unregister_percpu_thread(struct smp_hotplug_thread *plug_thread);
+int smpboot_thread_schedule(void);
+
+#endif
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index cff40aa..bf8c49f 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -114,6 +114,7 @@
void (*set_buffer_size)(struct rpc_xprt *xprt, size_t sndsize, size_t rcvsize);
int (*reserve_xprt)(struct rpc_xprt *xprt, struct rpc_task *task);
void (*release_xprt)(struct rpc_xprt *xprt, struct rpc_task *task);
+ void (*alloc_slot)(struct rpc_xprt *xprt, struct rpc_task *task);
void (*rpcbind)(struct rpc_task *task);
void (*set_port)(struct rpc_xprt *xprt, unsigned short port);
void (*connect)(struct rpc_task *task);
@@ -281,6 +282,8 @@
void xprt_reserve(struct rpc_task *task);
int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task);
int xprt_reserve_xprt_cong(struct rpc_xprt *xprt, struct rpc_task *task);
+void xprt_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task);
+void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task);
int xprt_prepare_transmit(struct rpc_task *task);
void xprt_transmit(struct rpc_task *task);
void xprt_end_transmit(struct rpc_task *task);
diff --git a/include/linux/task_work.h b/include/linux/task_work.h
index fb46b03..ca5a1cf 100644
--- a/include/linux/task_work.h
+++ b/include/linux/task_work.h
@@ -18,8 +18,7 @@
static inline void exit_task_work(struct task_struct *task)
{
- if (unlikely(task->task_works))
- task_work_run();
+ task_work_run();
}
#endif /* _LINUX_TASK_WORK_H */
diff --git a/include/linux/time.h b/include/linux/time.h
index c81c5e4..b51e664 100644
--- a/include/linux/time.h
+++ b/include/linux/time.h
@@ -107,11 +107,36 @@
return ts_delta;
}
+#define KTIME_MAX ((s64)~((u64)1 << 63))
+#if (BITS_PER_LONG == 64)
+# define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
+#else
+# define KTIME_SEC_MAX LONG_MAX
+#endif
+
/*
* Returns true if the timespec is norm, false if denorm:
*/
-#define timespec_valid(ts) \
- (((ts)->tv_sec >= 0) && (((unsigned long) (ts)->tv_nsec) < NSEC_PER_SEC))
+static inline bool timespec_valid(const struct timespec *ts)
+{
+ /* Dates before 1970 are bogus */
+ if (ts->tv_sec < 0)
+ return false;
+ /* Can't have more nanoseconds then a second */
+ if ((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)
+ return false;
+ return true;
+}
+
+static inline bool timespec_valid_strict(const struct timespec *ts)
+{
+ if (!timespec_valid(ts))
+ return false;
+ /* Disallow values that could overflow ktime_t */
+ if ((unsigned long long)ts->tv_sec >= KTIME_SEC_MAX)
+ return false;
+ return true;
+}
extern void read_persistent_clock(struct timespec *ts);
extern void read_boot_clock(struct timespec *ts);
diff --git a/include/linux/timer.h b/include/linux/timer.h
index 6abd913..8c5a197 100644
--- a/include/linux/timer.h
+++ b/include/linux/timer.h
@@ -49,147 +49,112 @@
#endif
/*
- * Note that all tvec_bases are 2 byte aligned and lower bit of
- * base in timer_list is guaranteed to be zero. Use the LSB to
- * indicate whether the timer is deferrable.
+ * Note that all tvec_bases are at least 4 byte aligned and lower two bits
+ * of base in timer_list is guaranteed to be zero. Use them for flags.
*
* A deferrable timer will work normally when the system is busy, but
* will not cause a CPU to come out of idle just to service it; instead,
* the timer will be serviced when the CPU eventually wakes up with a
* subsequent non-deferrable timer.
+ *
+ * An irqsafe timer is executed with IRQ disabled and it's safe to wait for
+ * the completion of the running instance from IRQ handlers, for example,
+ * by calling del_timer_sync().
+ *
+ * Note: The irq disabled callback execution is a special case for
+ * workqueue locking issues. It's not meant for executing random crap
+ * with interrupts disabled. Abuse is monitored!
*/
-#define TBASE_DEFERRABLE_FLAG (0x1)
+#define TIMER_DEFERRABLE 0x1LU
+#define TIMER_IRQSAFE 0x2LU
-#define TIMER_INITIALIZER(_function, _expires, _data) { \
+#define TIMER_FLAG_MASK 0x3LU
+
+#define __TIMER_INITIALIZER(_function, _expires, _data, _flags) { \
.entry = { .prev = TIMER_ENTRY_STATIC }, \
.function = (_function), \
.expires = (_expires), \
.data = (_data), \
- .base = &boot_tvec_bases, \
+ .base = (void *)((unsigned long)&boot_tvec_bases + (_flags)), \
.slack = -1, \
__TIMER_LOCKDEP_MAP_INITIALIZER( \
__FILE__ ":" __stringify(__LINE__)) \
}
-#define TBASE_MAKE_DEFERRED(ptr) ((struct tvec_base *) \
- ((unsigned char *)(ptr) + TBASE_DEFERRABLE_FLAG))
+#define TIMER_INITIALIZER(_function, _expires, _data) \
+ __TIMER_INITIALIZER((_function), (_expires), (_data), 0)
-#define TIMER_DEFERRED_INITIALIZER(_function, _expires, _data) {\
- .entry = { .prev = TIMER_ENTRY_STATIC }, \
- .function = (_function), \
- .expires = (_expires), \
- .data = (_data), \
- .base = TBASE_MAKE_DEFERRED(&boot_tvec_bases), \
- __TIMER_LOCKDEP_MAP_INITIALIZER( \
- __FILE__ ":" __stringify(__LINE__)) \
- }
+#define TIMER_DEFERRED_INITIALIZER(_function, _expires, _data) \
+ __TIMER_INITIALIZER((_function), (_expires), (_data), TIMER_DEFERRABLE)
#define DEFINE_TIMER(_name, _function, _expires, _data) \
struct timer_list _name = \
TIMER_INITIALIZER(_function, _expires, _data)
-void init_timer_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key);
-void init_timer_deferrable_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key);
-
-#ifdef CONFIG_LOCKDEP
-#define init_timer(timer) \
- do { \
- static struct lock_class_key __key; \
- init_timer_key((timer), #timer, &__key); \
- } while (0)
-
-#define init_timer_deferrable(timer) \
- do { \
- static struct lock_class_key __key; \
- init_timer_deferrable_key((timer), #timer, &__key); \
- } while (0)
-
-#define init_timer_on_stack(timer) \
- do { \
- static struct lock_class_key __key; \
- init_timer_on_stack_key((timer), #timer, &__key); \
- } while (0)
-
-#define setup_timer(timer, fn, data) \
- do { \
- static struct lock_class_key __key; \
- setup_timer_key((timer), #timer, &__key, (fn), (data));\
- } while (0)
-
-#define setup_timer_on_stack(timer, fn, data) \
- do { \
- static struct lock_class_key __key; \
- setup_timer_on_stack_key((timer), #timer, &__key, \
- (fn), (data)); \
- } while (0)
-#define setup_deferrable_timer_on_stack(timer, fn, data) \
- do { \
- static struct lock_class_key __key; \
- setup_deferrable_timer_on_stack_key((timer), #timer, \
- &__key, (fn), \
- (data)); \
- } while (0)
-#else
-#define init_timer(timer)\
- init_timer_key((timer), NULL, NULL)
-#define init_timer_deferrable(timer)\
- init_timer_deferrable_key((timer), NULL, NULL)
-#define init_timer_on_stack(timer)\
- init_timer_on_stack_key((timer), NULL, NULL)
-#define setup_timer(timer, fn, data)\
- setup_timer_key((timer), NULL, NULL, (fn), (data))
-#define setup_timer_on_stack(timer, fn, data)\
- setup_timer_on_stack_key((timer), NULL, NULL, (fn), (data))
-#define setup_deferrable_timer_on_stack(timer, fn, data)\
- setup_deferrable_timer_on_stack_key((timer), NULL, NULL, (fn), (data))
-#endif
+void init_timer_key(struct timer_list *timer, unsigned int flags,
+ const char *name, struct lock_class_key *key);
#ifdef CONFIG_DEBUG_OBJECTS_TIMERS
extern void init_timer_on_stack_key(struct timer_list *timer,
- const char *name,
+ unsigned int flags, const char *name,
struct lock_class_key *key);
extern void destroy_timer_on_stack(struct timer_list *timer);
#else
static inline void destroy_timer_on_stack(struct timer_list *timer) { }
static inline void init_timer_on_stack_key(struct timer_list *timer,
- const char *name,
+ unsigned int flags, const char *name,
struct lock_class_key *key)
{
- init_timer_key(timer, name, key);
+ init_timer_key(timer, flags, name, key);
}
#endif
-static inline void setup_timer_key(struct timer_list * timer,
- const char *name,
- struct lock_class_key *key,
- void (*function)(unsigned long),
- unsigned long data)
-{
- timer->function = function;
- timer->data = data;
- init_timer_key(timer, name, key);
-}
+#ifdef CONFIG_LOCKDEP
+#define __init_timer(_timer, _flags) \
+ do { \
+ static struct lock_class_key __key; \
+ init_timer_key((_timer), (_flags), #_timer, &__key); \
+ } while (0)
-static inline void setup_timer_on_stack_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key,
- void (*function)(unsigned long),
- unsigned long data)
-{
- timer->function = function;
- timer->data = data;
- init_timer_on_stack_key(timer, name, key);
-}
+#define __init_timer_on_stack(_timer, _flags) \
+ do { \
+ static struct lock_class_key __key; \
+ init_timer_on_stack_key((_timer), (_flags), #_timer, &__key); \
+ } while (0)
+#else
+#define __init_timer(_timer, _flags) \
+ init_timer_key((_timer), (_flags), NULL, NULL)
+#define __init_timer_on_stack(_timer, _flags) \
+ init_timer_on_stack_key((_timer), (_flags), NULL, NULL)
+#endif
-extern void setup_deferrable_timer_on_stack_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key,
- void (*function)(unsigned long),
- unsigned long data);
+#define init_timer(timer) \
+ __init_timer((timer), 0)
+#define init_timer_deferrable(timer) \
+ __init_timer((timer), TIMER_DEFERRABLE)
+#define init_timer_on_stack(timer) \
+ __init_timer_on_stack((timer), 0)
+
+#define __setup_timer(_timer, _fn, _data, _flags) \
+ do { \
+ __init_timer((_timer), (_flags)); \
+ (_timer)->function = (_fn); \
+ (_timer)->data = (_data); \
+ } while (0)
+
+#define __setup_timer_on_stack(_timer, _fn, _data, _flags) \
+ do { \
+ __init_timer_on_stack((_timer), (_flags)); \
+ (_timer)->function = (_fn); \
+ (_timer)->data = (_data); \
+ } while (0)
+
+#define setup_timer(timer, fn, data) \
+ __setup_timer((timer), (fn), (data), 0)
+#define setup_timer_on_stack(timer, fn, data) \
+ __setup_timer_on_stack((timer), (fn), (data), 0)
+#define setup_deferrable_timer_on_stack(timer, fn, data) \
+ __setup_timer_on_stack((timer), (fn), (data), TIMER_DEFERRABLE)
/**
* timer_pending - is a timer pending?
diff --git a/include/linux/topology.h b/include/linux/topology.h
index fec12d6..d3cf0d6 100644
--- a/include/linux/topology.h
+++ b/include/linux/topology.h
@@ -129,7 +129,6 @@
| 1*SD_BALANCE_FORK \
| 0*SD_BALANCE_WAKE \
| 1*SD_WAKE_AFFINE \
- | 0*SD_PREFER_LOCAL \
| 0*SD_SHARE_CPUPOWER \
| 1*SD_SHARE_PKG_RESOURCES \
| 0*SD_SERIALIZE \
@@ -160,7 +159,6 @@
| 1*SD_BALANCE_FORK \
| 0*SD_BALANCE_WAKE \
| 1*SD_WAKE_AFFINE \
- | 0*SD_PREFER_LOCAL \
| 0*SD_SHARE_CPUPOWER \
| 0*SD_SHARE_PKG_RESOURCES \
| 0*SD_SERIALIZE \
diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h
index 802de56..2f322c3 100644
--- a/include/linux/tracepoint.h
+++ b/include/linux/tracepoint.h
@@ -136,6 +136,22 @@
postrcu; \
} while (0)
+#ifndef MODULE
+#define __DECLARE_TRACE_RCU(name, proto, args, cond, data_proto, data_args) \
+ static inline void trace_##name##_rcuidle(proto) \
+ { \
+ if (static_key_false(&__tracepoint_##name.key)) \
+ __DO_TRACE(&__tracepoint_##name, \
+ TP_PROTO(data_proto), \
+ TP_ARGS(data_args), \
+ TP_CONDITION(cond), \
+ rcu_idle_exit(), \
+ rcu_idle_enter()); \
+ }
+#else
+#define __DECLARE_TRACE_RCU(name, proto, args, cond, data_proto, data_args)
+#endif
+
/*
* Make sure the alignment of the structure in the __tracepoints section will
* not add unwanted padding between the beginning of the section and the
@@ -151,16 +167,8 @@
TP_ARGS(data_args), \
TP_CONDITION(cond),,); \
} \
- static inline void trace_##name##_rcuidle(proto) \
- { \
- if (static_key_false(&__tracepoint_##name.key)) \
- __DO_TRACE(&__tracepoint_##name, \
- TP_PROTO(data_proto), \
- TP_ARGS(data_args), \
- TP_CONDITION(cond), \
- rcu_idle_exit(), \
- rcu_idle_enter()); \
- } \
+ __DECLARE_TRACE_RCU(name, PARAMS(proto), PARAMS(args), \
+ PARAMS(cond), PARAMS(data_proto), PARAMS(data_args)) \
static inline int \
register_trace_##name(void (*probe)(data_proto), void *data) \
{ \
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index efe4b33..e6f0331 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -99,25 +99,27 @@
struct uprobes_state {
struct xol_area *xol_area;
- atomic_t count;
};
+
extern int __weak set_swbp(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
-extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr, bool verify);
+extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
extern bool __weak is_swbp_insn(uprobe_opcode_t *insn);
extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern void uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
extern int uprobe_mmap(struct vm_area_struct *vma);
extern void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end);
+extern void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm);
extern void uprobe_free_utask(struct task_struct *t);
extern void uprobe_copy_process(struct task_struct *t);
extern unsigned long __weak uprobe_get_swbp_addr(struct pt_regs *regs);
+extern void __weak arch_uprobe_enable_step(struct arch_uprobe *arch);
+extern void __weak arch_uprobe_disable_step(struct arch_uprobe *arch);
extern int uprobe_post_sstep_notifier(struct pt_regs *regs);
extern int uprobe_pre_sstep_notifier(struct pt_regs *regs);
extern void uprobe_notify_resume(struct pt_regs *regs);
extern bool uprobe_deny_signal(void);
extern bool __weak arch_uprobe_skip_sstep(struct arch_uprobe *aup, struct pt_regs *regs);
extern void uprobe_clear_state(struct mm_struct *mm);
-extern void uprobe_reset_state(struct mm_struct *mm);
#else /* !CONFIG_UPROBES */
struct uprobes_state {
};
@@ -138,6 +140,10 @@
uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end)
{
}
+static inline void
+uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
+{
+}
static inline void uprobe_notify_resume(struct pt_regs *regs)
{
}
@@ -158,8 +164,5 @@
static inline void uprobe_clear_state(struct mm_struct *mm)
{
}
-static inline void uprobe_reset_state(struct mm_struct *mm)
-{
-}
#endif /* !CONFIG_UPROBES */
#endif /* _LINUX_UPROBES_H */
diff --git a/include/linux/xfrm.h b/include/linux/xfrm.h
index 22e61fd..28e493b 100644
--- a/include/linux/xfrm.h
+++ b/include/linux/xfrm.h
@@ -84,6 +84,8 @@
__u32 bitmap;
};
+#define XFRMA_REPLAY_ESN_MAX 4096
+
struct xfrm_replay_state_esn {
unsigned int bmp_len;
__u32 oseq;
diff --git a/include/net/bluetooth/smp.h b/include/net/bluetooth/smp.h
index ca356a7..8b27927 100644
--- a/include/net/bluetooth/smp.h
+++ b/include/net/bluetooth/smp.h
@@ -136,7 +136,7 @@
};
/* SMP Commands */
-int smp_conn_security(struct l2cap_conn *conn, __u8 sec_level);
+int smp_conn_security(struct hci_conn *hcon, __u8 sec_level);
int smp_sig_channel(struct l2cap_conn *conn, struct sk_buff *skb);
int smp_distribute_keys(struct l2cap_conn *conn, __u8 force);
int smp_user_confirm_reply(struct hci_conn *conn, u16 mgmt_op, __le32 passkey);
diff --git a/include/net/ip6_fib.h b/include/net/ip6_fib.h
index 0fedbd8..9fc7114 100644
--- a/include/net/ip6_fib.h
+++ b/include/net/ip6_fib.h
@@ -111,9 +111,8 @@
struct inet6_dev *rt6i_idev;
unsigned long _rt6i_peer;
-#ifdef CONFIG_XFRM
- u32 rt6i_flow_cache_genid;
-#endif
+ u32 rt6i_genid;
+
/* more non-fragment space at head required */
unsigned short rt6i_nfheader_len;
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index ae1cd6c..fd87963 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -102,6 +102,7 @@
#endif
struct netns_ipvs *ipvs;
struct sock *diag_nlsk;
+ atomic_t rt_genid;
};
@@ -300,5 +301,14 @@
}
#endif
+static inline int rt_genid(struct net *net)
+{
+ return atomic_read(&net->rt_genid);
+}
+
+static inline void rt_genid_bump(struct net *net)
+{
+ atomic_inc(&net->rt_genid);
+}
#endif /* __NET_NET_NAMESPACE_H */
diff --git a/include/net/netfilter/nf_conntrack_ecache.h b/include/net/netfilter/nf_conntrack_ecache.h
index e1ce104..4a045cd 100644
--- a/include/net/netfilter/nf_conntrack_ecache.h
+++ b/include/net/netfilter/nf_conntrack_ecache.h
@@ -18,6 +18,7 @@
u16 ctmask; /* bitmask of ct events to be delivered */
u16 expmask; /* bitmask of expect events to be delivered */
u32 pid; /* netlink pid of destroyer */
+ struct timer_list timeout;
};
static inline struct nf_conntrack_ecache *
diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h
index 1474dd6..eb24dbc 100644
--- a/include/net/netns/ipv4.h
+++ b/include/net/netns/ipv4.h
@@ -65,7 +65,6 @@
unsigned int sysctl_ping_group_range[2];
long sysctl_tcp_mem[3];
- atomic_t rt_genid;
atomic_t dev_addr_genid;
#ifdef CONFIG_IP_MROUTE
diff --git a/include/net/route.h b/include/net/route.h
index 776a27f..da22243 100644
--- a/include/net/route.h
+++ b/include/net/route.h
@@ -108,7 +108,7 @@
struct in_device;
extern int ip_rt_init(void);
-extern void rt_cache_flush(struct net *net, int how);
+extern void rt_cache_flush(struct net *net);
extern void rt_flush_dev(struct net_device *dev);
extern struct rtable *__ip_route_output_key(struct net *, struct flowi4 *flp);
extern struct rtable *ip_route_output_flow(struct net *, struct flowi4 *flp,
diff --git a/include/net/sock.h b/include/net/sock.h
index 72132ae..adb7da2 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1332,7 +1332,7 @@
}
static inline bool
-sk_rmem_schedule(struct sock *sk, struct sk_buff *skb, unsigned int size)
+sk_rmem_schedule(struct sock *sk, struct sk_buff *skb, int size)
{
if (!sk_has_account(sk))
return true;
diff --git a/include/net/xfrm.h b/include/net/xfrm.h
index 976a81a..639dd13 100644
--- a/include/net/xfrm.h
+++ b/include/net/xfrm.h
@@ -273,6 +273,9 @@
int (*check)(struct xfrm_state *x,
struct sk_buff *skb,
__be32 net_seq);
+ int (*recheck)(struct xfrm_state *x,
+ struct sk_buff *skb,
+ __be32 net_seq);
void (*notify)(struct xfrm_state *x, int event);
int (*overflow)(struct xfrm_state *x, struct sk_buff *skb);
};
diff --git a/include/target/target_core_backend.h b/include/target/target_core_backend.h
index f1405d33..941c84bf 100644
--- a/include/target/target_core_backend.h
+++ b/include/target/target_core_backend.h
@@ -23,7 +23,9 @@
struct se_device *(*create_virtdevice)(struct se_hba *,
struct se_subsystem_dev *, void *);
void (*free_device)(void *);
- int (*transport_complete)(struct se_cmd *cmd, struct scatterlist *);
+ void (*transport_complete)(struct se_cmd *cmd,
+ struct scatterlist *,
+ unsigned char *);
int (*parse_cdb)(struct se_cmd *cmd);
ssize_t (*check_configfs_dev_params)(struct se_hba *,
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 015cea0..5be8937 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -121,6 +121,7 @@
#define SE_INQUIRY_BUF 512
#define SE_MODE_PAGE_BUF 512
+#define SE_SENSE_BUF 96
/* struct se_hba->hba_flags */
enum hba_flags_table {
diff --git a/include/trace/define_trace.h b/include/trace/define_trace.h
index b0b4eb2..1905ca8 100644
--- a/include/trace/define_trace.h
+++ b/include/trace/define_trace.h
@@ -1,5 +1,5 @@
/*
- * Trace files that want to automate creationg of all tracepoints defined
+ * Trace files that want to automate creation of all tracepoints defined
* in their file should include this file. The following are macros that the
* trace file may define:
*
diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h
index 5f889f1..08fa272 100644
--- a/include/trace/events/kmem.h
+++ b/include/trace/events/kmem.h
@@ -214,7 +214,7 @@
TP_printk("page=%p pfn=%lu order=%d migratetype=%d gfp_flags=%s",
__entry->page,
- page_to_pfn(__entry->page),
+ __entry->page ? page_to_pfn(__entry->page) : 0,
__entry->order,
__entry->migratetype,
show_gfp_flags(__entry->gfp_flags))
@@ -240,7 +240,7 @@
TP_printk("page=%p pfn=%lu order=%u migratetype=%d percpu_refill=%d",
__entry->page,
- page_to_pfn(__entry->page),
+ __entry->page ? page_to_pfn(__entry->page) : 0,
__entry->order,
__entry->migratetype,
__entry->order == 0)
diff --git a/include/xen/events.h b/include/xen/events.h
index 9c641de..04399b2 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -58,8 +58,6 @@
void xen_irq_resume(void);
-void xen_hvm_prepare_kexec(struct shared_info *sip, unsigned long pfn);
-
/* Clear an irq's pending state, in preparation for polling on it */
void xen_clear_irq_pending(int irq);
void xen_set_irq_pending(int irq);
diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h
index 11e27c3..f19fff8 100644
--- a/include/xen/grant_table.h
+++ b/include/xen/grant_table.h
@@ -187,6 +187,7 @@
struct gnttab_map_grant_ref *kmap_ops,
struct page **pages, unsigned int count);
int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops,
- struct page **pages, unsigned int count, bool clear_pte);
+ struct gnttab_map_grant_ref *kunmap_ops,
+ struct page **pages, unsigned int count);
#endif /* __ASM_GNTTAB_H__ */
diff --git a/init/Kconfig b/init/Kconfig
index af6c7f8..495e6e9 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -267,31 +267,6 @@
depends on SYSCTL
default y
-config BSD_PROCESS_ACCT
- bool "BSD Process Accounting"
- help
- If you say Y here, a user level program will be able to instruct the
- kernel (via a special system call) to write process accounting
- information to a file: whenever a process exits, information about
- that process will be appended to the file by the kernel. The
- information includes things such as creation time, owning user,
- command name, memory usage, controlling terminal etc. (the complete
- list is in the struct acct in <file:include/linux/acct.h>). It is
- up to the user level program to do useful things with this
- information. This is generally a good idea, so say Y.
-
-config BSD_PROCESS_ACCT_V3
- bool "BSD Process Accounting version 3 file format"
- depends on BSD_PROCESS_ACCT
- default n
- help
- If you say Y here, the process accounting information is written
- in a new file format that also logs the process IDs of each
- process and it's parent. Note that this file format is incompatible
- with previous v0/v1/v2 file formats, so you will need updated tools
- for processing it. A preliminary version of these tools is available
- at <http://www.gnu.org/software/acct/>.
-
config FHANDLE
bool "open by fhandle syscalls"
select EXPORTFS
@@ -304,48 +279,6 @@
get renamed. Enables open_by_handle_at(2) and name_to_handle_at(2)
syscalls.
-config TASKSTATS
- bool "Export task/process statistics through netlink (EXPERIMENTAL)"
- depends on NET
- default n
- help
- Export selected statistics for tasks/processes through the
- generic netlink interface. Unlike BSD process accounting, the
- statistics are available during the lifetime of tasks/processes as
- responses to commands. Like BSD accounting, they are sent to user
- space on task exit.
-
- Say N if unsure.
-
-config TASK_DELAY_ACCT
- bool "Enable per-task delay accounting (EXPERIMENTAL)"
- depends on TASKSTATS
- help
- Collect information on time spent by a task waiting for system
- resources like cpu, synchronous block I/O completion and swapping
- in pages. Such statistics can help in setting a task's priorities
- relative to other tasks for cpu, io, rss limits etc.
-
- Say N if unsure.
-
-config TASK_XACCT
- bool "Enable extended accounting over taskstats (EXPERIMENTAL)"
- depends on TASKSTATS
- help
- Collect extended task accounting data and send the data
- to userland for processing over the taskstats interface.
-
- Say N if unsure.
-
-config TASK_IO_ACCOUNTING
- bool "Enable per-task storage I/O accounting (EXPERIMENTAL)"
- depends on TASK_XACCT
- help
- Collect information on the number of bytes of storage I/O which this
- task has caused.
-
- Say N if unsure.
-
config AUDIT
bool "Auditing support"
depends on NET
@@ -391,6 +324,118 @@
source "kernel/irq/Kconfig"
source "kernel/time/Kconfig"
+menu "CPU/Task time and stats accounting"
+
+choice
+ prompt "Cputime accounting"
+ default TICK_CPU_ACCOUNTING if !PPC64
+ default VIRT_CPU_ACCOUNTING if PPC64
+
+# Kind of a stub config for the pure tick based cputime accounting
+config TICK_CPU_ACCOUNTING
+ bool "Simple tick based cputime accounting"
+ depends on !S390
+ help
+ This is the basic tick based cputime accounting that maintains
+ statistics about user, system and idle time spent on per jiffies
+ granularity.
+
+ If unsure, say Y.
+
+config VIRT_CPU_ACCOUNTING
+ bool "Deterministic task and CPU time accounting"
+ depends on HAVE_VIRT_CPU_ACCOUNTING
+ help
+ Select this option to enable more accurate task and CPU time
+ accounting. This is done by reading a CPU counter on each
+ kernel entry and exit and on transitions within the kernel
+ between system, softirq and hardirq state, so there is a
+ small performance impact. In the case of s390 or IBM POWER > 5,
+ this also enables accounting of stolen time on logically-partitioned
+ systems.
+
+config IRQ_TIME_ACCOUNTING
+ bool "Fine granularity task level IRQ time accounting"
+ depends on HAVE_IRQ_TIME_ACCOUNTING
+ help
+ Select this option to enable fine granularity task irq time
+ accounting. This is done by reading a timestamp on each
+ transitions between softirq and hardirq state, so there can be a
+ small performance impact.
+
+ If in doubt, say N here.
+
+endchoice
+
+config BSD_PROCESS_ACCT
+ bool "BSD Process Accounting"
+ help
+ If you say Y here, a user level program will be able to instruct the
+ kernel (via a special system call) to write process accounting
+ information to a file: whenever a process exits, information about
+ that process will be appended to the file by the kernel. The
+ information includes things such as creation time, owning user,
+ command name, memory usage, controlling terminal etc. (the complete
+ list is in the struct acct in <file:include/linux/acct.h>). It is
+ up to the user level program to do useful things with this
+ information. This is generally a good idea, so say Y.
+
+config BSD_PROCESS_ACCT_V3
+ bool "BSD Process Accounting version 3 file format"
+ depends on BSD_PROCESS_ACCT
+ default n
+ help
+ If you say Y here, the process accounting information is written
+ in a new file format that also logs the process IDs of each
+ process and it's parent. Note that this file format is incompatible
+ with previous v0/v1/v2 file formats, so you will need updated tools
+ for processing it. A preliminary version of these tools is available
+ at <http://www.gnu.org/software/acct/>.
+
+config TASKSTATS
+ bool "Export task/process statistics through netlink (EXPERIMENTAL)"
+ depends on NET
+ default n
+ help
+ Export selected statistics for tasks/processes through the
+ generic netlink interface. Unlike BSD process accounting, the
+ statistics are available during the lifetime of tasks/processes as
+ responses to commands. Like BSD accounting, they are sent to user
+ space on task exit.
+
+ Say N if unsure.
+
+config TASK_DELAY_ACCT
+ bool "Enable per-task delay accounting (EXPERIMENTAL)"
+ depends on TASKSTATS
+ help
+ Collect information on time spent by a task waiting for system
+ resources like cpu, synchronous block I/O completion and swapping
+ in pages. Such statistics can help in setting a task's priorities
+ relative to other tasks for cpu, io, rss limits etc.
+
+ Say N if unsure.
+
+config TASK_XACCT
+ bool "Enable extended accounting over taskstats (EXPERIMENTAL)"
+ depends on TASKSTATS
+ help
+ Collect extended task accounting data and send the data
+ to userland for processing over the taskstats interface.
+
+ Say N if unsure.
+
+config TASK_IO_ACCOUNTING
+ bool "Enable per-task storage I/O accounting (EXPERIMENTAL)"
+ depends on TASK_XACCT
+ help
+ Collect information on the number of bytes of storage I/O which this
+ task has caused.
+
+ Say N if unsure.
+
+endmenu # "CPU/Task time and stats accounting"
+
menu "RCU Subsystem"
choice
@@ -441,6 +486,24 @@
This option enables preemptible-RCU code that is common between
the TREE_PREEMPT_RCU and TINY_PREEMPT_RCU implementations.
+config RCU_USER_QS
+ bool "Consider userspace as in RCU extended quiescent state"
+ depends on HAVE_RCU_USER_QS && SMP
+ help
+ This option sets hooks on kernel / userspace boundaries and
+ puts RCU in extended quiescent state when the CPU runs in
+ userspace. It means that when a CPU runs in userspace, it is
+ excluded from the global RCU state machine and thus doesn't
+ to keep the timer tick on for RCU.
+
+config RCU_USER_QS_FORCE
+ bool "Force userspace extended QS by default"
+ depends on RCU_USER_QS
+ help
+ Set the hooks in user/kernel boundaries by default in order to
+ test this feature that treats userspace as an extended quiescent
+ state until we have a real user like a full adaptive nohz option.
+
config RCU_FANOUT
int "Tree-based hierarchical RCU fanout value"
range 2 64 if 64BIT
@@ -1153,7 +1216,8 @@
config UID16
bool "Enable 16-bit UID system calls" if EXPERT
- depends on ARM || BLACKFIN || CRIS || FRV || H8300 || X86_32 || M68K || (S390 && !64BIT) || SUPERH || SPARC32 || (SPARC64 && COMPAT) || UML || (X86_64 && IA32_EMULATION)
+ depends on ARM || BLACKFIN || CRIS || FRV || H8300 || X86_32 || M68K || (S390 && !64BIT) || SUPERH || SPARC32 || (SPARC64 && COMPAT) || UML || (X86_64 && IA32_EMULATION) \
+ || AARCH32_EMULATION
default y
help
This enables the legacy 16-bit UID syscall wrappers.
diff --git a/init/main.c b/init/main.c
index b286730..db34c0e 100644
--- a/init/main.c
+++ b/init/main.c
@@ -631,6 +631,11 @@
acpi_early_init(); /* before LAPIC and SMP init */
sfi_init_late();
+ if (efi_enabled) {
+ efi_late_init();
+ efi_free_boot_services();
+ }
+
ftrace_init();
/* Do the rest non-__init'ed, we're now alive */
diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks
index 2251882..44511d1 100644
--- a/kernel/Kconfig.locks
+++ b/kernel/Kconfig.locks
@@ -87,6 +87,9 @@
config ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE
bool
+config UNINLINE_SPIN_UNLOCK
+ bool
+
#
# lock_* functions are inlined when:
# - DEBUG_SPINLOCK=n and GENERIC_LOCKBREAK=n and ARCH_INLINE_*LOCK=y
@@ -103,100 +106,120 @@
# - DEBUG_SPINLOCK=n and ARCH_INLINE_*LOCK=y
#
+if !DEBUG_SPINLOCK
+
config INLINE_SPIN_TRYLOCK
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_SPIN_TRYLOCK
+ def_bool y
+ depends on ARCH_INLINE_SPIN_TRYLOCK
config INLINE_SPIN_TRYLOCK_BH
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_SPIN_TRYLOCK_BH
+ def_bool y
+ depends on ARCH_INLINE_SPIN_TRYLOCK_BH
config INLINE_SPIN_LOCK
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && ARCH_INLINE_SPIN_LOCK
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_SPIN_LOCK
config INLINE_SPIN_LOCK_BH
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_SPIN_LOCK_BH
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_SPIN_LOCK_BH
config INLINE_SPIN_LOCK_IRQ
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_SPIN_LOCK_IRQ
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_SPIN_LOCK_IRQ
config INLINE_SPIN_LOCK_IRQSAVE
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_SPIN_LOCK_IRQSAVE
-
-config UNINLINE_SPIN_UNLOCK
- bool
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_SPIN_LOCK_IRQSAVE
config INLINE_SPIN_UNLOCK_BH
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_SPIN_UNLOCK_BH
+ def_bool y
+ depends on ARCH_INLINE_SPIN_UNLOCK_BH
config INLINE_SPIN_UNLOCK_IRQ
- def_bool !DEBUG_SPINLOCK && (!PREEMPT || ARCH_INLINE_SPIN_UNLOCK_BH)
+ def_bool y
+ depends on !PREEMPT || ARCH_INLINE_SPIN_UNLOCK_BH
config INLINE_SPIN_UNLOCK_IRQRESTORE
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE
+ def_bool y
+ depends on ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE
config INLINE_READ_TRYLOCK
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_READ_TRYLOCK
+ def_bool y
+ depends on ARCH_INLINE_READ_TRYLOCK
config INLINE_READ_LOCK
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && ARCH_INLINE_READ_LOCK
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_READ_LOCK
config INLINE_READ_LOCK_BH
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_READ_LOCK_BH
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_READ_LOCK_BH
config INLINE_READ_LOCK_IRQ
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_READ_LOCK_IRQ
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_READ_LOCK_IRQ
config INLINE_READ_LOCK_IRQSAVE
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_READ_LOCK_IRQSAVE
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_READ_LOCK_IRQSAVE
config INLINE_READ_UNLOCK
- def_bool !DEBUG_SPINLOCK && (!PREEMPT || ARCH_INLINE_READ_UNLOCK)
+ def_bool y
+ depends on !PREEMPT || ARCH_INLINE_READ_UNLOCK
config INLINE_READ_UNLOCK_BH
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_READ_UNLOCK_BH
+ def_bool y
+ depends on ARCH_INLINE_READ_UNLOCK_BH
config INLINE_READ_UNLOCK_IRQ
- def_bool !DEBUG_SPINLOCK && (!PREEMPT || ARCH_INLINE_READ_UNLOCK_BH)
+ def_bool y
+ depends on !PREEMPT || ARCH_INLINE_READ_UNLOCK_BH
config INLINE_READ_UNLOCK_IRQRESTORE
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_READ_UNLOCK_IRQRESTORE
+ def_bool y
+ depends on ARCH_INLINE_READ_UNLOCK_IRQRESTORE
config INLINE_WRITE_TRYLOCK
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_WRITE_TRYLOCK
+ def_bool y
+ depends on ARCH_INLINE_WRITE_TRYLOCK
config INLINE_WRITE_LOCK
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && ARCH_INLINE_WRITE_LOCK
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_WRITE_LOCK
config INLINE_WRITE_LOCK_BH
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_WRITE_LOCK_BH
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_WRITE_LOCK_BH
config INLINE_WRITE_LOCK_IRQ
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_WRITE_LOCK_IRQ
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_WRITE_LOCK_IRQ
config INLINE_WRITE_LOCK_IRQSAVE
- def_bool !DEBUG_SPINLOCK && !GENERIC_LOCKBREAK && \
- ARCH_INLINE_WRITE_LOCK_IRQSAVE
+ def_bool y
+ depends on !GENERIC_LOCKBREAK && ARCH_INLINE_WRITE_LOCK_IRQSAVE
config INLINE_WRITE_UNLOCK
- def_bool !DEBUG_SPINLOCK && (!PREEMPT || ARCH_INLINE_WRITE_UNLOCK)
+ def_bool y
+ depends on !PREEMPT || ARCH_INLINE_WRITE_UNLOCK
config INLINE_WRITE_UNLOCK_BH
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_WRITE_UNLOCK_BH
+ def_bool y
+ depends on ARCH_INLINE_WRITE_UNLOCK_BH
config INLINE_WRITE_UNLOCK_IRQ
- def_bool !DEBUG_SPINLOCK && (!PREEMPT || ARCH_INLINE_WRITE_UNLOCK_BH)
+ def_bool y
+ depends on !PREEMPT || ARCH_INLINE_WRITE_UNLOCK_BH
config INLINE_WRITE_UNLOCK_IRQRESTORE
- def_bool !DEBUG_SPINLOCK && ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE
+ def_bool y
+ depends on ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE
+
+endif
config MUTEX_SPIN_ON_OWNER
- def_bool SMP && !DEBUG_MUTEXES
+ def_bool y
+ depends on SMP && !DEBUG_MUTEXES
diff --git a/kernel/Makefile b/kernel/Makefile
index c0cc67a..5404911 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -10,7 +10,7 @@
kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \
hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \
notifier.o ksysfs.o cred.o \
- async.o range.o groups.o lglock.o
+ async.o range.o groups.o lglock.o smpboot.o
ifdef CONFIG_FUNCTION_TRACER
# Do not trace debug files and internal ftrace files
@@ -46,7 +46,6 @@
obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o
obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o
obj-$(CONFIG_SMP) += smp.o
-obj-$(CONFIG_SMP) += smpboot.o
ifneq ($(CONFIG_SMP),y)
obj-y += up.o
endif
@@ -98,7 +97,7 @@
obj-$(CONFIG_BINFMT_ELF_FDPIC) += elfcore.o
obj-$(CONFIG_FUNCTION_TRACER) += trace/
obj-$(CONFIG_TRACING) += trace/
-obj-$(CONFIG_X86_DS) += trace/
+obj-$(CONFIG_TRACE_CLOCK) += trace/
obj-$(CONFIG_RING_BUFFER) += trace/
obj-$(CONFIG_TRACEPOINTS) += trace/
obj-$(CONFIG_IRQ_WORK) += irq_work.o
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 14d3258..f560598 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -280,12 +280,13 @@
__func__, cpu);
goto out_release;
}
+ smpboot_park_threads(cpu);
err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
if (err) {
/* CPU didn't die: tell everyone. Can't complain. */
+ smpboot_unpark_threads(cpu);
cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
-
goto out_release;
}
BUG_ON(cpu_online(cpu));
@@ -354,6 +355,10 @@
goto out;
}
+ ret = smpboot_create_threads(cpu);
+ if (ret)
+ goto out;
+
ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
if (ret) {
nr_calls--;
@@ -368,6 +373,9 @@
goto out_notify;
BUG_ON(!cpu_online(cpu));
+ /* Wake the per cpu threads */
+ smpboot_unpark_threads(cpu);
+
/* Now call notifier in preparation. */
cpu_notify(CPU_ONLINE | mod, hcpu);
@@ -439,14 +447,6 @@
#ifdef CONFIG_PM_SLEEP_SMP
static cpumask_var_t frozen_cpus;
-void __weak arch_disable_nonboot_cpus_begin(void)
-{
-}
-
-void __weak arch_disable_nonboot_cpus_end(void)
-{
-}
-
int disable_nonboot_cpus(void)
{
int cpu, first_cpu, error = 0;
@@ -458,7 +458,6 @@
* with the userspace trying to use the CPU hotplug at the same time
*/
cpumask_clear(frozen_cpus);
- arch_disable_nonboot_cpus_begin();
printk("Disabling non-boot CPUs ...\n");
for_each_online_cpu(cpu) {
@@ -474,8 +473,6 @@
}
}
- arch_disable_nonboot_cpus_end();
-
if (!error) {
BUG_ON(num_online_cpus() > 1);
/* Make sure the CPUs won't be enabled by someone else */
diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c
index 98d4597..c772061 100644
--- a/kernel/events/callchain.c
+++ b/kernel/events/callchain.c
@@ -159,6 +159,11 @@
int rctx;
struct perf_callchain_entry *entry;
+ int kernel = !event->attr.exclude_callchain_kernel;
+ int user = !event->attr.exclude_callchain_user;
+
+ if (!kernel && !user)
+ return NULL;
entry = get_callchain_entry(&rctx);
if (rctx == -1)
@@ -169,24 +174,29 @@
entry->nr = 0;
- if (!user_mode(regs)) {
+ if (kernel && !user_mode(regs)) {
perf_callchain_store(entry, PERF_CONTEXT_KERNEL);
perf_callchain_kernel(entry, regs);
- if (current->mm)
- regs = task_pt_regs(current);
- else
- regs = NULL;
}
- if (regs) {
- /*
- * Disallow cross-task user callchains.
- */
- if (event->ctx->task && event->ctx->task != current)
- goto exit_put;
+ if (user) {
+ if (!user_mode(regs)) {
+ if (current->mm)
+ regs = task_pt_regs(current);
+ else
+ regs = NULL;
+ }
- perf_callchain_store(entry, PERF_CONTEXT_USER);
- perf_callchain_user(entry, regs);
+ if (regs) {
+ /*
+ * Disallow cross-task user callchains.
+ */
+ if (event->ctx->task && event->ctx->task != current)
+ goto exit_put;
+
+ perf_callchain_store(entry, PERF_CONTEXT_USER);
+ perf_callchain_user(entry, regs);
+ }
}
exit_put:
diff --git a/kernel/events/core.c b/kernel/events/core.c
index b7935fc..7b9df35 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -36,6 +36,7 @@
#include <linux/perf_event.h>
#include <linux/ftrace_event.h>
#include <linux/hw_breakpoint.h>
+#include <linux/mm_types.h>
#include "internal.h"
@@ -1253,7 +1254,7 @@
/*
* Cross CPU call to disable a performance event
*/
-static int __perf_event_disable(void *info)
+int __perf_event_disable(void *info)
{
struct perf_event *event = info;
struct perf_event_context *ctx = event->ctx;
@@ -2935,12 +2936,12 @@
/*
* Called when the last reference to the file is gone.
*/
-static int perf_release(struct inode *inode, struct file *file)
+static void put_event(struct perf_event *event)
{
- struct perf_event *event = file->private_data;
struct task_struct *owner;
- file->private_data = NULL;
+ if (!atomic_long_dec_and_test(&event->refcount))
+ return;
rcu_read_lock();
owner = ACCESS_ONCE(event->owner);
@@ -2975,7 +2976,13 @@
put_task_struct(owner);
}
- return perf_event_release_kernel(event);
+ perf_event_release_kernel(event);
+}
+
+static int perf_release(struct inode *inode, struct file *file)
+{
+ put_event(file->private_data);
+ return 0;
}
u64 perf_event_read_value(struct perf_event *event, u64 *enabled, u64 *running)
@@ -3227,7 +3234,7 @@
static const struct file_operations perf_fops;
-static struct perf_event *perf_fget_light(int fd, int *fput_needed)
+static struct file *perf_fget_light(int fd, int *fput_needed)
{
struct file *file;
@@ -3241,7 +3248,7 @@
return ERR_PTR(-EBADF);
}
- return file->private_data;
+ return file;
}
static int perf_event_set_output(struct perf_event *event,
@@ -3273,19 +3280,21 @@
case PERF_EVENT_IOC_SET_OUTPUT:
{
+ struct file *output_file = NULL;
struct perf_event *output_event = NULL;
int fput_needed = 0;
int ret;
if (arg != -1) {
- output_event = perf_fget_light(arg, &fput_needed);
- if (IS_ERR(output_event))
- return PTR_ERR(output_event);
+ output_file = perf_fget_light(arg, &fput_needed);
+ if (IS_ERR(output_file))
+ return PTR_ERR(output_file);
+ output_event = output_file->private_data;
}
ret = perf_event_set_output(event, output_event);
if (output_event)
- fput_light(output_event->filp, fput_needed);
+ fput_light(output_file, fput_needed);
return ret;
}
@@ -3756,6 +3765,132 @@
}
EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
+static void
+perf_output_sample_regs(struct perf_output_handle *handle,
+ struct pt_regs *regs, u64 mask)
+{
+ int bit;
+
+ for_each_set_bit(bit, (const unsigned long *) &mask,
+ sizeof(mask) * BITS_PER_BYTE) {
+ u64 val;
+
+ val = perf_reg_value(regs, bit);
+ perf_output_put(handle, val);
+ }
+}
+
+static void perf_sample_regs_user(struct perf_regs_user *regs_user,
+ struct pt_regs *regs)
+{
+ if (!user_mode(regs)) {
+ if (current->mm)
+ regs = task_pt_regs(current);
+ else
+ regs = NULL;
+ }
+
+ if (regs) {
+ regs_user->regs = regs;
+ regs_user->abi = perf_reg_abi(current);
+ }
+}
+
+/*
+ * Get remaining task size from user stack pointer.
+ *
+ * It'd be better to take stack vma map and limit this more
+ * precisly, but there's no way to get it safely under interrupt,
+ * so using TASK_SIZE as limit.
+ */
+static u64 perf_ustack_task_size(struct pt_regs *regs)
+{
+ unsigned long addr = perf_user_stack_pointer(regs);
+
+ if (!addr || addr >= TASK_SIZE)
+ return 0;
+
+ return TASK_SIZE - addr;
+}
+
+static u16
+perf_sample_ustack_size(u16 stack_size, u16 header_size,
+ struct pt_regs *regs)
+{
+ u64 task_size;
+
+ /* No regs, no stack pointer, no dump. */
+ if (!regs)
+ return 0;
+
+ /*
+ * Check if we fit in with the requested stack size into the:
+ * - TASK_SIZE
+ * If we don't, we limit the size to the TASK_SIZE.
+ *
+ * - remaining sample size
+ * If we don't, we customize the stack size to
+ * fit in to the remaining sample size.
+ */
+
+ task_size = min((u64) USHRT_MAX, perf_ustack_task_size(regs));
+ stack_size = min(stack_size, (u16) task_size);
+
+ /* Current header size plus static size and dynamic size. */
+ header_size += 2 * sizeof(u64);
+
+ /* Do we fit in with the current stack dump size? */
+ if ((u16) (header_size + stack_size) < header_size) {
+ /*
+ * If we overflow the maximum size for the sample,
+ * we customize the stack dump size to fit in.
+ */
+ stack_size = USHRT_MAX - header_size - sizeof(u64);
+ stack_size = round_up(stack_size, sizeof(u64));
+ }
+
+ return stack_size;
+}
+
+static void
+perf_output_sample_ustack(struct perf_output_handle *handle, u64 dump_size,
+ struct pt_regs *regs)
+{
+ /* Case of a kernel thread, nothing to dump */
+ if (!regs) {
+ u64 size = 0;
+ perf_output_put(handle, size);
+ } else {
+ unsigned long sp;
+ unsigned int rem;
+ u64 dyn_size;
+
+ /*
+ * We dump:
+ * static size
+ * - the size requested by user or the best one we can fit
+ * in to the sample max size
+ * data
+ * - user stack dump data
+ * dynamic size
+ * - the actual dumped size
+ */
+
+ /* Static size. */
+ perf_output_put(handle, dump_size);
+
+ /* Data. */
+ sp = perf_user_stack_pointer(regs);
+ rem = __output_copy_user(handle, (void *) sp, dump_size);
+ dyn_size = dump_size - rem;
+
+ perf_output_skip(handle, rem);
+
+ /* Dynamic size. */
+ perf_output_put(handle, dyn_size);
+ }
+}
+
static void __perf_event_header__init_id(struct perf_event_header *header,
struct perf_sample_data *data,
struct perf_event *event)
@@ -4016,6 +4151,28 @@
perf_output_put(handle, nr);
}
}
+
+ if (sample_type & PERF_SAMPLE_REGS_USER) {
+ u64 abi = data->regs_user.abi;
+
+ /*
+ * If there are no regs to dump, notice it through
+ * first u64 being zero (PERF_SAMPLE_REGS_ABI_NONE).
+ */
+ perf_output_put(handle, abi);
+
+ if (abi) {
+ u64 mask = event->attr.sample_regs_user;
+ perf_output_sample_regs(handle,
+ data->regs_user.regs,
+ mask);
+ }
+ }
+
+ if (sample_type & PERF_SAMPLE_STACK_USER)
+ perf_output_sample_ustack(handle,
+ data->stack_user_size,
+ data->regs_user.regs);
}
void perf_prepare_sample(struct perf_event_header *header,
@@ -4067,6 +4224,49 @@
}
header->size += size;
}
+
+ if (sample_type & PERF_SAMPLE_REGS_USER) {
+ /* regs dump ABI info */
+ int size = sizeof(u64);
+
+ perf_sample_regs_user(&data->regs_user, regs);
+
+ if (data->regs_user.regs) {
+ u64 mask = event->attr.sample_regs_user;
+ size += hweight64(mask) * sizeof(u64);
+ }
+
+ header->size += size;
+ }
+
+ if (sample_type & PERF_SAMPLE_STACK_USER) {
+ /*
+ * Either we need PERF_SAMPLE_STACK_USER bit to be allways
+ * processed as the last one or have additional check added
+ * in case new sample type is added, because we could eat
+ * up the rest of the sample size.
+ */
+ struct perf_regs_user *uregs = &data->regs_user;
+ u16 stack_size = event->attr.sample_stack_user;
+ u16 size = sizeof(u64);
+
+ if (!uregs->abi)
+ perf_sample_regs_user(uregs, regs);
+
+ stack_size = perf_sample_ustack_size(stack_size, header->size,
+ uregs->regs);
+
+ /*
+ * If there is something to dump, add space for the dump
+ * itself and for the field that tells the dynamic size,
+ * which is how many have been actually dumped.
+ */
+ if (stack_size)
+ size += sizeof(u64) + stack_size;
+
+ data->stack_user_size = stack_size;
+ header->size += size;
+ }
}
static void perf_event_output(struct perf_event *event,
@@ -5950,6 +6150,7 @@
mutex_init(&event->mmap_mutex);
+ atomic_long_set(&event->refcount, 1);
event->cpu = cpu;
event->attr = *attr;
event->group_leader = group_leader;
@@ -6142,6 +6343,28 @@
attr->branch_sample_type = mask;
}
}
+
+ if (attr->sample_type & PERF_SAMPLE_REGS_USER) {
+ ret = perf_reg_validate(attr->sample_regs_user);
+ if (ret)
+ return ret;
+ }
+
+ if (attr->sample_type & PERF_SAMPLE_STACK_USER) {
+ if (!arch_perf_have_user_stack_dump())
+ return -ENOSYS;
+
+ /*
+ * We have __u32 type for the size, but so far
+ * we can only use __u16 as maximum due to the
+ * __u16 sample size limit.
+ */
+ if (attr->sample_stack_user >= USHRT_MAX)
+ ret = -EINVAL;
+ else if (!IS_ALIGNED(attr->sample_stack_user, sizeof(u64)))
+ ret = -EINVAL;
+ }
+
out:
return ret;
@@ -6260,12 +6483,12 @@
return event_fd;
if (group_fd != -1) {
- group_leader = perf_fget_light(group_fd, &fput_needed);
- if (IS_ERR(group_leader)) {
- err = PTR_ERR(group_leader);
+ group_file = perf_fget_light(group_fd, &fput_needed);
+ if (IS_ERR(group_file)) {
+ err = PTR_ERR(group_file);
goto err_fd;
}
- group_file = group_leader->filp;
+ group_leader = group_file->private_data;
if (flags & PERF_FLAG_FD_OUTPUT)
output_event = group_leader;
if (flags & PERF_FLAG_FD_NO_GROUP)
@@ -6402,7 +6625,6 @@
put_ctx(gctx);
}
- event->filp = event_file;
WARN_ON_ONCE(ctx->parent_ctx);
mutex_lock(&ctx->mutex);
@@ -6496,7 +6718,6 @@
goto err_free;
}
- event->filp = NULL;
WARN_ON_ONCE(ctx->parent_ctx);
mutex_lock(&ctx->mutex);
perf_install_in_context(ctx, event, cpu);
@@ -6578,7 +6799,7 @@
* Release the parent event, if this was the last
* reference to it.
*/
- fput(parent_event->filp);
+ put_event(parent_event);
}
static void
@@ -6654,9 +6875,8 @@
*
* __perf_event_exit_task()
* sync_child_event()
- * fput(parent_event->filp)
- * perf_release()
- * mutex_lock(&ctx->mutex)
+ * put_event()
+ * mutex_lock(&ctx->mutex)
*
* But since its the parent context it won't be the same instance.
*/
@@ -6724,7 +6944,7 @@
list_del_init(&event->child_list);
mutex_unlock(&parent->child_mutex);
- fput(parent->filp);
+ put_event(parent);
perf_group_detach(event);
list_del_event(event, ctx);
@@ -6804,6 +7024,12 @@
NULL, NULL);
if (IS_ERR(child_event))
return child_event;
+
+ if (!atomic_long_inc_not_zero(&parent_event->refcount)) {
+ free_event(child_event);
+ return NULL;
+ }
+
get_ctx(child_ctx);
/*
@@ -6845,14 +7071,6 @@
raw_spin_unlock_irqrestore(&child_ctx->lock, flags);
/*
- * Get a reference to the parent filp - we will fput it
- * when the child event exits. This is safe to do because
- * we are in the parent and we know that the filp still
- * exists and has a nonzero count:
- */
- atomic_long_inc(&parent_event->filp->f_count);
-
- /*
* Link this into the parent event's child list
*/
WARN_ON_ONCE(parent_event->ctx->parent_ctx);
diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c
index bb38c4d..9a7b487 100644
--- a/kernel/events/hw_breakpoint.c
+++ b/kernel/events/hw_breakpoint.c
@@ -453,7 +453,16 @@
int old_type = bp->attr.bp_type;
int err = 0;
- perf_event_disable(bp);
+ /*
+ * modify_user_hw_breakpoint can be invoked with IRQs disabled and hence it
+ * will not be possible to raise IPIs that invoke __perf_event_disable.
+ * So call the function directly after making sure we are targeting the
+ * current task.
+ */
+ if (irqs_disabled() && bp->ctx && bp->ctx->task == current)
+ __perf_event_disable(bp);
+ else
+ perf_event_disable(bp);
bp->attr.bp_addr = attr->bp_addr;
bp->attr.bp_type = attr->bp_type;
diff --git a/kernel/events/internal.h b/kernel/events/internal.h
index a096c19..d56a64c 100644
--- a/kernel/events/internal.h
+++ b/kernel/events/internal.h
@@ -2,6 +2,7 @@
#define _KERNEL_EVENTS_INTERNAL_H
#include <linux/hardirq.h>
+#include <linux/uaccess.h>
/* Buffer handling */
@@ -76,30 +77,53 @@
return rb->nr_pages << (PAGE_SHIFT + page_order(rb));
}
-static inline void
-__output_copy(struct perf_output_handle *handle,
- const void *buf, unsigned int len)
-{
- do {
- unsigned long size = min_t(unsigned long, handle->size, len);
-
- memcpy(handle->addr, buf, size);
-
- len -= size;
- handle->addr += size;
- buf += size;
- handle->size -= size;
- if (!handle->size) {
- struct ring_buffer *rb = handle->rb;
-
- handle->page++;
- handle->page &= rb->nr_pages - 1;
- handle->addr = rb->data_pages[handle->page];
- handle->size = PAGE_SIZE << page_order(rb);
- }
- } while (len);
+#define DEFINE_OUTPUT_COPY(func_name, memcpy_func) \
+static inline unsigned int \
+func_name(struct perf_output_handle *handle, \
+ const void *buf, unsigned int len) \
+{ \
+ unsigned long size, written; \
+ \
+ do { \
+ size = min_t(unsigned long, handle->size, len); \
+ \
+ written = memcpy_func(handle->addr, buf, size); \
+ \
+ len -= written; \
+ handle->addr += written; \
+ buf += written; \
+ handle->size -= written; \
+ if (!handle->size) { \
+ struct ring_buffer *rb = handle->rb; \
+ \
+ handle->page++; \
+ handle->page &= rb->nr_pages - 1; \
+ handle->addr = rb->data_pages[handle->page]; \
+ handle->size = PAGE_SIZE << page_order(rb); \
+ } \
+ } while (len && written == size); \
+ \
+ return len; \
}
+static inline int memcpy_common(void *dst, const void *src, size_t n)
+{
+ memcpy(dst, src, n);
+ return n;
+}
+
+DEFINE_OUTPUT_COPY(__output_copy, memcpy_common)
+
+#define MEMCPY_SKIP(dst, src, n) (n)
+
+DEFINE_OUTPUT_COPY(__output_skip, MEMCPY_SKIP)
+
+#ifndef arch_perf_out_copy_user
+#define arch_perf_out_copy_user __copy_from_user_inatomic
+#endif
+
+DEFINE_OUTPUT_COPY(__output_copy_user, arch_perf_out_copy_user)
+
/* Callchain handling */
extern struct perf_callchain_entry *
perf_callchain(struct perf_event *event, struct pt_regs *regs);
@@ -134,4 +158,20 @@
recursion[rctx]--;
}
+#ifdef CONFIG_HAVE_PERF_USER_STACK_DUMP
+static inline bool arch_perf_have_user_stack_dump(void)
+{
+ return true;
+}
+
+#define perf_user_stack_pointer(regs) user_stack_pointer(regs)
+#else
+static inline bool arch_perf_have_user_stack_dump(void)
+{
+ return false;
+}
+
+#define perf_user_stack_pointer(regs) 0
+#endif /* CONFIG_HAVE_PERF_USER_STACK_DUMP */
+
#endif /* _KERNEL_EVENTS_INTERNAL_H */
diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 6ddaba4..23cb34f 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -182,10 +182,16 @@
return -ENOSPC;
}
-void perf_output_copy(struct perf_output_handle *handle,
+unsigned int perf_output_copy(struct perf_output_handle *handle,
const void *buf, unsigned int len)
{
- __output_copy(handle, buf, len);
+ return __output_copy(handle, buf, len);
+}
+
+unsigned int perf_output_skip(struct perf_output_handle *handle,
+ unsigned int len)
+{
+ return __output_skip(handle, NULL, len);
}
void perf_output_end(struct perf_output_handle *handle)
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index c08a22d..912ef48 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -280,12 +280,10 @@
if (ret <= 0)
return ret;
- lock_page(page);
vaddr_new = kmap_atomic(page);
vaddr &= ~PAGE_MASK;
memcpy(opcode, vaddr_new + vaddr, UPROBE_SWBP_INSN_SIZE);
kunmap_atomic(vaddr_new);
- unlock_page(page);
put_page(page);
@@ -334,7 +332,7 @@
*/
result = is_swbp_at_addr(mm, vaddr);
if (result == 1)
- return -EEXIST;
+ return 0;
if (result)
return result;
@@ -347,24 +345,22 @@
* @mm: the probed process address space.
* @auprobe: arch specific probepoint information.
* @vaddr: the virtual address to insert the opcode.
- * @verify: if true, verify existance of breakpoint instruction.
*
* For mm @mm, restore the original opcode (opcode) at @vaddr.
* Return 0 (success) or a negative errno.
*/
int __weak
-set_orig_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned long vaddr, bool verify)
+set_orig_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, unsigned long vaddr)
{
- if (verify) {
- int result;
+ int result;
- result = is_swbp_at_addr(mm, vaddr);
- if (!result)
- return -EINVAL;
+ result = is_swbp_at_addr(mm, vaddr);
+ if (!result)
+ return -EINVAL;
- if (result != 1)
- return result;
- }
+ if (result != 1)
+ return result;
+
return write_opcode(auprobe, mm, vaddr, *(uprobe_opcode_t *)auprobe->insn);
}
@@ -415,11 +411,10 @@
static struct uprobe *find_uprobe(struct inode *inode, loff_t offset)
{
struct uprobe *uprobe;
- unsigned long flags;
- spin_lock_irqsave(&uprobes_treelock, flags);
+ spin_lock(&uprobes_treelock);
uprobe = __find_uprobe(inode, offset);
- spin_unlock_irqrestore(&uprobes_treelock, flags);
+ spin_unlock(&uprobes_treelock);
return uprobe;
}
@@ -466,12 +461,11 @@
*/
static struct uprobe *insert_uprobe(struct uprobe *uprobe)
{
- unsigned long flags;
struct uprobe *u;
- spin_lock_irqsave(&uprobes_treelock, flags);
+ spin_lock(&uprobes_treelock);
u = __insert_uprobe(uprobe);
- spin_unlock_irqrestore(&uprobes_treelock, flags);
+ spin_unlock(&uprobes_treelock);
/* For now assume that the instruction need not be single-stepped */
uprobe->flags |= UPROBE_SKIP_SSTEP;
@@ -649,6 +643,7 @@
install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
struct vm_area_struct *vma, unsigned long vaddr)
{
+ bool first_uprobe;
int ret;
/*
@@ -659,7 +654,7 @@
* Hence behave as if probe already existed.
*/
if (!uprobe->consumers)
- return -EEXIST;
+ return 0;
if (!(uprobe->flags & UPROBE_COPY_INSN)) {
ret = copy_insn(uprobe, vma->vm_file);
@@ -681,17 +676,18 @@
}
/*
- * Ideally, should be updating the probe count after the breakpoint
- * has been successfully inserted. However a thread could hit the
- * breakpoint we just inserted even before the probe count is
- * incremented. If this is the first breakpoint placed, breakpoint
- * notifier might ignore uprobes and pass the trap to the thread.
- * Hence increment before and decrement on failure.
+ * set MMF_HAS_UPROBES in advance for uprobe_pre_sstep_notifier(),
+ * the task can hit this breakpoint right after __replace_page().
*/
- atomic_inc(&mm->uprobes_state.count);
+ first_uprobe = !test_bit(MMF_HAS_UPROBES, &mm->flags);
+ if (first_uprobe)
+ set_bit(MMF_HAS_UPROBES, &mm->flags);
+
ret = set_swbp(&uprobe->arch, mm, vaddr);
- if (ret)
- atomic_dec(&mm->uprobes_state.count);
+ if (!ret)
+ clear_bit(MMF_RECALC_UPROBES, &mm->flags);
+ else if (first_uprobe)
+ clear_bit(MMF_HAS_UPROBES, &mm->flags);
return ret;
}
@@ -699,8 +695,12 @@
static void
remove_breakpoint(struct uprobe *uprobe, struct mm_struct *mm, unsigned long vaddr)
{
- if (!set_orig_insn(&uprobe->arch, mm, vaddr, true))
- atomic_dec(&mm->uprobes_state.count);
+ /* can happen if uprobe_register() fails */
+ if (!test_bit(MMF_HAS_UPROBES, &mm->flags))
+ return;
+
+ set_bit(MMF_RECALC_UPROBES, &mm->flags);
+ set_orig_insn(&uprobe->arch, mm, vaddr);
}
/*
@@ -710,11 +710,9 @@
*/
static void delete_uprobe(struct uprobe *uprobe)
{
- unsigned long flags;
-
- spin_lock_irqsave(&uprobes_treelock, flags);
+ spin_lock(&uprobes_treelock);
rb_erase(&uprobe->rb_node, &uprobes_tree);
- spin_unlock_irqrestore(&uprobes_treelock, flags);
+ spin_unlock(&uprobes_treelock);
iput(uprobe->inode);
put_uprobe(uprobe);
atomic_dec(&uprobe_events);
@@ -831,17 +829,11 @@
vaddr_to_offset(vma, info->vaddr) != uprobe->offset)
goto unlock;
- if (is_register) {
+ if (is_register)
err = install_breakpoint(uprobe, mm, vma, info->vaddr);
- /*
- * We can race against uprobe_mmap(), see the
- * comment near uprobe_hash().
- */
- if (err == -EEXIST)
- err = 0;
- } else {
+ else
remove_breakpoint(uprobe, mm, info->vaddr);
- }
+
unlock:
up_write(&mm->mmap_sem);
free:
@@ -908,7 +900,8 @@
}
mutex_unlock(uprobes_hash(inode));
- put_uprobe(uprobe);
+ if (uprobe)
+ put_uprobe(uprobe);
return ret;
}
@@ -978,7 +971,6 @@
struct list_head *head)
{
loff_t min, max;
- unsigned long flags;
struct rb_node *n, *t;
struct uprobe *u;
@@ -986,7 +978,7 @@
min = vaddr_to_offset(vma, start);
max = min + (end - start) - 1;
- spin_lock_irqsave(&uprobes_treelock, flags);
+ spin_lock(&uprobes_treelock);
n = find_node_in_range(inode, min, max);
if (n) {
for (t = n; t; t = rb_prev(t)) {
@@ -1004,27 +996,20 @@
atomic_inc(&u->ref);
}
}
- spin_unlock_irqrestore(&uprobes_treelock, flags);
+ spin_unlock(&uprobes_treelock);
}
/*
- * Called from mmap_region.
- * called with mm->mmap_sem acquired.
+ * Called from mmap_region/vma_adjust with mm->mmap_sem acquired.
*
- * Return -ve no if we fail to insert probes and we cannot
- * bail-out.
- * Return 0 otherwise. i.e:
- *
- * - successful insertion of probes
- * - (or) no possible probes to be inserted.
- * - (or) insertion of probes failed but we can bail-out.
+ * Currently we ignore all errors and always return 0, the callers
+ * can't handle the failure anyway.
*/
int uprobe_mmap(struct vm_area_struct *vma)
{
struct list_head tmp_list;
struct uprobe *uprobe, *u;
struct inode *inode;
- int ret, count;
if (!atomic_read(&uprobe_events) || !valid_vma(vma, true))
return 0;
@@ -1036,44 +1021,35 @@
mutex_lock(uprobes_mmap_hash(inode));
build_probe_list(inode, vma, vma->vm_start, vma->vm_end, &tmp_list);
- ret = 0;
- count = 0;
-
list_for_each_entry_safe(uprobe, u, &tmp_list, pending_list) {
- if (!ret) {
+ if (!fatal_signal_pending(current)) {
unsigned long vaddr = offset_to_vaddr(vma, uprobe->offset);
-
- ret = install_breakpoint(uprobe, vma->vm_mm, vma, vaddr);
- /*
- * We can race against uprobe_register(), see the
- * comment near uprobe_hash().
- */
- if (ret == -EEXIST) {
- ret = 0;
-
- if (!is_swbp_at_addr(vma->vm_mm, vaddr))
- continue;
-
- /*
- * Unable to insert a breakpoint, but
- * breakpoint lies underneath. Increment the
- * probe count.
- */
- atomic_inc(&vma->vm_mm->uprobes_state.count);
- }
-
- if (!ret)
- count++;
+ install_breakpoint(uprobe, vma->vm_mm, vma, vaddr);
}
put_uprobe(uprobe);
}
-
mutex_unlock(uprobes_mmap_hash(inode));
- if (ret)
- atomic_sub(count, &vma->vm_mm->uprobes_state.count);
+ return 0;
+}
- return ret;
+static bool
+vma_has_uprobes(struct vm_area_struct *vma, unsigned long start, unsigned long end)
+{
+ loff_t min, max;
+ struct inode *inode;
+ struct rb_node *n;
+
+ inode = vma->vm_file->f_mapping->host;
+
+ min = vaddr_to_offset(vma, start);
+ max = min + (end - start) - 1;
+
+ spin_lock(&uprobes_treelock);
+ n = find_node_in_range(inode, min, max);
+ spin_unlock(&uprobes_treelock);
+
+ return !!n;
}
/*
@@ -1081,37 +1057,18 @@
*/
void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end)
{
- struct list_head tmp_list;
- struct uprobe *uprobe, *u;
- struct inode *inode;
-
if (!atomic_read(&uprobe_events) || !valid_vma(vma, false))
return;
if (!atomic_read(&vma->vm_mm->mm_users)) /* called by mmput() ? */
return;
- if (!atomic_read(&vma->vm_mm->uprobes_state.count))
+ if (!test_bit(MMF_HAS_UPROBES, &vma->vm_mm->flags) ||
+ test_bit(MMF_RECALC_UPROBES, &vma->vm_mm->flags))
return;
- inode = vma->vm_file->f_mapping->host;
- if (!inode)
- return;
-
- mutex_lock(uprobes_mmap_hash(inode));
- build_probe_list(inode, vma, start, end, &tmp_list);
-
- list_for_each_entry_safe(uprobe, u, &tmp_list, pending_list) {
- unsigned long vaddr = offset_to_vaddr(vma, uprobe->offset);
- /*
- * An unregister could have removed the probe before
- * unmap. So check before we decrement the count.
- */
- if (is_swbp_at_addr(vma->vm_mm, vaddr) == 1)
- atomic_dec(&vma->vm_mm->uprobes_state.count);
- put_uprobe(uprobe);
- }
- mutex_unlock(uprobes_mmap_hash(inode));
+ if (vma_has_uprobes(vma, start, end))
+ set_bit(MMF_RECALC_UPROBES, &vma->vm_mm->flags);
}
/* Slot allocation for XOL */
@@ -1213,13 +1170,15 @@
kfree(area);
}
-/*
- * uprobe_reset_state - Free the area allocated for slots.
- */
-void uprobe_reset_state(struct mm_struct *mm)
+void uprobe_dup_mmap(struct mm_struct *oldmm, struct mm_struct *newmm)
{
- mm->uprobes_state.xol_area = NULL;
- atomic_set(&mm->uprobes_state.count, 0);
+ newmm->uprobes_state.xol_area = NULL;
+
+ if (test_bit(MMF_HAS_UPROBES, &oldmm->flags)) {
+ set_bit(MMF_HAS_UPROBES, &newmm->flags);
+ /* unconditionally, dup_mmap() skips VM_DONTCOPY vmas */
+ set_bit(MMF_RECALC_UPROBES, &newmm->flags);
+ }
}
/*
@@ -1437,6 +1396,25 @@
return false;
}
+static void mmf_recalc_uprobes(struct mm_struct *mm)
+{
+ struct vm_area_struct *vma;
+
+ for (vma = mm->mmap; vma; vma = vma->vm_next) {
+ if (!valid_vma(vma, false))
+ continue;
+ /*
+ * This is not strictly accurate, we can race with
+ * uprobe_unregister() and see the already removed
+ * uprobe if delete_uprobe() was not yet called.
+ */
+ if (vma_has_uprobes(vma, vma->vm_start, vma->vm_end))
+ return;
+ }
+
+ clear_bit(MMF_HAS_UPROBES, &mm->flags);
+}
+
static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp)
{
struct mm_struct *mm = current->mm;
@@ -1458,11 +1436,24 @@
} else {
*is_swbp = -EFAULT;
}
+
+ if (!uprobe && test_and_clear_bit(MMF_RECALC_UPROBES, &mm->flags))
+ mmf_recalc_uprobes(mm);
up_read(&mm->mmap_sem);
return uprobe;
}
+void __weak arch_uprobe_enable_step(struct arch_uprobe *arch)
+{
+ user_enable_single_step(current);
+}
+
+void __weak arch_uprobe_disable_step(struct arch_uprobe *arch)
+{
+ user_disable_single_step(current);
+}
+
/*
* Run handler and ask thread to singlestep.
* Ensure all non-fatal signals cannot interrupt thread while it singlesteps.
@@ -1509,7 +1500,7 @@
utask->state = UTASK_SSTEP;
if (!pre_ssout(uprobe, regs, bp_vaddr)) {
- user_enable_single_step(current);
+ arch_uprobe_enable_step(&uprobe->arch);
return;
}
@@ -1518,17 +1509,15 @@
utask->active_uprobe = NULL;
utask->state = UTASK_RUNNING;
}
- if (uprobe) {
- if (!(uprobe->flags & UPROBE_SKIP_SSTEP))
+ if (!(uprobe->flags & UPROBE_SKIP_SSTEP))
- /*
- * cannot singlestep; cannot skip instruction;
- * re-execute the instruction.
- */
- instruction_pointer_set(regs, bp_vaddr);
+ /*
+ * cannot singlestep; cannot skip instruction;
+ * re-execute the instruction.
+ */
+ instruction_pointer_set(regs, bp_vaddr);
- put_uprobe(uprobe);
- }
+ put_uprobe(uprobe);
}
/*
@@ -1547,10 +1536,10 @@
else
WARN_ON_ONCE(1);
+ arch_uprobe_disable_step(&uprobe->arch);
put_uprobe(uprobe);
utask->active_uprobe = NULL;
utask->state = UTASK_RUNNING;
- user_disable_single_step(current);
xol_free_insn_slot(current);
spin_lock_irq(¤t->sighand->siglock);
@@ -1589,8 +1578,7 @@
{
struct uprobe_task *utask;
- if (!current->mm || !atomic_read(¤t->mm->uprobes_state.count))
- /* task is currently not uprobed */
+ if (!current->mm || !test_bit(MMF_HAS_UPROBES, ¤t->mm->flags))
return 0;
utask = current->utask;
diff --git a/kernel/fork.c b/kernel/fork.c
index 3bd2280..5a0e74d 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -353,6 +353,7 @@
down_write(&oldmm->mmap_sem);
flush_cache_dup_mm(oldmm);
+ uprobe_dup_mmap(oldmm, mm);
/*
* Not linked in yet - no deadlock potential:
*/
@@ -454,9 +455,6 @@
if (retval)
goto out;
-
- if (file && uprobe_mmap(tmp))
- goto out;
}
/* a new mm has just been created */
arch_dup_mmap(oldmm, mm);
@@ -839,8 +837,6 @@
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
mm->pmd_huge_pte = NULL;
#endif
- uprobe_reset_state(mm);
-
if (!mm_init(mm, tsk))
goto fail_nomem;
@@ -1280,11 +1276,7 @@
#endif
#ifdef CONFIG_TRACE_IRQFLAGS
p->irq_events = 0;
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
- p->hardirqs_enabled = 1;
-#else
p->hardirqs_enabled = 0;
-#endif
p->hardirq_enable_ip = 0;
p->hardirq_enable_event = 0;
p->hardirq_disable_ip = _THIS_IP_;
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index eebd6d5..57d86d0 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -671,6 +671,7 @@
irq_set_chip(irq, chip);
__irq_set_handler(irq, handle, 0, name);
}
+EXPORT_SYMBOL_GPL(irq_set_chip_and_handler_name);
void irq_modify_status(unsigned int irq, unsigned long clr, unsigned long set)
{
diff --git a/kernel/irq/dummychip.c b/kernel/irq/dummychip.c
index b5fcd96..988dc58 100644
--- a/kernel/irq/dummychip.c
+++ b/kernel/irq/dummychip.c
@@ -6,6 +6,7 @@
*/
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/export.h>
#include "internals.h"
@@ -57,3 +58,4 @@
.irq_mask = noop,
.irq_unmask = noop,
};
+EXPORT_SYMBOL_GPL(dummy_irq_chip);
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index c62b854..098f396 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -561,9 +561,9 @@
{
LIST_HEAD(free_list);
+ mutex_lock(&kprobe_mutex);
/* Lock modules while optimizing kprobes */
mutex_lock(&module_mutex);
- mutex_lock(&kprobe_mutex);
/*
* Step 1: Unoptimize kprobes and collect cleaned (unused and disarmed)
@@ -586,8 +586,8 @@
/* Step 4: Free cleaned kprobes after quiesence period */
do_free_cleaned_kprobes(&free_list);
- mutex_unlock(&kprobe_mutex);
mutex_unlock(&module_mutex);
+ mutex_unlock(&kprobe_mutex);
/* Step 5: Kick optimizer again if needed */
if (!list_empty(&optimizing_list) || !list_empty(&unoptimizing_list))
@@ -759,20 +759,32 @@
struct kprobe *ap;
struct optimized_kprobe *op;
+ /* Impossible to optimize ftrace-based kprobe */
+ if (kprobe_ftrace(p))
+ return;
+
+ /* For preparing optimization, jump_label_text_reserved() is called */
+ jump_label_lock();
+ mutex_lock(&text_mutex);
+
ap = alloc_aggr_kprobe(p);
if (!ap)
- return;
+ goto out;
op = container_of(ap, struct optimized_kprobe, kp);
if (!arch_prepared_optinsn(&op->optinsn)) {
/* If failed to setup optimizing, fallback to kprobe */
arch_remove_optimized_kprobe(op);
kfree(op);
- return;
+ goto out;
}
init_aggr_kprobe(ap, p);
- optimize_kprobe(ap);
+ optimize_kprobe(ap); /* This just kicks optimizer thread */
+
+out:
+ mutex_unlock(&text_mutex);
+ jump_label_unlock();
}
#ifdef CONFIG_SYSCTL
@@ -907,9 +919,64 @@
}
#endif /* CONFIG_OPTPROBES */
+#ifdef KPROBES_CAN_USE_FTRACE
+static struct ftrace_ops kprobe_ftrace_ops __read_mostly = {
+ .func = kprobe_ftrace_handler,
+ .flags = FTRACE_OPS_FL_SAVE_REGS,
+};
+static int kprobe_ftrace_enabled;
+
+/* Must ensure p->addr is really on ftrace */
+static int __kprobes prepare_kprobe(struct kprobe *p)
+{
+ if (!kprobe_ftrace(p))
+ return arch_prepare_kprobe(p);
+
+ return arch_prepare_kprobe_ftrace(p);
+}
+
+/* Caller must lock kprobe_mutex */
+static void __kprobes arm_kprobe_ftrace(struct kprobe *p)
+{
+ int ret;
+
+ ret = ftrace_set_filter_ip(&kprobe_ftrace_ops,
+ (unsigned long)p->addr, 0, 0);
+ WARN(ret < 0, "Failed to arm kprobe-ftrace at %p (%d)\n", p->addr, ret);
+ kprobe_ftrace_enabled++;
+ if (kprobe_ftrace_enabled == 1) {
+ ret = register_ftrace_function(&kprobe_ftrace_ops);
+ WARN(ret < 0, "Failed to init kprobe-ftrace (%d)\n", ret);
+ }
+}
+
+/* Caller must lock kprobe_mutex */
+static void __kprobes disarm_kprobe_ftrace(struct kprobe *p)
+{
+ int ret;
+
+ kprobe_ftrace_enabled--;
+ if (kprobe_ftrace_enabled == 0) {
+ ret = unregister_ftrace_function(&kprobe_ftrace_ops);
+ WARN(ret < 0, "Failed to init kprobe-ftrace (%d)\n", ret);
+ }
+ ret = ftrace_set_filter_ip(&kprobe_ftrace_ops,
+ (unsigned long)p->addr, 1, 0);
+ WARN(ret < 0, "Failed to disarm kprobe-ftrace at %p (%d)\n", p->addr, ret);
+}
+#else /* !KPROBES_CAN_USE_FTRACE */
+#define prepare_kprobe(p) arch_prepare_kprobe(p)
+#define arm_kprobe_ftrace(p) do {} while (0)
+#define disarm_kprobe_ftrace(p) do {} while (0)
+#endif
+
/* Arm a kprobe with text_mutex */
static void __kprobes arm_kprobe(struct kprobe *kp)
{
+ if (unlikely(kprobe_ftrace(kp))) {
+ arm_kprobe_ftrace(kp);
+ return;
+ }
/*
* Here, since __arm_kprobe() doesn't use stop_machine(),
* this doesn't cause deadlock on text_mutex. So, we don't
@@ -921,11 +988,15 @@
}
/* Disarm a kprobe with text_mutex */
-static void __kprobes disarm_kprobe(struct kprobe *kp)
+static void __kprobes disarm_kprobe(struct kprobe *kp, bool reopt)
{
+ if (unlikely(kprobe_ftrace(kp))) {
+ disarm_kprobe_ftrace(kp);
+ return;
+ }
/* Ditto */
mutex_lock(&text_mutex);
- __disarm_kprobe(kp, true);
+ __disarm_kprobe(kp, reopt);
mutex_unlock(&text_mutex);
}
@@ -1144,12 +1215,6 @@
if (p->post_handler && !ap->post_handler)
ap->post_handler = aggr_post_handler;
- if (kprobe_disabled(ap) && !kprobe_disabled(p)) {
- ap->flags &= ~KPROBE_FLAG_DISABLED;
- if (!kprobes_all_disarmed)
- /* Arm the breakpoint again. */
- __arm_kprobe(ap);
- }
return 0;
}
@@ -1189,11 +1254,22 @@
int ret = 0;
struct kprobe *ap = orig_p;
+ /* For preparing optimization, jump_label_text_reserved() is called */
+ jump_label_lock();
+ /*
+ * Get online CPUs to avoid text_mutex deadlock.with stop machine,
+ * which is invoked by unoptimize_kprobe() in add_new_kprobe()
+ */
+ get_online_cpus();
+ mutex_lock(&text_mutex);
+
if (!kprobe_aggrprobe(orig_p)) {
/* If orig_p is not an aggr_kprobe, create new aggr_kprobe. */
ap = alloc_aggr_kprobe(orig_p);
- if (!ap)
- return -ENOMEM;
+ if (!ap) {
+ ret = -ENOMEM;
+ goto out;
+ }
init_aggr_kprobe(ap, orig_p);
} else if (kprobe_unused(ap))
/* This probe is going to die. Rescue it */
@@ -1213,7 +1289,7 @@
* free aggr_probe. It will be used next time, or
* freed by unregister_kprobe.
*/
- return ret;
+ goto out;
/* Prepare optimized instructions if possible. */
prepare_optimized_kprobe(ap);
@@ -1228,7 +1304,20 @@
/* Copy ap's insn slot to p */
copy_kprobe(ap, p);
- return add_new_kprobe(ap, p);
+ ret = add_new_kprobe(ap, p);
+
+out:
+ mutex_unlock(&text_mutex);
+ put_online_cpus();
+ jump_label_unlock();
+
+ if (ret == 0 && kprobe_disabled(ap) && !kprobe_disabled(p)) {
+ ap->flags &= ~KPROBE_FLAG_DISABLED;
+ if (!kprobes_all_disarmed)
+ /* Arm the breakpoint again. */
+ arm_kprobe(ap);
+ }
+ return ret;
}
static int __kprobes in_kprobes_functions(unsigned long addr)
@@ -1313,13 +1402,77 @@
return ret;
}
-int __kprobes register_kprobe(struct kprobe *p)
+static __kprobes int check_kprobe_address_safe(struct kprobe *p,
+ struct module **probed_mod)
{
int ret = 0;
+ unsigned long ftrace_addr;
+
+ /*
+ * If the address is located on a ftrace nop, set the
+ * breakpoint to the following instruction.
+ */
+ ftrace_addr = ftrace_location((unsigned long)p->addr);
+ if (ftrace_addr) {
+#ifdef KPROBES_CAN_USE_FTRACE
+ /* Given address is not on the instruction boundary */
+ if ((unsigned long)p->addr != ftrace_addr)
+ return -EILSEQ;
+ p->flags |= KPROBE_FLAG_FTRACE;
+#else /* !KPROBES_CAN_USE_FTRACE */
+ return -EINVAL;
+#endif
+ }
+
+ jump_label_lock();
+ preempt_disable();
+
+ /* Ensure it is not in reserved area nor out of text */
+ if (!kernel_text_address((unsigned long) p->addr) ||
+ in_kprobes_functions((unsigned long) p->addr) ||
+ jump_label_text_reserved(p->addr, p->addr)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ /* Check if are we probing a module */
+ *probed_mod = __module_text_address((unsigned long) p->addr);
+ if (*probed_mod) {
+ /*
+ * We must hold a refcount of the probed module while updating
+ * its code to prohibit unexpected unloading.
+ */
+ if (unlikely(!try_module_get(*probed_mod))) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ /*
+ * If the module freed .init.text, we couldn't insert
+ * kprobes in there.
+ */
+ if (within_module_init((unsigned long)p->addr, *probed_mod) &&
+ (*probed_mod)->state != MODULE_STATE_COMING) {
+ module_put(*probed_mod);
+ *probed_mod = NULL;
+ ret = -ENOENT;
+ }
+ }
+out:
+ preempt_enable();
+ jump_label_unlock();
+
+ return ret;
+}
+
+int __kprobes register_kprobe(struct kprobe *p)
+{
+ int ret;
struct kprobe *old_p;
struct module *probed_mod;
kprobe_opcode_t *addr;
+ /* Adjust probe address from symbol */
addr = kprobe_addr(p);
if (IS_ERR(addr))
return PTR_ERR(addr);
@@ -1329,56 +1482,17 @@
if (ret)
return ret;
- jump_label_lock();
- preempt_disable();
- if (!kernel_text_address((unsigned long) p->addr) ||
- in_kprobes_functions((unsigned long) p->addr) ||
- ftrace_text_reserved(p->addr, p->addr) ||
- jump_label_text_reserved(p->addr, p->addr)) {
- ret = -EINVAL;
- goto cannot_probe;
- }
-
/* User can pass only KPROBE_FLAG_DISABLED to register_kprobe */
p->flags &= KPROBE_FLAG_DISABLED;
-
- /*
- * Check if are we probing a module.
- */
- probed_mod = __module_text_address((unsigned long) p->addr);
- if (probed_mod) {
- /* Return -ENOENT if fail. */
- ret = -ENOENT;
- /*
- * We must hold a refcount of the probed module while updating
- * its code to prohibit unexpected unloading.
- */
- if (unlikely(!try_module_get(probed_mod)))
- goto cannot_probe;
-
- /*
- * If the module freed .init.text, we couldn't insert
- * kprobes in there.
- */
- if (within_module_init((unsigned long)p->addr, probed_mod) &&
- probed_mod->state != MODULE_STATE_COMING) {
- module_put(probed_mod);
- goto cannot_probe;
- }
- /* ret will be updated by following code */
- }
- preempt_enable();
- jump_label_unlock();
-
p->nmissed = 0;
INIT_LIST_HEAD(&p->list);
+
+ ret = check_kprobe_address_safe(p, &probed_mod);
+ if (ret)
+ return ret;
+
mutex_lock(&kprobe_mutex);
- jump_label_lock(); /* needed to call jump_label_text_reserved() */
-
- get_online_cpus(); /* For avoiding text_mutex deadlock. */
- mutex_lock(&text_mutex);
-
old_p = get_kprobe(p->addr);
if (old_p) {
/* Since this may unoptimize old_p, locking text_mutex. */
@@ -1386,7 +1500,9 @@
goto out;
}
- ret = arch_prepare_kprobe(p);
+ mutex_lock(&text_mutex); /* Avoiding text modification */
+ ret = prepare_kprobe(p);
+ mutex_unlock(&text_mutex);
if (ret)
goto out;
@@ -1395,26 +1511,18 @@
&kprobe_table[hash_ptr(p->addr, KPROBE_HASH_BITS)]);
if (!kprobes_all_disarmed && !kprobe_disabled(p))
- __arm_kprobe(p);
+ arm_kprobe(p);
/* Try to optimize kprobe */
try_to_optimize_kprobe(p);
out:
- mutex_unlock(&text_mutex);
- put_online_cpus();
- jump_label_unlock();
mutex_unlock(&kprobe_mutex);
if (probed_mod)
module_put(probed_mod);
return ret;
-
-cannot_probe:
- preempt_enable();
- jump_label_unlock();
- return ret;
}
EXPORT_SYMBOL_GPL(register_kprobe);
@@ -1451,7 +1559,7 @@
/* Try to disarm and disable this/parent probe */
if (p == orig_p || aggr_kprobe_disabled(orig_p)) {
- disarm_kprobe(orig_p);
+ disarm_kprobe(orig_p, true);
orig_p->flags |= KPROBE_FLAG_DISABLED;
}
}
@@ -2049,10 +2157,11 @@
if (!pp)
pp = p;
- seq_printf(pi, "%s%s%s\n",
+ seq_printf(pi, "%s%s%s%s\n",
(kprobe_gone(p) ? "[GONE]" : ""),
((kprobe_disabled(p) && !kprobe_gone(p)) ? "[DISABLED]" : ""),
- (kprobe_optimized(pp) ? "[OPTIMIZED]" : ""));
+ (kprobe_optimized(pp) ? "[OPTIMIZED]" : ""),
+ (kprobe_ftrace(pp) ? "[FTRACE]" : ""));
}
static void __kprobes *kprobe_seq_start(struct seq_file *f, loff_t *pos)
@@ -2131,14 +2240,12 @@
goto already_enabled;
/* Arming kprobes doesn't optimize kprobe itself */
- mutex_lock(&text_mutex);
for (i = 0; i < KPROBE_TABLE_SIZE; i++) {
head = &kprobe_table[i];
hlist_for_each_entry_rcu(p, node, head, hlist)
if (!kprobe_disabled(p))
- __arm_kprobe(p);
+ arm_kprobe(p);
}
- mutex_unlock(&text_mutex);
kprobes_all_disarmed = false;
printk(KERN_INFO "Kprobes globally enabled\n");
@@ -2166,15 +2273,13 @@
kprobes_all_disarmed = true;
printk(KERN_INFO "Kprobes globally disabled\n");
- mutex_lock(&text_mutex);
for (i = 0; i < KPROBE_TABLE_SIZE; i++) {
head = &kprobe_table[i];
hlist_for_each_entry_rcu(p, node, head, hlist) {
if (!arch_trampoline_kprobe(p) && !kprobe_disabled(p))
- __disarm_kprobe(p, false);
+ disarm_kprobe(p, false);
}
}
- mutex_unlock(&text_mutex);
mutex_unlock(&kprobe_mutex);
/* Wait for disarming all kprobes by optimizer */
diff --git a/kernel/kthread.c b/kernel/kthread.c
index b579af5..146a6fa 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -37,11 +37,20 @@
};
struct kthread {
- int should_stop;
+ unsigned long flags;
+ unsigned int cpu;
void *data;
+ struct completion parked;
struct completion exited;
};
+enum KTHREAD_BITS {
+ KTHREAD_IS_PER_CPU = 0,
+ KTHREAD_SHOULD_STOP,
+ KTHREAD_SHOULD_PARK,
+ KTHREAD_IS_PARKED,
+};
+
#define to_kthread(tsk) \
container_of((tsk)->vfork_done, struct kthread, exited)
@@ -52,13 +61,29 @@
* and this will return true. You should then return, and your return
* value will be passed through to kthread_stop().
*/
-int kthread_should_stop(void)
+bool kthread_should_stop(void)
{
- return to_kthread(current)->should_stop;
+ return test_bit(KTHREAD_SHOULD_STOP, &to_kthread(current)->flags);
}
EXPORT_SYMBOL(kthread_should_stop);
/**
+ * kthread_should_park - should this kthread park now?
+ *
+ * When someone calls kthread_park() on your kthread, it will be woken
+ * and this will return true. You should then do the necessary
+ * cleanup and call kthread_parkme()
+ *
+ * Similar to kthread_should_stop(), but this keeps the thread alive
+ * and in a park position. kthread_unpark() "restarts" the thread and
+ * calls the thread function again.
+ */
+bool kthread_should_park(void)
+{
+ return test_bit(KTHREAD_SHOULD_PARK, &to_kthread(current)->flags);
+}
+
+/**
* kthread_freezable_should_stop - should this freezable kthread return now?
* @was_frozen: optional out parameter, indicates whether %current was frozen
*
@@ -96,6 +121,24 @@
return to_kthread(task)->data;
}
+static void __kthread_parkme(struct kthread *self)
+{
+ __set_current_state(TASK_INTERRUPTIBLE);
+ while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
+ if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
+ complete(&self->parked);
+ schedule();
+ __set_current_state(TASK_INTERRUPTIBLE);
+ }
+ clear_bit(KTHREAD_IS_PARKED, &self->flags);
+ __set_current_state(TASK_RUNNING);
+}
+
+void kthread_parkme(void)
+{
+ __kthread_parkme(to_kthread(current));
+}
+
static int kthread(void *_create)
{
/* Copy data: it's on kthread's stack */
@@ -105,9 +148,10 @@
struct kthread self;
int ret;
- self.should_stop = 0;
+ self.flags = 0;
self.data = data;
init_completion(&self.exited);
+ init_completion(&self.parked);
current->vfork_done = &self.exited;
/* OK, tell user we're spawned, wait for stop or wakeup */
@@ -117,9 +161,11 @@
schedule();
ret = -EINTR;
- if (!self.should_stop)
- ret = threadfn(data);
+ if (!test_bit(KTHREAD_SHOULD_STOP, &self.flags)) {
+ __kthread_parkme(&self);
+ ret = threadfn(data);
+ }
/* we can't just return, we must preserve "self" on stack */
do_exit(ret);
}
@@ -172,8 +218,7 @@
* Returns a task_struct or ERR_PTR(-ENOMEM).
*/
struct task_struct *kthread_create_on_node(int (*threadfn)(void *data),
- void *data,
- int node,
+ void *data, int node,
const char namefmt[],
...)
{
@@ -210,6 +255,13 @@
}
EXPORT_SYMBOL(kthread_create_on_node);
+static void __kthread_bind(struct task_struct *p, unsigned int cpu)
+{
+ /* It's safe because the task is inactive. */
+ do_set_cpus_allowed(p, cpumask_of(cpu));
+ p->flags |= PF_THREAD_BOUND;
+}
+
/**
* kthread_bind - bind a just-created kthread to a cpu.
* @p: thread created by kthread_create().
@@ -226,14 +278,112 @@
WARN_ON(1);
return;
}
-
- /* It's safe because the task is inactive. */
- do_set_cpus_allowed(p, cpumask_of(cpu));
- p->flags |= PF_THREAD_BOUND;
+ __kthread_bind(p, cpu);
}
EXPORT_SYMBOL(kthread_bind);
/**
+ * kthread_create_on_cpu - Create a cpu bound kthread
+ * @threadfn: the function to run until signal_pending(current).
+ * @data: data ptr for @threadfn.
+ * @cpu: The cpu on which the thread should be bound,
+ * @namefmt: printf-style name for the thread. Format is restricted
+ * to "name.*%u". Code fills in cpu number.
+ *
+ * Description: This helper function creates and names a kernel thread
+ * The thread will be woken and put into park mode.
+ */
+struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data),
+ void *data, unsigned int cpu,
+ const char *namefmt)
+{
+ struct task_struct *p;
+
+ p = kthread_create_on_node(threadfn, data, cpu_to_node(cpu), namefmt,
+ cpu);
+ if (IS_ERR(p))
+ return p;
+ set_bit(KTHREAD_IS_PER_CPU, &to_kthread(p)->flags);
+ to_kthread(p)->cpu = cpu;
+ /* Park the thread to get it out of TASK_UNINTERRUPTIBLE state */
+ kthread_park(p);
+ return p;
+}
+
+static struct kthread *task_get_live_kthread(struct task_struct *k)
+{
+ struct kthread *kthread;
+
+ get_task_struct(k);
+ kthread = to_kthread(k);
+ /* It might have exited */
+ barrier();
+ if (k->vfork_done != NULL)
+ return kthread;
+ return NULL;
+}
+
+/**
+ * kthread_unpark - unpark a thread created by kthread_create().
+ * @k: thread created by kthread_create().
+ *
+ * Sets kthread_should_park() for @k to return false, wakes it, and
+ * waits for it to return. If the thread is marked percpu then its
+ * bound to the cpu again.
+ */
+void kthread_unpark(struct task_struct *k)
+{
+ struct kthread *kthread = task_get_live_kthread(k);
+
+ if (kthread) {
+ clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
+ /*
+ * We clear the IS_PARKED bit here as we don't wait
+ * until the task has left the park code. So if we'd
+ * park before that happens we'd see the IS_PARKED bit
+ * which might be about to be cleared.
+ */
+ if (test_and_clear_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
+ if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
+ __kthread_bind(k, kthread->cpu);
+ wake_up_process(k);
+ }
+ }
+ put_task_struct(k);
+}
+
+/**
+ * kthread_park - park a thread created by kthread_create().
+ * @k: thread created by kthread_create().
+ *
+ * Sets kthread_should_park() for @k to return true, wakes it, and
+ * waits for it to return. This can also be called after kthread_create()
+ * instead of calling wake_up_process(): the thread will park without
+ * calling threadfn().
+ *
+ * Returns 0 if the thread is parked, -ENOSYS if the thread exited.
+ * If called by the kthread itself just the park bit is set.
+ */
+int kthread_park(struct task_struct *k)
+{
+ struct kthread *kthread = task_get_live_kthread(k);
+ int ret = -ENOSYS;
+
+ if (kthread) {
+ if (!test_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
+ set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
+ if (k != current) {
+ wake_up_process(k);
+ wait_for_completion(&kthread->parked);
+ }
+ }
+ ret = 0;
+ }
+ put_task_struct(k);
+ return ret;
+}
+
+/**
* kthread_stop - stop a thread created by kthread_create().
* @k: thread created by kthread_create().
*
@@ -250,16 +400,13 @@
*/
int kthread_stop(struct task_struct *k)
{
- struct kthread *kthread;
+ struct kthread *kthread = task_get_live_kthread(k);
int ret;
trace_sched_kthread_stop(k);
- get_task_struct(k);
-
- kthread = to_kthread(k);
- barrier(); /* it might have exited */
- if (k->vfork_done != NULL) {
- kthread->should_stop = 1;
+ if (kthread) {
+ set_bit(KTHREAD_SHOULD_STOP, &kthread->flags);
+ clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
wake_up_process(k);
wait_for_completion(&kthread->exited);
}
diff --git a/kernel/lockdep.c b/kernel/lockdep.c
index ea9ee45..7981e5b 100644
--- a/kernel/lockdep.c
+++ b/kernel/lockdep.c
@@ -2998,6 +2998,42 @@
struct lock_class_key __lockdep_no_validate__;
+static int
+print_lock_nested_lock_not_held(struct task_struct *curr,
+ struct held_lock *hlock,
+ unsigned long ip)
+{
+ if (!debug_locks_off())
+ return 0;
+ if (debug_locks_silent)
+ return 0;
+
+ printk("\n");
+ printk("==================================\n");
+ printk("[ BUG: Nested lock was not taken ]\n");
+ print_kernel_ident();
+ printk("----------------------------------\n");
+
+ printk("%s/%d is trying to lock:\n", curr->comm, task_pid_nr(curr));
+ print_lock(hlock);
+
+ printk("\nbut this task is not holding:\n");
+ printk("%s\n", hlock->nest_lock->name);
+
+ printk("\nstack backtrace:\n");
+ dump_stack();
+
+ printk("\nother info that might help us debug this:\n");
+ lockdep_print_held_locks(curr);
+
+ printk("\nstack backtrace:\n");
+ dump_stack();
+
+ return 0;
+}
+
+static int __lock_is_held(struct lockdep_map *lock);
+
/*
* This gets called for every mutex_lock*()/spin_lock*() operation.
* We maintain the dependency maps and validate the locking attempt:
@@ -3139,6 +3175,9 @@
}
chain_key = iterate_chain_key(chain_key, id);
+ if (nest_lock && !__lock_is_held(nest_lock))
+ return print_lock_nested_lock_not_held(curr, hlock, ip);
+
if (!validate_chain(curr, lock, hlock, chain_head, chain_key))
return 0;
diff --git a/kernel/pid_namespace.c b/kernel/pid_namespace.c
index b3c7fd5..6144bab 100644
--- a/kernel/pid_namespace.c
+++ b/kernel/pid_namespace.c
@@ -232,15 +232,19 @@
*/
tmp.data = ¤t->nsproxy->pid_ns->last_pid;
- return proc_dointvec(&tmp, write, buffer, lenp, ppos);
+ return proc_dointvec_minmax(&tmp, write, buffer, lenp, ppos);
}
+extern int pid_max;
+static int zero = 0;
static struct ctl_table pid_ns_ctl_table[] = {
{
.procname = "ns_last_pid",
.maxlen = sizeof(int),
.mode = 0666, /* permissions are checked in the handler */
.proc_handler = pid_ns_ctl_handler,
+ .extra1 = &zero,
+ .extra2 = &pid_max,
},
{ }
};
diff --git a/kernel/rcupdate.c b/kernel/rcupdate.c
index 4e6a61b..29ca1c6 100644
--- a/kernel/rcupdate.c
+++ b/kernel/rcupdate.c
@@ -45,6 +45,7 @@
#include <linux/mutex.h>
#include <linux/export.h>
#include <linux/hardirq.h>
+#include <linux/delay.h>
#define CREATE_TRACE_POINTS
#include <trace/events/rcu.h>
@@ -81,6 +82,9 @@
} else {
barrier(); /* critical section before exit code. */
t->rcu_read_lock_nesting = INT_MIN;
+#ifdef CONFIG_PROVE_RCU_DELAY
+ udelay(10); /* Make preemption more probable. */
+#endif /* #ifdef CONFIG_PROVE_RCU_DELAY */
barrier(); /* assign before ->rcu_read_unlock_special load */
if (unlikely(ACCESS_ONCE(t->rcu_read_unlock_special)))
rcu_read_unlock_special(t);
diff --git a/kernel/rcutiny.c b/kernel/rcutiny.c
index 547b1fe..e4c6a59 100644
--- a/kernel/rcutiny.c
+++ b/kernel/rcutiny.c
@@ -56,25 +56,28 @@
static long long rcu_dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
/* Common code for rcu_idle_enter() and rcu_irq_exit(), see kernel/rcutree.c. */
-static void rcu_idle_enter_common(long long oldval)
+static void rcu_idle_enter_common(long long newval)
{
- if (rcu_dynticks_nesting) {
+ if (newval) {
RCU_TRACE(trace_rcu_dyntick("--=",
- oldval, rcu_dynticks_nesting));
+ rcu_dynticks_nesting, newval));
+ rcu_dynticks_nesting = newval;
return;
}
- RCU_TRACE(trace_rcu_dyntick("Start", oldval, rcu_dynticks_nesting));
+ RCU_TRACE(trace_rcu_dyntick("Start", rcu_dynticks_nesting, newval));
if (!is_idle_task(current)) {
struct task_struct *idle = idle_task(smp_processor_id());
RCU_TRACE(trace_rcu_dyntick("Error on entry: not idle task",
- oldval, rcu_dynticks_nesting));
+ rcu_dynticks_nesting, newval));
ftrace_dump(DUMP_ALL);
WARN_ONCE(1, "Current pid: %d comm: %s / Idle pid: %d comm: %s",
current->pid, current->comm,
idle->pid, idle->comm); /* must be idle task! */
}
rcu_sched_qs(0); /* implies rcu_bh_qsctr_inc(0) */
+ barrier();
+ rcu_dynticks_nesting = newval;
}
/*
@@ -84,17 +87,16 @@
void rcu_idle_enter(void)
{
unsigned long flags;
- long long oldval;
+ long long newval;
local_irq_save(flags);
- oldval = rcu_dynticks_nesting;
WARN_ON_ONCE((rcu_dynticks_nesting & DYNTICK_TASK_NEST_MASK) == 0);
if ((rcu_dynticks_nesting & DYNTICK_TASK_NEST_MASK) ==
DYNTICK_TASK_NEST_VALUE)
- rcu_dynticks_nesting = 0;
+ newval = 0;
else
- rcu_dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
- rcu_idle_enter_common(oldval);
+ newval = rcu_dynticks_nesting - DYNTICK_TASK_NEST_VALUE;
+ rcu_idle_enter_common(newval);
local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(rcu_idle_enter);
@@ -105,15 +107,15 @@
void rcu_irq_exit(void)
{
unsigned long flags;
- long long oldval;
+ long long newval;
local_irq_save(flags);
- oldval = rcu_dynticks_nesting;
- rcu_dynticks_nesting--;
- WARN_ON_ONCE(rcu_dynticks_nesting < 0);
- rcu_idle_enter_common(oldval);
+ newval = rcu_dynticks_nesting - 1;
+ WARN_ON_ONCE(newval < 0);
+ rcu_idle_enter_common(newval);
local_irq_restore(flags);
}
+EXPORT_SYMBOL_GPL(rcu_irq_exit);
/* Common code for rcu_idle_exit() and rcu_irq_enter(), see kernel/rcutree.c. */
static void rcu_idle_exit_common(long long oldval)
@@ -171,6 +173,7 @@
rcu_idle_exit_common(oldval);
local_irq_restore(flags);
}
+EXPORT_SYMBOL_GPL(rcu_irq_enter);
#ifdef CONFIG_DEBUG_LOCK_ALLOC
diff --git a/kernel/rcutiny_plugin.h b/kernel/rcutiny_plugin.h
index 918fd1e..3d01902 100644
--- a/kernel/rcutiny_plugin.h
+++ b/kernel/rcutiny_plugin.h
@@ -278,7 +278,7 @@
rcu_preempt_ctrlblk.exp_tasks == NULL)
return 0; /* Nothing to boost. */
- raw_local_irq_save(flags);
+ local_irq_save(flags);
/*
* Recheck with irqs disabled: all tasks in need of boosting
@@ -287,7 +287,7 @@
*/
if (rcu_preempt_ctrlblk.boost_tasks == NULL &&
rcu_preempt_ctrlblk.exp_tasks == NULL) {
- raw_local_irq_restore(flags);
+ local_irq_restore(flags);
return 0;
}
@@ -317,7 +317,7 @@
t = container_of(tb, struct task_struct, rcu_node_entry);
rt_mutex_init_proxy_locked(&mtx, t);
t->rcu_boost_mutex = &mtx;
- raw_local_irq_restore(flags);
+ local_irq_restore(flags);
rt_mutex_lock(&mtx);
rt_mutex_unlock(&mtx); /* Keep lockdep happy. */
@@ -991,9 +991,9 @@
{
unsigned long flags;
- raw_local_irq_save(flags);
+ local_irq_save(flags);
rcp->qlen -= n;
- raw_local_irq_restore(flags);
+ local_irq_restore(flags);
}
/*
diff --git a/kernel/rcutorture.c b/kernel/rcutorture.c
index 25b1503..aaa7b9f 100644
--- a/kernel/rcutorture.c
+++ b/kernel/rcutorture.c
@@ -53,10 +53,11 @@
static int nreaders = -1; /* # reader threads, defaults to 2*ncpus */
static int nfakewriters = 4; /* # fake writer threads */
-static int stat_interval; /* Interval between stats, in seconds. */
- /* Defaults to "only at end of test". */
+static int stat_interval = 60; /* Interval between stats, in seconds. */
+ /* Zero means "only at end of test". */
static bool verbose; /* Print more debug info. */
-static bool test_no_idle_hz; /* Test RCU's support for tickless idle CPUs. */
+static bool test_no_idle_hz = true;
+ /* Test RCU support for tickless idle CPUs. */
static int shuffle_interval = 3; /* Interval between shuffles (in sec)*/
static int stutter = 5; /* Start/stop testing interval (in sec) */
static int irqreader = 1; /* RCU readers from irq (timers). */
@@ -119,11 +120,11 @@
#define TORTURE_FLAG "-torture:"
#define PRINTK_STRING(s) \
- do { printk(KERN_ALERT "%s" TORTURE_FLAG s "\n", torture_type); } while (0)
+ do { pr_alert("%s" TORTURE_FLAG s "\n", torture_type); } while (0)
#define VERBOSE_PRINTK_STRING(s) \
- do { if (verbose) printk(KERN_ALERT "%s" TORTURE_FLAG s "\n", torture_type); } while (0)
+ do { if (verbose) pr_alert("%s" TORTURE_FLAG s "\n", torture_type); } while (0)
#define VERBOSE_PRINTK_ERRSTRING(s) \
- do { if (verbose) printk(KERN_ALERT "%s" TORTURE_FLAG "!!! " s "\n", torture_type); } while (0)
+ do { if (verbose) pr_alert("%s" TORTURE_FLAG "!!! " s "\n", torture_type); } while (0)
static char printk_buf[4096];
@@ -176,8 +177,14 @@
static long n_rcu_torture_timers;
static long n_offline_attempts;
static long n_offline_successes;
+static unsigned long sum_offline;
+static int min_offline = -1;
+static int max_offline;
static long n_online_attempts;
static long n_online_successes;
+static unsigned long sum_online;
+static int min_online = -1;
+static int max_online;
static long n_barrier_attempts;
static long n_barrier_successes;
static struct list_head rcu_torture_removed;
@@ -235,7 +242,7 @@
if (fullstop == FULLSTOP_DONTSTOP)
fullstop = FULLSTOP_SHUTDOWN;
else
- printk(KERN_WARNING /* but going down anyway, so... */
+ pr_warn(/* but going down anyway, so... */
"Concurrent 'rmmod rcutorture' and shutdown illegal!\n");
mutex_unlock(&fullstop_mutex);
return NOTIFY_DONE;
@@ -248,7 +255,7 @@
static void rcutorture_shutdown_absorb(char *title)
{
if (ACCESS_ONCE(fullstop) == FULLSTOP_SHUTDOWN) {
- printk(KERN_NOTICE
+ pr_notice(
"rcutorture thread %s parking due to system shutdown\n",
title);
schedule_timeout_uninterruptible(MAX_SCHEDULE_TIMEOUT);
@@ -1214,11 +1221,13 @@
n_rcu_torture_boost_failure,
n_rcu_torture_boosts,
n_rcu_torture_timers);
- cnt += sprintf(&page[cnt], "onoff: %ld/%ld:%ld/%ld ",
- n_online_successes,
- n_online_attempts,
- n_offline_successes,
- n_offline_attempts);
+ cnt += sprintf(&page[cnt],
+ "onoff: %ld/%ld:%ld/%ld %d,%d:%d,%d %lu:%lu (HZ=%d) ",
+ n_online_successes, n_online_attempts,
+ n_offline_successes, n_offline_attempts,
+ min_online, max_online,
+ min_offline, max_offline,
+ sum_online, sum_offline, HZ);
cnt += sprintf(&page[cnt], "barrier: %ld/%ld:%ld",
n_barrier_successes,
n_barrier_attempts,
@@ -1267,7 +1276,7 @@
int cnt;
cnt = rcu_torture_printk(printk_buf);
- printk(KERN_ALERT "%s", printk_buf);
+ pr_alert("%s", printk_buf);
}
/*
@@ -1380,20 +1389,20 @@
static inline void
rcu_torture_print_module_parms(struct rcu_torture_ops *cur_ops, char *tag)
{
- printk(KERN_ALERT "%s" TORTURE_FLAG
- "--- %s: nreaders=%d nfakewriters=%d "
- "stat_interval=%d verbose=%d test_no_idle_hz=%d "
- "shuffle_interval=%d stutter=%d irqreader=%d "
- "fqs_duration=%d fqs_holdoff=%d fqs_stutter=%d "
- "test_boost=%d/%d test_boost_interval=%d "
- "test_boost_duration=%d shutdown_secs=%d "
- "onoff_interval=%d onoff_holdoff=%d\n",
- torture_type, tag, nrealreaders, nfakewriters,
- stat_interval, verbose, test_no_idle_hz, shuffle_interval,
- stutter, irqreader, fqs_duration, fqs_holdoff, fqs_stutter,
- test_boost, cur_ops->can_boost,
- test_boost_interval, test_boost_duration, shutdown_secs,
- onoff_interval, onoff_holdoff);
+ pr_alert("%s" TORTURE_FLAG
+ "--- %s: nreaders=%d nfakewriters=%d "
+ "stat_interval=%d verbose=%d test_no_idle_hz=%d "
+ "shuffle_interval=%d stutter=%d irqreader=%d "
+ "fqs_duration=%d fqs_holdoff=%d fqs_stutter=%d "
+ "test_boost=%d/%d test_boost_interval=%d "
+ "test_boost_duration=%d shutdown_secs=%d "
+ "onoff_interval=%d onoff_holdoff=%d\n",
+ torture_type, tag, nrealreaders, nfakewriters,
+ stat_interval, verbose, test_no_idle_hz, shuffle_interval,
+ stutter, irqreader, fqs_duration, fqs_holdoff, fqs_stutter,
+ test_boost, cur_ops->can_boost,
+ test_boost_interval, test_boost_duration, shutdown_secs,
+ onoff_interval, onoff_holdoff);
}
static struct notifier_block rcutorture_shutdown_nb = {
@@ -1460,9 +1469,9 @@
!kthread_should_stop()) {
delta = shutdown_time - jiffies_snap;
if (verbose)
- printk(KERN_ALERT "%s" TORTURE_FLAG
- "rcu_torture_shutdown task: %lu jiffies remaining\n",
- torture_type, delta);
+ pr_alert("%s" TORTURE_FLAG
+ "rcu_torture_shutdown task: %lu jiffies remaining\n",
+ torture_type, delta);
schedule_timeout_interruptible(delta);
jiffies_snap = ACCESS_ONCE(jiffies);
}
@@ -1490,8 +1499,10 @@
rcu_torture_onoff(void *arg)
{
int cpu;
+ unsigned long delta;
int maxcpu = -1;
DEFINE_RCU_RANDOM(rand);
+ unsigned long starttime;
VERBOSE_PRINTK_STRING("rcu_torture_onoff task started");
for_each_online_cpu(cpu)
@@ -1506,29 +1517,51 @@
cpu = (rcu_random(&rand) >> 4) % (maxcpu + 1);
if (cpu_online(cpu) && cpu_is_hotpluggable(cpu)) {
if (verbose)
- printk(KERN_ALERT "%s" TORTURE_FLAG
- "rcu_torture_onoff task: offlining %d\n",
- torture_type, cpu);
+ pr_alert("%s" TORTURE_FLAG
+ "rcu_torture_onoff task: offlining %d\n",
+ torture_type, cpu);
+ starttime = jiffies;
n_offline_attempts++;
if (cpu_down(cpu) == 0) {
if (verbose)
- printk(KERN_ALERT "%s" TORTURE_FLAG
- "rcu_torture_onoff task: offlined %d\n",
- torture_type, cpu);
+ pr_alert("%s" TORTURE_FLAG
+ "rcu_torture_onoff task: offlined %d\n",
+ torture_type, cpu);
n_offline_successes++;
+ delta = jiffies - starttime;
+ sum_offline += delta;
+ if (min_offline < 0) {
+ min_offline = delta;
+ max_offline = delta;
+ }
+ if (min_offline > delta)
+ min_offline = delta;
+ if (max_offline < delta)
+ max_offline = delta;
}
} else if (cpu_is_hotpluggable(cpu)) {
if (verbose)
- printk(KERN_ALERT "%s" TORTURE_FLAG
- "rcu_torture_onoff task: onlining %d\n",
- torture_type, cpu);
+ pr_alert("%s" TORTURE_FLAG
+ "rcu_torture_onoff task: onlining %d\n",
+ torture_type, cpu);
+ starttime = jiffies;
n_online_attempts++;
if (cpu_up(cpu) == 0) {
if (verbose)
- printk(KERN_ALERT "%s" TORTURE_FLAG
- "rcu_torture_onoff task: onlined %d\n",
- torture_type, cpu);
+ pr_alert("%s" TORTURE_FLAG
+ "rcu_torture_onoff task: onlined %d\n",
+ torture_type, cpu);
n_online_successes++;
+ delta = jiffies - starttime;
+ sum_online += delta;
+ if (min_online < 0) {
+ min_online = delta;
+ max_online = delta;
+ }
+ if (min_online > delta)
+ min_online = delta;
+ if (max_online < delta)
+ max_online = delta;
}
}
schedule_timeout_interruptible(onoff_interval * HZ);
@@ -1593,14 +1626,14 @@
if (!kthread_should_stop()) {
stop_at = get_seconds() + stall_cpu;
/* RCU CPU stall is expected behavior in following code. */
- printk(KERN_ALERT "rcu_torture_stall start.\n");
+ pr_alert("rcu_torture_stall start.\n");
rcu_read_lock();
preempt_disable();
while (ULONG_CMP_LT(get_seconds(), stop_at))
continue; /* Induce RCU CPU stall warning. */
preempt_enable();
rcu_read_unlock();
- printk(KERN_ALERT "rcu_torture_stall end.\n");
+ pr_alert("rcu_torture_stall end.\n");
}
rcutorture_shutdown_absorb("rcu_torture_stall");
while (!kthread_should_stop())
@@ -1716,12 +1749,12 @@
if (n_barrier_cbs == 0)
return 0;
if (cur_ops->call == NULL || cur_ops->cb_barrier == NULL) {
- printk(KERN_ALERT "%s" TORTURE_FLAG
- " Call or barrier ops missing for %s,\n",
- torture_type, cur_ops->name);
- printk(KERN_ALERT "%s" TORTURE_FLAG
- " RCU barrier testing omitted from run.\n",
- torture_type);
+ pr_alert("%s" TORTURE_FLAG
+ " Call or barrier ops missing for %s,\n",
+ torture_type, cur_ops->name);
+ pr_alert("%s" TORTURE_FLAG
+ " RCU barrier testing omitted from run.\n",
+ torture_type);
return 0;
}
atomic_set(&barrier_cbs_count, 0);
@@ -1814,7 +1847,7 @@
mutex_lock(&fullstop_mutex);
rcutorture_record_test_transition();
if (fullstop == FULLSTOP_SHUTDOWN) {
- printk(KERN_WARNING /* but going down anyway, so... */
+ pr_warn(/* but going down anyway, so... */
"Concurrent 'rmmod rcutorture' and shutdown illegal!\n");
mutex_unlock(&fullstop_mutex);
schedule_timeout_uninterruptible(10);
@@ -1938,17 +1971,17 @@
break;
}
if (i == ARRAY_SIZE(torture_ops)) {
- printk(KERN_ALERT "rcu-torture: invalid torture type: \"%s\"\n",
- torture_type);
- printk(KERN_ALERT "rcu-torture types:");
+ pr_alert("rcu-torture: invalid torture type: \"%s\"\n",
+ torture_type);
+ pr_alert("rcu-torture types:");
for (i = 0; i < ARRAY_SIZE(torture_ops); i++)
- printk(KERN_ALERT " %s", torture_ops[i]->name);
- printk(KERN_ALERT "\n");
+ pr_alert(" %s", torture_ops[i]->name);
+ pr_alert("\n");
mutex_unlock(&fullstop_mutex);
return -EINVAL;
}
if (cur_ops->fqs == NULL && fqs_duration != 0) {
- printk(KERN_ALERT "rcu-torture: ->fqs NULL and non-zero fqs_duration, fqs disabled.\n");
+ pr_alert("rcu-torture: ->fqs NULL and non-zero fqs_duration, fqs disabled.\n");
fqs_duration = 0;
}
if (cur_ops->init)
@@ -1996,14 +2029,15 @@
/* Start up the kthreads. */
VERBOSE_PRINTK_STRING("Creating rcu_torture_writer task");
- writer_task = kthread_run(rcu_torture_writer, NULL,
- "rcu_torture_writer");
+ writer_task = kthread_create(rcu_torture_writer, NULL,
+ "rcu_torture_writer");
if (IS_ERR(writer_task)) {
firsterr = PTR_ERR(writer_task);
VERBOSE_PRINTK_ERRSTRING("Failed to create writer");
writer_task = NULL;
goto unwind;
}
+ wake_up_process(writer_task);
fakewriter_tasks = kzalloc(nfakewriters * sizeof(fakewriter_tasks[0]),
GFP_KERNEL);
if (fakewriter_tasks == NULL) {
@@ -2118,14 +2152,15 @@
}
if (shutdown_secs > 0) {
shutdown_time = jiffies + shutdown_secs * HZ;
- shutdown_task = kthread_run(rcu_torture_shutdown, NULL,
- "rcu_torture_shutdown");
+ shutdown_task = kthread_create(rcu_torture_shutdown, NULL,
+ "rcu_torture_shutdown");
if (IS_ERR(shutdown_task)) {
firsterr = PTR_ERR(shutdown_task);
VERBOSE_PRINTK_ERRSTRING("Failed to create shutdown");
shutdown_task = NULL;
goto unwind;
}
+ wake_up_process(shutdown_task);
}
i = rcu_torture_onoff_init();
if (i != 0) {
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index f280e54..4fb2376 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -52,6 +52,7 @@
#include <linux/prefetch.h>
#include <linux/delay.h>
#include <linux/stop_machine.h>
+#include <linux/random.h>
#include "rcutree.h"
#include <trace/events/rcu.h>
@@ -61,6 +62,7 @@
/* Data structures. */
static struct lock_class_key rcu_node_class[RCU_NUM_LVLS];
+static struct lock_class_key rcu_fqs_class[RCU_NUM_LVLS];
#define RCU_STATE_INITIALIZER(sname, cr) { \
.level = { &sname##_state.node[0] }, \
@@ -72,7 +74,6 @@
.orphan_nxttail = &sname##_state.orphan_nxtlist, \
.orphan_donetail = &sname##_state.orphan_donelist, \
.barrier_mutex = __MUTEX_INITIALIZER(sname##_state.barrier_mutex), \
- .fqslock = __RAW_SPIN_LOCK_UNLOCKED(&sname##_state.fqslock), \
.name = #sname, \
}
@@ -88,7 +89,7 @@
/* Increase (but not decrease) the CONFIG_RCU_FANOUT_LEAF at boot time. */
static int rcu_fanout_leaf = CONFIG_RCU_FANOUT_LEAF;
-module_param(rcu_fanout_leaf, int, 0);
+module_param(rcu_fanout_leaf, int, 0444);
int rcu_num_lvls __read_mostly = RCU_NUM_LVLS;
static int num_rcu_lvl[] = { /* Number of rcu_nodes at specified level. */
NUM_RCU_LVL_0,
@@ -133,13 +134,12 @@
*/
static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task);
DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_status);
-DEFINE_PER_CPU(int, rcu_cpu_kthread_cpu);
DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops);
DEFINE_PER_CPU(char, rcu_cpu_has_work);
#endif /* #ifdef CONFIG_RCU_BOOST */
-static void rcu_node_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu);
+static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu);
static void invoke_rcu_core(void);
static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp);
@@ -175,8 +175,6 @@
{
struct rcu_data *rdp = &per_cpu(rcu_sched_data, cpu);
- rdp->passed_quiesce_gpnum = rdp->gpnum;
- barrier();
if (rdp->passed_quiesce == 0)
trace_rcu_grace_period("rcu_sched", rdp->gpnum, "cpuqs");
rdp->passed_quiesce = 1;
@@ -186,8 +184,6 @@
{
struct rcu_data *rdp = &per_cpu(rcu_bh_data, cpu);
- rdp->passed_quiesce_gpnum = rdp->gpnum;
- barrier();
if (rdp->passed_quiesce == 0)
trace_rcu_grace_period("rcu_bh", rdp->gpnum, "cpuqs");
rdp->passed_quiesce = 1;
@@ -210,15 +206,18 @@
DEFINE_PER_CPU(struct rcu_dynticks, rcu_dynticks) = {
.dynticks_nesting = DYNTICK_TASK_EXIT_IDLE,
.dynticks = ATOMIC_INIT(1),
+#if defined(CONFIG_RCU_USER_QS) && !defined(CONFIG_RCU_USER_QS_FORCE)
+ .ignore_user_qs = true,
+#endif
};
static int blimit = 10; /* Maximum callbacks per rcu_do_batch. */
static int qhimark = 10000; /* If this many pending, ignore blimit. */
static int qlowmark = 100; /* Once only this many pending, use blimit. */
-module_param(blimit, int, 0);
-module_param(qhimark, int, 0);
-module_param(qlowmark, int, 0);
+module_param(blimit, int, 0444);
+module_param(qhimark, int, 0444);
+module_param(qlowmark, int, 0444);
int rcu_cpu_stall_suppress __read_mostly; /* 1 = suppress stall warnings. */
int rcu_cpu_stall_timeout __read_mostly = CONFIG_RCU_CPU_STALL_TIMEOUT;
@@ -226,7 +225,14 @@
module_param(rcu_cpu_stall_suppress, int, 0644);
module_param(rcu_cpu_stall_timeout, int, 0644);
-static void force_quiescent_state(struct rcu_state *rsp, int relaxed);
+static ulong jiffies_till_first_fqs = RCU_JIFFIES_TILL_FORCE_QS;
+static ulong jiffies_till_next_fqs = RCU_JIFFIES_TILL_FORCE_QS;
+
+module_param(jiffies_till_first_fqs, ulong, 0644);
+module_param(jiffies_till_next_fqs, ulong, 0644);
+
+static void force_qs_rnp(struct rcu_state *rsp, int (*f)(struct rcu_data *));
+static void force_quiescent_state(struct rcu_state *rsp);
static int rcu_pending(int cpu);
/*
@@ -252,7 +258,7 @@
*/
void rcu_bh_force_quiescent_state(void)
{
- force_quiescent_state(&rcu_bh_state, 0);
+ force_quiescent_state(&rcu_bh_state);
}
EXPORT_SYMBOL_GPL(rcu_bh_force_quiescent_state);
@@ -286,7 +292,7 @@
*/
void rcu_sched_force_quiescent_state(void)
{
- force_quiescent_state(&rcu_sched_state, 0);
+ force_quiescent_state(&rcu_sched_state);
}
EXPORT_SYMBOL_GPL(rcu_sched_force_quiescent_state);
@@ -305,7 +311,9 @@
static int
cpu_needs_another_gp(struct rcu_state *rsp, struct rcu_data *rdp)
{
- return *rdp->nxttail[RCU_DONE_TAIL] && !rcu_gp_in_progress(rsp);
+ return *rdp->nxttail[RCU_DONE_TAIL +
+ ACCESS_ONCE(rsp->completed) != rdp->completed] &&
+ !rcu_gp_in_progress(rsp);
}
/*
@@ -317,45 +325,17 @@
}
/*
- * If the specified CPU is offline, tell the caller that it is in
- * a quiescent state. Otherwise, whack it with a reschedule IPI.
- * Grace periods can end up waiting on an offline CPU when that
- * CPU is in the process of coming online -- it will be added to the
- * rcu_node bitmasks before it actually makes it online. The same thing
- * can happen while a CPU is in the process of coming online. Because this
- * race is quite rare, we check for it after detecting that the grace
- * period has been delayed rather than checking each and every CPU
- * each and every time we start a new grace period.
- */
-static int rcu_implicit_offline_qs(struct rcu_data *rdp)
-{
- /*
- * If the CPU is offline for more than a jiffy, it is in a quiescent
- * state. We can trust its state not to change because interrupts
- * are disabled. The reason for the jiffy's worth of slack is to
- * handle CPUs initializing on the way up and finding their way
- * to the idle loop on the way down.
- */
- if (cpu_is_offline(rdp->cpu) &&
- ULONG_CMP_LT(rdp->rsp->gp_start + 2, jiffies)) {
- trace_rcu_fqs(rdp->rsp->name, rdp->gpnum, rdp->cpu, "ofl");
- rdp->offline_fqs++;
- return 1;
- }
- return 0;
-}
-
-/*
- * rcu_idle_enter_common - inform RCU that current CPU is moving towards idle
+ * rcu_eqs_enter_common - current CPU is moving towards extended quiescent state
*
* If the new value of the ->dynticks_nesting counter now is zero,
* we really have entered idle, and must do the appropriate accounting.
* The caller must have disabled interrupts.
*/
-static void rcu_idle_enter_common(struct rcu_dynticks *rdtp, long long oldval)
+static void rcu_eqs_enter_common(struct rcu_dynticks *rdtp, long long oldval,
+ bool user)
{
trace_rcu_dyntick("Start", oldval, 0);
- if (!is_idle_task(current)) {
+ if (!user && !is_idle_task(current)) {
struct task_struct *idle = idle_task(smp_processor_id());
trace_rcu_dyntick("Error on entry: not idle task", oldval, 0);
@@ -372,7 +352,7 @@
WARN_ON_ONCE(atomic_read(&rdtp->dynticks) & 0x1);
/*
- * The idle task is not permitted to enter the idle loop while
+ * It is illegal to enter an extended quiescent state while
* in an RCU read-side critical section.
*/
rcu_lockdep_assert(!lock_is_held(&rcu_lock_map),
@@ -383,6 +363,25 @@
"Illegal idle entry in RCU-sched read-side critical section.");
}
+/*
+ * Enter an RCU extended quiescent state, which can be either the
+ * idle loop or adaptive-tickless usermode execution.
+ */
+static void rcu_eqs_enter(bool user)
+{
+ long long oldval;
+ struct rcu_dynticks *rdtp;
+
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ oldval = rdtp->dynticks_nesting;
+ WARN_ON_ONCE((oldval & DYNTICK_TASK_NEST_MASK) == 0);
+ if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE)
+ rdtp->dynticks_nesting = 0;
+ else
+ rdtp->dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
+ rcu_eqs_enter_common(rdtp, oldval, user);
+}
+
/**
* rcu_idle_enter - inform RCU that current CPU is entering idle
*
@@ -398,21 +397,70 @@
void rcu_idle_enter(void)
{
unsigned long flags;
- long long oldval;
+
+ local_irq_save(flags);
+ rcu_eqs_enter(false);
+ local_irq_restore(flags);
+}
+EXPORT_SYMBOL_GPL(rcu_idle_enter);
+
+#ifdef CONFIG_RCU_USER_QS
+/**
+ * rcu_user_enter - inform RCU that we are resuming userspace.
+ *
+ * Enter RCU idle mode right before resuming userspace. No use of RCU
+ * is permitted between this call and rcu_user_exit(). This way the
+ * CPU doesn't need to maintain the tick for RCU maintenance purposes
+ * when the CPU runs in userspace.
+ */
+void rcu_user_enter(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ /*
+ * Some contexts may involve an exception occuring in an irq,
+ * leading to that nesting:
+ * rcu_irq_enter() rcu_user_exit() rcu_user_exit() rcu_irq_exit()
+ * This would mess up the dyntick_nesting count though. And rcu_irq_*()
+ * helpers are enough to protect RCU uses inside the exception. So
+ * just return immediately if we detect we are in an IRQ.
+ */
+ if (in_interrupt())
+ return;
+
+ WARN_ON_ONCE(!current->mm);
+
+ local_irq_save(flags);
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ if (!rdtp->ignore_user_qs && !rdtp->in_user) {
+ rdtp->in_user = true;
+ rcu_eqs_enter(true);
+ }
+ local_irq_restore(flags);
+}
+
+/**
+ * rcu_user_enter_after_irq - inform RCU that we are going to resume userspace
+ * after the current irq returns.
+ *
+ * This is similar to rcu_user_enter() but in the context of a non-nesting
+ * irq. After this call, RCU enters into idle mode when the interrupt
+ * returns.
+ */
+void rcu_user_enter_after_irq(void)
+{
+ unsigned long flags;
struct rcu_dynticks *rdtp;
local_irq_save(flags);
rdtp = &__get_cpu_var(rcu_dynticks);
- oldval = rdtp->dynticks_nesting;
- WARN_ON_ONCE((oldval & DYNTICK_TASK_NEST_MASK) == 0);
- if ((oldval & DYNTICK_TASK_NEST_MASK) == DYNTICK_TASK_NEST_VALUE)
- rdtp->dynticks_nesting = 0;
- else
- rdtp->dynticks_nesting -= DYNTICK_TASK_NEST_VALUE;
- rcu_idle_enter_common(rdtp, oldval);
+ /* Ensure this irq is interrupting a non-idle RCU state. */
+ WARN_ON_ONCE(!(rdtp->dynticks_nesting & DYNTICK_TASK_MASK));
+ rdtp->dynticks_nesting = 1;
local_irq_restore(flags);
}
-EXPORT_SYMBOL_GPL(rcu_idle_enter);
+#endif /* CONFIG_RCU_USER_QS */
/**
* rcu_irq_exit - inform RCU that current CPU is exiting irq towards idle
@@ -444,18 +492,19 @@
if (rdtp->dynticks_nesting)
trace_rcu_dyntick("--=", oldval, rdtp->dynticks_nesting);
else
- rcu_idle_enter_common(rdtp, oldval);
+ rcu_eqs_enter_common(rdtp, oldval, true);
local_irq_restore(flags);
}
/*
- * rcu_idle_exit_common - inform RCU that current CPU is moving away from idle
+ * rcu_eqs_exit_common - current CPU moving away from extended quiescent state
*
* If the new value of the ->dynticks_nesting counter was previously zero,
* we really have exited idle, and must do the appropriate accounting.
* The caller must have disabled interrupts.
*/
-static void rcu_idle_exit_common(struct rcu_dynticks *rdtp, long long oldval)
+static void rcu_eqs_exit_common(struct rcu_dynticks *rdtp, long long oldval,
+ int user)
{
smp_mb__before_atomic_inc(); /* Force ordering w/previous sojourn. */
atomic_inc(&rdtp->dynticks);
@@ -464,7 +513,7 @@
WARN_ON_ONCE(!(atomic_read(&rdtp->dynticks) & 0x1));
rcu_cleanup_after_idle(smp_processor_id());
trace_rcu_dyntick("End", oldval, rdtp->dynticks_nesting);
- if (!is_idle_task(current)) {
+ if (!user && !is_idle_task(current)) {
struct task_struct *idle = idle_task(smp_processor_id());
trace_rcu_dyntick("Error on exit: not idle task",
@@ -476,6 +525,25 @@
}
}
+/*
+ * Exit an RCU extended quiescent state, which can be either the
+ * idle loop or adaptive-tickless usermode execution.
+ */
+static void rcu_eqs_exit(bool user)
+{
+ struct rcu_dynticks *rdtp;
+ long long oldval;
+
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ oldval = rdtp->dynticks_nesting;
+ WARN_ON_ONCE(oldval < 0);
+ if (oldval & DYNTICK_TASK_NEST_MASK)
+ rdtp->dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
+ else
+ rdtp->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
+ rcu_eqs_exit_common(rdtp, oldval, user);
+}
+
/**
* rcu_idle_exit - inform RCU that current CPU is leaving idle
*
@@ -490,22 +558,68 @@
void rcu_idle_exit(void)
{
unsigned long flags;
- struct rcu_dynticks *rdtp;
- long long oldval;
local_irq_save(flags);
- rdtp = &__get_cpu_var(rcu_dynticks);
- oldval = rdtp->dynticks_nesting;
- WARN_ON_ONCE(oldval < 0);
- if (oldval & DYNTICK_TASK_NEST_MASK)
- rdtp->dynticks_nesting += DYNTICK_TASK_NEST_VALUE;
- else
- rdtp->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
- rcu_idle_exit_common(rdtp, oldval);
+ rcu_eqs_exit(false);
local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(rcu_idle_exit);
+#ifdef CONFIG_RCU_USER_QS
+/**
+ * rcu_user_exit - inform RCU that we are exiting userspace.
+ *
+ * Exit RCU idle mode while entering the kernel because it can
+ * run a RCU read side critical section anytime.
+ */
+void rcu_user_exit(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ /*
+ * Some contexts may involve an exception occuring in an irq,
+ * leading to that nesting:
+ * rcu_irq_enter() rcu_user_exit() rcu_user_exit() rcu_irq_exit()
+ * This would mess up the dyntick_nesting count though. And rcu_irq_*()
+ * helpers are enough to protect RCU uses inside the exception. So
+ * just return immediately if we detect we are in an IRQ.
+ */
+ if (in_interrupt())
+ return;
+
+ local_irq_save(flags);
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ if (rdtp->in_user) {
+ rdtp->in_user = false;
+ rcu_eqs_exit(true);
+ }
+ local_irq_restore(flags);
+}
+
+/**
+ * rcu_user_exit_after_irq - inform RCU that we won't resume to userspace
+ * idle mode after the current non-nesting irq returns.
+ *
+ * This is similar to rcu_user_exit() but in the context of an irq.
+ * This is called when the irq has interrupted a userspace RCU idle mode
+ * context. When the current non-nesting interrupt returns after this call,
+ * the CPU won't restore the RCU idle mode.
+ */
+void rcu_user_exit_after_irq(void)
+{
+ unsigned long flags;
+ struct rcu_dynticks *rdtp;
+
+ local_irq_save(flags);
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ /* Ensure we are interrupting an RCU idle mode. */
+ WARN_ON_ONCE(rdtp->dynticks_nesting & DYNTICK_TASK_NEST_MASK);
+ rdtp->dynticks_nesting += DYNTICK_TASK_EXIT_IDLE;
+ local_irq_restore(flags);
+}
+#endif /* CONFIG_RCU_USER_QS */
+
/**
* rcu_irq_enter - inform RCU that current CPU is entering irq away from idle
*
@@ -539,7 +653,7 @@
if (oldval)
trace_rcu_dyntick("++=", oldval, rdtp->dynticks_nesting);
else
- rcu_idle_exit_common(rdtp, oldval);
+ rcu_eqs_exit_common(rdtp, oldval, true);
local_irq_restore(flags);
}
@@ -603,6 +717,21 @@
}
EXPORT_SYMBOL(rcu_is_cpu_idle);
+#ifdef CONFIG_RCU_USER_QS
+void rcu_user_hooks_switch(struct task_struct *prev,
+ struct task_struct *next)
+{
+ struct rcu_dynticks *rdtp;
+
+ /* Interrupts are disabled in context switch */
+ rdtp = &__get_cpu_var(rcu_dynticks);
+ if (!rdtp->ignore_user_qs) {
+ clear_tsk_thread_flag(prev, TIF_NOHZ);
+ set_tsk_thread_flag(next, TIF_NOHZ);
+ }
+}
+#endif /* #ifdef CONFIG_RCU_USER_QS */
+
#if defined(CONFIG_PROVE_RCU) && defined(CONFIG_HOTPLUG_CPU)
/*
@@ -673,7 +802,7 @@
* Return true if the specified CPU has passed through a quiescent
* state by virtue of being in or having passed through an dynticks
* idle state since the last call to dyntick_save_progress_counter()
- * for this same CPU.
+ * for this same CPU, or by virtue of having been offline.
*/
static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
{
@@ -697,8 +826,26 @@
return 1;
}
- /* Go check for the CPU being offline. */
- return rcu_implicit_offline_qs(rdp);
+ /*
+ * Check for the CPU being offline, but only if the grace period
+ * is old enough. We don't need to worry about the CPU changing
+ * state: If we see it offline even once, it has been through a
+ * quiescent state.
+ *
+ * The reason for insisting that the grace period be at least
+ * one jiffy old is that CPUs that are not quite online and that
+ * have just gone offline can still execute RCU read-side critical
+ * sections.
+ */
+ if (ULONG_CMP_GE(rdp->rsp->gp_start + 2, jiffies))
+ return 0; /* Grace period is not old enough. */
+ barrier();
+ if (cpu_is_offline(rdp->cpu)) {
+ trace_rcu_fqs(rdp->rsp->name, rdp->gpnum, rdp->cpu, "ofl");
+ rdp->offline_fqs++;
+ return 1;
+ }
+ return 0;
}
static int jiffies_till_stall_check(void)
@@ -755,14 +902,15 @@
rcu_for_each_leaf_node(rsp, rnp) {
raw_spin_lock_irqsave(&rnp->lock, flags);
ndetected += rcu_print_task_stall(rnp);
+ if (rnp->qsmask != 0) {
+ for (cpu = 0; cpu <= rnp->grphi - rnp->grplo; cpu++)
+ if (rnp->qsmask & (1UL << cpu)) {
+ print_cpu_stall_info(rsp,
+ rnp->grplo + cpu);
+ ndetected++;
+ }
+ }
raw_spin_unlock_irqrestore(&rnp->lock, flags);
- if (rnp->qsmask == 0)
- continue;
- for (cpu = 0; cpu <= rnp->grphi - rnp->grplo; cpu++)
- if (rnp->qsmask & (1UL << cpu)) {
- print_cpu_stall_info(rsp, rnp->grplo + cpu);
- ndetected++;
- }
}
/*
@@ -782,11 +930,11 @@
else if (!trigger_all_cpu_backtrace())
dump_stack();
- /* If so configured, complain about tasks blocking the grace period. */
+ /* Complain about tasks blocking the grace period. */
rcu_print_detail_task_stall(rsp);
- force_quiescent_state(rsp, 0); /* Kick them all. */
+ force_quiescent_state(rsp); /* Kick them all. */
}
static void print_cpu_stall(struct rcu_state *rsp)
@@ -827,7 +975,8 @@
j = ACCESS_ONCE(jiffies);
js = ACCESS_ONCE(rsp->jiffies_stall);
rnp = rdp->mynode;
- if ((ACCESS_ONCE(rnp->qsmask) & rdp->grpmask) && ULONG_CMP_GE(j, js)) {
+ if (rcu_gp_in_progress(rsp) &&
+ (ACCESS_ONCE(rnp->qsmask) & rdp->grpmask) && ULONG_CMP_GE(j, js)) {
/* We haven't checked in, so go dump stack. */
print_cpu_stall(rsp);
@@ -889,12 +1038,8 @@
*/
rdp->gpnum = rnp->gpnum;
trace_rcu_grace_period(rsp->name, rdp->gpnum, "cpustart");
- if (rnp->qsmask & rdp->grpmask) {
- rdp->qs_pending = 1;
- rdp->passed_quiesce = 0;
- } else {
- rdp->qs_pending = 0;
- }
+ rdp->passed_quiesce = 0;
+ rdp->qs_pending = !!(rnp->qsmask & rdp->grpmask);
zero_cpu_stall_ticks(rdp);
}
}
@@ -974,10 +1119,13 @@
* our behalf. Catch up with this state to avoid noting
* spurious new grace periods. If another grace period
* has started, then rnp->gpnum will have advanced, so
- * we will detect this later on.
+ * we will detect this later on. Of course, any quiescent
+ * states we found for the old GP are now invalid.
*/
- if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
+ if (ULONG_CMP_LT(rdp->gpnum, rdp->completed)) {
rdp->gpnum = rdp->completed;
+ rdp->passed_quiesce = 0;
+ }
/*
* If RCU does not need a quiescent state from this CPU,
@@ -1021,25 +1169,220 @@
/* Prior grace period ended, so advance callbacks for current CPU. */
__rcu_process_gp_end(rsp, rnp, rdp);
- /*
- * Because this CPU just now started the new grace period, we know
- * that all of its callbacks will be covered by this upcoming grace
- * period, even the ones that were registered arbitrarily recently.
- * Therefore, advance all outstanding callbacks to RCU_WAIT_TAIL.
- *
- * Other CPUs cannot be sure exactly when the grace period started.
- * Therefore, their recently registered callbacks must pass through
- * an additional RCU_NEXT_READY stage, so that they will be handled
- * by the next RCU grace period.
- */
- rdp->nxttail[RCU_NEXT_READY_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
- rdp->nxttail[RCU_WAIT_TAIL] = rdp->nxttail[RCU_NEXT_TAIL];
-
/* Set state so that this CPU will detect the next quiescent state. */
__note_new_gpnum(rsp, rnp, rdp);
}
/*
+ * Initialize a new grace period.
+ */
+static int rcu_gp_init(struct rcu_state *rsp)
+{
+ struct rcu_data *rdp;
+ struct rcu_node *rnp = rcu_get_root(rsp);
+
+ raw_spin_lock_irq(&rnp->lock);
+ rsp->gp_flags = 0; /* Clear all flags: New grace period. */
+
+ if (rcu_gp_in_progress(rsp)) {
+ /* Grace period already in progress, don't start another. */
+ raw_spin_unlock_irq(&rnp->lock);
+ return 0;
+ }
+
+ /* Advance to a new grace period and initialize state. */
+ rsp->gpnum++;
+ trace_rcu_grace_period(rsp->name, rsp->gpnum, "start");
+ record_gp_stall_check_time(rsp);
+ raw_spin_unlock_irq(&rnp->lock);
+
+ /* Exclude any concurrent CPU-hotplug operations. */
+ get_online_cpus();
+
+ /*
+ * Set the quiescent-state-needed bits in all the rcu_node
+ * structures for all currently online CPUs in breadth-first order,
+ * starting from the root rcu_node structure, relying on the layout
+ * of the tree within the rsp->node[] array. Note that other CPUs
+ * will access only the leaves of the hierarchy, thus seeing that no
+ * grace period is in progress, at least until the corresponding
+ * leaf node has been initialized. In addition, we have excluded
+ * CPU-hotplug operations.
+ *
+ * The grace period cannot complete until the initialization
+ * process finishes, because this kthread handles both.
+ */
+ rcu_for_each_node_breadth_first(rsp, rnp) {
+ raw_spin_lock_irq(&rnp->lock);
+ rdp = this_cpu_ptr(rsp->rda);
+ rcu_preempt_check_blocked_tasks(rnp);
+ rnp->qsmask = rnp->qsmaskinit;
+ rnp->gpnum = rsp->gpnum;
+ WARN_ON_ONCE(rnp->completed != rsp->completed);
+ rnp->completed = rsp->completed;
+ if (rnp == rdp->mynode)
+ rcu_start_gp_per_cpu(rsp, rnp, rdp);
+ rcu_preempt_boost_start_gp(rnp);
+ trace_rcu_grace_period_init(rsp->name, rnp->gpnum,
+ rnp->level, rnp->grplo,
+ rnp->grphi, rnp->qsmask);
+ raw_spin_unlock_irq(&rnp->lock);
+#ifdef CONFIG_PROVE_RCU_DELAY
+ if ((random32() % (rcu_num_nodes * 8)) == 0)
+ schedule_timeout_uninterruptible(2);
+#endif /* #ifdef CONFIG_PROVE_RCU_DELAY */
+ cond_resched();
+ }
+
+ put_online_cpus();
+ return 1;
+}
+
+/*
+ * Do one round of quiescent-state forcing.
+ */
+int rcu_gp_fqs(struct rcu_state *rsp, int fqs_state_in)
+{
+ int fqs_state = fqs_state_in;
+ struct rcu_node *rnp = rcu_get_root(rsp);
+
+ rsp->n_force_qs++;
+ if (fqs_state == RCU_SAVE_DYNTICK) {
+ /* Collect dyntick-idle snapshots. */
+ force_qs_rnp(rsp, dyntick_save_progress_counter);
+ fqs_state = RCU_FORCE_QS;
+ } else {
+ /* Handle dyntick-idle and offline CPUs. */
+ force_qs_rnp(rsp, rcu_implicit_dynticks_qs);
+ }
+ /* Clear flag to prevent immediate re-entry. */
+ if (ACCESS_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) {
+ raw_spin_lock_irq(&rnp->lock);
+ rsp->gp_flags &= ~RCU_GP_FLAG_FQS;
+ raw_spin_unlock_irq(&rnp->lock);
+ }
+ return fqs_state;
+}
+
+/*
+ * Clean up after the old grace period.
+ */
+static void rcu_gp_cleanup(struct rcu_state *rsp)
+{
+ unsigned long gp_duration;
+ struct rcu_data *rdp;
+ struct rcu_node *rnp = rcu_get_root(rsp);
+
+ raw_spin_lock_irq(&rnp->lock);
+ gp_duration = jiffies - rsp->gp_start;
+ if (gp_duration > rsp->gp_max)
+ rsp->gp_max = gp_duration;
+
+ /*
+ * We know the grace period is complete, but to everyone else
+ * it appears to still be ongoing. But it is also the case
+ * that to everyone else it looks like there is nothing that
+ * they can do to advance the grace period. It is therefore
+ * safe for us to drop the lock in order to mark the grace
+ * period as completed in all of the rcu_node structures.
+ */
+ raw_spin_unlock_irq(&rnp->lock);
+
+ /*
+ * Propagate new ->completed value to rcu_node structures so
+ * that other CPUs don't have to wait until the start of the next
+ * grace period to process their callbacks. This also avoids
+ * some nasty RCU grace-period initialization races by forcing
+ * the end of the current grace period to be completely recorded in
+ * all of the rcu_node structures before the beginning of the next
+ * grace period is recorded in any of the rcu_node structures.
+ */
+ rcu_for_each_node_breadth_first(rsp, rnp) {
+ raw_spin_lock_irq(&rnp->lock);
+ rnp->completed = rsp->gpnum;
+ raw_spin_unlock_irq(&rnp->lock);
+ cond_resched();
+ }
+ rnp = rcu_get_root(rsp);
+ raw_spin_lock_irq(&rnp->lock);
+
+ rsp->completed = rsp->gpnum; /* Declare grace period done. */
+ trace_rcu_grace_period(rsp->name, rsp->completed, "end");
+ rsp->fqs_state = RCU_GP_IDLE;
+ rdp = this_cpu_ptr(rsp->rda);
+ if (cpu_needs_another_gp(rsp, rdp))
+ rsp->gp_flags = 1;
+ raw_spin_unlock_irq(&rnp->lock);
+}
+
+/*
+ * Body of kthread that handles grace periods.
+ */
+static int __noreturn rcu_gp_kthread(void *arg)
+{
+ int fqs_state;
+ unsigned long j;
+ int ret;
+ struct rcu_state *rsp = arg;
+ struct rcu_node *rnp = rcu_get_root(rsp);
+
+ for (;;) {
+
+ /* Handle grace-period start. */
+ for (;;) {
+ wait_event_interruptible(rsp->gp_wq,
+ rsp->gp_flags &
+ RCU_GP_FLAG_INIT);
+ if ((rsp->gp_flags & RCU_GP_FLAG_INIT) &&
+ rcu_gp_init(rsp))
+ break;
+ cond_resched();
+ flush_signals(current);
+ }
+
+ /* Handle quiescent-state forcing. */
+ fqs_state = RCU_SAVE_DYNTICK;
+ j = jiffies_till_first_fqs;
+ if (j > HZ) {
+ j = HZ;
+ jiffies_till_first_fqs = HZ;
+ }
+ for (;;) {
+ rsp->jiffies_force_qs = jiffies + j;
+ ret = wait_event_interruptible_timeout(rsp->gp_wq,
+ (rsp->gp_flags & RCU_GP_FLAG_FQS) ||
+ (!ACCESS_ONCE(rnp->qsmask) &&
+ !rcu_preempt_blocked_readers_cgp(rnp)),
+ j);
+ /* If grace period done, leave loop. */
+ if (!ACCESS_ONCE(rnp->qsmask) &&
+ !rcu_preempt_blocked_readers_cgp(rnp))
+ break;
+ /* If time for quiescent-state forcing, do it. */
+ if (ret == 0 || (rsp->gp_flags & RCU_GP_FLAG_FQS)) {
+ fqs_state = rcu_gp_fqs(rsp, fqs_state);
+ cond_resched();
+ } else {
+ /* Deal with stray signal. */
+ cond_resched();
+ flush_signals(current);
+ }
+ j = jiffies_till_next_fqs;
+ if (j > HZ) {
+ j = HZ;
+ jiffies_till_next_fqs = HZ;
+ } else if (j < 1) {
+ j = 1;
+ jiffies_till_next_fqs = 1;
+ }
+ }
+
+ /* Handle grace-period end. */
+ rcu_gp_cleanup(rsp);
+ }
+}
+
+/*
* Start a new RCU grace period if warranted, re-initializing the hierarchy
* in preparation for detecting the next grace period. The caller must hold
* the root node's ->lock, which is released before return. Hard irqs must
@@ -1056,77 +1399,20 @@
struct rcu_data *rdp = this_cpu_ptr(rsp->rda);
struct rcu_node *rnp = rcu_get_root(rsp);
- if (!rcu_scheduler_fully_active ||
+ if (!rsp->gp_kthread ||
!cpu_needs_another_gp(rsp, rdp)) {
/*
- * Either the scheduler hasn't yet spawned the first
- * non-idle task or this CPU does not need another
- * grace period. Either way, don't start a new grace
- * period.
+ * Either we have not yet spawned the grace-period
+ * task or this CPU does not need another grace period.
+ * Either way, don't start a new grace period.
*/
raw_spin_unlock_irqrestore(&rnp->lock, flags);
return;
}
- if (rsp->fqs_active) {
- /*
- * This CPU needs a grace period, but force_quiescent_state()
- * is running. Tell it to start one on this CPU's behalf.
- */
- rsp->fqs_need_gp = 1;
- raw_spin_unlock_irqrestore(&rnp->lock, flags);
- return;
- }
-
- /* Advance to a new grace period and initialize state. */
- rsp->gpnum++;
- trace_rcu_grace_period(rsp->name, rsp->gpnum, "start");
- WARN_ON_ONCE(rsp->fqs_state == RCU_GP_INIT);
- rsp->fqs_state = RCU_GP_INIT; /* Hold off force_quiescent_state. */
- rsp->jiffies_force_qs = jiffies + RCU_JIFFIES_TILL_FORCE_QS;
- record_gp_stall_check_time(rsp);
- raw_spin_unlock(&rnp->lock); /* leave irqs disabled. */
-
- /* Exclude any concurrent CPU-hotplug operations. */
- raw_spin_lock(&rsp->onofflock); /* irqs already disabled. */
-
- /*
- * Set the quiescent-state-needed bits in all the rcu_node
- * structures for all currently online CPUs in breadth-first
- * order, starting from the root rcu_node structure. This
- * operation relies on the layout of the hierarchy within the
- * rsp->node[] array. Note that other CPUs will access only
- * the leaves of the hierarchy, which still indicate that no
- * grace period is in progress, at least until the corresponding
- * leaf node has been initialized. In addition, we have excluded
- * CPU-hotplug operations.
- *
- * Note that the grace period cannot complete until we finish
- * the initialization process, as there will be at least one
- * qsmask bit set in the root node until that time, namely the
- * one corresponding to this CPU, due to the fact that we have
- * irqs disabled.
- */
- rcu_for_each_node_breadth_first(rsp, rnp) {
- raw_spin_lock(&rnp->lock); /* irqs already disabled. */
- rcu_preempt_check_blocked_tasks(rnp);
- rnp->qsmask = rnp->qsmaskinit;
- rnp->gpnum = rsp->gpnum;
- rnp->completed = rsp->completed;
- if (rnp == rdp->mynode)
- rcu_start_gp_per_cpu(rsp, rnp, rdp);
- rcu_preempt_boost_start_gp(rnp);
- trace_rcu_grace_period_init(rsp->name, rnp->gpnum,
- rnp->level, rnp->grplo,
- rnp->grphi, rnp->qsmask);
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
- }
-
- rnp = rcu_get_root(rsp);
- raw_spin_lock(&rnp->lock); /* irqs already disabled. */
- rsp->fqs_state = RCU_SIGNAL_INIT; /* force_quiescent_state now OK. */
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
- raw_spin_unlock_irqrestore(&rsp->onofflock, flags);
+ rsp->gp_flags = RCU_GP_FLAG_INIT;
+ raw_spin_unlock_irqrestore(&rnp->lock, flags);
+ wake_up(&rsp->gp_wq);
}
/*
@@ -1139,57 +1425,9 @@
static void rcu_report_qs_rsp(struct rcu_state *rsp, unsigned long flags)
__releases(rcu_get_root(rsp)->lock)
{
- unsigned long gp_duration;
- struct rcu_node *rnp = rcu_get_root(rsp);
- struct rcu_data *rdp = this_cpu_ptr(rsp->rda);
-
WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
-
- /*
- * Ensure that all grace-period and pre-grace-period activity
- * is seen before the assignment to rsp->completed.
- */
- smp_mb(); /* See above block comment. */
- gp_duration = jiffies - rsp->gp_start;
- if (gp_duration > rsp->gp_max)
- rsp->gp_max = gp_duration;
-
- /*
- * We know the grace period is complete, but to everyone else
- * it appears to still be ongoing. But it is also the case
- * that to everyone else it looks like there is nothing that
- * they can do to advance the grace period. It is therefore
- * safe for us to drop the lock in order to mark the grace
- * period as completed in all of the rcu_node structures.
- *
- * But if this CPU needs another grace period, it will take
- * care of this while initializing the next grace period.
- * We use RCU_WAIT_TAIL instead of the usual RCU_DONE_TAIL
- * because the callbacks have not yet been advanced: Those
- * callbacks are waiting on the grace period that just now
- * completed.
- */
- if (*rdp->nxttail[RCU_WAIT_TAIL] == NULL) {
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
-
- /*
- * Propagate new ->completed value to rcu_node structures
- * so that other CPUs don't have to wait until the start
- * of the next grace period to process their callbacks.
- */
- rcu_for_each_node_breadth_first(rsp, rnp) {
- raw_spin_lock(&rnp->lock); /* irqs already disabled. */
- rnp->completed = rsp->gpnum;
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
- }
- rnp = rcu_get_root(rsp);
- raw_spin_lock(&rnp->lock); /* irqs already disabled. */
- }
-
- rsp->completed = rsp->gpnum; /* Declare the grace period complete. */
- trace_rcu_grace_period(rsp->name, rsp->completed, "end");
- rsp->fqs_state = RCU_GP_IDLE;
- rcu_start_gp(rsp, flags); /* releases root node's rnp->lock. */
+ raw_spin_unlock_irqrestore(&rcu_get_root(rsp)->lock, flags);
+ wake_up(&rsp->gp_wq); /* Memory barrier implied by wake_up() path. */
}
/*
@@ -1258,7 +1496,7 @@
* based on quiescent states detected in an earlier grace period!
*/
static void
-rcu_report_qs_rdp(int cpu, struct rcu_state *rsp, struct rcu_data *rdp, long lastgp)
+rcu_report_qs_rdp(int cpu, struct rcu_state *rsp, struct rcu_data *rdp)
{
unsigned long flags;
unsigned long mask;
@@ -1266,7 +1504,8 @@
rnp = rdp->mynode;
raw_spin_lock_irqsave(&rnp->lock, flags);
- if (lastgp != rnp->gpnum || rnp->completed == rnp->gpnum) {
+ if (rdp->passed_quiesce == 0 || rdp->gpnum != rnp->gpnum ||
+ rnp->completed == rnp->gpnum) {
/*
* The grace period in which this quiescent state was
@@ -1325,7 +1564,7 @@
* Tell RCU we are done (but rcu_report_qs_rdp() will be the
* judge of that).
*/
- rcu_report_qs_rdp(rdp->cpu, rsp, rdp, rdp->passed_quiesce_gpnum);
+ rcu_report_qs_rdp(rdp->cpu, rsp, rdp);
}
#ifdef CONFIG_HOTPLUG_CPU
@@ -1390,17 +1629,6 @@
int i;
struct rcu_data *rdp = __this_cpu_ptr(rsp->rda);
- /*
- * If there is an rcu_barrier() operation in progress, then
- * only the task doing that operation is permitted to adopt
- * callbacks. To do otherwise breaks rcu_barrier() and friends
- * by causing them to fail to wait for the callbacks in the
- * orphanage.
- */
- if (rsp->rcu_barrier_in_progress &&
- rsp->rcu_barrier_in_progress != current)
- return;
-
/* Do the accounting first. */
rdp->qlen_lazy += rsp->qlen_lazy;
rdp->qlen += rsp->qlen;
@@ -1455,9 +1683,8 @@
* The CPU has been completely removed, and some other CPU is reporting
* this fact from process context. Do the remainder of the cleanup,
* including orphaning the outgoing CPU's RCU callbacks, and also
- * adopting them, if there is no _rcu_barrier() instance running.
- * There can only be one CPU hotplug operation at a time, so no other
- * CPU can be attempting to update rcu_cpu_kthread_task.
+ * adopting them. There can only be one CPU hotplug operation at a time,
+ * so no other CPU can be attempting to update rcu_cpu_kthread_task.
*/
static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)
{
@@ -1468,8 +1695,7 @@
struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */
/* Adjust any no-longer-needed kthreads. */
- rcu_stop_cpu_kthread(cpu);
- rcu_node_kthread_setaffinity(rnp, -1);
+ rcu_boost_kthread_setaffinity(rnp, -1);
/* Remove the dead CPU from the bitmasks in the rcu_node hierarchy. */
@@ -1515,14 +1741,13 @@
WARN_ONCE(rdp->qlen != 0 || rdp->nxtlist != NULL,
"rcu_cleanup_dead_cpu: Callbacks on offline CPU %d: qlen=%lu, nxtlist=%p\n",
cpu, rdp->qlen, rdp->nxtlist);
+ init_callback_list(rdp);
+ /* Disallow further callbacks on this CPU. */
+ rdp->nxttail[RCU_NEXT_TAIL] = NULL;
}
#else /* #ifdef CONFIG_HOTPLUG_CPU */
-static void rcu_adopt_orphan_cbs(struct rcu_state *rsp)
-{
-}
-
static void rcu_cleanup_dying_cpu(struct rcu_state *rsp)
{
}
@@ -1687,6 +1912,7 @@
struct rcu_node *rnp;
rcu_for_each_leaf_node(rsp, rnp) {
+ cond_resched();
mask = 0;
raw_spin_lock_irqsave(&rnp->lock, flags);
if (!rcu_gp_in_progress(rsp)) {
@@ -1723,72 +1949,39 @@
* Force quiescent states on reluctant CPUs, and also detect which
* CPUs are in dyntick-idle mode.
*/
-static void force_quiescent_state(struct rcu_state *rsp, int relaxed)
+static void force_quiescent_state(struct rcu_state *rsp)
{
unsigned long flags;
- struct rcu_node *rnp = rcu_get_root(rsp);
+ bool ret;
+ struct rcu_node *rnp;
+ struct rcu_node *rnp_old = NULL;
- trace_rcu_utilization("Start fqs");
- if (!rcu_gp_in_progress(rsp)) {
- trace_rcu_utilization("End fqs");
- return; /* No grace period in progress, nothing to force. */
+ /* Funnel through hierarchy to reduce memory contention. */
+ rnp = per_cpu_ptr(rsp->rda, raw_smp_processor_id())->mynode;
+ for (; rnp != NULL; rnp = rnp->parent) {
+ ret = (ACCESS_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) ||
+ !raw_spin_trylock(&rnp->fqslock);
+ if (rnp_old != NULL)
+ raw_spin_unlock(&rnp_old->fqslock);
+ if (ret) {
+ rsp->n_force_qs_lh++;
+ return;
+ }
+ rnp_old = rnp;
}
- if (!raw_spin_trylock_irqsave(&rsp->fqslock, flags)) {
- rsp->n_force_qs_lh++; /* Inexact, can lose counts. Tough! */
- trace_rcu_utilization("End fqs");
- return; /* Someone else is already on the job. */
+ /* rnp_old == rcu_get_root(rsp), rnp == NULL. */
+
+ /* Reached the root of the rcu_node tree, acquire lock. */
+ raw_spin_lock_irqsave(&rnp_old->lock, flags);
+ raw_spin_unlock(&rnp_old->fqslock);
+ if (ACCESS_ONCE(rsp->gp_flags) & RCU_GP_FLAG_FQS) {
+ rsp->n_force_qs_lh++;
+ raw_spin_unlock_irqrestore(&rnp_old->lock, flags);
+ return; /* Someone beat us to it. */
}
- if (relaxed && ULONG_CMP_GE(rsp->jiffies_force_qs, jiffies))
- goto unlock_fqs_ret; /* no emergency and done recently. */
- rsp->n_force_qs++;
- raw_spin_lock(&rnp->lock); /* irqs already disabled */
- rsp->jiffies_force_qs = jiffies + RCU_JIFFIES_TILL_FORCE_QS;
- if(!rcu_gp_in_progress(rsp)) {
- rsp->n_force_qs_ngp++;
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
- goto unlock_fqs_ret; /* no GP in progress, time updated. */
- }
- rsp->fqs_active = 1;
- switch (rsp->fqs_state) {
- case RCU_GP_IDLE:
- case RCU_GP_INIT:
-
- break; /* grace period idle or initializing, ignore. */
-
- case RCU_SAVE_DYNTICK:
-
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
-
- /* Record dyntick-idle state. */
- force_qs_rnp(rsp, dyntick_save_progress_counter);
- raw_spin_lock(&rnp->lock); /* irqs already disabled */
- if (rcu_gp_in_progress(rsp))
- rsp->fqs_state = RCU_FORCE_QS;
- break;
-
- case RCU_FORCE_QS:
-
- /* Check dyntick-idle state, send IPI to laggarts. */
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
- force_qs_rnp(rsp, rcu_implicit_dynticks_qs);
-
- /* Leave state in case more forcing is required. */
-
- raw_spin_lock(&rnp->lock); /* irqs already disabled */
- break;
- }
- rsp->fqs_active = 0;
- if (rsp->fqs_need_gp) {
- raw_spin_unlock(&rsp->fqslock); /* irqs remain disabled */
- rsp->fqs_need_gp = 0;
- rcu_start_gp(rsp, flags); /* releases rnp->lock */
- trace_rcu_utilization("End fqs");
- return;
- }
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
-unlock_fqs_ret:
- raw_spin_unlock_irqrestore(&rsp->fqslock, flags);
- trace_rcu_utilization("End fqs");
+ rsp->gp_flags |= RCU_GP_FLAG_FQS;
+ raw_spin_unlock_irqrestore(&rnp_old->lock, flags);
+ wake_up(&rsp->gp_wq); /* Memory barrier implied by wake_up() path. */
}
/*
@@ -1805,13 +1998,6 @@
WARN_ON_ONCE(rdp->beenonline == 0);
/*
- * If an RCU GP has gone long enough, go check for dyntick
- * idle CPUs and, if needed, send resched IPIs.
- */
- if (ULONG_CMP_LT(ACCESS_ONCE(rsp->jiffies_force_qs), jiffies))
- force_quiescent_state(rsp, 1);
-
- /*
* Advance callbacks in response to end of earlier grace
* period that some other CPU ended.
*/
@@ -1838,6 +2024,8 @@
{
struct rcu_state *rsp;
+ if (cpu_is_offline(smp_processor_id()))
+ return;
trace_rcu_utilization("Start RCU core");
for_each_rcu_flavor(rsp)
__rcu_process_callbacks(rsp);
@@ -1909,12 +2097,11 @@
rdp->blimit = LONG_MAX;
if (rsp->n_force_qs == rdp->n_force_qs_snap &&
*rdp->nxttail[RCU_DONE_TAIL] != head)
- force_quiescent_state(rsp, 0);
+ force_quiescent_state(rsp);
rdp->n_force_qs_snap = rsp->n_force_qs;
rdp->qlen_last_fqs_check = rdp->qlen;
}
- } else if (ULONG_CMP_LT(ACCESS_ONCE(rsp->jiffies_force_qs), jiffies))
- force_quiescent_state(rsp, 1);
+ }
}
static void
@@ -1929,8 +2116,6 @@
head->func = func;
head->next = NULL;
- smp_mb(); /* Ensure RCU update seen before callback registry. */
-
/*
* Opportunistically note grace-period endings and beginnings.
* Note that we might see a beginning right after we see an
@@ -1941,6 +2126,12 @@
rdp = this_cpu_ptr(rsp->rda);
/* Add the callback to our list. */
+ if (unlikely(rdp->nxttail[RCU_NEXT_TAIL] == NULL)) {
+ /* _call_rcu() is illegal on offline CPU; leak the callback. */
+ WARN_ON_ONCE(1);
+ local_irq_restore(flags);
+ return;
+ }
ACCESS_ONCE(rdp->qlen)++;
if (lazy)
rdp->qlen_lazy++;
@@ -2195,17 +2386,7 @@
/* Is the RCU core waiting for a quiescent state from this CPU? */
if (rcu_scheduler_fully_active &&
rdp->qs_pending && !rdp->passed_quiesce) {
-
- /*
- * If force_quiescent_state() coming soon and this CPU
- * needs a quiescent state, and this is either RCU-sched
- * or RCU-bh, force a local reschedule.
- */
rdp->n_rp_qs_pending++;
- if (!rdp->preemptible &&
- ULONG_CMP_LT(ACCESS_ONCE(rsp->jiffies_force_qs) - 1,
- jiffies))
- set_need_resched();
} else if (rdp->qs_pending && rdp->passed_quiesce) {
rdp->n_rp_report_qs++;
return 1;
@@ -2235,13 +2416,6 @@
return 1;
}
- /* Has an RCU GP gone long enough to send resched IPIs &c? */
- if (rcu_gp_in_progress(rsp) &&
- ULONG_CMP_LT(ACCESS_ONCE(rsp->jiffies_force_qs), jiffies)) {
- rdp->n_rp_need_fqs++;
- return 1;
- }
-
/* nothing to do */
rdp->n_rp_need_nothing++;
return 0;
@@ -2326,13 +2500,10 @@
static void _rcu_barrier(struct rcu_state *rsp)
{
int cpu;
- unsigned long flags;
struct rcu_data *rdp;
- struct rcu_data rd;
unsigned long snap = ACCESS_ONCE(rsp->n_barrier_done);
unsigned long snap_done;
- init_rcu_head_on_stack(&rd.barrier_head);
_rcu_barrier_trace(rsp, "Begin", -1, snap);
/* Take mutex to serialize concurrent rcu_barrier() requests. */
@@ -2372,70 +2543,30 @@
/*
* Initialize the count to one rather than to zero in order to
* avoid a too-soon return to zero in case of a short grace period
- * (or preemption of this task). Also flag this task as doing
- * an rcu_barrier(). This will prevent anyone else from adopting
- * orphaned callbacks, which could cause otherwise failure if a
- * CPU went offline and quickly came back online. To see this,
- * consider the following sequence of events:
- *
- * 1. We cause CPU 0 to post an rcu_barrier_callback() callback.
- * 2. CPU 1 goes offline, orphaning its callbacks.
- * 3. CPU 0 adopts CPU 1's orphaned callbacks.
- * 4. CPU 1 comes back online.
- * 5. We cause CPU 1 to post an rcu_barrier_callback() callback.
- * 6. Both rcu_barrier_callback() callbacks are invoked, awakening
- * us -- but before CPU 1's orphaned callbacks are invoked!!!
+ * (or preemption of this task). Exclude CPU-hotplug operations
+ * to ensure that no offline CPU has callbacks queued.
*/
init_completion(&rsp->barrier_completion);
atomic_set(&rsp->barrier_cpu_count, 1);
- raw_spin_lock_irqsave(&rsp->onofflock, flags);
- rsp->rcu_barrier_in_progress = current;
- raw_spin_unlock_irqrestore(&rsp->onofflock, flags);
+ get_online_cpus();
/*
- * Force every CPU with callbacks to register a new callback
- * that will tell us when all the preceding callbacks have
- * been invoked. If an offline CPU has callbacks, wait for
- * it to either come back online or to finish orphaning those
- * callbacks.
+ * Force each CPU with callbacks to register a new callback.
+ * When that callback is invoked, we will know that all of the
+ * corresponding CPU's preceding callbacks have been invoked.
*/
- for_each_possible_cpu(cpu) {
- preempt_disable();
+ for_each_online_cpu(cpu) {
rdp = per_cpu_ptr(rsp->rda, cpu);
- if (cpu_is_offline(cpu)) {
- _rcu_barrier_trace(rsp, "Offline", cpu,
- rsp->n_barrier_done);
- preempt_enable();
- while (cpu_is_offline(cpu) && ACCESS_ONCE(rdp->qlen))
- schedule_timeout_interruptible(1);
- } else if (ACCESS_ONCE(rdp->qlen)) {
+ if (ACCESS_ONCE(rdp->qlen)) {
_rcu_barrier_trace(rsp, "OnlineQ", cpu,
rsp->n_barrier_done);
smp_call_function_single(cpu, rcu_barrier_func, rsp, 1);
- preempt_enable();
} else {
_rcu_barrier_trace(rsp, "OnlineNQ", cpu,
rsp->n_barrier_done);
- preempt_enable();
}
}
-
- /*
- * Now that all online CPUs have rcu_barrier_callback() callbacks
- * posted, we can adopt all of the orphaned callbacks and place
- * an rcu_barrier_callback() callback after them. When that is done,
- * we are guaranteed to have an rcu_barrier_callback() callback
- * following every callback that could possibly have been
- * registered before _rcu_barrier() was called.
- */
- raw_spin_lock_irqsave(&rsp->onofflock, flags);
- rcu_adopt_orphan_cbs(rsp);
- rsp->rcu_barrier_in_progress = NULL;
- raw_spin_unlock_irqrestore(&rsp->onofflock, flags);
- atomic_inc(&rsp->barrier_cpu_count);
- smp_mb__after_atomic_inc(); /* Ensure atomic_inc() before callback. */
- rd.rsp = rsp;
- rsp->call(&rd.barrier_head, rcu_barrier_callback);
+ put_online_cpus();
/*
* Now that we have an rcu_barrier_callback() callback on each
@@ -2456,8 +2587,6 @@
/* Other rcu_barrier() invocations can now safely proceed. */
mutex_unlock(&rsp->barrier_mutex);
-
- destroy_rcu_head_on_stack(&rd.barrier_head);
}
/**
@@ -2497,6 +2626,9 @@
rdp->dynticks = &per_cpu(rcu_dynticks, cpu);
WARN_ON_ONCE(rdp->dynticks->dynticks_nesting != DYNTICK_TASK_EXIT_IDLE);
WARN_ON_ONCE(atomic_read(&rdp->dynticks->dynticks) != 1);
+#ifdef CONFIG_RCU_USER_QS
+ WARN_ON_ONCE(rdp->dynticks->in_user);
+#endif
rdp->cpu = cpu;
rdp->rsp = rsp;
raw_spin_unlock_irqrestore(&rnp->lock, flags);
@@ -2523,6 +2655,7 @@
rdp->qlen_last_fqs_check = 0;
rdp->n_force_qs_snap = rsp->n_force_qs;
rdp->blimit = blimit;
+ init_callback_list(rdp); /* Re-enable callbacks on this CPU. */
rdp->dynticks->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
atomic_set(&rdp->dynticks->dynticks,
(atomic_read(&rdp->dynticks->dynticks) & ~0x1) + 1);
@@ -2555,7 +2688,6 @@
rdp->completed = rnp->completed;
rdp->passed_quiesce = 0;
rdp->qs_pending = 0;
- rdp->passed_quiesce_gpnum = rnp->gpnum - 1;
trace_rcu_grace_period(rsp->name, rdp->gpnum, "cpuonl");
}
raw_spin_unlock(&rnp->lock); /* irqs already disabled. */
@@ -2594,12 +2726,10 @@
break;
case CPU_ONLINE:
case CPU_DOWN_FAILED:
- rcu_node_kthread_setaffinity(rnp, -1);
- rcu_cpu_kthread_setrt(cpu, 1);
+ rcu_boost_kthread_setaffinity(rnp, -1);
break;
case CPU_DOWN_PREPARE:
- rcu_node_kthread_setaffinity(rnp, cpu);
- rcu_cpu_kthread_setrt(cpu, 0);
+ rcu_boost_kthread_setaffinity(rnp, cpu);
break;
case CPU_DYING:
case CPU_DYING_FROZEN:
@@ -2627,6 +2757,28 @@
}
/*
+ * Spawn the kthread that handles this RCU flavor's grace periods.
+ */
+static int __init rcu_spawn_gp_kthread(void)
+{
+ unsigned long flags;
+ struct rcu_node *rnp;
+ struct rcu_state *rsp;
+ struct task_struct *t;
+
+ for_each_rcu_flavor(rsp) {
+ t = kthread_run(rcu_gp_kthread, rsp, rsp->name);
+ BUG_ON(IS_ERR(t));
+ rnp = rcu_get_root(rsp);
+ raw_spin_lock_irqsave(&rnp->lock, flags);
+ rsp->gp_kthread = t;
+ raw_spin_unlock_irqrestore(&rnp->lock, flags);
+ }
+ return 0;
+}
+early_initcall(rcu_spawn_gp_kthread);
+
+/*
* This function is invoked towards the end of the scheduler's initialization
* process. Before this is called, the idle task might contain
* RCU read-side critical sections (during which time, this idle
@@ -2661,7 +2813,7 @@
int cprv;
int i;
- cprv = NR_CPUS;
+ cprv = nr_cpu_ids;
for (i = rcu_num_lvls - 1; i >= 0; i--) {
ccur = rsp->levelcnt[i];
rsp->levelspread[i] = (cprv + ccur - 1) / ccur;
@@ -2676,10 +2828,14 @@
static void __init rcu_init_one(struct rcu_state *rsp,
struct rcu_data __percpu *rda)
{
- static char *buf[] = { "rcu_node_level_0",
- "rcu_node_level_1",
- "rcu_node_level_2",
- "rcu_node_level_3" }; /* Match MAX_RCU_LVLS */
+ static char *buf[] = { "rcu_node_0",
+ "rcu_node_1",
+ "rcu_node_2",
+ "rcu_node_3" }; /* Match MAX_RCU_LVLS */
+ static char *fqs[] = { "rcu_node_fqs_0",
+ "rcu_node_fqs_1",
+ "rcu_node_fqs_2",
+ "rcu_node_fqs_3" }; /* Match MAX_RCU_LVLS */
int cpustride = 1;
int i;
int j;
@@ -2704,7 +2860,11 @@
raw_spin_lock_init(&rnp->lock);
lockdep_set_class_and_name(&rnp->lock,
&rcu_node_class[i], buf[i]);
- rnp->gpnum = 0;
+ raw_spin_lock_init(&rnp->fqslock);
+ lockdep_set_class_and_name(&rnp->fqslock,
+ &rcu_fqs_class[i], fqs[i]);
+ rnp->gpnum = rsp->gpnum;
+ rnp->completed = rsp->completed;
rnp->qsmask = 0;
rnp->qsmaskinit = 0;
rnp->grplo = j * cpustride;
@@ -2727,6 +2887,7 @@
}
rsp->rda = rda;
+ init_waitqueue_head(&rsp->gp_wq);
rnp = rsp->level[rcu_num_lvls - 1];
for_each_possible_cpu(i) {
while (i > rnp->grphi)
@@ -2750,7 +2911,8 @@
int rcu_capacity[MAX_RCU_LVLS + 1];
/* If the compile-time values are accurate, just leave. */
- if (rcu_fanout_leaf == CONFIG_RCU_FANOUT_LEAF)
+ if (rcu_fanout_leaf == CONFIG_RCU_FANOUT_LEAF &&
+ nr_cpu_ids == NR_CPUS)
return;
/*
diff --git a/kernel/rcutree.h b/kernel/rcutree.h
index 4d29169..5faf05d 100644
--- a/kernel/rcutree.h
+++ b/kernel/rcutree.h
@@ -102,6 +102,10 @@
/* idle-period nonlazy_posted snapshot. */
int tick_nohz_enabled_snap; /* Previously seen value from sysfs. */
#endif /* #ifdef CONFIG_RCU_FAST_NO_HZ */
+#ifdef CONFIG_RCU_USER_QS
+ bool ignore_user_qs; /* Treat userspace as extended QS or not */
+ bool in_user; /* Is the CPU in userland from RCU POV? */
+#endif
};
/* RCU's kthread states for tracing. */
@@ -196,12 +200,7 @@
/* Refused to boost: not sure why, though. */
/* This can happen due to race conditions. */
#endif /* #ifdef CONFIG_RCU_BOOST */
- struct task_struct *node_kthread_task;
- /* kthread that takes care of this rcu_node */
- /* structure, for example, awakening the */
- /* per-CPU kthreads as needed. */
- unsigned int node_kthread_status;
- /* State of node_kthread_task for tracing. */
+ raw_spinlock_t fqslock ____cacheline_internodealigned_in_smp;
} ____cacheline_internodealigned_in_smp;
/*
@@ -245,8 +244,6 @@
/* in order to detect GP end. */
unsigned long gpnum; /* Highest gp number that this CPU */
/* is aware of having started. */
- unsigned long passed_quiesce_gpnum;
- /* gpnum at time of quiescent state. */
bool passed_quiesce; /* User-mode/idle loop etc. */
bool qs_pending; /* Core waits for quiesc state. */
bool beenonline; /* CPU online at least once. */
@@ -312,11 +309,13 @@
unsigned long n_rp_cpu_needs_gp;
unsigned long n_rp_gp_completed;
unsigned long n_rp_gp_started;
- unsigned long n_rp_need_fqs;
unsigned long n_rp_need_nothing;
- /* 6) _rcu_barrier() callback. */
+ /* 6) _rcu_barrier() and OOM callbacks. */
struct rcu_head barrier_head;
+#ifdef CONFIG_RCU_FAST_NO_HZ
+ struct rcu_head oom_head;
+#endif /* #ifdef CONFIG_RCU_FAST_NO_HZ */
int cpu;
struct rcu_state *rsp;
@@ -375,20 +374,17 @@
u8 fqs_state ____cacheline_internodealigned_in_smp;
/* Force QS state. */
- u8 fqs_active; /* force_quiescent_state() */
- /* is running. */
- u8 fqs_need_gp; /* A CPU was prevented from */
- /* starting a new grace */
- /* period because */
- /* force_quiescent_state() */
- /* was running. */
u8 boost; /* Subject to priority boost. */
unsigned long gpnum; /* Current gp number. */
unsigned long completed; /* # of last completed gp. */
+ struct task_struct *gp_kthread; /* Task for grace periods. */
+ wait_queue_head_t gp_wq; /* Where GP task waits. */
+ int gp_flags; /* Commands for GP task. */
/* End of fields guarded by root rcu_node's lock. */
- raw_spinlock_t onofflock; /* exclude on/offline and */
+ raw_spinlock_t onofflock ____cacheline_internodealigned_in_smp;
+ /* exclude on/offline and */
/* starting new GP. */
struct rcu_head *orphan_nxtlist; /* Orphaned callbacks that */
/* need a grace period. */
@@ -398,16 +394,11 @@
struct rcu_head **orphan_donetail; /* Tail of above. */
long qlen_lazy; /* Number of lazy callbacks. */
long qlen; /* Total number of callbacks. */
- struct task_struct *rcu_barrier_in_progress;
- /* Task doing rcu_barrier(), */
- /* or NULL if no barrier. */
struct mutex barrier_mutex; /* Guards barrier fields. */
atomic_t barrier_cpu_count; /* # CPUs waiting on. */
struct completion barrier_completion; /* Wake at barrier end. */
unsigned long n_barrier_done; /* ++ at start and end of */
/* _rcu_barrier(). */
- raw_spinlock_t fqslock; /* Only one task forcing */
- /* quiescent states. */
unsigned long jiffies_force_qs; /* Time at which to invoke */
/* force_quiescent_state(). */
unsigned long n_force_qs; /* Number of calls to */
@@ -426,6 +417,10 @@
struct list_head flavors; /* List of RCU flavors. */
};
+/* Values for rcu_state structure's gp_flags field. */
+#define RCU_GP_FLAG_INIT 0x1 /* Need grace-period initialization. */
+#define RCU_GP_FLAG_FQS 0x2 /* Need grace-period quiescent-state forcing. */
+
extern struct list_head rcu_struct_flavors;
#define for_each_rcu_flavor(rsp) \
list_for_each_entry((rsp), &rcu_struct_flavors, flavors)
@@ -468,7 +463,6 @@
#ifdef CONFIG_HOTPLUG_CPU
static void rcu_report_unblock_qs_rnp(struct rcu_node *rnp,
unsigned long flags);
-static void rcu_stop_cpu_kthread(int cpu);
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
static void rcu_print_detail_task_stall(struct rcu_state *rsp);
static int rcu_print_task_stall(struct rcu_node *rnp);
@@ -491,15 +485,9 @@
static bool rcu_is_callbacks_kthread(void);
#ifdef CONFIG_RCU_BOOST
static void rcu_preempt_do_callbacks(void);
-static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp,
- cpumask_var_t cm);
static int __cpuinit rcu_spawn_one_boost_kthread(struct rcu_state *rsp,
- struct rcu_node *rnp,
- int rnp_index);
-static void invoke_rcu_node_kthread(struct rcu_node *rnp);
-static void rcu_yield(void (*f)(unsigned long), unsigned long arg);
+ struct rcu_node *rnp);
#endif /* #ifdef CONFIG_RCU_BOOST */
-static void rcu_cpu_kthread_setrt(int cpu, int to_rt);
static void __cpuinit rcu_prepare_kthreads(int cpu);
static void rcu_prepare_for_idle_init(int cpu);
static void rcu_cleanup_after_idle(int cpu);
diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index 7f3244c..f921154 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -25,6 +25,8 @@
*/
#include <linux/delay.h>
+#include <linux/oom.h>
+#include <linux/smpboot.h>
#define RCU_KTHREAD_PRIO 1
@@ -118,7 +120,7 @@
*/
void rcu_force_quiescent_state(void)
{
- force_quiescent_state(&rcu_preempt_state, 0);
+ force_quiescent_state(&rcu_preempt_state);
}
EXPORT_SYMBOL_GPL(rcu_force_quiescent_state);
@@ -136,8 +138,6 @@
{
struct rcu_data *rdp = &per_cpu(rcu_preempt_data, cpu);
- rdp->passed_quiesce_gpnum = rdp->gpnum;
- barrier();
if (rdp->passed_quiesce == 0)
trace_rcu_grace_period("rcu_preempt", rdp->gpnum, "cpuqs");
rdp->passed_quiesce = 1;
@@ -422,9 +422,11 @@
unsigned long flags;
struct task_struct *t;
- if (!rcu_preempt_blocked_readers_cgp(rnp))
- return;
raw_spin_lock_irqsave(&rnp->lock, flags);
+ if (!rcu_preempt_blocked_readers_cgp(rnp)) {
+ raw_spin_unlock_irqrestore(&rnp->lock, flags);
+ return;
+ }
t = list_entry(rnp->gp_tasks,
struct task_struct, rcu_node_entry);
list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry)
@@ -584,17 +586,23 @@
raw_spin_unlock(&rnp_root->lock); /* irqs still disabled */
}
+ rnp->gp_tasks = NULL;
+ rnp->exp_tasks = NULL;
#ifdef CONFIG_RCU_BOOST
- /* In case root is being boosted and leaf is not. */
+ rnp->boost_tasks = NULL;
+ /*
+ * In case root is being boosted and leaf was not. Make sure
+ * that we boost the tasks blocking the current grace period
+ * in this case.
+ */
raw_spin_lock(&rnp_root->lock); /* irqs already disabled */
if (rnp_root->boost_tasks != NULL &&
- rnp_root->boost_tasks != rnp_root->gp_tasks)
+ rnp_root->boost_tasks != rnp_root->gp_tasks &&
+ rnp_root->boost_tasks != rnp_root->exp_tasks)
rnp_root->boost_tasks = rnp_root->gp_tasks;
raw_spin_unlock(&rnp_root->lock); /* irqs still disabled */
#endif /* #ifdef CONFIG_RCU_BOOST */
- rnp->gp_tasks = NULL;
- rnp->exp_tasks = NULL;
return retval;
}
@@ -676,7 +684,7 @@
EXPORT_SYMBOL_GPL(synchronize_rcu);
static DECLARE_WAIT_QUEUE_HEAD(sync_rcu_preempt_exp_wq);
-static long sync_rcu_preempt_exp_count;
+static unsigned long sync_rcu_preempt_exp_count;
static DEFINE_MUTEX(sync_rcu_preempt_exp_mutex);
/*
@@ -791,7 +799,7 @@
unsigned long flags;
struct rcu_node *rnp;
struct rcu_state *rsp = &rcu_preempt_state;
- long snap;
+ unsigned long snap;
int trycount = 0;
smp_mb(); /* Caller's modifications seen first by other CPUs. */
@@ -799,33 +807,47 @@
smp_mb(); /* Above access cannot bleed into critical section. */
/*
+ * Block CPU-hotplug operations. This means that any CPU-hotplug
+ * operation that finds an rcu_node structure with tasks in the
+ * process of being boosted will know that all tasks blocking
+ * this expedited grace period will already be in the process of
+ * being boosted. This simplifies the process of moving tasks
+ * from leaf to root rcu_node structures.
+ */
+ get_online_cpus();
+
+ /*
* Acquire lock, falling back to synchronize_rcu() if too many
* lock-acquisition failures. Of course, if someone does the
* expedited grace period for us, just leave.
*/
while (!mutex_trylock(&sync_rcu_preempt_exp_mutex)) {
+ if (ULONG_CMP_LT(snap,
+ ACCESS_ONCE(sync_rcu_preempt_exp_count))) {
+ put_online_cpus();
+ goto mb_ret; /* Others did our work for us. */
+ }
if (trycount++ < 10) {
udelay(trycount * num_online_cpus());
} else {
+ put_online_cpus();
synchronize_rcu();
return;
}
- if ((ACCESS_ONCE(sync_rcu_preempt_exp_count) - snap) > 0)
- goto mb_ret; /* Others did our work for us. */
}
- if ((ACCESS_ONCE(sync_rcu_preempt_exp_count) - snap) > 0)
+ if (ULONG_CMP_LT(snap, ACCESS_ONCE(sync_rcu_preempt_exp_count))) {
+ put_online_cpus();
goto unlock_mb_ret; /* Others did our work for us. */
+ }
/* force all RCU readers onto ->blkd_tasks lists. */
synchronize_sched_expedited();
- raw_spin_lock_irqsave(&rsp->onofflock, flags);
-
/* Initialize ->expmask for all non-leaf rcu_node structures. */
rcu_for_each_nonleaf_node_breadth_first(rsp, rnp) {
- raw_spin_lock(&rnp->lock); /* irqs already disabled. */
+ raw_spin_lock_irqsave(&rnp->lock, flags);
rnp->expmask = rnp->qsmaskinit;
- raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
+ raw_spin_unlock_irqrestore(&rnp->lock, flags);
}
/* Snapshot current state of ->blkd_tasks lists. */
@@ -834,7 +856,7 @@
if (NUM_RCU_NODES > 1)
sync_rcu_preempt_exp_init(rsp, rcu_get_root(rsp));
- raw_spin_unlock_irqrestore(&rsp->onofflock, flags);
+ put_online_cpus();
/* Wait for snapshotted ->blkd_tasks lists to drain. */
rnp = rcu_get_root(rsp);
@@ -1069,6 +1091,16 @@
#endif /* #else #ifdef CONFIG_RCU_TRACE */
+static void rcu_wake_cond(struct task_struct *t, int status)
+{
+ /*
+ * If the thread is yielding, only wake it when this
+ * is invoked from idle
+ */
+ if (status != RCU_KTHREAD_YIELDING || is_idle_task(current))
+ wake_up_process(t);
+}
+
/*
* Carry out RCU priority boosting on the task indicated by ->exp_tasks
* or ->boost_tasks, advancing the pointer to the next task in the
@@ -1141,17 +1173,6 @@
}
/*
- * Timer handler to initiate waking up of boost kthreads that
- * have yielded the CPU due to excessive numbers of tasks to
- * boost. We wake up the per-rcu_node kthread, which in turn
- * will wake up the booster kthread.
- */
-static void rcu_boost_kthread_timer(unsigned long arg)
-{
- invoke_rcu_node_kthread((struct rcu_node *)arg);
-}
-
-/*
* Priority-boosting kthread. One per leaf rcu_node and one for the
* root rcu_node.
*/
@@ -1174,8 +1195,9 @@
else
spincnt = 0;
if (spincnt > 10) {
+ rnp->boost_kthread_status = RCU_KTHREAD_YIELDING;
trace_rcu_utilization("End boost kthread@rcu_yield");
- rcu_yield(rcu_boost_kthread_timer, (unsigned long)rnp);
+ schedule_timeout_interruptible(2);
trace_rcu_utilization("Start boost kthread@rcu_yield");
spincnt = 0;
}
@@ -1191,9 +1213,9 @@
* kthread to start boosting them. If there is an expedited grace
* period in progress, it is always time to boost.
*
- * The caller must hold rnp->lock, which this function releases,
- * but irqs remain disabled. The ->boost_kthread_task is immortal,
- * so we don't need to worry about it going away.
+ * The caller must hold rnp->lock, which this function releases.
+ * The ->boost_kthread_task is immortal, so we don't need to worry
+ * about it going away.
*/
static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
{
@@ -1213,8 +1235,8 @@
rnp->boost_tasks = rnp->gp_tasks;
raw_spin_unlock_irqrestore(&rnp->lock, flags);
t = rnp->boost_kthread_task;
- if (t != NULL)
- wake_up_process(t);
+ if (t)
+ rcu_wake_cond(t, rnp->boost_kthread_status);
} else {
rcu_initiate_boost_trace(rnp);
raw_spin_unlock_irqrestore(&rnp->lock, flags);
@@ -1231,8 +1253,10 @@
local_irq_save(flags);
__this_cpu_write(rcu_cpu_has_work, 1);
if (__this_cpu_read(rcu_cpu_kthread_task) != NULL &&
- current != __this_cpu_read(rcu_cpu_kthread_task))
- wake_up_process(__this_cpu_read(rcu_cpu_kthread_task));
+ current != __this_cpu_read(rcu_cpu_kthread_task)) {
+ rcu_wake_cond(__this_cpu_read(rcu_cpu_kthread_task),
+ __this_cpu_read(rcu_cpu_kthread_status));
+ }
local_irq_restore(flags);
}
@@ -1245,21 +1269,6 @@
return __get_cpu_var(rcu_cpu_kthread_task) == current;
}
-/*
- * Set the affinity of the boost kthread. The CPU-hotplug locks are
- * held, so no one should be messing with the existence of the boost
- * kthread.
- */
-static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp,
- cpumask_var_t cm)
-{
- struct task_struct *t;
-
- t = rnp->boost_kthread_task;
- if (t != NULL)
- set_cpus_allowed_ptr(rnp->boost_kthread_task, cm);
-}
-
#define RCU_BOOST_DELAY_JIFFIES DIV_ROUND_UP(CONFIG_RCU_BOOST_DELAY * HZ, 1000)
/*
@@ -1276,15 +1285,19 @@
* Returns zero if all is well, a negated errno otherwise.
*/
static int __cpuinit rcu_spawn_one_boost_kthread(struct rcu_state *rsp,
- struct rcu_node *rnp,
- int rnp_index)
+ struct rcu_node *rnp)
{
+ int rnp_index = rnp - &rsp->node[0];
unsigned long flags;
struct sched_param sp;
struct task_struct *t;
if (&rcu_preempt_state != rsp)
return 0;
+
+ if (!rcu_scheduler_fully_active || rnp->qsmaskinit == 0)
+ return 0;
+
rsp->boost = 1;
if (rnp->boost_kthread_task != NULL)
return 0;
@@ -1301,25 +1314,6 @@
return 0;
}
-#ifdef CONFIG_HOTPLUG_CPU
-
-/*
- * Stop the RCU's per-CPU kthread when its CPU goes offline,.
- */
-static void rcu_stop_cpu_kthread(int cpu)
-{
- struct task_struct *t;
-
- /* Stop the CPU's kthread. */
- t = per_cpu(rcu_cpu_kthread_task, cpu);
- if (t != NULL) {
- per_cpu(rcu_cpu_kthread_task, cpu) = NULL;
- kthread_stop(t);
- }
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-
static void rcu_kthread_do_work(void)
{
rcu_do_batch(&rcu_sched_state, &__get_cpu_var(rcu_sched_data));
@@ -1327,112 +1321,22 @@
rcu_preempt_do_callbacks();
}
-/*
- * Wake up the specified per-rcu_node-structure kthread.
- * Because the per-rcu_node kthreads are immortal, we don't need
- * to do anything to keep them alive.
- */
-static void invoke_rcu_node_kthread(struct rcu_node *rnp)
-{
- struct task_struct *t;
-
- t = rnp->node_kthread_task;
- if (t != NULL)
- wake_up_process(t);
-}
-
-/*
- * Set the specified CPU's kthread to run RT or not, as specified by
- * the to_rt argument. The CPU-hotplug locks are held, so the task
- * is not going away.
- */
-static void rcu_cpu_kthread_setrt(int cpu, int to_rt)
-{
- int policy;
- struct sched_param sp;
- struct task_struct *t;
-
- t = per_cpu(rcu_cpu_kthread_task, cpu);
- if (t == NULL)
- return;
- if (to_rt) {
- policy = SCHED_FIFO;
- sp.sched_priority = RCU_KTHREAD_PRIO;
- } else {
- policy = SCHED_NORMAL;
- sp.sched_priority = 0;
- }
- sched_setscheduler_nocheck(t, policy, &sp);
-}
-
-/*
- * Timer handler to initiate the waking up of per-CPU kthreads that
- * have yielded the CPU due to excess numbers of RCU callbacks.
- * We wake up the per-rcu_node kthread, which in turn will wake up
- * the booster kthread.
- */
-static void rcu_cpu_kthread_timer(unsigned long arg)
-{
- struct rcu_data *rdp = per_cpu_ptr(rcu_state->rda, arg);
- struct rcu_node *rnp = rdp->mynode;
-
- atomic_or(rdp->grpmask, &rnp->wakemask);
- invoke_rcu_node_kthread(rnp);
-}
-
-/*
- * Drop to non-real-time priority and yield, but only after posting a
- * timer that will cause us to regain our real-time priority if we
- * remain preempted. Either way, we restore our real-time priority
- * before returning.
- */
-static void rcu_yield(void (*f)(unsigned long), unsigned long arg)
+static void rcu_cpu_kthread_setup(unsigned int cpu)
{
struct sched_param sp;
- struct timer_list yield_timer;
- int prio = current->rt_priority;
- setup_timer_on_stack(&yield_timer, f, arg);
- mod_timer(&yield_timer, jiffies + 2);
- sp.sched_priority = 0;
- sched_setscheduler_nocheck(current, SCHED_NORMAL, &sp);
- set_user_nice(current, 19);
- schedule();
- set_user_nice(current, 0);
- sp.sched_priority = prio;
+ sp.sched_priority = RCU_KTHREAD_PRIO;
sched_setscheduler_nocheck(current, SCHED_FIFO, &sp);
- del_timer(&yield_timer);
}
-/*
- * Handle cases where the rcu_cpu_kthread() ends up on the wrong CPU.
- * This can happen while the corresponding CPU is either coming online
- * or going offline. We cannot wait until the CPU is fully online
- * before starting the kthread, because the various notifier functions
- * can wait for RCU grace periods. So we park rcu_cpu_kthread() until
- * the corresponding CPU is online.
- *
- * Return 1 if the kthread needs to stop, 0 otherwise.
- *
- * Caller must disable bh. This function can momentarily enable it.
- */
-static int rcu_cpu_kthread_should_stop(int cpu)
+static void rcu_cpu_kthread_park(unsigned int cpu)
{
- while (cpu_is_offline(cpu) ||
- !cpumask_equal(¤t->cpus_allowed, cpumask_of(cpu)) ||
- smp_processor_id() != cpu) {
- if (kthread_should_stop())
- return 1;
- per_cpu(rcu_cpu_kthread_status, cpu) = RCU_KTHREAD_OFFCPU;
- per_cpu(rcu_cpu_kthread_cpu, cpu) = raw_smp_processor_id();
- local_bh_enable();
- schedule_timeout_uninterruptible(1);
- if (!cpumask_equal(¤t->cpus_allowed, cpumask_of(cpu)))
- set_cpus_allowed_ptr(current, cpumask_of(cpu));
- local_bh_disable();
- }
- per_cpu(rcu_cpu_kthread_cpu, cpu) = cpu;
- return 0;
+ per_cpu(rcu_cpu_kthread_status, cpu) = RCU_KTHREAD_OFFCPU;
+}
+
+static int rcu_cpu_kthread_should_run(unsigned int cpu)
+{
+ return __get_cpu_var(rcu_cpu_has_work);
}
/*
@@ -1440,138 +1344,35 @@
* RCU softirq used in flavors and configurations of RCU that do not
* support RCU priority boosting.
*/
-static int rcu_cpu_kthread(void *arg)
+static void rcu_cpu_kthread(unsigned int cpu)
{
- int cpu = (int)(long)arg;
- unsigned long flags;
- int spincnt = 0;
- unsigned int *statusp = &per_cpu(rcu_cpu_kthread_status, cpu);
- char work;
- char *workp = &per_cpu(rcu_cpu_has_work, cpu);
+ unsigned int *statusp = &__get_cpu_var(rcu_cpu_kthread_status);
+ char work, *workp = &__get_cpu_var(rcu_cpu_has_work);
+ int spincnt;
- trace_rcu_utilization("Start CPU kthread@init");
- for (;;) {
- *statusp = RCU_KTHREAD_WAITING;
- trace_rcu_utilization("End CPU kthread@rcu_wait");
- rcu_wait(*workp != 0 || kthread_should_stop());
+ for (spincnt = 0; spincnt < 10; spincnt++) {
trace_rcu_utilization("Start CPU kthread@rcu_wait");
local_bh_disable();
- if (rcu_cpu_kthread_should_stop(cpu)) {
- local_bh_enable();
- break;
- }
*statusp = RCU_KTHREAD_RUNNING;
- per_cpu(rcu_cpu_kthread_loops, cpu)++;
- local_irq_save(flags);
+ this_cpu_inc(rcu_cpu_kthread_loops);
+ local_irq_disable();
work = *workp;
*workp = 0;
- local_irq_restore(flags);
+ local_irq_enable();
if (work)
rcu_kthread_do_work();
local_bh_enable();
- if (*workp != 0)
- spincnt++;
- else
- spincnt = 0;
- if (spincnt > 10) {
- *statusp = RCU_KTHREAD_YIELDING;
- trace_rcu_utilization("End CPU kthread@rcu_yield");
- rcu_yield(rcu_cpu_kthread_timer, (unsigned long)cpu);
- trace_rcu_utilization("Start CPU kthread@rcu_yield");
- spincnt = 0;
+ if (*workp == 0) {
+ trace_rcu_utilization("End CPU kthread@rcu_wait");
+ *statusp = RCU_KTHREAD_WAITING;
+ return;
}
}
- *statusp = RCU_KTHREAD_STOPPED;
- trace_rcu_utilization("End CPU kthread@term");
- return 0;
-}
-
-/*
- * Spawn a per-CPU kthread, setting up affinity and priority.
- * Because the CPU hotplug lock is held, no other CPU will be attempting
- * to manipulate rcu_cpu_kthread_task. There might be another CPU
- * attempting to access it during boot, but the locking in kthread_bind()
- * will enforce sufficient ordering.
- *
- * Please note that we cannot simply refuse to wake up the per-CPU
- * kthread because kthreads are created in TASK_UNINTERRUPTIBLE state,
- * which can result in softlockup complaints if the task ends up being
- * idle for more than a couple of minutes.
- *
- * However, please note also that we cannot bind the per-CPU kthread to its
- * CPU until that CPU is fully online. We also cannot wait until the
- * CPU is fully online before we create its per-CPU kthread, as this would
- * deadlock the system when CPU notifiers tried waiting for grace
- * periods. So we bind the per-CPU kthread to its CPU only if the CPU
- * is online. If its CPU is not yet fully online, then the code in
- * rcu_cpu_kthread() will wait until it is fully online, and then do
- * the binding.
- */
-static int __cpuinit rcu_spawn_one_cpu_kthread(int cpu)
-{
- struct sched_param sp;
- struct task_struct *t;
-
- if (!rcu_scheduler_fully_active ||
- per_cpu(rcu_cpu_kthread_task, cpu) != NULL)
- return 0;
- t = kthread_create_on_node(rcu_cpu_kthread,
- (void *)(long)cpu,
- cpu_to_node(cpu),
- "rcuc/%d", cpu);
- if (IS_ERR(t))
- return PTR_ERR(t);
- if (cpu_online(cpu))
- kthread_bind(t, cpu);
- per_cpu(rcu_cpu_kthread_cpu, cpu) = cpu;
- WARN_ON_ONCE(per_cpu(rcu_cpu_kthread_task, cpu) != NULL);
- sp.sched_priority = RCU_KTHREAD_PRIO;
- sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
- per_cpu(rcu_cpu_kthread_task, cpu) = t;
- wake_up_process(t); /* Get to TASK_INTERRUPTIBLE quickly. */
- return 0;
-}
-
-/*
- * Per-rcu_node kthread, which is in charge of waking up the per-CPU
- * kthreads when needed. We ignore requests to wake up kthreads
- * for offline CPUs, which is OK because force_quiescent_state()
- * takes care of this case.
- */
-static int rcu_node_kthread(void *arg)
-{
- int cpu;
- unsigned long flags;
- unsigned long mask;
- struct rcu_node *rnp = (struct rcu_node *)arg;
- struct sched_param sp;
- struct task_struct *t;
-
- for (;;) {
- rnp->node_kthread_status = RCU_KTHREAD_WAITING;
- rcu_wait(atomic_read(&rnp->wakemask) != 0);
- rnp->node_kthread_status = RCU_KTHREAD_RUNNING;
- raw_spin_lock_irqsave(&rnp->lock, flags);
- mask = atomic_xchg(&rnp->wakemask, 0);
- rcu_initiate_boost(rnp, flags); /* releases rnp->lock. */
- for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask >>= 1) {
- if ((mask & 0x1) == 0)
- continue;
- preempt_disable();
- t = per_cpu(rcu_cpu_kthread_task, cpu);
- if (!cpu_online(cpu) || t == NULL) {
- preempt_enable();
- continue;
- }
- per_cpu(rcu_cpu_has_work, cpu) = 1;
- sp.sched_priority = RCU_KTHREAD_PRIO;
- sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
- preempt_enable();
- }
- }
- /* NOTREACHED */
- rnp->node_kthread_status = RCU_KTHREAD_STOPPED;
- return 0;
+ *statusp = RCU_KTHREAD_YIELDING;
+ trace_rcu_utilization("Start CPU kthread@rcu_yield");
+ schedule_timeout_interruptible(2);
+ trace_rcu_utilization("End CPU kthread@rcu_yield");
+ *statusp = RCU_KTHREAD_WAITING;
}
/*
@@ -1583,17 +1384,17 @@
* no outgoing CPU. If there are no CPUs left in the affinity set,
* this function allows the kthread to execute on any CPU.
*/
-static void rcu_node_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
+static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
{
+ struct task_struct *t = rnp->boost_kthread_task;
+ unsigned long mask = rnp->qsmaskinit;
cpumask_var_t cm;
int cpu;
- unsigned long mask = rnp->qsmaskinit;
- if (rnp->node_kthread_task == NULL)
+ if (!t)
return;
- if (!alloc_cpumask_var(&cm, GFP_KERNEL))
+ if (!zalloc_cpumask_var(&cm, GFP_KERNEL))
return;
- cpumask_clear(cm);
for (cpu = rnp->grplo; cpu <= rnp->grphi; cpu++, mask >>= 1)
if ((mask & 0x1) && cpu != outgoingcpu)
cpumask_set_cpu(cpu, cm);
@@ -1603,62 +1404,36 @@
cpumask_clear_cpu(cpu, cm);
WARN_ON_ONCE(cpumask_weight(cm) == 0);
}
- set_cpus_allowed_ptr(rnp->node_kthread_task, cm);
- rcu_boost_kthread_setaffinity(rnp, cm);
+ set_cpus_allowed_ptr(t, cm);
free_cpumask_var(cm);
}
-/*
- * Spawn a per-rcu_node kthread, setting priority and affinity.
- * Called during boot before online/offline can happen, or, if
- * during runtime, with the main CPU-hotplug locks held. So only
- * one of these can be executing at a time.
- */
-static int __cpuinit rcu_spawn_one_node_kthread(struct rcu_state *rsp,
- struct rcu_node *rnp)
-{
- unsigned long flags;
- int rnp_index = rnp - &rsp->node[0];
- struct sched_param sp;
- struct task_struct *t;
-
- if (!rcu_scheduler_fully_active ||
- rnp->qsmaskinit == 0)
- return 0;
- if (rnp->node_kthread_task == NULL) {
- t = kthread_create(rcu_node_kthread, (void *)rnp,
- "rcun/%d", rnp_index);
- if (IS_ERR(t))
- return PTR_ERR(t);
- raw_spin_lock_irqsave(&rnp->lock, flags);
- rnp->node_kthread_task = t;
- raw_spin_unlock_irqrestore(&rnp->lock, flags);
- sp.sched_priority = 99;
- sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
- wake_up_process(t); /* get to TASK_INTERRUPTIBLE quickly. */
- }
- return rcu_spawn_one_boost_kthread(rsp, rnp, rnp_index);
-}
+static struct smp_hotplug_thread rcu_cpu_thread_spec = {
+ .store = &rcu_cpu_kthread_task,
+ .thread_should_run = rcu_cpu_kthread_should_run,
+ .thread_fn = rcu_cpu_kthread,
+ .thread_comm = "rcuc/%u",
+ .setup = rcu_cpu_kthread_setup,
+ .park = rcu_cpu_kthread_park,
+};
/*
* Spawn all kthreads -- called as soon as the scheduler is running.
*/
static int __init rcu_spawn_kthreads(void)
{
- int cpu;
struct rcu_node *rnp;
+ int cpu;
rcu_scheduler_fully_active = 1;
- for_each_possible_cpu(cpu) {
+ for_each_possible_cpu(cpu)
per_cpu(rcu_cpu_has_work, cpu) = 0;
- if (cpu_online(cpu))
- (void)rcu_spawn_one_cpu_kthread(cpu);
- }
+ BUG_ON(smpboot_register_percpu_thread(&rcu_cpu_thread_spec));
rnp = rcu_get_root(rcu_state);
- (void)rcu_spawn_one_node_kthread(rcu_state, rnp);
+ (void)rcu_spawn_one_boost_kthread(rcu_state, rnp);
if (NUM_RCU_NODES > 1) {
rcu_for_each_leaf_node(rcu_state, rnp)
- (void)rcu_spawn_one_node_kthread(rcu_state, rnp);
+ (void)rcu_spawn_one_boost_kthread(rcu_state, rnp);
}
return 0;
}
@@ -1670,11 +1445,8 @@
struct rcu_node *rnp = rdp->mynode;
/* Fire up the incoming CPU's kthread and leaf rcu_node kthread. */
- if (rcu_scheduler_fully_active) {
- (void)rcu_spawn_one_cpu_kthread(cpu);
- if (rnp->node_kthread_task == NULL)
- (void)rcu_spawn_one_node_kthread(rcu_state, rnp);
- }
+ if (rcu_scheduler_fully_active)
+ (void)rcu_spawn_one_boost_kthread(rcu_state, rnp);
}
#else /* #ifdef CONFIG_RCU_BOOST */
@@ -1698,19 +1470,7 @@
{
}
-#ifdef CONFIG_HOTPLUG_CPU
-
-static void rcu_stop_cpu_kthread(int cpu)
-{
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-
-static void rcu_node_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
-{
-}
-
-static void rcu_cpu_kthread_setrt(int cpu, int to_rt)
+static void rcu_boost_kthread_setaffinity(struct rcu_node *rnp, int outgoingcpu)
{
}
@@ -1997,6 +1757,26 @@
if (!tne)
return;
+ /* Adaptive-tick mode, where usermode execution is idle to RCU. */
+ if (!is_idle_task(current)) {
+ rdtp->dyntick_holdoff = jiffies - 1;
+ if (rcu_cpu_has_nonlazy_callbacks(cpu)) {
+ trace_rcu_prep_idle("User dyntick with callbacks");
+ rdtp->idle_gp_timer_expires =
+ round_up(jiffies + RCU_IDLE_GP_DELAY,
+ RCU_IDLE_GP_DELAY);
+ } else if (rcu_cpu_has_callbacks(cpu)) {
+ rdtp->idle_gp_timer_expires =
+ round_jiffies(jiffies + RCU_IDLE_LAZY_GP_DELAY);
+ trace_rcu_prep_idle("User dyntick with lazy callbacks");
+ } else {
+ return;
+ }
+ tp = &rdtp->idle_gp_timer;
+ mod_timer_pinned(tp, rdtp->idle_gp_timer_expires);
+ return;
+ }
+
/*
* If this is an idle re-entry, for example, due to use of
* RCU_NONIDLE() or the new idle-loop tracing API within the idle
@@ -2075,16 +1855,16 @@
#ifdef CONFIG_TREE_PREEMPT_RCU
if (per_cpu(rcu_preempt_data, cpu).nxtlist) {
rcu_preempt_qs(cpu);
- force_quiescent_state(&rcu_preempt_state, 0);
+ force_quiescent_state(&rcu_preempt_state);
}
#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */
if (per_cpu(rcu_sched_data, cpu).nxtlist) {
rcu_sched_qs(cpu);
- force_quiescent_state(&rcu_sched_state, 0);
+ force_quiescent_state(&rcu_sched_state);
}
if (per_cpu(rcu_bh_data, cpu).nxtlist) {
rcu_bh_qs(cpu);
- force_quiescent_state(&rcu_bh_state, 0);
+ force_quiescent_state(&rcu_bh_state);
}
/*
@@ -2112,6 +1892,88 @@
__this_cpu_add(rcu_dynticks.nonlazy_posted, 1);
}
+/*
+ * Data for flushing lazy RCU callbacks at OOM time.
+ */
+static atomic_t oom_callback_count;
+static DECLARE_WAIT_QUEUE_HEAD(oom_callback_wq);
+
+/*
+ * RCU OOM callback -- decrement the outstanding count and deliver the
+ * wake-up if we are the last one.
+ */
+static void rcu_oom_callback(struct rcu_head *rhp)
+{
+ if (atomic_dec_and_test(&oom_callback_count))
+ wake_up(&oom_callback_wq);
+}
+
+/*
+ * Post an rcu_oom_notify callback on the current CPU if it has at
+ * least one lazy callback. This will unnecessarily post callbacks
+ * to CPUs that already have a non-lazy callback at the end of their
+ * callback list, but this is an infrequent operation, so accept some
+ * extra overhead to keep things simple.
+ */
+static void rcu_oom_notify_cpu(void *unused)
+{
+ struct rcu_state *rsp;
+ struct rcu_data *rdp;
+
+ for_each_rcu_flavor(rsp) {
+ rdp = __this_cpu_ptr(rsp->rda);
+ if (rdp->qlen_lazy != 0) {
+ atomic_inc(&oom_callback_count);
+ rsp->call(&rdp->oom_head, rcu_oom_callback);
+ }
+ }
+}
+
+/*
+ * If low on memory, ensure that each CPU has a non-lazy callback.
+ * This will wake up CPUs that have only lazy callbacks, in turn
+ * ensuring that they free up the corresponding memory in a timely manner.
+ * Because an uncertain amount of memory will be freed in some uncertain
+ * timeframe, we do not claim to have freed anything.
+ */
+static int rcu_oom_notify(struct notifier_block *self,
+ unsigned long notused, void *nfreed)
+{
+ int cpu;
+
+ /* Wait for callbacks from earlier instance to complete. */
+ wait_event(oom_callback_wq, atomic_read(&oom_callback_count) == 0);
+
+ /*
+ * Prevent premature wakeup: ensure that all increments happen
+ * before there is a chance of the counter reaching zero.
+ */
+ atomic_set(&oom_callback_count, 1);
+
+ get_online_cpus();
+ for_each_online_cpu(cpu) {
+ smp_call_function_single(cpu, rcu_oom_notify_cpu, NULL, 1);
+ cond_resched();
+ }
+ put_online_cpus();
+
+ /* Unconditionally decrement: no need to wake ourselves up. */
+ atomic_dec(&oom_callback_count);
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block rcu_oom_nb = {
+ .notifier_call = rcu_oom_notify
+};
+
+static int __init rcu_register_oom_notifier(void)
+{
+ register_oom_notifier(&rcu_oom_nb);
+ return 0;
+}
+early_initcall(rcu_register_oom_notifier);
+
#endif /* #else #if !defined(CONFIG_RCU_FAST_NO_HZ) */
#ifdef CONFIG_RCU_CPU_STALL_INFO
@@ -2122,11 +1984,15 @@
{
struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
struct timer_list *tltp = &rdtp->idle_gp_timer;
+ char c;
- sprintf(cp, "drain=%d %c timer=%lu",
- rdtp->dyntick_drain,
- rdtp->dyntick_holdoff == jiffies ? 'H' : '.',
- timer_pending(tltp) ? tltp->expires - jiffies : -1);
+ c = rdtp->dyntick_holdoff == jiffies ? 'H' : '.';
+ if (timer_pending(tltp))
+ sprintf(cp, "drain=%d %c timer=%lu",
+ rdtp->dyntick_drain, c, tltp->expires - jiffies);
+ else
+ sprintf(cp, "drain=%d %c timer not pending",
+ rdtp->dyntick_drain, c);
}
#else /* #ifdef CONFIG_RCU_FAST_NO_HZ */
@@ -2194,11 +2060,10 @@
/* Increment ->ticks_this_gp for all flavors of RCU. */
static void increment_cpu_stall_ticks(void)
{
- __get_cpu_var(rcu_sched_data).ticks_this_gp++;
- __get_cpu_var(rcu_bh_data).ticks_this_gp++;
-#ifdef CONFIG_TREE_PREEMPT_RCU
- __get_cpu_var(rcu_preempt_data).ticks_this_gp++;
-#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */
+ struct rcu_state *rsp;
+
+ for_each_rcu_flavor(rsp)
+ __this_cpu_ptr(rsp->rda)->ticks_this_gp++;
}
#else /* #ifdef CONFIG_RCU_CPU_STALL_INFO */
diff --git a/kernel/rcutree_trace.c b/kernel/rcutree_trace.c
index abffb48..693513b 100644
--- a/kernel/rcutree_trace.c
+++ b/kernel/rcutree_trace.c
@@ -51,8 +51,8 @@
struct rcu_state *rsp;
for_each_rcu_flavor(rsp)
- seq_printf(m, "%s: %c bcc: %d nbd: %lu\n",
- rsp->name, rsp->rcu_barrier_in_progress ? 'B' : '.',
+ seq_printf(m, "%s: bcc: %d nbd: %lu\n",
+ rsp->name,
atomic_read(&rsp->barrier_cpu_count),
rsp->n_barrier_done);
return 0;
@@ -86,12 +86,11 @@
{
if (!rdp->beenonline)
return;
- seq_printf(m, "%3d%cc=%lu g=%lu pq=%d pgp=%lu qp=%d",
+ seq_printf(m, "%3d%cc=%lu g=%lu pq=%d qp=%d",
rdp->cpu,
cpu_is_offline(rdp->cpu) ? '!' : ' ',
rdp->completed, rdp->gpnum,
- rdp->passed_quiesce, rdp->passed_quiesce_gpnum,
- rdp->qs_pending);
+ rdp->passed_quiesce, rdp->qs_pending);
seq_printf(m, " dt=%d/%llx/%d df=%lu",
atomic_read(&rdp->dynticks->dynticks),
rdp->dynticks->dynticks_nesting,
@@ -108,11 +107,10 @@
rdp->nxttail[RCU_WAIT_TAIL]],
".D"[&rdp->nxtlist != rdp->nxttail[RCU_DONE_TAIL]]);
#ifdef CONFIG_RCU_BOOST
- seq_printf(m, " kt=%d/%c/%d ktl=%x",
+ seq_printf(m, " kt=%d/%c ktl=%x",
per_cpu(rcu_cpu_has_work, rdp->cpu),
convert_kthread_status(per_cpu(rcu_cpu_kthread_status,
rdp->cpu)),
- per_cpu(rcu_cpu_kthread_cpu, rdp->cpu),
per_cpu(rcu_cpu_kthread_loops, rdp->cpu) & 0xffff);
#endif /* #ifdef CONFIG_RCU_BOOST */
seq_printf(m, " b=%ld", rdp->blimit);
@@ -150,12 +148,11 @@
{
if (!rdp->beenonline)
return;
- seq_printf(m, "%d,%s,%lu,%lu,%d,%lu,%d",
+ seq_printf(m, "%d,%s,%lu,%lu,%d,%d",
rdp->cpu,
cpu_is_offline(rdp->cpu) ? "\"N\"" : "\"Y\"",
rdp->completed, rdp->gpnum,
- rdp->passed_quiesce, rdp->passed_quiesce_gpnum,
- rdp->qs_pending);
+ rdp->passed_quiesce, rdp->qs_pending);
seq_printf(m, ",%d,%llx,%d,%lu",
atomic_read(&rdp->dynticks->dynticks),
rdp->dynticks->dynticks_nesting,
@@ -186,7 +183,7 @@
int cpu;
struct rcu_state *rsp;
- seq_puts(m, "\"CPU\",\"Online?\",\"c\",\"g\",\"pq\",\"pgp\",\"pq\",");
+ seq_puts(m, "\"CPU\",\"Online?\",\"c\",\"g\",\"pq\",\"pq\",");
seq_puts(m, "\"dt\",\"dt nesting\",\"dt NMI nesting\",\"df\",");
seq_puts(m, "\"of\",\"qll\",\"ql\",\"qs\"");
#ifdef CONFIG_RCU_BOOST
@@ -386,10 +383,9 @@
rdp->n_rp_report_qs,
rdp->n_rp_cb_ready,
rdp->n_rp_cpu_needs_gp);
- seq_printf(m, "gpc=%ld gps=%ld nf=%ld nn=%ld\n",
+ seq_printf(m, "gpc=%ld gps=%ld nn=%ld\n",
rdp->n_rp_gp_completed,
rdp->n_rp_gp_started,
- rdp->n_rp_need_fqs,
rdp->n_rp_need_nothing);
}
diff --git a/kernel/sched/Makefile b/kernel/sched/Makefile
index 173ea52..f06d249 100644
--- a/kernel/sched/Makefile
+++ b/kernel/sched/Makefile
@@ -11,7 +11,7 @@
CFLAGS_core.o := $(PROFILING) -fno-omit-frame-pointer
endif
-obj-y += core.o clock.o idle_task.o fair.o rt.o stop_task.o
+obj-y += core.o clock.o cputime.o idle_task.o fair.o rt.o stop_task.o
obj-$(CONFIG_SMP) += cpupri.o
obj-$(CONFIG_SCHED_AUTOGROUP) += auto_group.o
obj-$(CONFIG_SCHEDSTATS) += stats.o
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index fbf1fd0..c177472 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -740,126 +740,6 @@
dequeue_task(rq, p, flags);
}
-#ifdef CONFIG_IRQ_TIME_ACCOUNTING
-
-/*
- * There are no locks covering percpu hardirq/softirq time.
- * They are only modified in account_system_vtime, on corresponding CPU
- * with interrupts disabled. So, writes are safe.
- * They are read and saved off onto struct rq in update_rq_clock().
- * This may result in other CPU reading this CPU's irq time and can
- * race with irq/account_system_vtime on this CPU. We would either get old
- * or new value with a side effect of accounting a slice of irq time to wrong
- * task when irq is in progress while we read rq->clock. That is a worthy
- * compromise in place of having locks on each irq in account_system_time.
- */
-static DEFINE_PER_CPU(u64, cpu_hardirq_time);
-static DEFINE_PER_CPU(u64, cpu_softirq_time);
-
-static DEFINE_PER_CPU(u64, irq_start_time);
-static int sched_clock_irqtime;
-
-void enable_sched_clock_irqtime(void)
-{
- sched_clock_irqtime = 1;
-}
-
-void disable_sched_clock_irqtime(void)
-{
- sched_clock_irqtime = 0;
-}
-
-#ifndef CONFIG_64BIT
-static DEFINE_PER_CPU(seqcount_t, irq_time_seq);
-
-static inline void irq_time_write_begin(void)
-{
- __this_cpu_inc(irq_time_seq.sequence);
- smp_wmb();
-}
-
-static inline void irq_time_write_end(void)
-{
- smp_wmb();
- __this_cpu_inc(irq_time_seq.sequence);
-}
-
-static inline u64 irq_time_read(int cpu)
-{
- u64 irq_time;
- unsigned seq;
-
- do {
- seq = read_seqcount_begin(&per_cpu(irq_time_seq, cpu));
- irq_time = per_cpu(cpu_softirq_time, cpu) +
- per_cpu(cpu_hardirq_time, cpu);
- } while (read_seqcount_retry(&per_cpu(irq_time_seq, cpu), seq));
-
- return irq_time;
-}
-#else /* CONFIG_64BIT */
-static inline void irq_time_write_begin(void)
-{
-}
-
-static inline void irq_time_write_end(void)
-{
-}
-
-static inline u64 irq_time_read(int cpu)
-{
- return per_cpu(cpu_softirq_time, cpu) + per_cpu(cpu_hardirq_time, cpu);
-}
-#endif /* CONFIG_64BIT */
-
-/*
- * Called before incrementing preempt_count on {soft,}irq_enter
- * and before decrementing preempt_count on {soft,}irq_exit.
- */
-void account_system_vtime(struct task_struct *curr)
-{
- unsigned long flags;
- s64 delta;
- int cpu;
-
- if (!sched_clock_irqtime)
- return;
-
- local_irq_save(flags);
-
- cpu = smp_processor_id();
- delta = sched_clock_cpu(cpu) - __this_cpu_read(irq_start_time);
- __this_cpu_add(irq_start_time, delta);
-
- irq_time_write_begin();
- /*
- * We do not account for softirq time from ksoftirqd here.
- * We want to continue accounting softirq time to ksoftirqd thread
- * in that case, so as not to confuse scheduler with a special task
- * that do not consume any time, but still wants to run.
- */
- if (hardirq_count())
- __this_cpu_add(cpu_hardirq_time, delta);
- else if (in_serving_softirq() && curr != this_cpu_ksoftirqd())
- __this_cpu_add(cpu_softirq_time, delta);
-
- irq_time_write_end();
- local_irq_restore(flags);
-}
-EXPORT_SYMBOL_GPL(account_system_vtime);
-
-#endif /* CONFIG_IRQ_TIME_ACCOUNTING */
-
-#ifdef CONFIG_PARAVIRT
-static inline u64 steal_ticks(u64 steal)
-{
- if (unlikely(steal > NSEC_PER_SEC))
- return div_u64(steal, TICK_NSEC);
-
- return __iter_div_u64_rem(steal, TICK_NSEC, &steal);
-}
-#endif
-
static void update_rq_clock_task(struct rq *rq, s64 delta)
{
/*
@@ -920,43 +800,6 @@
#endif
}
-#ifdef CONFIG_IRQ_TIME_ACCOUNTING
-static int irqtime_account_hi_update(void)
-{
- u64 *cpustat = kcpustat_this_cpu->cpustat;
- unsigned long flags;
- u64 latest_ns;
- int ret = 0;
-
- local_irq_save(flags);
- latest_ns = this_cpu_read(cpu_hardirq_time);
- if (nsecs_to_cputime64(latest_ns) > cpustat[CPUTIME_IRQ])
- ret = 1;
- local_irq_restore(flags);
- return ret;
-}
-
-static int irqtime_account_si_update(void)
-{
- u64 *cpustat = kcpustat_this_cpu->cpustat;
- unsigned long flags;
- u64 latest_ns;
- int ret = 0;
-
- local_irq_save(flags);
- latest_ns = this_cpu_read(cpu_softirq_time);
- if (nsecs_to_cputime64(latest_ns) > cpustat[CPUTIME_SOFTIRQ])
- ret = 1;
- local_irq_restore(flags);
- return ret;
-}
-
-#else /* CONFIG_IRQ_TIME_ACCOUNTING */
-
-#define sched_clock_irqtime (0)
-
-#endif
-
void sched_set_stop_task(int cpu, struct task_struct *stop)
{
struct sched_param param = { .sched_priority = MAX_RT_PRIO - 1 };
@@ -1518,25 +1361,6 @@
smp_send_reschedule(cpu);
}
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
-static int ttwu_activate_remote(struct task_struct *p, int wake_flags)
-{
- struct rq *rq;
- int ret = 0;
-
- rq = __task_rq_lock(p);
- if (p->on_cpu) {
- ttwu_activate(rq, p, ENQUEUE_WAKEUP);
- ttwu_do_wakeup(rq, p, wake_flags);
- ret = 1;
- }
- __task_rq_unlock(rq);
-
- return ret;
-
-}
-#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
-
bool cpus_share_cache(int this_cpu, int that_cpu)
{
return per_cpu(sd_llc_id, this_cpu) == per_cpu(sd_llc_id, that_cpu);
@@ -1597,21 +1421,8 @@
* If the owning (remote) cpu is still in the middle of schedule() with
* this task as prev, wait until its done referencing the task.
*/
- while (p->on_cpu) {
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
- /*
- * In case the architecture enables interrupts in
- * context_switch(), we cannot busy wait, since that
- * would lead to deadlocks when an interrupt hits and
- * tries to wake up @prev. So bail and do a complete
- * remote wakeup.
- */
- if (ttwu_activate_remote(p, wake_flags))
- goto stat;
-#else
+ while (p->on_cpu)
cpu_relax();
-#endif
- }
/*
* Pairs with the smp_wmb() in finish_lock_switch().
*/
@@ -1953,14 +1764,9 @@
* Manfred Spraul <manfred@colorfullife.com>
*/
prev_state = prev->state;
+ vtime_task_switch(prev);
finish_arch_switch(prev);
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
- local_irq_disable();
-#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
perf_event_task_sched_in(prev, current);
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
- local_irq_enable();
-#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
finish_lock_switch(rq, prev);
finish_arch_post_lock_switch();
@@ -2081,6 +1887,7 @@
#endif
/* Here we just switch the register state and the stack. */
+ rcu_switch(prev, next);
switch_to(prev, next, prev);
barrier();
@@ -2809,404 +2616,6 @@
return ns;
}
-#ifdef CONFIG_CGROUP_CPUACCT
-struct cgroup_subsys cpuacct_subsys;
-struct cpuacct root_cpuacct;
-#endif
-
-static inline void task_group_account_field(struct task_struct *p, int index,
- u64 tmp)
-{
-#ifdef CONFIG_CGROUP_CPUACCT
- struct kernel_cpustat *kcpustat;
- struct cpuacct *ca;
-#endif
- /*
- * Since all updates are sure to touch the root cgroup, we
- * get ourselves ahead and touch it first. If the root cgroup
- * is the only cgroup, then nothing else should be necessary.
- *
- */
- __get_cpu_var(kernel_cpustat).cpustat[index] += tmp;
-
-#ifdef CONFIG_CGROUP_CPUACCT
- if (unlikely(!cpuacct_subsys.active))
- return;
-
- rcu_read_lock();
- ca = task_ca(p);
- while (ca && (ca != &root_cpuacct)) {
- kcpustat = this_cpu_ptr(ca->cpustat);
- kcpustat->cpustat[index] += tmp;
- ca = parent_ca(ca);
- }
- rcu_read_unlock();
-#endif
-}
-
-
-/*
- * Account user cpu time to a process.
- * @p: the process that the cpu time gets accounted to
- * @cputime: the cpu time spent in user space since the last update
- * @cputime_scaled: cputime scaled by cpu frequency
- */
-void account_user_time(struct task_struct *p, cputime_t cputime,
- cputime_t cputime_scaled)
-{
- int index;
-
- /* Add user time to process. */
- p->utime += cputime;
- p->utimescaled += cputime_scaled;
- account_group_user_time(p, cputime);
-
- index = (TASK_NICE(p) > 0) ? CPUTIME_NICE : CPUTIME_USER;
-
- /* Add user time to cpustat. */
- task_group_account_field(p, index, (__force u64) cputime);
-
- /* Account for user time used */
- acct_update_integrals(p);
-}
-
-/*
- * Account guest cpu time to a process.
- * @p: the process that the cpu time gets accounted to
- * @cputime: the cpu time spent in virtual machine since the last update
- * @cputime_scaled: cputime scaled by cpu frequency
- */
-static void account_guest_time(struct task_struct *p, cputime_t cputime,
- cputime_t cputime_scaled)
-{
- u64 *cpustat = kcpustat_this_cpu->cpustat;
-
- /* Add guest time to process. */
- p->utime += cputime;
- p->utimescaled += cputime_scaled;
- account_group_user_time(p, cputime);
- p->gtime += cputime;
-
- /* Add guest time to cpustat. */
- if (TASK_NICE(p) > 0) {
- cpustat[CPUTIME_NICE] += (__force u64) cputime;
- cpustat[CPUTIME_GUEST_NICE] += (__force u64) cputime;
- } else {
- cpustat[CPUTIME_USER] += (__force u64) cputime;
- cpustat[CPUTIME_GUEST] += (__force u64) cputime;
- }
-}
-
-/*
- * Account system cpu time to a process and desired cpustat field
- * @p: the process that the cpu time gets accounted to
- * @cputime: the cpu time spent in kernel space since the last update
- * @cputime_scaled: cputime scaled by cpu frequency
- * @target_cputime64: pointer to cpustat field that has to be updated
- */
-static inline
-void __account_system_time(struct task_struct *p, cputime_t cputime,
- cputime_t cputime_scaled, int index)
-{
- /* Add system time to process. */
- p->stime += cputime;
- p->stimescaled += cputime_scaled;
- account_group_system_time(p, cputime);
-
- /* Add system time to cpustat. */
- task_group_account_field(p, index, (__force u64) cputime);
-
- /* Account for system time used */
- acct_update_integrals(p);
-}
-
-/*
- * Account system cpu time to a process.
- * @p: the process that the cpu time gets accounted to
- * @hardirq_offset: the offset to subtract from hardirq_count()
- * @cputime: the cpu time spent in kernel space since the last update
- * @cputime_scaled: cputime scaled by cpu frequency
- */
-void account_system_time(struct task_struct *p, int hardirq_offset,
- cputime_t cputime, cputime_t cputime_scaled)
-{
- int index;
-
- if ((p->flags & PF_VCPU) && (irq_count() - hardirq_offset == 0)) {
- account_guest_time(p, cputime, cputime_scaled);
- return;
- }
-
- if (hardirq_count() - hardirq_offset)
- index = CPUTIME_IRQ;
- else if (in_serving_softirq())
- index = CPUTIME_SOFTIRQ;
- else
- index = CPUTIME_SYSTEM;
-
- __account_system_time(p, cputime, cputime_scaled, index);
-}
-
-/*
- * Account for involuntary wait time.
- * @cputime: the cpu time spent in involuntary wait
- */
-void account_steal_time(cputime_t cputime)
-{
- u64 *cpustat = kcpustat_this_cpu->cpustat;
-
- cpustat[CPUTIME_STEAL] += (__force u64) cputime;
-}
-
-/*
- * Account for idle time.
- * @cputime: the cpu time spent in idle wait
- */
-void account_idle_time(cputime_t cputime)
-{
- u64 *cpustat = kcpustat_this_cpu->cpustat;
- struct rq *rq = this_rq();
-
- if (atomic_read(&rq->nr_iowait) > 0)
- cpustat[CPUTIME_IOWAIT] += (__force u64) cputime;
- else
- cpustat[CPUTIME_IDLE] += (__force u64) cputime;
-}
-
-static __always_inline bool steal_account_process_tick(void)
-{
-#ifdef CONFIG_PARAVIRT
- if (static_key_false(¶virt_steal_enabled)) {
- u64 steal, st = 0;
-
- steal = paravirt_steal_clock(smp_processor_id());
- steal -= this_rq()->prev_steal_time;
-
- st = steal_ticks(steal);
- this_rq()->prev_steal_time += st * TICK_NSEC;
-
- account_steal_time(st);
- return st;
- }
-#endif
- return false;
-}
-
-#ifndef CONFIG_VIRT_CPU_ACCOUNTING
-
-#ifdef CONFIG_IRQ_TIME_ACCOUNTING
-/*
- * Account a tick to a process and cpustat
- * @p: the process that the cpu time gets accounted to
- * @user_tick: is the tick from userspace
- * @rq: the pointer to rq
- *
- * Tick demultiplexing follows the order
- * - pending hardirq update
- * - pending softirq update
- * - user_time
- * - idle_time
- * - system time
- * - check for guest_time
- * - else account as system_time
- *
- * Check for hardirq is done both for system and user time as there is
- * no timer going off while we are on hardirq and hence we may never get an
- * opportunity to update it solely in system time.
- * p->stime and friends are only updated on system time and not on irq
- * softirq as those do not count in task exec_runtime any more.
- */
-static void irqtime_account_process_tick(struct task_struct *p, int user_tick,
- struct rq *rq)
-{
- cputime_t one_jiffy_scaled = cputime_to_scaled(cputime_one_jiffy);
- u64 *cpustat = kcpustat_this_cpu->cpustat;
-
- if (steal_account_process_tick())
- return;
-
- if (irqtime_account_hi_update()) {
- cpustat[CPUTIME_IRQ] += (__force u64) cputime_one_jiffy;
- } else if (irqtime_account_si_update()) {
- cpustat[CPUTIME_SOFTIRQ] += (__force u64) cputime_one_jiffy;
- } else if (this_cpu_ksoftirqd() == p) {
- /*
- * ksoftirqd time do not get accounted in cpu_softirq_time.
- * So, we have to handle it separately here.
- * Also, p->stime needs to be updated for ksoftirqd.
- */
- __account_system_time(p, cputime_one_jiffy, one_jiffy_scaled,
- CPUTIME_SOFTIRQ);
- } else if (user_tick) {
- account_user_time(p, cputime_one_jiffy, one_jiffy_scaled);
- } else if (p == rq->idle) {
- account_idle_time(cputime_one_jiffy);
- } else if (p->flags & PF_VCPU) { /* System time or guest time */
- account_guest_time(p, cputime_one_jiffy, one_jiffy_scaled);
- } else {
- __account_system_time(p, cputime_one_jiffy, one_jiffy_scaled,
- CPUTIME_SYSTEM);
- }
-}
-
-static void irqtime_account_idle_ticks(int ticks)
-{
- int i;
- struct rq *rq = this_rq();
-
- for (i = 0; i < ticks; i++)
- irqtime_account_process_tick(current, 0, rq);
-}
-#else /* CONFIG_IRQ_TIME_ACCOUNTING */
-static void irqtime_account_idle_ticks(int ticks) {}
-static void irqtime_account_process_tick(struct task_struct *p, int user_tick,
- struct rq *rq) {}
-#endif /* CONFIG_IRQ_TIME_ACCOUNTING */
-
-/*
- * Account a single tick of cpu time.
- * @p: the process that the cpu time gets accounted to
- * @user_tick: indicates if the tick is a user or a system tick
- */
-void account_process_tick(struct task_struct *p, int user_tick)
-{
- cputime_t one_jiffy_scaled = cputime_to_scaled(cputime_one_jiffy);
- struct rq *rq = this_rq();
-
- if (sched_clock_irqtime) {
- irqtime_account_process_tick(p, user_tick, rq);
- return;
- }
-
- if (steal_account_process_tick())
- return;
-
- if (user_tick)
- account_user_time(p, cputime_one_jiffy, one_jiffy_scaled);
- else if ((p != rq->idle) || (irq_count() != HARDIRQ_OFFSET))
- account_system_time(p, HARDIRQ_OFFSET, cputime_one_jiffy,
- one_jiffy_scaled);
- else
- account_idle_time(cputime_one_jiffy);
-}
-
-/*
- * Account multiple ticks of steal time.
- * @p: the process from which the cpu time has been stolen
- * @ticks: number of stolen ticks
- */
-void account_steal_ticks(unsigned long ticks)
-{
- account_steal_time(jiffies_to_cputime(ticks));
-}
-
-/*
- * Account multiple ticks of idle time.
- * @ticks: number of stolen ticks
- */
-void account_idle_ticks(unsigned long ticks)
-{
-
- if (sched_clock_irqtime) {
- irqtime_account_idle_ticks(ticks);
- return;
- }
-
- account_idle_time(jiffies_to_cputime(ticks));
-}
-
-#endif
-
-/*
- * Use precise platform statistics if available:
- */
-#ifdef CONFIG_VIRT_CPU_ACCOUNTING
-void task_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
-{
- *ut = p->utime;
- *st = p->stime;
-}
-
-void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
-{
- struct task_cputime cputime;
-
- thread_group_cputime(p, &cputime);
-
- *ut = cputime.utime;
- *st = cputime.stime;
-}
-#else
-
-#ifndef nsecs_to_cputime
-# define nsecs_to_cputime(__nsecs) nsecs_to_jiffies(__nsecs)
-#endif
-
-static cputime_t scale_utime(cputime_t utime, cputime_t rtime, cputime_t total)
-{
- u64 temp = (__force u64) rtime;
-
- temp *= (__force u64) utime;
-
- if (sizeof(cputime_t) == 4)
- temp = div_u64(temp, (__force u32) total);
- else
- temp = div64_u64(temp, (__force u64) total);
-
- return (__force cputime_t) temp;
-}
-
-void task_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
-{
- cputime_t rtime, utime = p->utime, total = utime + p->stime;
-
- /*
- * Use CFS's precise accounting:
- */
- rtime = nsecs_to_cputime(p->se.sum_exec_runtime);
-
- if (total)
- utime = scale_utime(utime, rtime, total);
- else
- utime = rtime;
-
- /*
- * Compare with previous values, to keep monotonicity:
- */
- p->prev_utime = max(p->prev_utime, utime);
- p->prev_stime = max(p->prev_stime, rtime - p->prev_utime);
-
- *ut = p->prev_utime;
- *st = p->prev_stime;
-}
-
-/*
- * Must be called with siglock held.
- */
-void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
-{
- struct signal_struct *sig = p->signal;
- struct task_cputime cputime;
- cputime_t rtime, utime, total;
-
- thread_group_cputime(p, &cputime);
-
- total = cputime.utime + cputime.stime;
- rtime = nsecs_to_cputime(cputime.sum_exec_runtime);
-
- if (total)
- utime = scale_utime(cputime.utime, rtime, total);
- else
- utime = rtime;
-
- sig->prev_utime = max(sig->prev_utime, utime);
- sig->prev_stime = max(sig->prev_stime, rtime - sig->prev_utime);
-
- *ut = sig->prev_utime;
- *st = sig->prev_stime;
-}
-#endif
-
/*
* This function gets called by the timer code, with HZ frequency.
* We call it with interrupts disabled.
@@ -3367,6 +2776,40 @@
/*
* __schedule() is the main scheduler function.
+ *
+ * The main means of driving the scheduler and thus entering this function are:
+ *
+ * 1. Explicit blocking: mutex, semaphore, waitqueue, etc.
+ *
+ * 2. TIF_NEED_RESCHED flag is checked on interrupt and userspace return
+ * paths. For example, see arch/x86/entry_64.S.
+ *
+ * To drive preemption between tasks, the scheduler sets the flag in timer
+ * interrupt handler scheduler_tick().
+ *
+ * 3. Wakeups don't really cause entry into schedule(). They add a
+ * task to the run-queue and that's it.
+ *
+ * Now, if the new task added to the run-queue preempts the current
+ * task, then the wakeup sets TIF_NEED_RESCHED and schedule() gets
+ * called on the nearest possible occasion:
+ *
+ * - If the kernel is preemptible (CONFIG_PREEMPT=y):
+ *
+ * - in syscall or exception context, at the next outmost
+ * preempt_enable(). (this might be as soon as the wake_up()'s
+ * spin_unlock()!)
+ *
+ * - in IRQ context, return from interrupt-handler to
+ * preemptible context
+ *
+ * - If the kernel is not preemptible (CONFIG_PREEMPT is not set)
+ * then at the next:
+ *
+ * - cond_resched() call
+ * - explicit schedule() call
+ * - return from syscall or exception to user-space
+ * - return from interrupt-handler to user-space
*/
static void __sched __schedule(void)
{
@@ -3468,6 +2911,21 @@
}
EXPORT_SYMBOL(schedule);
+#ifdef CONFIG_RCU_USER_QS
+asmlinkage void __sched schedule_user(void)
+{
+ /*
+ * If we come here after a random call to set_need_resched(),
+ * or we have been woken up remotely but the IPI has not yet arrived,
+ * we haven't yet exited the RCU idle mode. Do it here manually until
+ * we find a better solution.
+ */
+ rcu_user_exit();
+ schedule();
+ rcu_user_enter();
+}
+#endif
+
/**
* schedule_preempt_disabled - called with preemption disabled
*
@@ -3569,6 +3027,7 @@
/* Catch callers which need to be fixed */
BUG_ON(ti->preempt_count || !irqs_disabled());
+ rcu_user_exit();
do {
add_preempt_count(PREEMPT_ACTIVE);
local_irq_enable();
@@ -4868,13 +4327,6 @@
*/
if (preempt && rq != p_rq)
resched_task(p_rq->curr);
- } else {
- /*
- * We might have set it in task_yield_fair(), but are
- * not going to schedule(), so don't want to skip
- * the next update.
- */
- rq->skip_clock_update = 0;
}
out:
@@ -5304,27 +4756,17 @@
}
/*
- * While a dead CPU has no uninterruptible tasks queued at this point,
- * it might still have a nonzero ->nr_uninterruptible counter, because
- * for performance reasons the counter is not stricly tracking tasks to
- * their home CPUs. So we just add the counter to another CPU's counter,
- * to keep the global sum constant after CPU-down:
+ * Since this CPU is going 'away' for a while, fold any nr_active delta
+ * we might have. Assumes we're called after migrate_tasks() so that the
+ * nr_active count is stable.
+ *
+ * Also see the comment "Global load-average calculations".
*/
-static void migrate_nr_uninterruptible(struct rq *rq_src)
+static void calc_load_migrate(struct rq *rq)
{
- struct rq *rq_dest = cpu_rq(cpumask_any(cpu_active_mask));
-
- rq_dest->nr_uninterruptible += rq_src->nr_uninterruptible;
- rq_src->nr_uninterruptible = 0;
-}
-
-/*
- * remove the tasks which were accounted by rq from calc_load_tasks.
- */
-static void calc_global_load_remove(struct rq *rq)
-{
- atomic_long_sub(rq->calc_load_active, &calc_load_tasks);
- rq->calc_load_active = 0;
+ long delta = calc_load_fold_active(rq);
+ if (delta)
+ atomic_long_add(delta, &calc_load_tasks);
}
/*
@@ -5352,9 +4794,6 @@
*/
rq->stop = NULL;
- /* Ensure any throttled groups are reachable by pick_next_task */
- unthrottle_offline_cfs_rqs(rq);
-
for ( ; ; ) {
/*
* There's this thread running, bail when that's the only
@@ -5429,16 +4868,25 @@
*tablep = NULL;
}
+static int min_load_idx = 0;
+static int max_load_idx = CPU_LOAD_IDX_MAX;
+
static void
set_table_entry(struct ctl_table *entry,
const char *procname, void *data, int maxlen,
- umode_t mode, proc_handler *proc_handler)
+ umode_t mode, proc_handler *proc_handler,
+ bool load_idx)
{
entry->procname = procname;
entry->data = data;
entry->maxlen = maxlen;
entry->mode = mode;
entry->proc_handler = proc_handler;
+
+ if (load_idx) {
+ entry->extra1 = &min_load_idx;
+ entry->extra2 = &max_load_idx;
+ }
}
static struct ctl_table *
@@ -5450,30 +4898,30 @@
return NULL;
set_table_entry(&table[0], "min_interval", &sd->min_interval,
- sizeof(long), 0644, proc_doulongvec_minmax);
+ sizeof(long), 0644, proc_doulongvec_minmax, false);
set_table_entry(&table[1], "max_interval", &sd->max_interval,
- sizeof(long), 0644, proc_doulongvec_minmax);
+ sizeof(long), 0644, proc_doulongvec_minmax, false);
set_table_entry(&table[2], "busy_idx", &sd->busy_idx,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, true);
set_table_entry(&table[3], "idle_idx", &sd->idle_idx,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, true);
set_table_entry(&table[4], "newidle_idx", &sd->newidle_idx,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, true);
set_table_entry(&table[5], "wake_idx", &sd->wake_idx,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, true);
set_table_entry(&table[6], "forkexec_idx", &sd->forkexec_idx,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, true);
set_table_entry(&table[7], "busy_factor", &sd->busy_factor,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, false);
set_table_entry(&table[8], "imbalance_pct", &sd->imbalance_pct,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, false);
set_table_entry(&table[9], "cache_nice_tries",
&sd->cache_nice_tries,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, false);
set_table_entry(&table[10], "flags", &sd->flags,
- sizeof(int), 0644, proc_dointvec_minmax);
+ sizeof(int), 0644, proc_dointvec_minmax, false);
set_table_entry(&table[11], "name", sd->name,
- CORENAME_MAX_SIZE, 0444, proc_dostring);
+ CORENAME_MAX_SIZE, 0444, proc_dostring, false);
/* &table[12] is terminator */
return table;
@@ -5617,9 +5065,10 @@
migrate_tasks(cpu);
BUG_ON(rq->nr_running != 1); /* the migration thread */
raw_spin_unlock_irqrestore(&rq->lock, flags);
+ break;
- migrate_nr_uninterruptible(rq);
- calc_global_load_remove(rq);
+ case CPU_DEAD:
+ calc_load_migrate(rq);
break;
#endif
}
@@ -6028,11 +5477,6 @@
* SD_SHARE_PKG_RESOURCE set (Last Level Cache Domain) for this
* allows us to avoid some pointer chasing select_idle_sibling().
*
- * Iterate domains and sched_groups downward, assigning CPUs to be
- * select_idle_sibling() hw buddy. Cross-wiring hw makes bouncing
- * due to random perturbation self canceling, ie sw buddies pull
- * their counterpart to their CPU's hw counterpart.
- *
* Also keep a unique ID per domain (we use the first cpu number in
* the cpumask of the domain), this allows us to quickly tell if
* two cpus are in the same cache domain, see cpus_share_cache().
@@ -6046,40 +5490,8 @@
int id = cpu;
sd = highest_flag_domain(cpu, SD_SHARE_PKG_RESOURCES);
- if (sd) {
- struct sched_domain *tmp = sd;
- struct sched_group *sg, *prev;
- bool right;
-
- /*
- * Traverse to first CPU in group, and count hops
- * to cpu from there, switching direction on each
- * hop, never ever pointing the last CPU rightward.
- */
- do {
- id = cpumask_first(sched_domain_span(tmp));
- prev = sg = tmp->groups;
- right = 1;
-
- while (cpumask_first(sched_group_cpus(sg)) != id)
- sg = sg->next;
-
- while (!cpumask_test_cpu(cpu, sched_group_cpus(sg))) {
- prev = sg;
- sg = sg->next;
- right = !right;
- }
-
- /* A CPU went down, never point back to domain start. */
- if (right && cpumask_first(sched_group_cpus(sg->next)) == id)
- right = false;
-
- sg = right ? sg->next : prev;
- tmp->idle_buddy = cpumask_first(sched_group_cpus(sg));
- } while ((tmp = tmp->child));
-
+ if (sd)
id = cpumask_first(sched_domain_span(sd));
- }
rcu_assign_pointer(per_cpu(sd_llc, cpu), sd);
per_cpu(sd_llc_id, cpu) = id;
@@ -6588,7 +6000,6 @@
| 0*SD_BALANCE_FORK
| 0*SD_BALANCE_WAKE
| 0*SD_WAKE_AFFINE
- | 0*SD_PREFER_LOCAL
| 0*SD_SHARE_CPUPOWER
| 0*SD_SHARE_PKG_RESOURCES
| 1*SD_SERIALIZE
@@ -8386,6 +7797,8 @@
* (balbir@in.ibm.com).
*/
+struct cpuacct root_cpuacct;
+
/* create a new cpu accounting group */
static struct cgroup_subsys_state *cpuacct_create(struct cgroup *cgrp)
{
diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c
new file mode 100644
index 0000000..81b763b
--- /dev/null
+++ b/kernel/sched/cputime.c
@@ -0,0 +1,530 @@
+#include <linux/export.h>
+#include <linux/sched.h>
+#include <linux/tsacct_kern.h>
+#include <linux/kernel_stat.h>
+#include <linux/static_key.h>
+#include "sched.h"
+
+
+#ifdef CONFIG_IRQ_TIME_ACCOUNTING
+
+/*
+ * There are no locks covering percpu hardirq/softirq time.
+ * They are only modified in vtime_account, on corresponding CPU
+ * with interrupts disabled. So, writes are safe.
+ * They are read and saved off onto struct rq in update_rq_clock().
+ * This may result in other CPU reading this CPU's irq time and can
+ * race with irq/vtime_account on this CPU. We would either get old
+ * or new value with a side effect of accounting a slice of irq time to wrong
+ * task when irq is in progress while we read rq->clock. That is a worthy
+ * compromise in place of having locks on each irq in account_system_time.
+ */
+DEFINE_PER_CPU(u64, cpu_hardirq_time);
+DEFINE_PER_CPU(u64, cpu_softirq_time);
+
+static DEFINE_PER_CPU(u64, irq_start_time);
+static int sched_clock_irqtime;
+
+void enable_sched_clock_irqtime(void)
+{
+ sched_clock_irqtime = 1;
+}
+
+void disable_sched_clock_irqtime(void)
+{
+ sched_clock_irqtime = 0;
+}
+
+#ifndef CONFIG_64BIT
+DEFINE_PER_CPU(seqcount_t, irq_time_seq);
+#endif /* CONFIG_64BIT */
+
+/*
+ * Called before incrementing preempt_count on {soft,}irq_enter
+ * and before decrementing preempt_count on {soft,}irq_exit.
+ */
+void vtime_account(struct task_struct *curr)
+{
+ unsigned long flags;
+ s64 delta;
+ int cpu;
+
+ if (!sched_clock_irqtime)
+ return;
+
+ local_irq_save(flags);
+
+ cpu = smp_processor_id();
+ delta = sched_clock_cpu(cpu) - __this_cpu_read(irq_start_time);
+ __this_cpu_add(irq_start_time, delta);
+
+ irq_time_write_begin();
+ /*
+ * We do not account for softirq time from ksoftirqd here.
+ * We want to continue accounting softirq time to ksoftirqd thread
+ * in that case, so as not to confuse scheduler with a special task
+ * that do not consume any time, but still wants to run.
+ */
+ if (hardirq_count())
+ __this_cpu_add(cpu_hardirq_time, delta);
+ else if (in_serving_softirq() && curr != this_cpu_ksoftirqd())
+ __this_cpu_add(cpu_softirq_time, delta);
+
+ irq_time_write_end();
+ local_irq_restore(flags);
+}
+EXPORT_SYMBOL_GPL(vtime_account);
+
+static int irqtime_account_hi_update(void)
+{
+ u64 *cpustat = kcpustat_this_cpu->cpustat;
+ unsigned long flags;
+ u64 latest_ns;
+ int ret = 0;
+
+ local_irq_save(flags);
+ latest_ns = this_cpu_read(cpu_hardirq_time);
+ if (nsecs_to_cputime64(latest_ns) > cpustat[CPUTIME_IRQ])
+ ret = 1;
+ local_irq_restore(flags);
+ return ret;
+}
+
+static int irqtime_account_si_update(void)
+{
+ u64 *cpustat = kcpustat_this_cpu->cpustat;
+ unsigned long flags;
+ u64 latest_ns;
+ int ret = 0;
+
+ local_irq_save(flags);
+ latest_ns = this_cpu_read(cpu_softirq_time);
+ if (nsecs_to_cputime64(latest_ns) > cpustat[CPUTIME_SOFTIRQ])
+ ret = 1;
+ local_irq_restore(flags);
+ return ret;
+}
+
+#else /* CONFIG_IRQ_TIME_ACCOUNTING */
+
+#define sched_clock_irqtime (0)
+
+#endif /* !CONFIG_IRQ_TIME_ACCOUNTING */
+
+static inline void task_group_account_field(struct task_struct *p, int index,
+ u64 tmp)
+{
+#ifdef CONFIG_CGROUP_CPUACCT
+ struct kernel_cpustat *kcpustat;
+ struct cpuacct *ca;
+#endif
+ /*
+ * Since all updates are sure to touch the root cgroup, we
+ * get ourselves ahead and touch it first. If the root cgroup
+ * is the only cgroup, then nothing else should be necessary.
+ *
+ */
+ __get_cpu_var(kernel_cpustat).cpustat[index] += tmp;
+
+#ifdef CONFIG_CGROUP_CPUACCT
+ if (unlikely(!cpuacct_subsys.active))
+ return;
+
+ rcu_read_lock();
+ ca = task_ca(p);
+ while (ca && (ca != &root_cpuacct)) {
+ kcpustat = this_cpu_ptr(ca->cpustat);
+ kcpustat->cpustat[index] += tmp;
+ ca = parent_ca(ca);
+ }
+ rcu_read_unlock();
+#endif
+}
+
+/*
+ * Account user cpu time to a process.
+ * @p: the process that the cpu time gets accounted to
+ * @cputime: the cpu time spent in user space since the last update
+ * @cputime_scaled: cputime scaled by cpu frequency
+ */
+void account_user_time(struct task_struct *p, cputime_t cputime,
+ cputime_t cputime_scaled)
+{
+ int index;
+
+ /* Add user time to process. */
+ p->utime += cputime;
+ p->utimescaled += cputime_scaled;
+ account_group_user_time(p, cputime);
+
+ index = (TASK_NICE(p) > 0) ? CPUTIME_NICE : CPUTIME_USER;
+
+ /* Add user time to cpustat. */
+ task_group_account_field(p, index, (__force u64) cputime);
+
+ /* Account for user time used */
+ acct_update_integrals(p);
+}
+
+/*
+ * Account guest cpu time to a process.
+ * @p: the process that the cpu time gets accounted to
+ * @cputime: the cpu time spent in virtual machine since the last update
+ * @cputime_scaled: cputime scaled by cpu frequency
+ */
+static void account_guest_time(struct task_struct *p, cputime_t cputime,
+ cputime_t cputime_scaled)
+{
+ u64 *cpustat = kcpustat_this_cpu->cpustat;
+
+ /* Add guest time to process. */
+ p->utime += cputime;
+ p->utimescaled += cputime_scaled;
+ account_group_user_time(p, cputime);
+ p->gtime += cputime;
+
+ /* Add guest time to cpustat. */
+ if (TASK_NICE(p) > 0) {
+ cpustat[CPUTIME_NICE] += (__force u64) cputime;
+ cpustat[CPUTIME_GUEST_NICE] += (__force u64) cputime;
+ } else {
+ cpustat[CPUTIME_USER] += (__force u64) cputime;
+ cpustat[CPUTIME_GUEST] += (__force u64) cputime;
+ }
+}
+
+/*
+ * Account system cpu time to a process and desired cpustat field
+ * @p: the process that the cpu time gets accounted to
+ * @cputime: the cpu time spent in kernel space since the last update
+ * @cputime_scaled: cputime scaled by cpu frequency
+ * @target_cputime64: pointer to cpustat field that has to be updated
+ */
+static inline
+void __account_system_time(struct task_struct *p, cputime_t cputime,
+ cputime_t cputime_scaled, int index)
+{
+ /* Add system time to process. */
+ p->stime += cputime;
+ p->stimescaled += cputime_scaled;
+ account_group_system_time(p, cputime);
+
+ /* Add system time to cpustat. */
+ task_group_account_field(p, index, (__force u64) cputime);
+
+ /* Account for system time used */
+ acct_update_integrals(p);
+}
+
+/*
+ * Account system cpu time to a process.
+ * @p: the process that the cpu time gets accounted to
+ * @hardirq_offset: the offset to subtract from hardirq_count()
+ * @cputime: the cpu time spent in kernel space since the last update
+ * @cputime_scaled: cputime scaled by cpu frequency
+ */
+void account_system_time(struct task_struct *p, int hardirq_offset,
+ cputime_t cputime, cputime_t cputime_scaled)
+{
+ int index;
+
+ if ((p->flags & PF_VCPU) && (irq_count() - hardirq_offset == 0)) {
+ account_guest_time(p, cputime, cputime_scaled);
+ return;
+ }
+
+ if (hardirq_count() - hardirq_offset)
+ index = CPUTIME_IRQ;
+ else if (in_serving_softirq())
+ index = CPUTIME_SOFTIRQ;
+ else
+ index = CPUTIME_SYSTEM;
+
+ __account_system_time(p, cputime, cputime_scaled, index);
+}
+
+/*
+ * Account for involuntary wait time.
+ * @cputime: the cpu time spent in involuntary wait
+ */
+void account_steal_time(cputime_t cputime)
+{
+ u64 *cpustat = kcpustat_this_cpu->cpustat;
+
+ cpustat[CPUTIME_STEAL] += (__force u64) cputime;
+}
+
+/*
+ * Account for idle time.
+ * @cputime: the cpu time spent in idle wait
+ */
+void account_idle_time(cputime_t cputime)
+{
+ u64 *cpustat = kcpustat_this_cpu->cpustat;
+ struct rq *rq = this_rq();
+
+ if (atomic_read(&rq->nr_iowait) > 0)
+ cpustat[CPUTIME_IOWAIT] += (__force u64) cputime;
+ else
+ cpustat[CPUTIME_IDLE] += (__force u64) cputime;
+}
+
+static __always_inline bool steal_account_process_tick(void)
+{
+#ifdef CONFIG_PARAVIRT
+ if (static_key_false(¶virt_steal_enabled)) {
+ u64 steal, st = 0;
+
+ steal = paravirt_steal_clock(smp_processor_id());
+ steal -= this_rq()->prev_steal_time;
+
+ st = steal_ticks(steal);
+ this_rq()->prev_steal_time += st * TICK_NSEC;
+
+ account_steal_time(st);
+ return st;
+ }
+#endif
+ return false;
+}
+
+#ifndef CONFIG_VIRT_CPU_ACCOUNTING
+
+#ifdef CONFIG_IRQ_TIME_ACCOUNTING
+/*
+ * Account a tick to a process and cpustat
+ * @p: the process that the cpu time gets accounted to
+ * @user_tick: is the tick from userspace
+ * @rq: the pointer to rq
+ *
+ * Tick demultiplexing follows the order
+ * - pending hardirq update
+ * - pending softirq update
+ * - user_time
+ * - idle_time
+ * - system time
+ * - check for guest_time
+ * - else account as system_time
+ *
+ * Check for hardirq is done both for system and user time as there is
+ * no timer going off while we are on hardirq and hence we may never get an
+ * opportunity to update it solely in system time.
+ * p->stime and friends are only updated on system time and not on irq
+ * softirq as those do not count in task exec_runtime any more.
+ */
+static void irqtime_account_process_tick(struct task_struct *p, int user_tick,
+ struct rq *rq)
+{
+ cputime_t one_jiffy_scaled = cputime_to_scaled(cputime_one_jiffy);
+ u64 *cpustat = kcpustat_this_cpu->cpustat;
+
+ if (steal_account_process_tick())
+ return;
+
+ if (irqtime_account_hi_update()) {
+ cpustat[CPUTIME_IRQ] += (__force u64) cputime_one_jiffy;
+ } else if (irqtime_account_si_update()) {
+ cpustat[CPUTIME_SOFTIRQ] += (__force u64) cputime_one_jiffy;
+ } else if (this_cpu_ksoftirqd() == p) {
+ /*
+ * ksoftirqd time do not get accounted in cpu_softirq_time.
+ * So, we have to handle it separately here.
+ * Also, p->stime needs to be updated for ksoftirqd.
+ */
+ __account_system_time(p, cputime_one_jiffy, one_jiffy_scaled,
+ CPUTIME_SOFTIRQ);
+ } else if (user_tick) {
+ account_user_time(p, cputime_one_jiffy, one_jiffy_scaled);
+ } else if (p == rq->idle) {
+ account_idle_time(cputime_one_jiffy);
+ } else if (p->flags & PF_VCPU) { /* System time or guest time */
+ account_guest_time(p, cputime_one_jiffy, one_jiffy_scaled);
+ } else {
+ __account_system_time(p, cputime_one_jiffy, one_jiffy_scaled,
+ CPUTIME_SYSTEM);
+ }
+}
+
+static void irqtime_account_idle_ticks(int ticks)
+{
+ int i;
+ struct rq *rq = this_rq();
+
+ for (i = 0; i < ticks; i++)
+ irqtime_account_process_tick(current, 0, rq);
+}
+#else /* CONFIG_IRQ_TIME_ACCOUNTING */
+static void irqtime_account_idle_ticks(int ticks) {}
+static void irqtime_account_process_tick(struct task_struct *p, int user_tick,
+ struct rq *rq) {}
+#endif /* CONFIG_IRQ_TIME_ACCOUNTING */
+
+/*
+ * Account a single tick of cpu time.
+ * @p: the process that the cpu time gets accounted to
+ * @user_tick: indicates if the tick is a user or a system tick
+ */
+void account_process_tick(struct task_struct *p, int user_tick)
+{
+ cputime_t one_jiffy_scaled = cputime_to_scaled(cputime_one_jiffy);
+ struct rq *rq = this_rq();
+
+ if (sched_clock_irqtime) {
+ irqtime_account_process_tick(p, user_tick, rq);
+ return;
+ }
+
+ if (steal_account_process_tick())
+ return;
+
+ if (user_tick)
+ account_user_time(p, cputime_one_jiffy, one_jiffy_scaled);
+ else if ((p != rq->idle) || (irq_count() != HARDIRQ_OFFSET))
+ account_system_time(p, HARDIRQ_OFFSET, cputime_one_jiffy,
+ one_jiffy_scaled);
+ else
+ account_idle_time(cputime_one_jiffy);
+}
+
+/*
+ * Account multiple ticks of steal time.
+ * @p: the process from which the cpu time has been stolen
+ * @ticks: number of stolen ticks
+ */
+void account_steal_ticks(unsigned long ticks)
+{
+ account_steal_time(jiffies_to_cputime(ticks));
+}
+
+/*
+ * Account multiple ticks of idle time.
+ * @ticks: number of stolen ticks
+ */
+void account_idle_ticks(unsigned long ticks)
+{
+
+ if (sched_clock_irqtime) {
+ irqtime_account_idle_ticks(ticks);
+ return;
+ }
+
+ account_idle_time(jiffies_to_cputime(ticks));
+}
+
+#endif
+
+/*
+ * Use precise platform statistics if available:
+ */
+#ifdef CONFIG_VIRT_CPU_ACCOUNTING
+void task_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
+{
+ *ut = p->utime;
+ *st = p->stime;
+}
+
+void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
+{
+ struct task_cputime cputime;
+
+ thread_group_cputime(p, &cputime);
+
+ *ut = cputime.utime;
+ *st = cputime.stime;
+}
+
+/*
+ * Archs that account the whole time spent in the idle task
+ * (outside irq) as idle time can rely on this and just implement
+ * vtime_account_system() and vtime_account_idle(). Archs that
+ * have other meaning of the idle time (s390 only includes the
+ * time spent by the CPU when it's in low power mode) must override
+ * vtime_account().
+ */
+#ifndef __ARCH_HAS_VTIME_ACCOUNT
+void vtime_account(struct task_struct *tsk)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ if (in_interrupt() || !is_idle_task(tsk))
+ vtime_account_system(tsk);
+ else
+ vtime_account_idle(tsk);
+
+ local_irq_restore(flags);
+}
+EXPORT_SYMBOL_GPL(vtime_account);
+#endif /* __ARCH_HAS_VTIME_ACCOUNT */
+
+#else
+
+#ifndef nsecs_to_cputime
+# define nsecs_to_cputime(__nsecs) nsecs_to_jiffies(__nsecs)
+#endif
+
+static cputime_t scale_utime(cputime_t utime, cputime_t rtime, cputime_t total)
+{
+ u64 temp = (__force u64) rtime;
+
+ temp *= (__force u64) utime;
+
+ if (sizeof(cputime_t) == 4)
+ temp = div_u64(temp, (__force u32) total);
+ else
+ temp = div64_u64(temp, (__force u64) total);
+
+ return (__force cputime_t) temp;
+}
+
+void task_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
+{
+ cputime_t rtime, utime = p->utime, total = utime + p->stime;
+
+ /*
+ * Use CFS's precise accounting:
+ */
+ rtime = nsecs_to_cputime(p->se.sum_exec_runtime);
+
+ if (total)
+ utime = scale_utime(utime, rtime, total);
+ else
+ utime = rtime;
+
+ /*
+ * Compare with previous values, to keep monotonicity:
+ */
+ p->prev_utime = max(p->prev_utime, utime);
+ p->prev_stime = max(p->prev_stime, rtime - p->prev_utime);
+
+ *ut = p->prev_utime;
+ *st = p->prev_stime;
+}
+
+/*
+ * Must be called with siglock held.
+ */
+void thread_group_times(struct task_struct *p, cputime_t *ut, cputime_t *st)
+{
+ struct signal_struct *sig = p->signal;
+ struct task_cputime cputime;
+ cputime_t rtime, utime, total;
+
+ thread_group_cputime(p, &cputime);
+
+ total = cputime.utime + cputime.stime;
+ rtime = nsecs_to_cputime(cputime.sum_exec_runtime);
+
+ if (total)
+ utime = scale_utime(cputime.utime, rtime, total);
+ else
+ utime = rtime;
+
+ sig->prev_utime = max(sig->prev_utime, utime);
+ sig->prev_stime = max(sig->prev_stime, rtime - sig->prev_utime);
+
+ *ut = sig->prev_utime;
+ *st = sig->prev_stime;
+}
+#endif
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c219bf8..6b800a1 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -597,7 +597,7 @@
/*
* The idea is to set a period in which each task runs once.
*
- * When there are too many tasks (sysctl_sched_nr_latency) we have to stretch
+ * When there are too many tasks (sched_nr_latency) we have to stretch
* this period because otherwise the slices get too small.
*
* p = (nr <= nl) ? l : l*nr/nl
@@ -2052,7 +2052,7 @@
hrtimer_cancel(&cfs_b->slack_timer);
}
-void unthrottle_offline_cfs_rqs(struct rq *rq)
+static void unthrottle_offline_cfs_rqs(struct rq *rq)
{
struct cfs_rq *cfs_rq;
@@ -2106,7 +2106,7 @@
return NULL;
}
static inline void destroy_cfs_bandwidth(struct cfs_bandwidth *cfs_b) {}
-void unthrottle_offline_cfs_rqs(struct rq *rq) {}
+static inline void unthrottle_offline_cfs_rqs(struct rq *rq) {}
#endif /* CONFIG_CFS_BANDWIDTH */
@@ -2637,6 +2637,8 @@
int cpu = smp_processor_id();
int prev_cpu = task_cpu(p);
struct sched_domain *sd;
+ struct sched_group *sg;
+ int i;
/*
* If the task is going to be woken-up on this cpu and if it is
@@ -2653,17 +2655,29 @@
return prev_cpu;
/*
- * Otherwise, check assigned siblings to find an elegible idle cpu.
+ * Otherwise, iterate the domains and find an elegible idle cpu.
*/
sd = rcu_dereference(per_cpu(sd_llc, target));
-
for_each_lower_domain(sd) {
- if (!cpumask_test_cpu(sd->idle_buddy, tsk_cpus_allowed(p)))
- continue;
- if (idle_cpu(sd->idle_buddy))
- return sd->idle_buddy;
- }
+ sg = sd->groups;
+ do {
+ if (!cpumask_intersects(sched_group_cpus(sg),
+ tsk_cpus_allowed(p)))
+ goto next;
+ for_each_cpu(i, sched_group_cpus(sg)) {
+ if (!idle_cpu(i))
+ goto next;
+ }
+
+ target = cpumask_first_and(sched_group_cpus(sg),
+ tsk_cpus_allowed(p));
+ goto done;
+next:
+ sg = sg->next;
+ } while (sg != sd->groups);
+ }
+done:
return target;
}
@@ -2686,7 +2700,6 @@
int prev_cpu = task_cpu(p);
int new_cpu = cpu;
int want_affine = 0;
- int want_sd = 1;
int sync = wake_flags & WF_SYNC;
if (p->nr_cpus_allowed == 1)
@@ -2704,48 +2717,21 @@
continue;
/*
- * If power savings logic is enabled for a domain, see if we
- * are not overloaded, if so, don't balance wider.
- */
- if (tmp->flags & (SD_PREFER_LOCAL)) {
- unsigned long power = 0;
- unsigned long nr_running = 0;
- unsigned long capacity;
- int i;
-
- for_each_cpu(i, sched_domain_span(tmp)) {
- power += power_of(i);
- nr_running += cpu_rq(i)->cfs.nr_running;
- }
-
- capacity = DIV_ROUND_CLOSEST(power, SCHED_POWER_SCALE);
-
- if (nr_running < capacity)
- want_sd = 0;
- }
-
- /*
* If both cpu and prev_cpu are part of this domain,
* cpu is a valid SD_WAKE_AFFINE target.
*/
if (want_affine && (tmp->flags & SD_WAKE_AFFINE) &&
cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) {
affine_sd = tmp;
- want_affine = 0;
+ break;
}
- if (!want_sd && !want_affine)
- break;
-
- if (!(tmp->flags & sd_flag))
- continue;
-
- if (want_sd)
+ if (tmp->flags & sd_flag)
sd = tmp;
}
if (affine_sd) {
- if (cpu == prev_cpu || wake_affine(affine_sd, p, sync))
+ if (cpu != prev_cpu && wake_affine(affine_sd, p, sync))
prev_cpu = cpu;
new_cpu = select_idle_sibling(p, prev_cpu);
@@ -3658,7 +3644,6 @@
* @group: sched_group whose statistics are to be updated.
* @load_idx: Load index of sched_domain of this_cpu for load calc.
* @local_group: Does group contain this_cpu.
- * @cpus: Set of cpus considered for load balancing.
* @balance: Should we balance.
* @sgs: variable to hold the statistics for this group.
*/
@@ -3805,7 +3790,6 @@
/**
* update_sd_lb_stats - Update sched_domain's statistics for load balancing.
* @env: The load balancing environment.
- * @cpus: Set of cpus considered for load balancing.
* @balance: Should we balance.
* @sds: variable to hold the statistics for this sched_domain.
*/
@@ -4283,7 +4267,7 @@
goto out_balanced;
}
- BUG_ON(busiest == this_rq);
+ BUG_ON(busiest == env.dst_rq);
schedstat_add(sd, lb_imbalance[idle], env.imbalance);
@@ -4304,7 +4288,7 @@
update_h_load(env.src_cpu);
more_balance:
local_irq_save(flags);
- double_rq_lock(this_rq, busiest);
+ double_rq_lock(env.dst_rq, busiest);
/*
* cur_ld_moved - load moved in current iteration
@@ -4312,7 +4296,7 @@
*/
cur_ld_moved = move_tasks(&env);
ld_moved += cur_ld_moved;
- double_rq_unlock(this_rq, busiest);
+ double_rq_unlock(env.dst_rq, busiest);
local_irq_restore(flags);
if (env.flags & LBF_NEED_BREAK) {
@@ -4348,8 +4332,7 @@
if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0 &&
lb_iterations++ < max_lb_iterations) {
- this_rq = cpu_rq(env.new_dst_cpu);
- env.dst_rq = this_rq;
+ env.dst_rq = cpu_rq(env.new_dst_cpu);
env.dst_cpu = env.new_dst_cpu;
env.flags &= ~LBF_SOME_PINNED;
env.loop = 0;
@@ -4634,7 +4617,7 @@
return;
}
-static inline void clear_nohz_tick_stopped(int cpu)
+static inline void nohz_balance_exit_idle(int cpu)
{
if (unlikely(test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))) {
cpumask_clear_cpu(cpu, nohz.idle_cpus_mask);
@@ -4674,28 +4657,23 @@
}
/*
- * This routine will record that this cpu is going idle with tick stopped.
+ * This routine will record that the cpu is going idle with tick stopped.
* This info will be used in performing idle load balancing in the future.
*/
-void select_nohz_load_balancer(int stop_tick)
+void nohz_balance_enter_idle(int cpu)
{
- int cpu = smp_processor_id();
-
/*
* If this cpu is going down, then nothing needs to be done.
*/
if (!cpu_active(cpu))
return;
- if (stop_tick) {
- if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
- return;
+ if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
+ return;
- cpumask_set_cpu(cpu, nohz.idle_cpus_mask);
- atomic_inc(&nohz.nr_cpus);
- set_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu));
- }
- return;
+ cpumask_set_cpu(cpu, nohz.idle_cpus_mask);
+ atomic_inc(&nohz.nr_cpus);
+ set_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu));
}
static int __cpuinit sched_ilb_notifier(struct notifier_block *nfb,
@@ -4703,7 +4681,7 @@
{
switch (action & ~CPU_TASKS_FROZEN) {
case CPU_DYING:
- clear_nohz_tick_stopped(smp_processor_id());
+ nohz_balance_exit_idle(smp_processor_id());
return NOTIFY_OK;
default:
return NOTIFY_DONE;
@@ -4825,14 +4803,15 @@
if (need_resched())
break;
- raw_spin_lock_irq(&this_rq->lock);
- update_rq_clock(this_rq);
- update_idle_cpu_load(this_rq);
- raw_spin_unlock_irq(&this_rq->lock);
+ rq = cpu_rq(balance_cpu);
+
+ raw_spin_lock_irq(&rq->lock);
+ update_rq_clock(rq);
+ update_idle_cpu_load(rq);
+ raw_spin_unlock_irq(&rq->lock);
rebalance_domains(balance_cpu, CPU_IDLE);
- rq = cpu_rq(balance_cpu);
if (time_after(this_rq->next_balance, rq->next_balance))
this_rq->next_balance = rq->next_balance;
}
@@ -4863,7 +4842,7 @@
* busy tick after returning from idle, we will update the busy stats.
*/
set_cpu_sd_state_busy();
- clear_nohz_tick_stopped(cpu);
+ nohz_balance_exit_idle(cpu);
/*
* None are in tickless mode and hence no need for NOHZ idle load
@@ -4956,6 +4935,9 @@
static void rq_offline_fair(struct rq *rq)
{
update_sysctl();
+
+ /* Ensure any throttled groups are reachable by pick_next_task */
+ unthrottle_offline_cfs_rqs(rq);
}
#endif /* CONFIG_SMP */
diff --git a/kernel/sched/features.h b/kernel/sched/features.h
index de00a48..eebefca 100644
--- a/kernel/sched/features.h
+++ b/kernel/sched/features.h
@@ -12,14 +12,6 @@
SCHED_FEAT(START_DEBIT, true)
/*
- * Based on load and program behaviour, see if it makes sense to place
- * a newly woken task on the same cpu as the task that woke it --
- * improve cache locality. Typically used with SYNC wakeups as
- * generated by pipes and the like, see also SYNC_WAKEUPS.
- */
-SCHED_FEAT(AFFINE_WAKEUPS, true)
-
-/*
* Prefer to schedule the task we woke last (assuming it failed
* wakeup-preemption), since its likely going to consume data we
* touched, increases cache locality.
@@ -42,7 +34,7 @@
/*
* Use arch dependent cpu power functions
*/
-SCHED_FEAT(ARCH_POWER, false)
+SCHED_FEAT(ARCH_POWER, true)
SCHED_FEAT(HRTICK, false)
SCHED_FEAT(DOUBLE_TICK, false)
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 944cb68..418feb0 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -691,6 +691,7 @@
* runtime - in which case borrowing doesn't make sense.
*/
rt_rq->rt_runtime = RUNTIME_INF;
+ rt_rq->rt_throttled = 0;
raw_spin_unlock(&rt_rq->rt_runtime_lock);
raw_spin_unlock(&rt_b->rt_runtime_lock);
}
@@ -1631,11 +1632,6 @@
if (!next_task)
return 0;
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
- if (unlikely(task_running(rq, next_task)))
- return 0;
-#endif
-
retry:
if (unlikely(next_task == rq->curr)) {
WARN_ON(1);
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index f6714d0..7a7db09 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -737,11 +737,7 @@
*/
next->on_cpu = 1;
#endif
-#ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
- raw_spin_unlock_irq(&rq->lock);
-#else
raw_spin_unlock(&rq->lock);
-#endif
}
static inline void finish_lock_switch(struct rq *rq, struct task_struct *prev)
@@ -755,9 +751,7 @@
smp_wmb();
prev->on_cpu = 0;
#endif
-#ifndef __ARCH_WANT_INTERRUPTS_ON_CTXSW
local_irq_enable();
-#endif
}
#endif /* __ARCH_WANT_UNLOCKED_CTXSW */
@@ -891,6 +885,9 @@
struct kernel_cpustat __percpu *cpustat;
};
+extern struct cgroup_subsys cpuacct_subsys;
+extern struct cpuacct root_cpuacct;
+
/* return cpu accounting group corresponding to this container */
static inline struct cpuacct *cgroup_ca(struct cgroup *cgrp)
{
@@ -917,6 +914,16 @@
static inline void cpuacct_charge(struct task_struct *tsk, u64 cputime) {}
#endif
+#ifdef CONFIG_PARAVIRT
+static inline u64 steal_ticks(u64 steal)
+{
+ if (unlikely(steal > NSEC_PER_SEC))
+ return div_u64(steal, TICK_NSEC);
+
+ return __iter_div_u64_rem(steal, TICK_NSEC, &steal);
+}
+#endif
+
static inline void inc_nr_running(struct rq *rq)
{
rq->nr_running++;
@@ -1144,7 +1151,6 @@
extern void init_cfs_rq(struct cfs_rq *cfs_rq);
extern void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq);
-extern void unthrottle_offline_cfs_rqs(struct rq *rq);
extern void account_cfs_bandwidth_used(int enabled, int was_enabled);
@@ -1157,3 +1163,53 @@
#define nohz_flags(cpu) (&cpu_rq(cpu)->nohz_flags)
#endif
+
+#ifdef CONFIG_IRQ_TIME_ACCOUNTING
+
+DECLARE_PER_CPU(u64, cpu_hardirq_time);
+DECLARE_PER_CPU(u64, cpu_softirq_time);
+
+#ifndef CONFIG_64BIT
+DECLARE_PER_CPU(seqcount_t, irq_time_seq);
+
+static inline void irq_time_write_begin(void)
+{
+ __this_cpu_inc(irq_time_seq.sequence);
+ smp_wmb();
+}
+
+static inline void irq_time_write_end(void)
+{
+ smp_wmb();
+ __this_cpu_inc(irq_time_seq.sequence);
+}
+
+static inline u64 irq_time_read(int cpu)
+{
+ u64 irq_time;
+ unsigned seq;
+
+ do {
+ seq = read_seqcount_begin(&per_cpu(irq_time_seq, cpu));
+ irq_time = per_cpu(cpu_softirq_time, cpu) +
+ per_cpu(cpu_hardirq_time, cpu);
+ } while (read_seqcount_retry(&per_cpu(irq_time_seq, cpu), seq));
+
+ return irq_time;
+}
+#else /* CONFIG_64BIT */
+static inline void irq_time_write_begin(void)
+{
+}
+
+static inline void irq_time_write_end(void)
+{
+}
+
+static inline u64 irq_time_read(int cpu)
+{
+ return per_cpu(cpu_softirq_time, cpu) + per_cpu(cpu_hardirq_time, cpu);
+}
+#endif /* CONFIG_64BIT */
+#endif /* CONFIG_IRQ_TIME_ACCOUNTING */
+
diff --git a/kernel/signal.c b/kernel/signal.c
index be4f856..2c681f1 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -1971,13 +1971,8 @@
void ptrace_notify(int exit_code)
{
BUG_ON((exit_code & (0x7f | ~0xffff)) != SIGTRAP);
- if (unlikely(current->task_works)) {
- if (test_and_clear_ti_thread_flag(current_thread_info(),
- TIF_NOTIFY_RESUME)) {
- smp_mb__after_clear_bit();
- task_work_run();
- }
- }
+ if (unlikely(current->task_works))
+ task_work_run();
spin_lock_irq(¤t->sighand->siglock);
ptrace_do_notify(SIGTRAP, exit_code, CLD_TRAPPED);
@@ -2198,13 +2193,8 @@
struct signal_struct *signal = current->signal;
int signr;
- if (unlikely(current->task_works)) {
- if (test_and_clear_ti_thread_flag(current_thread_info(),
- TIF_NOTIFY_RESUME)) {
- smp_mb__after_clear_bit();
- task_work_run();
- }
- }
+ if (unlikely(current->task_works))
+ task_work_run();
if (unlikely(uprobe_deny_signal()))
return 0;
diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index 98f60c5..d6c5fc0 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -1,14 +1,22 @@
/*
* Common SMP CPU bringup/teardown functions
*/
+#include <linux/cpu.h>
#include <linux/err.h>
#include <linux/smp.h>
#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/sched.h>
+#include <linux/export.h>
#include <linux/percpu.h>
+#include <linux/kthread.h>
+#include <linux/smpboot.h>
#include "smpboot.h"
+#ifdef CONFIG_SMP
+
#ifdef CONFIG_GENERIC_SMP_IDLE_THREAD
/*
* For the hotplug case we keep the task structs around and reuse
@@ -65,3 +73,228 @@
}
}
#endif
+
+#endif /* #ifdef CONFIG_SMP */
+
+static LIST_HEAD(hotplug_threads);
+static DEFINE_MUTEX(smpboot_threads_lock);
+
+struct smpboot_thread_data {
+ unsigned int cpu;
+ unsigned int status;
+ struct smp_hotplug_thread *ht;
+};
+
+enum {
+ HP_THREAD_NONE = 0,
+ HP_THREAD_ACTIVE,
+ HP_THREAD_PARKED,
+};
+
+/**
+ * smpboot_thread_fn - percpu hotplug thread loop function
+ * @data: thread data pointer
+ *
+ * Checks for thread stop and park conditions. Calls the necessary
+ * setup, cleanup, park and unpark functions for the registered
+ * thread.
+ *
+ * Returns 1 when the thread should exit, 0 otherwise.
+ */
+static int smpboot_thread_fn(void *data)
+{
+ struct smpboot_thread_data *td = data;
+ struct smp_hotplug_thread *ht = td->ht;
+
+ while (1) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ preempt_disable();
+ if (kthread_should_stop()) {
+ set_current_state(TASK_RUNNING);
+ preempt_enable();
+ if (ht->cleanup)
+ ht->cleanup(td->cpu, cpu_online(td->cpu));
+ kfree(td);
+ return 0;
+ }
+
+ if (kthread_should_park()) {
+ __set_current_state(TASK_RUNNING);
+ preempt_enable();
+ if (ht->park && td->status == HP_THREAD_ACTIVE) {
+ BUG_ON(td->cpu != smp_processor_id());
+ ht->park(td->cpu);
+ td->status = HP_THREAD_PARKED;
+ }
+ kthread_parkme();
+ /* We might have been woken for stop */
+ continue;
+ }
+
+ BUG_ON(td->cpu != smp_processor_id());
+
+ /* Check for state change setup */
+ switch (td->status) {
+ case HP_THREAD_NONE:
+ preempt_enable();
+ if (ht->setup)
+ ht->setup(td->cpu);
+ td->status = HP_THREAD_ACTIVE;
+ preempt_disable();
+ break;
+ case HP_THREAD_PARKED:
+ preempt_enable();
+ if (ht->unpark)
+ ht->unpark(td->cpu);
+ td->status = HP_THREAD_ACTIVE;
+ preempt_disable();
+ break;
+ }
+
+ if (!ht->thread_should_run(td->cpu)) {
+ preempt_enable();
+ schedule();
+ } else {
+ set_current_state(TASK_RUNNING);
+ preempt_enable();
+ ht->thread_fn(td->cpu);
+ }
+ }
+}
+
+static int
+__smpboot_create_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
+{
+ struct task_struct *tsk = *per_cpu_ptr(ht->store, cpu);
+ struct smpboot_thread_data *td;
+
+ if (tsk)
+ return 0;
+
+ td = kzalloc_node(sizeof(*td), GFP_KERNEL, cpu_to_node(cpu));
+ if (!td)
+ return -ENOMEM;
+ td->cpu = cpu;
+ td->ht = ht;
+
+ tsk = kthread_create_on_cpu(smpboot_thread_fn, td, cpu,
+ ht->thread_comm);
+ if (IS_ERR(tsk)) {
+ kfree(td);
+ return PTR_ERR(tsk);
+ }
+
+ get_task_struct(tsk);
+ *per_cpu_ptr(ht->store, cpu) = tsk;
+ return 0;
+}
+
+int smpboot_create_threads(unsigned int cpu)
+{
+ struct smp_hotplug_thread *cur;
+ int ret = 0;
+
+ mutex_lock(&smpboot_threads_lock);
+ list_for_each_entry(cur, &hotplug_threads, list) {
+ ret = __smpboot_create_thread(cur, cpu);
+ if (ret)
+ break;
+ }
+ mutex_unlock(&smpboot_threads_lock);
+ return ret;
+}
+
+static void smpboot_unpark_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
+{
+ struct task_struct *tsk = *per_cpu_ptr(ht->store, cpu);
+
+ kthread_unpark(tsk);
+}
+
+void smpboot_unpark_threads(unsigned int cpu)
+{
+ struct smp_hotplug_thread *cur;
+
+ mutex_lock(&smpboot_threads_lock);
+ list_for_each_entry(cur, &hotplug_threads, list)
+ smpboot_unpark_thread(cur, cpu);
+ mutex_unlock(&smpboot_threads_lock);
+}
+
+static void smpboot_park_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
+{
+ struct task_struct *tsk = *per_cpu_ptr(ht->store, cpu);
+
+ if (tsk)
+ kthread_park(tsk);
+}
+
+void smpboot_park_threads(unsigned int cpu)
+{
+ struct smp_hotplug_thread *cur;
+
+ mutex_lock(&smpboot_threads_lock);
+ list_for_each_entry_reverse(cur, &hotplug_threads, list)
+ smpboot_park_thread(cur, cpu);
+ mutex_unlock(&smpboot_threads_lock);
+}
+
+static void smpboot_destroy_threads(struct smp_hotplug_thread *ht)
+{
+ unsigned int cpu;
+
+ /* We need to destroy also the parked threads of offline cpus */
+ for_each_possible_cpu(cpu) {
+ struct task_struct *tsk = *per_cpu_ptr(ht->store, cpu);
+
+ if (tsk) {
+ kthread_stop(tsk);
+ put_task_struct(tsk);
+ *per_cpu_ptr(ht->store, cpu) = NULL;
+ }
+ }
+}
+
+/**
+ * smpboot_register_percpu_thread - Register a per_cpu thread related to hotplug
+ * @plug_thread: Hotplug thread descriptor
+ *
+ * Creates and starts the threads on all online cpus.
+ */
+int smpboot_register_percpu_thread(struct smp_hotplug_thread *plug_thread)
+{
+ unsigned int cpu;
+ int ret = 0;
+
+ mutex_lock(&smpboot_threads_lock);
+ for_each_online_cpu(cpu) {
+ ret = __smpboot_create_thread(plug_thread, cpu);
+ if (ret) {
+ smpboot_destroy_threads(plug_thread);
+ goto out;
+ }
+ smpboot_unpark_thread(plug_thread, cpu);
+ }
+ list_add(&plug_thread->list, &hotplug_threads);
+out:
+ mutex_unlock(&smpboot_threads_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(smpboot_register_percpu_thread);
+
+/**
+ * smpboot_unregister_percpu_thread - Unregister a per_cpu thread related to hotplug
+ * @plug_thread: Hotplug thread descriptor
+ *
+ * Stops all threads on all possible cpus.
+ */
+void smpboot_unregister_percpu_thread(struct smp_hotplug_thread *plug_thread)
+{
+ get_online_cpus();
+ mutex_lock(&smpboot_threads_lock);
+ list_del(&plug_thread->list);
+ smpboot_destroy_threads(plug_thread);
+ mutex_unlock(&smpboot_threads_lock);
+ put_online_cpus();
+}
+EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
diff --git a/kernel/smpboot.h b/kernel/smpboot.h
index 6ef9433..72415a0 100644
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -13,4 +13,8 @@
static inline void idle_threads_init(void) { }
#endif
+int smpboot_create_threads(unsigned int cpu);
+void smpboot_park_threads(unsigned int cpu);
+void smpboot_unpark_threads(unsigned int cpu);
+
#endif
diff --git a/kernel/softirq.c b/kernel/softirq.c
index b73e681..cc96bdc 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -23,6 +23,7 @@
#include <linux/rcupdate.h>
#include <linux/ftrace.h>
#include <linux/smp.h>
+#include <linux/smpboot.h>
#include <linux/tick.h>
#define CREATE_TRACE_POINTS
@@ -220,7 +221,7 @@
current->flags &= ~PF_MEMALLOC;
pending = local_softirq_pending();
- account_system_vtime(current);
+ vtime_account(current);
__local_bh_disable((unsigned long)__builtin_return_address(0),
SOFTIRQ_OFFSET);
@@ -271,7 +272,7 @@
lockdep_softirq_exit();
- account_system_vtime(current);
+ vtime_account(current);
__local_bh_enable(SOFTIRQ_OFFSET);
tsk_restore_flags(current, old_flags, PF_MEMALLOC);
}
@@ -340,7 +341,7 @@
*/
void irq_exit(void)
{
- account_system_vtime(current);
+ vtime_account(current);
trace_hardirq_exit();
sub_preempt_count(IRQ_EXIT_OFFSET);
if (!in_interrupt() && local_softirq_pending())
@@ -742,49 +743,22 @@
open_softirq(HI_SOFTIRQ, tasklet_hi_action);
}
-static int run_ksoftirqd(void * __bind_cpu)
+static int ksoftirqd_should_run(unsigned int cpu)
{
- set_current_state(TASK_INTERRUPTIBLE);
+ return local_softirq_pending();
+}
- while (!kthread_should_stop()) {
- preempt_disable();
- if (!local_softirq_pending()) {
- schedule_preempt_disabled();
- }
-
- __set_current_state(TASK_RUNNING);
-
- while (local_softirq_pending()) {
- /* Preempt disable stops cpu going offline.
- If already offline, we'll be on wrong CPU:
- don't process */
- if (cpu_is_offline((long)__bind_cpu))
- goto wait_to_die;
- local_irq_disable();
- if (local_softirq_pending())
- __do_softirq();
- local_irq_enable();
- sched_preempt_enable_no_resched();
- cond_resched();
- preempt_disable();
- rcu_note_context_switch((long)__bind_cpu);
- }
- preempt_enable();
- set_current_state(TASK_INTERRUPTIBLE);
+static void run_ksoftirqd(unsigned int cpu)
+{
+ local_irq_disable();
+ if (local_softirq_pending()) {
+ __do_softirq();
+ rcu_note_context_switch(cpu);
+ local_irq_enable();
+ cond_resched();
+ return;
}
- __set_current_state(TASK_RUNNING);
- return 0;
-
-wait_to_die:
- preempt_enable();
- /* Wait for kthread_stop */
- set_current_state(TASK_INTERRUPTIBLE);
- while (!kthread_should_stop()) {
- schedule();
- set_current_state(TASK_INTERRUPTIBLE);
- }
- __set_current_state(TASK_RUNNING);
- return 0;
+ local_irq_enable();
}
#ifdef CONFIG_HOTPLUG_CPU
@@ -850,50 +824,14 @@
unsigned long action,
void *hcpu)
{
- int hotcpu = (unsigned long)hcpu;
- struct task_struct *p;
-
switch (action) {
- case CPU_UP_PREPARE:
- case CPU_UP_PREPARE_FROZEN:
- p = kthread_create_on_node(run_ksoftirqd,
- hcpu,
- cpu_to_node(hotcpu),
- "ksoftirqd/%d", hotcpu);
- if (IS_ERR(p)) {
- printk("ksoftirqd for %i failed\n", hotcpu);
- return notifier_from_errno(PTR_ERR(p));
- }
- kthread_bind(p, hotcpu);
- per_cpu(ksoftirqd, hotcpu) = p;
- break;
- case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
- wake_up_process(per_cpu(ksoftirqd, hotcpu));
- break;
#ifdef CONFIG_HOTPLUG_CPU
- case CPU_UP_CANCELED:
- case CPU_UP_CANCELED_FROZEN:
- if (!per_cpu(ksoftirqd, hotcpu))
- break;
- /* Unbind so it can run. Fall thru. */
- kthread_bind(per_cpu(ksoftirqd, hotcpu),
- cpumask_any(cpu_online_mask));
case CPU_DEAD:
- case CPU_DEAD_FROZEN: {
- static const struct sched_param param = {
- .sched_priority = MAX_RT_PRIO-1
- };
-
- p = per_cpu(ksoftirqd, hotcpu);
- per_cpu(ksoftirqd, hotcpu) = NULL;
- sched_setscheduler_nocheck(p, SCHED_FIFO, ¶m);
- kthread_stop(p);
- takeover_tasklets(hotcpu);
+ case CPU_DEAD_FROZEN:
+ takeover_tasklets((unsigned long)hcpu);
break;
- }
#endif /* CONFIG_HOTPLUG_CPU */
- }
+ }
return NOTIFY_OK;
}
@@ -901,14 +839,19 @@
.notifier_call = cpu_callback
};
+static struct smp_hotplug_thread softirq_threads = {
+ .store = &ksoftirqd,
+ .thread_should_run = ksoftirqd_should_run,
+ .thread_fn = run_ksoftirqd,
+ .thread_comm = "ksoftirqd/%u",
+};
+
static __init int spawn_ksoftirqd(void)
{
- void *cpu = (void *)(long)smp_processor_id();
- int err = cpu_callback(&cpu_nfb, CPU_UP_PREPARE, cpu);
-
- BUG_ON(err != NOTIFY_OK);
- cpu_callback(&cpu_nfb, CPU_ONLINE, cpu);
register_cpu_notifier(&cpu_nfb);
+
+ BUG_ON(smpboot_register_percpu_thread(&softirq_threads));
+
return 0;
}
early_initcall(spawn_ksoftirqd);
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index 87174ef..84c76a3 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -307,7 +307,7 @@
.extra2 = &max_sched_tunable_scaling,
},
{
- .procname = "sched_migration_cost",
+ .procname = "sched_migration_cost_ns",
.data = &sysctl_sched_migration_cost,
.maxlen = sizeof(unsigned int),
.mode = 0644,
@@ -321,14 +321,14 @@
.proc_handler = proc_dointvec,
},
{
- .procname = "sched_time_avg",
+ .procname = "sched_time_avg_ms",
.data = &sysctl_sched_time_avg,
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
- .procname = "sched_shares_window",
+ .procname = "sched_shares_window_ns",
.data = &sysctl_sched_shares_window,
.maxlen = sizeof(unsigned int),
.mode = 0644,
@@ -1544,7 +1544,7 @@
static struct ctl_table debug_table[] = {
#if defined(CONFIG_X86) || defined(CONFIG_PPC) || defined(CONFIG_SPARC) || \
- defined(CONFIG_S390) || defined(CONFIG_TILE)
+ defined(CONFIG_S390) || defined(CONFIG_TILE) || defined(CONFIG_ARM64)
{
.procname = "exception-trace",
.data = &show_unhandled_signals,
diff --git a/kernel/task_work.c b/kernel/task_work.c
index d320d44..65bd3c9 100644
--- a/kernel/task_work.c
+++ b/kernel/task_work.c
@@ -2,26 +2,20 @@
#include <linux/task_work.h>
#include <linux/tracehook.h>
+static struct callback_head work_exited; /* all we need is ->next == NULL */
+
int
-task_work_add(struct task_struct *task, struct callback_head *twork, bool notify)
+task_work_add(struct task_struct *task, struct callback_head *work, bool notify)
{
- struct callback_head *last, *first;
- unsigned long flags;
+ struct callback_head *head;
- /*
- * Not inserting the new work if the task has already passed
- * exit_task_work() is the responisbility of callers.
- */
- raw_spin_lock_irqsave(&task->pi_lock, flags);
- last = task->task_works;
- first = last ? last->next : twork;
- twork->next = first;
- if (last)
- last->next = twork;
- task->task_works = twork;
- raw_spin_unlock_irqrestore(&task->pi_lock, flags);
+ do {
+ head = ACCESS_ONCE(task->task_works);
+ if (unlikely(head == &work_exited))
+ return -ESRCH;
+ work->next = head;
+ } while (cmpxchg(&task->task_works, head, work) != head);
- /* test_and_set_bit() implies mb(), see tracehook_notify_resume(). */
if (notify)
set_notify_resume(task);
return 0;
@@ -30,52 +24,69 @@
struct callback_head *
task_work_cancel(struct task_struct *task, task_work_func_t func)
{
+ struct callback_head **pprev = &task->task_works;
+ struct callback_head *work = NULL;
unsigned long flags;
- struct callback_head *last, *res = NULL;
-
+ /*
+ * If cmpxchg() fails we continue without updating pprev.
+ * Either we raced with task_work_add() which added the
+ * new entry before this work, we will find it again. Or
+ * we raced with task_work_run(), *pprev == NULL/exited.
+ */
raw_spin_lock_irqsave(&task->pi_lock, flags);
- last = task->task_works;
- if (last) {
- struct callback_head *q = last, *p = q->next;
- while (1) {
- if (p->func == func) {
- q->next = p->next;
- if (p == last)
- task->task_works = q == p ? NULL : q;
- res = p;
- break;
- }
- if (p == last)
- break;
- q = p;
- p = q->next;
- }
+ while ((work = ACCESS_ONCE(*pprev))) {
+ read_barrier_depends();
+ if (work->func != func)
+ pprev = &work->next;
+ else if (cmpxchg(pprev, work, work->next) == work)
+ break;
}
raw_spin_unlock_irqrestore(&task->pi_lock, flags);
- return res;
+
+ return work;
}
void task_work_run(void)
{
struct task_struct *task = current;
- struct callback_head *p, *q;
+ struct callback_head *work, *head, *next;
- while (1) {
- raw_spin_lock_irq(&task->pi_lock);
- p = task->task_works;
- task->task_works = NULL;
- raw_spin_unlock_irq(&task->pi_lock);
+ for (;;) {
+ /*
+ * work->func() can do task_work_add(), do not set
+ * work_exited unless the list is empty.
+ */
+ do {
+ work = ACCESS_ONCE(task->task_works);
+ head = !work && (task->flags & PF_EXITING) ?
+ &work_exited : NULL;
+ } while (cmpxchg(&task->task_works, work, head) != work);
- if (unlikely(!p))
- return;
+ if (!work)
+ break;
+ /*
+ * Synchronize with task_work_cancel(). It can't remove
+ * the first entry == work, cmpxchg(task_works) should
+ * fail, but it can play with *work and other entries.
+ */
+ raw_spin_unlock_wait(&task->pi_lock);
+ smp_mb();
- q = p->next; /* head */
- p->next = NULL; /* cut it */
- while (q) {
- p = q->next;
- q->func(q);
- q = p;
+ /* Reverse the list to run the works in fifo order */
+ head = NULL;
+ do {
+ next = work->next;
+ work->next = head;
+ head = work;
+ work = next;
+ } while (work);
+
+ work = head;
+ do {
+ next = work->next;
+ work->func(work);
+ work = next;
cond_resched();
- }
+ } while (work);
}
}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 024540f..f423bdd0 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -372,7 +372,7 @@
* the scheduler tick in nohz_restart_sched_tick.
*/
if (!ts->tick_stopped) {
- select_nohz_load_balancer(1);
+ nohz_balance_enter_idle(cpu);
calc_load_enter_idle();
ts->last_tick = hrtimer_get_expires(&ts->sched_timer);
@@ -436,7 +436,8 @@
if (unlikely(local_softirq_pending() && cpu_online(cpu))) {
static int ratelimit;
- if (ratelimit < 10) {
+ if (ratelimit < 10 &&
+ (local_softirq_pending() & SOFTIRQ_STOP_IDLE_MASK)) {
printk(KERN_ERR "NOHZ: local_softirq_pending %02x\n",
(unsigned int) local_softirq_pending());
ratelimit++;
@@ -569,10 +570,10 @@
static void tick_nohz_restart_sched_tick(struct tick_sched *ts, ktime_t now)
{
/* Update jiffies first */
- select_nohz_load_balancer(0);
tick_do_update_jiffies64(now);
update_cpu_load_nohz();
+ calc_load_exit_idle();
touch_softlockup_watchdog();
/*
* Cancel the scheduled timer and restore the tick
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index e16af19..d3b91e7 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -115,6 +115,7 @@
{
tk->xtime_sec += ts->tv_sec;
tk->xtime_nsec += (u64)ts->tv_nsec << tk->shift;
+ tk_normalize_xtime(tk);
}
static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec wtm)
@@ -276,7 +277,7 @@
tk->xtime_nsec += cycle_delta * tk->mult;
/* If arch requires, add in gettimeoffset() */
- tk->xtime_nsec += arch_gettimeoffset() << tk->shift;
+ tk->xtime_nsec += (u64)arch_gettimeoffset() << tk->shift;
tk_normalize_xtime(tk);
@@ -302,10 +303,11 @@
seq = read_seqbegin(&tk->lock);
ts->tv_sec = tk->xtime_sec;
- ts->tv_nsec = timekeeping_get_ns(tk);
+ nsecs = timekeeping_get_ns(tk);
} while (read_seqretry(&tk->lock, seq));
+ ts->tv_nsec = 0;
timespec_add_ns(ts, nsecs);
}
EXPORT_SYMBOL(getnstimeofday);
@@ -344,6 +346,7 @@
{
struct timekeeper *tk = &timekeeper;
struct timespec tomono;
+ s64 nsec;
unsigned int seq;
WARN_ON(timekeeping_suspended);
@@ -351,13 +354,14 @@
do {
seq = read_seqbegin(&tk->lock);
ts->tv_sec = tk->xtime_sec;
- ts->tv_nsec = timekeeping_get_ns(tk);
+ nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
} while (read_seqretry(&tk->lock, seq));
- set_normalized_timespec(ts, ts->tv_sec + tomono.tv_sec,
- ts->tv_nsec + tomono.tv_nsec);
+ ts->tv_sec += tomono.tv_sec;
+ ts->tv_nsec = 0;
+ timespec_add_ns(ts, nsec + tomono.tv_nsec);
}
EXPORT_SYMBOL_GPL(ktime_get_ts);
@@ -427,7 +431,7 @@
struct timespec ts_delta, xt;
unsigned long flags;
- if ((unsigned long)tv->tv_nsec >= NSEC_PER_SEC)
+ if (!timespec_valid_strict(tv))
return -EINVAL;
write_seqlock_irqsave(&tk->lock, flags);
@@ -463,6 +467,8 @@
{
struct timekeeper *tk = &timekeeper;
unsigned long flags;
+ struct timespec tmp;
+ int ret = 0;
if ((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)
return -EINVAL;
@@ -471,10 +477,17 @@
timekeeping_forward_now(tk);
+ /* Make sure the proposed value is valid */
+ tmp = timespec_add(tk_xtime(tk), *ts);
+ if (!timespec_valid_strict(&tmp)) {
+ ret = -EINVAL;
+ goto error;
+ }
tk_xtime_add(tk, ts);
tk_set_wall_to_mono(tk, timespec_sub(tk->wall_to_monotonic, *ts));
+error: /* even if we error out, we forwarded the time, so call update */
timekeeping_update(tk, true);
write_sequnlock_irqrestore(&tk->lock, flags);
@@ -482,7 +495,7 @@
/* signal hrtimers about time change */
clock_was_set();
- return 0;
+ return ret;
}
EXPORT_SYMBOL(timekeeping_inject_offset);
@@ -649,7 +662,20 @@
struct timespec now, boot, tmp;
read_persistent_clock(&now);
+ if (!timespec_valid_strict(&now)) {
+ pr_warn("WARNING: Persistent clock returned invalid value!\n"
+ " Check your CMOS/BIOS settings.\n");
+ now.tv_sec = 0;
+ now.tv_nsec = 0;
+ }
+
read_boot_clock(&boot);
+ if (!timespec_valid_strict(&boot)) {
+ pr_warn("WARNING: Boot clock returned invalid value!\n"
+ " Check your CMOS/BIOS settings.\n");
+ boot.tv_sec = 0;
+ boot.tv_nsec = 0;
+ }
seqlock_init(&tk->lock);
@@ -690,7 +716,7 @@
static void __timekeeping_inject_sleeptime(struct timekeeper *tk,
struct timespec *delta)
{
- if (!timespec_valid(delta)) {
+ if (!timespec_valid_strict(delta)) {
printk(KERN_WARNING "__timekeeping_inject_sleeptime: Invalid "
"sleep delta value!\n");
return;
@@ -1129,6 +1155,10 @@
offset = (clock->read(clock) - clock->cycle_last) & clock->mask;
#endif
+ /* Check if there's really nothing to do */
+ if (offset < tk->cycle_interval)
+ goto out;
+
/*
* With NO_HZ we may have to accumulate many cycle_intervals
* (think "ticks") worth of time at once. To do this efficiently,
@@ -1161,9 +1191,9 @@
* the vsyscall implementations are converted to use xtime_nsec
* (shifted nanoseconds), this can be killed.
*/
- remainder = tk->xtime_nsec & ((1 << tk->shift) - 1);
+ remainder = tk->xtime_nsec & ((1ULL << tk->shift) - 1);
tk->xtime_nsec -= remainder;
- tk->xtime_nsec += 1 << tk->shift;
+ tk->xtime_nsec += 1ULL << tk->shift;
tk->ntp_error += remainder << tk->ntp_error_shift;
/*
@@ -1217,6 +1247,7 @@
{
struct timekeeper *tk = &timekeeper;
struct timespec tomono, sleep;
+ s64 nsec;
unsigned int seq;
WARN_ON(timekeeping_suspended);
@@ -1224,14 +1255,15 @@
do {
seq = read_seqbegin(&tk->lock);
ts->tv_sec = tk->xtime_sec;
- ts->tv_nsec = timekeeping_get_ns(tk);
+ nsec = timekeeping_get_ns(tk);
tomono = tk->wall_to_monotonic;
sleep = tk->total_sleep_time;
} while (read_seqretry(&tk->lock, seq));
- set_normalized_timespec(ts, ts->tv_sec + tomono.tv_sec + sleep.tv_sec,
- ts->tv_nsec + tomono.tv_nsec + sleep.tv_nsec);
+ ts->tv_sec += tomono.tv_sec + sleep.tv_sec;
+ ts->tv_nsec = 0;
+ timespec_add_ns(ts, nsec + tomono.tv_nsec + sleep.tv_nsec);
}
EXPORT_SYMBOL_GPL(get_monotonic_boottime);
diff --git a/kernel/timer.c b/kernel/timer.c
index 8c5e7b9..d5de1b2 100644
--- a/kernel/timer.c
+++ b/kernel/timer.c
@@ -92,24 +92,25 @@
/* Functions below help us manage 'deferrable' flag */
static inline unsigned int tbase_get_deferrable(struct tvec_base *base)
{
- return ((unsigned int)(unsigned long)base & TBASE_DEFERRABLE_FLAG);
+ return ((unsigned int)(unsigned long)base & TIMER_DEFERRABLE);
+}
+
+static inline unsigned int tbase_get_irqsafe(struct tvec_base *base)
+{
+ return ((unsigned int)(unsigned long)base & TIMER_IRQSAFE);
}
static inline struct tvec_base *tbase_get_base(struct tvec_base *base)
{
- return ((struct tvec_base *)((unsigned long)base & ~TBASE_DEFERRABLE_FLAG));
-}
-
-static inline void timer_set_deferrable(struct timer_list *timer)
-{
- timer->base = TBASE_MAKE_DEFERRED(timer->base);
+ return ((struct tvec_base *)((unsigned long)base & ~TIMER_FLAG_MASK));
}
static inline void
timer_set_base(struct timer_list *timer, struct tvec_base *new_base)
{
- timer->base = (struct tvec_base *)((unsigned long)(new_base) |
- tbase_get_deferrable(timer->base));
+ unsigned long flags = (unsigned long)timer->base & TIMER_FLAG_MASK;
+
+ timer->base = (struct tvec_base *)((unsigned long)(new_base) | flags);
}
static unsigned long round_jiffies_common(unsigned long j, int cpu,
@@ -563,16 +564,14 @@
debug_object_assert_init(timer, &timer_debug_descr);
}
-static void __init_timer(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key);
+static void do_init_timer(struct timer_list *timer, unsigned int flags,
+ const char *name, struct lock_class_key *key);
-void init_timer_on_stack_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key)
+void init_timer_on_stack_key(struct timer_list *timer, unsigned int flags,
+ const char *name, struct lock_class_key *key)
{
debug_object_init_on_stack(timer, &timer_debug_descr);
- __init_timer(timer, name, key);
+ do_init_timer(timer, flags, name, key);
}
EXPORT_SYMBOL_GPL(init_timer_on_stack_key);
@@ -613,12 +612,13 @@
debug_timer_assert_init(timer);
}
-static void __init_timer(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key)
+static void do_init_timer(struct timer_list *timer, unsigned int flags,
+ const char *name, struct lock_class_key *key)
{
+ struct tvec_base *base = __raw_get_cpu_var(tvec_bases);
+
timer->entry.next = NULL;
- timer->base = __raw_get_cpu_var(tvec_bases);
+ timer->base = (void *)((unsigned long)base | flags);
timer->slack = -1;
#ifdef CONFIG_TIMER_STATS
timer->start_site = NULL;
@@ -628,22 +628,10 @@
lockdep_init_map(&timer->lockdep_map, name, key, 0);
}
-void setup_deferrable_timer_on_stack_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key,
- void (*function)(unsigned long),
- unsigned long data)
-{
- timer->function = function;
- timer->data = data;
- init_timer_on_stack_key(timer, name, key);
- timer_set_deferrable(timer);
-}
-EXPORT_SYMBOL_GPL(setup_deferrable_timer_on_stack_key);
-
/**
* init_timer_key - initialize a timer
* @timer: the timer to be initialized
+ * @flags: timer flags
* @name: name of the timer
* @key: lockdep class key of the fake lock used for tracking timer
* sync lock dependencies
@@ -651,24 +639,14 @@
* init_timer_key() must be done to a timer prior calling *any* of the
* other timer functions.
*/
-void init_timer_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key)
+void init_timer_key(struct timer_list *timer, unsigned int flags,
+ const char *name, struct lock_class_key *key)
{
debug_init(timer);
- __init_timer(timer, name, key);
+ do_init_timer(timer, flags, name, key);
}
EXPORT_SYMBOL(init_timer_key);
-void init_timer_deferrable_key(struct timer_list *timer,
- const char *name,
- struct lock_class_key *key)
-{
- init_timer_key(timer, name, key);
- timer_set_deferrable(timer);
-}
-EXPORT_SYMBOL(init_timer_deferrable_key);
-
static inline void detach_timer(struct timer_list *timer, bool clear_pending)
{
struct list_head *entry = &timer->entry;
@@ -686,7 +664,7 @@
{
detach_timer(timer, true);
if (!tbase_get_deferrable(timer->base))
- timer->base->active_timers--;
+ base->active_timers--;
}
static int detach_if_pending(struct timer_list *timer, struct tvec_base *base,
@@ -697,7 +675,7 @@
detach_timer(timer, clear_pending);
if (!tbase_get_deferrable(timer->base)) {
- timer->base->active_timers--;
+ base->active_timers--;
if (timer->expires == base->next_timer)
base->next_timer = base->timer_jiffies;
}
@@ -1029,14 +1007,14 @@
*
* Synchronization rules: Callers must prevent restarting of the timer,
* otherwise this function is meaningless. It must not be called from
- * interrupt contexts. The caller must not hold locks which would prevent
- * completion of the timer's handler. The timer's handler must not call
- * add_timer_on(). Upon exit the timer is not queued and the handler is
- * not running on any CPU.
+ * interrupt contexts unless the timer is an irqsafe one. The caller must
+ * not hold locks which would prevent completion of the timer's
+ * handler. The timer's handler must not call add_timer_on(). Upon exit the
+ * timer is not queued and the handler is not running on any CPU.
*
- * Note: You must not hold locks that are held in interrupt context
- * while calling this function. Even if the lock has nothing to do
- * with the timer in question. Here's why:
+ * Note: For !irqsafe timers, you must not hold locks that are held in
+ * interrupt context while calling this function. Even if the lock has
+ * nothing to do with the timer in question. Here's why:
*
* CPU0 CPU1
* ---- ----
@@ -1073,7 +1051,7 @@
* don't use it in hardirq context, because it
* could lead to deadlock.
*/
- WARN_ON(in_irq());
+ WARN_ON(in_irq() && !tbase_get_irqsafe(timer->base));
for (;;) {
int ret = try_to_del_timer_sync(timer);
if (ret >= 0)
@@ -1180,19 +1158,27 @@
while (!list_empty(head)) {
void (*fn)(unsigned long);
unsigned long data;
+ bool irqsafe;
timer = list_first_entry(head, struct timer_list,entry);
fn = timer->function;
data = timer->data;
+ irqsafe = tbase_get_irqsafe(timer->base);
timer_stats_account_timer(timer);
base->running_timer = timer;
detach_expired_timer(timer, base);
- spin_unlock_irq(&base->lock);
- call_timer_fn(timer, fn, data);
- spin_lock_irq(&base->lock);
+ if (irqsafe) {
+ spin_unlock(&base->lock);
+ call_timer_fn(timer, fn, data);
+ spin_lock(&base->lock);
+ } else {
+ spin_unlock_irq(&base->lock);
+ call_timer_fn(timer, fn, data);
+ spin_lock_irq(&base->lock);
+ }
}
}
base->running_timer = NULL;
@@ -1791,9 +1777,13 @@
void __init init_timers(void)
{
- int err = timer_cpu_notify(&timers_nb, (unsigned long)CPU_UP_PREPARE,
- (void *)(long)smp_processor_id());
+ int err;
+ /* ensure there are enough low bits for flags in timer->base pointer */
+ BUILD_BUG_ON(__alignof__(struct tvec_base) & TIMER_FLAG_MASK);
+
+ err = timer_cpu_notify(&timers_nb, (unsigned long)CPU_UP_PREPARE,
+ (void *)(long)smp_processor_id());
init_timer_stats();
BUG_ON(err != NOTIFY_OK);
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 8c4c070..4cea4f4 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -49,6 +49,11 @@
help
See Documentation/trace/ftrace-design.txt
+config HAVE_FENTRY
+ bool
+ help
+ Arch supports the gcc options -pg with -mfentry
+
config HAVE_C_RECORDMCOUNT
bool
help
@@ -57,8 +62,12 @@
config TRACER_MAX_TRACE
bool
+config TRACE_CLOCK
+ bool
+
config RING_BUFFER
bool
+ select TRACE_CLOCK
config FTRACE_NMI_ENTER
bool
@@ -109,6 +118,7 @@
select NOP_TRACER
select BINARY_PRINTF
select EVENT_TRACING
+ select TRACE_CLOCK
config GENERIC_TRACER
bool
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index b831087..d7e2068 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -5,10 +5,12 @@
ORIG_CFLAGS := $(KBUILD_CFLAGS)
KBUILD_CFLAGS = $(subst -pg,,$(ORIG_CFLAGS))
+ifdef CONFIG_FTRACE_SELFTEST
# selftest needs instrumentation
CFLAGS_trace_selftest_dynamic.o = -pg
obj-y += trace_selftest_dynamic.o
endif
+endif
# If unlikely tracing is enabled, do not trace these files
ifdef CONFIG_TRACING_BRANCHES
@@ -17,11 +19,7 @@
CFLAGS_trace_events_filter.o := -I$(src)
-#
-# Make the trace clocks available generally: it's infrastructure
-# relied on by ptrace for example:
-#
-obj-y += trace_clock.o
+obj-$(CONFIG_TRACE_CLOCK) += trace_clock.o
obj-$(CONFIG_FUNCTION_TRACER) += libftrace.o
obj-$(CONFIG_RING_BUFFER) += ring_buffer.o
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index b4f20fb..9dcf15d 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -64,12 +64,20 @@
#define FL_GLOBAL_CONTROL_MASK (FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_CONTROL)
+static struct ftrace_ops ftrace_list_end __read_mostly = {
+ .func = ftrace_stub,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
+};
+
/* ftrace_enabled is a method to turn ftrace on or off */
int ftrace_enabled __read_mostly;
static int last_ftrace_enabled;
/* Quick disabling of function tracer. */
-int function_trace_stop;
+int function_trace_stop __read_mostly;
+
+/* Current function tracing op */
+struct ftrace_ops *function_trace_op __read_mostly = &ftrace_list_end;
/* List for set_ftrace_pid's pids. */
LIST_HEAD(ftrace_pids);
@@ -86,22 +94,43 @@
static DEFINE_MUTEX(ftrace_lock);
-static struct ftrace_ops ftrace_list_end __read_mostly = {
- .func = ftrace_stub,
-};
-
static struct ftrace_ops *ftrace_global_list __read_mostly = &ftrace_list_end;
static struct ftrace_ops *ftrace_control_list __read_mostly = &ftrace_list_end;
static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
-static ftrace_func_t __ftrace_trace_function_delay __read_mostly = ftrace_stub;
-ftrace_func_t __ftrace_trace_function __read_mostly = ftrace_stub;
ftrace_func_t ftrace_pid_function __read_mostly = ftrace_stub;
static struct ftrace_ops global_ops;
static struct ftrace_ops control_ops;
-static void
-ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip);
+#if ARCH_SUPPORTS_FTRACE_OPS
+static void ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *regs);
+#else
+/* See comment below, where ftrace_ops_list_func is defined */
+static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip);
+#define ftrace_ops_list_func ((ftrace_func_t)ftrace_ops_no_ops)
+#endif
+
+/**
+ * ftrace_nr_registered_ops - return number of ops registered
+ *
+ * Returns the number of ftrace_ops registered and tracing functions
+ */
+int ftrace_nr_registered_ops(void)
+{
+ struct ftrace_ops *ops;
+ int cnt = 0;
+
+ mutex_lock(&ftrace_lock);
+
+ for (ops = ftrace_ops_list;
+ ops != &ftrace_list_end; ops = ops->next)
+ cnt++;
+
+ mutex_unlock(&ftrace_lock);
+
+ return cnt;
+}
/*
* Traverse the ftrace_global_list, invoking all entries. The reason that we
@@ -112,29 +141,29 @@
*
* Silly Alpha and silly pointer-speculation compiler optimizations!
*/
-static void ftrace_global_list_func(unsigned long ip,
- unsigned long parent_ip)
+static void
+ftrace_global_list_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *regs)
{
- struct ftrace_ops *op;
-
if (unlikely(trace_recursion_test(TRACE_GLOBAL_BIT)))
return;
trace_recursion_set(TRACE_GLOBAL_BIT);
op = rcu_dereference_raw(ftrace_global_list); /*see above*/
while (op != &ftrace_list_end) {
- op->func(ip, parent_ip);
+ op->func(ip, parent_ip, op, regs);
op = rcu_dereference_raw(op->next); /*see above*/
};
trace_recursion_clear(TRACE_GLOBAL_BIT);
}
-static void ftrace_pid_func(unsigned long ip, unsigned long parent_ip)
+static void ftrace_pid_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *regs)
{
if (!test_tsk_trace_trace(current))
return;
- ftrace_pid_function(ip, parent_ip);
+ ftrace_pid_function(ip, parent_ip, op, regs);
}
static void set_ftrace_pid_function(ftrace_func_t func)
@@ -153,25 +182,9 @@
void clear_ftrace_function(void)
{
ftrace_trace_function = ftrace_stub;
- __ftrace_trace_function = ftrace_stub;
- __ftrace_trace_function_delay = ftrace_stub;
ftrace_pid_function = ftrace_stub;
}
-#ifndef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
-/*
- * For those archs that do not test ftrace_trace_stop in their
- * mcount call site, we need to do it from C.
- */
-static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip)
-{
- if (function_trace_stop)
- return;
-
- __ftrace_trace_function(ip, parent_ip);
-}
-#endif
-
static void control_ops_disable_all(struct ftrace_ops *ops)
{
int cpu;
@@ -230,28 +243,27 @@
/*
* If we are at the end of the list and this ops is
- * not dynamic, then have the mcount trampoline call
- * the function directly
+ * recursion safe and not dynamic and the arch supports passing ops,
+ * then have the mcount trampoline call the function directly.
*/
if (ftrace_ops_list == &ftrace_list_end ||
(ftrace_ops_list->next == &ftrace_list_end &&
- !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC)))
+ !(ftrace_ops_list->flags & FTRACE_OPS_FL_DYNAMIC) &&
+ (ftrace_ops_list->flags & FTRACE_OPS_FL_RECURSION_SAFE) &&
+ !FTRACE_FORCE_LIST_FUNC)) {
+ /* Set the ftrace_ops that the arch callback uses */
+ if (ftrace_ops_list == &global_ops)
+ function_trace_op = ftrace_global_list;
+ else
+ function_trace_op = ftrace_ops_list;
func = ftrace_ops_list->func;
- else
+ } else {
+ /* Just use the default ftrace_ops */
+ function_trace_op = &ftrace_list_end;
func = ftrace_ops_list_func;
+ }
-#ifdef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
ftrace_trace_function = func;
-#else
-#ifdef CONFIG_DYNAMIC_FTRACE
- /* do not update till all functions have been modified */
- __ftrace_trace_function_delay = func;
-#else
- __ftrace_trace_function = func;
-#endif
- ftrace_trace_function =
- (func == ftrace_stub) ? func : ftrace_test_stop_func;
-#endif
}
static void add_ftrace_ops(struct ftrace_ops **list, struct ftrace_ops *ops)
@@ -325,6 +337,20 @@
if ((ops->flags & FL_GLOBAL_CONTROL_MASK) == FL_GLOBAL_CONTROL_MASK)
return -EINVAL;
+#ifndef ARCH_SUPPORTS_FTRACE_SAVE_REGS
+ /*
+ * If the ftrace_ops specifies SAVE_REGS, then it only can be used
+ * if the arch supports it, or SAVE_REGS_IF_SUPPORTED is also set.
+ * Setting SAVE_REGS_IF_SUPPORTED makes SAVE_REGS irrelevant.
+ */
+ if (ops->flags & FTRACE_OPS_FL_SAVE_REGS &&
+ !(ops->flags & FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED))
+ return -EINVAL;
+
+ if (ops->flags & FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED)
+ ops->flags |= FTRACE_OPS_FL_SAVE_REGS;
+#endif
+
if (!core_kernel_data((unsigned long)ops))
ops->flags |= FTRACE_OPS_FL_DYNAMIC;
@@ -773,7 +799,8 @@
}
static void
-function_profile_call(unsigned long ip, unsigned long parent_ip)
+function_profile_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *ops, struct pt_regs *regs)
{
struct ftrace_profile_stat *stat;
struct ftrace_profile *rec;
@@ -803,7 +830,7 @@
#ifdef CONFIG_FUNCTION_GRAPH_TRACER
static int profile_graph_entry(struct ftrace_graph_ent *trace)
{
- function_profile_call(trace->func, 0);
+ function_profile_call(trace->func, 0, NULL, NULL);
return 1;
}
@@ -863,6 +890,7 @@
#else
static struct ftrace_ops ftrace_profile_ops __read_mostly = {
.func = function_profile_call,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static int register_ftrace_profiler(void)
@@ -1045,6 +1073,7 @@
.func = ftrace_stub,
.notrace_hash = EMPTY_HASH,
.filter_hash = EMPTY_HASH,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static DEFINE_MUTEX(ftrace_regex_lock);
@@ -1525,6 +1554,12 @@
rec->flags++;
if (FTRACE_WARN_ON((rec->flags & ~FTRACE_FL_MASK) == FTRACE_REF_MAX))
return;
+ /*
+ * If any ops wants regs saved for this function
+ * then all ops will get saved regs.
+ */
+ if (ops->flags & FTRACE_OPS_FL_SAVE_REGS)
+ rec->flags |= FTRACE_FL_REGS;
} else {
if (FTRACE_WARN_ON((rec->flags & ~FTRACE_FL_MASK) == 0))
return;
@@ -1616,18 +1651,59 @@
if (enable && (rec->flags & ~FTRACE_FL_MASK))
flag = FTRACE_FL_ENABLED;
+ /*
+ * If enabling and the REGS flag does not match the REGS_EN, then
+ * do not ignore this record. Set flags to fail the compare against
+ * ENABLED.
+ */
+ if (flag &&
+ (!(rec->flags & FTRACE_FL_REGS) != !(rec->flags & FTRACE_FL_REGS_EN)))
+ flag |= FTRACE_FL_REGS;
+
/* If the state of this record hasn't changed, then do nothing */
if ((rec->flags & FTRACE_FL_ENABLED) == flag)
return FTRACE_UPDATE_IGNORE;
if (flag) {
- if (update)
+ /* Save off if rec is being enabled (for return value) */
+ flag ^= rec->flags & FTRACE_FL_ENABLED;
+
+ if (update) {
rec->flags |= FTRACE_FL_ENABLED;
- return FTRACE_UPDATE_MAKE_CALL;
+ if (flag & FTRACE_FL_REGS) {
+ if (rec->flags & FTRACE_FL_REGS)
+ rec->flags |= FTRACE_FL_REGS_EN;
+ else
+ rec->flags &= ~FTRACE_FL_REGS_EN;
+ }
+ }
+
+ /*
+ * If this record is being updated from a nop, then
+ * return UPDATE_MAKE_CALL.
+ * Otherwise, if the EN flag is set, then return
+ * UPDATE_MODIFY_CALL_REGS to tell the caller to convert
+ * from the non-save regs, to a save regs function.
+ * Otherwise,
+ * return UPDATE_MODIFY_CALL to tell the caller to convert
+ * from the save regs, to a non-save regs function.
+ */
+ if (flag & FTRACE_FL_ENABLED)
+ return FTRACE_UPDATE_MAKE_CALL;
+ else if (rec->flags & FTRACE_FL_REGS_EN)
+ return FTRACE_UPDATE_MODIFY_CALL_REGS;
+ else
+ return FTRACE_UPDATE_MODIFY_CALL;
}
- if (update)
- rec->flags &= ~FTRACE_FL_ENABLED;
+ if (update) {
+ /* If there's no more users, clear all flags */
+ if (!(rec->flags & ~FTRACE_FL_MASK))
+ rec->flags = 0;
+ else
+ /* Just disable the record (keep REGS state) */
+ rec->flags &= ~FTRACE_FL_ENABLED;
+ }
return FTRACE_UPDATE_MAKE_NOP;
}
@@ -1662,13 +1738,17 @@
static int
__ftrace_replace_code(struct dyn_ftrace *rec, int enable)
{
+ unsigned long ftrace_old_addr;
unsigned long ftrace_addr;
int ret;
- ftrace_addr = (unsigned long)FTRACE_ADDR;
-
ret = ftrace_update_record(rec, enable);
+ if (rec->flags & FTRACE_FL_REGS)
+ ftrace_addr = (unsigned long)FTRACE_REGS_ADDR;
+ else
+ ftrace_addr = (unsigned long)FTRACE_ADDR;
+
switch (ret) {
case FTRACE_UPDATE_IGNORE:
return 0;
@@ -1678,6 +1758,15 @@
case FTRACE_UPDATE_MAKE_NOP:
return ftrace_make_nop(NULL, rec, ftrace_addr);
+
+ case FTRACE_UPDATE_MODIFY_CALL_REGS:
+ case FTRACE_UPDATE_MODIFY_CALL:
+ if (rec->flags & FTRACE_FL_REGS)
+ ftrace_old_addr = (unsigned long)FTRACE_ADDR;
+ else
+ ftrace_old_addr = (unsigned long)FTRACE_REGS_ADDR;
+
+ return ftrace_modify_call(rec, ftrace_old_addr, ftrace_addr);
}
return -1; /* unknow ftrace bug */
@@ -1882,16 +1971,6 @@
*/
arch_ftrace_update_code(command);
-#ifndef CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST
- /*
- * For archs that call ftrace_test_stop_func(), we must
- * wait till after we update all the function callers
- * before we update the callback. This keeps different
- * ops that record different functions from corrupting
- * each other.
- */
- __ftrace_trace_function = __ftrace_trace_function_delay;
-#endif
function_trace_stop--;
ret = ftrace_arch_code_modify_post_process();
@@ -2441,8 +2520,9 @@
seq_printf(m, "%ps", (void *)rec->ip);
if (iter->flags & FTRACE_ITER_ENABLED)
- seq_printf(m, " (%ld)",
- rec->flags & ~FTRACE_FL_MASK);
+ seq_printf(m, " (%ld)%s",
+ rec->flags & ~FTRACE_FL_MASK,
+ rec->flags & FTRACE_FL_REGS ? " R" : "");
seq_printf(m, "\n");
return 0;
@@ -2790,8 +2870,8 @@
}
device_initcall(ftrace_mod_cmd_init);
-static void
-function_trace_probe_call(unsigned long ip, unsigned long parent_ip)
+static void function_trace_probe_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
struct ftrace_func_probe *entry;
struct hlist_head *hhd;
@@ -3162,8 +3242,27 @@
}
static int
-ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
- int reset, int enable)
+ftrace_match_addr(struct ftrace_hash *hash, unsigned long ip, int remove)
+{
+ struct ftrace_func_entry *entry;
+
+ if (!ftrace_location(ip))
+ return -EINVAL;
+
+ if (remove) {
+ entry = ftrace_lookup_ip(hash, ip);
+ if (!entry)
+ return -ENOENT;
+ free_hash_entry(hash, entry);
+ return 0;
+ }
+
+ return add_hash_entry(hash, ip);
+}
+
+static int
+ftrace_set_hash(struct ftrace_ops *ops, unsigned char *buf, int len,
+ unsigned long ip, int remove, int reset, int enable)
{
struct ftrace_hash **orig_hash;
struct ftrace_hash *hash;
@@ -3192,6 +3291,11 @@
ret = -EINVAL;
goto out_regex_unlock;
}
+ if (ip) {
+ ret = ftrace_match_addr(hash, ip, remove);
+ if (ret < 0)
+ goto out_regex_unlock;
+ }
mutex_lock(&ftrace_lock);
ret = ftrace_hash_move(ops, enable, orig_hash, hash);
@@ -3208,6 +3312,37 @@
return ret;
}
+static int
+ftrace_set_addr(struct ftrace_ops *ops, unsigned long ip, int remove,
+ int reset, int enable)
+{
+ return ftrace_set_hash(ops, 0, 0, ip, remove, reset, enable);
+}
+
+/**
+ * ftrace_set_filter_ip - set a function to filter on in ftrace by address
+ * @ops - the ops to set the filter with
+ * @ip - the address to add to or remove from the filter.
+ * @remove - non zero to remove the ip from the filter
+ * @reset - non zero to reset all filters before applying this filter.
+ *
+ * Filters denote which functions should be enabled when tracing is enabled
+ * If @ip is NULL, it failes to update filter.
+ */
+int ftrace_set_filter_ip(struct ftrace_ops *ops, unsigned long ip,
+ int remove, int reset)
+{
+ return ftrace_set_addr(ops, ip, remove, reset, 1);
+}
+EXPORT_SYMBOL_GPL(ftrace_set_filter_ip);
+
+static int
+ftrace_set_regex(struct ftrace_ops *ops, unsigned char *buf, int len,
+ int reset, int enable)
+{
+ return ftrace_set_hash(ops, buf, len, 0, 0, reset, enable);
+}
+
/**
* ftrace_set_filter - set a function to filter on in ftrace
* @ops - the ops to set the filter with
@@ -3912,6 +4047,7 @@
static struct ftrace_ops global_ops = {
.func = ftrace_stub,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static int __init ftrace_nodyn_init(void)
@@ -3942,10 +4078,9 @@
#endif /* CONFIG_DYNAMIC_FTRACE */
static void
-ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip)
+ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *regs)
{
- struct ftrace_ops *op;
-
if (unlikely(trace_recursion_test(TRACE_CONTROL_BIT)))
return;
@@ -3959,7 +4094,7 @@
while (op != &ftrace_list_end) {
if (!ftrace_function_local_disabled(op) &&
ftrace_ops_test(op, ip))
- op->func(ip, parent_ip);
+ op->func(ip, parent_ip, op, regs);
op = rcu_dereference_raw(op->next);
};
@@ -3969,13 +4104,18 @@
static struct ftrace_ops control_ops = {
.func = ftrace_ops_control_func,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
-static void
-ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip)
+static inline void
+__ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *ignored, struct pt_regs *regs)
{
struct ftrace_ops *op;
+ if (function_trace_stop)
+ return;
+
if (unlikely(trace_recursion_test(TRACE_INTERNAL_BIT)))
return;
@@ -3988,13 +4128,39 @@
op = rcu_dereference_raw(ftrace_ops_list);
while (op != &ftrace_list_end) {
if (ftrace_ops_test(op, ip))
- op->func(ip, parent_ip);
+ op->func(ip, parent_ip, op, regs);
op = rcu_dereference_raw(op->next);
};
preempt_enable_notrace();
trace_recursion_clear(TRACE_INTERNAL_BIT);
}
+/*
+ * Some archs only support passing ip and parent_ip. Even though
+ * the list function ignores the op parameter, we do not want any
+ * C side effects, where a function is called without the caller
+ * sending a third parameter.
+ * Archs are to support both the regs and ftrace_ops at the same time.
+ * If they support ftrace_ops, it is assumed they support regs.
+ * If call backs want to use regs, they must either check for regs
+ * being NULL, or ARCH_SUPPORTS_FTRACE_SAVE_REGS.
+ * Note, ARCH_SUPPORT_SAVE_REGS expects a full regs to be saved.
+ * An architecture can pass partial regs with ftrace_ops and still
+ * set the ARCH_SUPPORT_FTARCE_OPS.
+ */
+#if ARCH_SUPPORTS_FTRACE_OPS
+static void ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *regs)
+{
+ __ftrace_ops_list_func(ip, parent_ip, NULL, regs);
+}
+#else
+static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)
+{
+ __ftrace_ops_list_func(ip, parent_ip, NULL, NULL);
+}
+#endif
+
static void clear_ftrace_swapper(void)
{
struct task_struct *p;
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 49491fa..b32ed0e 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -2816,7 +2816,7 @@
* to the buffer after this will fail and return NULL.
*
* This is different than ring_buffer_record_disable() as
- * it works like an on/off switch, where as the disable() verison
+ * it works like an on/off switch, where as the disable() version
* must be paired with a enable().
*/
void ring_buffer_record_off(struct ring_buffer *buffer)
@@ -2839,7 +2839,7 @@
* ring_buffer_record_off().
*
* This is different than ring_buffer_record_enable() as
- * it works like an on/off switch, where as the enable() verison
+ * it works like an on/off switch, where as the enable() version
* must be paired with a disable().
*/
void ring_buffer_record_on(struct ring_buffer *buffer)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 5c38c81..1ec5c1d 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -328,7 +328,7 @@
unsigned long trace_flags = TRACE_ITER_PRINT_PARENT | TRACE_ITER_PRINTK |
TRACE_ITER_ANNOTATE | TRACE_ITER_CONTEXT_INFO | TRACE_ITER_SLEEP_TIME |
TRACE_ITER_GRAPH_TIME | TRACE_ITER_RECORD_CMD | TRACE_ITER_OVERWRITE |
- TRACE_ITER_IRQ_INFO;
+ TRACE_ITER_IRQ_INFO | TRACE_ITER_MARKERS;
static int trace_stop_count;
static DEFINE_RAW_SPINLOCK(tracing_start_lock);
@@ -426,15 +426,15 @@
static int __init set_tracing_thresh(char *str)
{
- unsigned long threshhold;
+ unsigned long threshold;
int ret;
if (!str)
return 0;
- ret = strict_strtoul(str, 0, &threshhold);
+ ret = strict_strtoul(str, 0, &threshold);
if (ret < 0)
return 0;
- tracing_thresh = threshhold * 1000;
+ tracing_thresh = threshold * 1000;
return 1;
}
__setup("tracing_thresh=", set_tracing_thresh);
@@ -470,6 +470,7 @@
"overwrite",
"disable_on_free",
"irq-info",
+ "markers",
NULL
};
@@ -3886,6 +3887,9 @@
if (tracing_disabled)
return -EINVAL;
+ if (!(trace_flags & TRACE_ITER_MARKERS))
+ return -EINVAL;
+
if (cnt > TRACE_BUF_SIZE)
cnt = TRACE_BUF_SIZE;
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 55e1f7f..63a2da0 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -472,11 +472,11 @@
#ifdef CONFIG_DYNAMIC_FTRACE
extern unsigned long ftrace_update_tot_cnt;
+#endif
#define DYN_FTRACE_TEST_NAME trace_selftest_dynamic_test_func
extern int DYN_FTRACE_TEST_NAME(void);
#define DYN_FTRACE_TEST_NAME2 trace_selftest_dynamic_test_func2
extern int DYN_FTRACE_TEST_NAME2(void);
-#endif
extern int ring_buffer_expanded;
extern bool tracing_selftest_disabled;
@@ -680,6 +680,7 @@
TRACE_ITER_OVERWRITE = 0x200000,
TRACE_ITER_STOP_ON_FREE = 0x400000,
TRACE_ITER_IRQ_INFO = 0x800000,
+ TRACE_ITER_MARKERS = 0x1000000,
};
/*
diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
index 8a6d2ee..84b1e04 100644
--- a/kernel/trace/trace_event_perf.c
+++ b/kernel/trace/trace_event_perf.c
@@ -258,7 +258,8 @@
#ifdef CONFIG_FUNCTION_TRACER
static void
-perf_ftrace_function_call(unsigned long ip, unsigned long parent_ip)
+perf_ftrace_function_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *ops, struct pt_regs *pt_regs)
{
struct ftrace_entry *entry;
struct hlist_head *head;
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c
index 29111da..d608d09 100644
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -1199,6 +1199,31 @@
return 0;
}
+static void event_remove(struct ftrace_event_call *call)
+{
+ ftrace_event_enable_disable(call, 0);
+ if (call->event.funcs)
+ __unregister_ftrace_event(&call->event);
+ list_del(&call->list);
+}
+
+static int event_init(struct ftrace_event_call *call)
+{
+ int ret = 0;
+
+ if (WARN_ON(!call->name))
+ return -EINVAL;
+
+ if (call->class->raw_init) {
+ ret = call->class->raw_init(call);
+ if (ret < 0 && ret != -ENOSYS)
+ pr_warn("Could not initialize trace events/%s\n",
+ call->name);
+ }
+
+ return ret;
+}
+
static int
__trace_add_event_call(struct ftrace_event_call *call, struct module *mod,
const struct file_operations *id,
@@ -1209,19 +1234,9 @@
struct dentry *d_events;
int ret;
- /* The linker may leave blanks */
- if (!call->name)
- return -EINVAL;
-
- if (call->class->raw_init) {
- ret = call->class->raw_init(call);
- if (ret < 0) {
- if (ret != -ENOSYS)
- pr_warning("Could not initialize trace events/%s\n",
- call->name);
- return ret;
- }
- }
+ ret = event_init(call);
+ if (ret < 0)
+ return ret;
d_events = event_trace_events_dir();
if (!d_events)
@@ -1272,13 +1287,10 @@
*/
static void __trace_remove_event_call(struct ftrace_event_call *call)
{
- ftrace_event_enable_disable(call, 0);
- if (call->event.funcs)
- __unregister_ftrace_event(&call->event);
- debugfs_remove_recursive(call->dir);
- list_del(&call->list);
+ event_remove(call);
trace_destroy_fields(call);
destroy_preds(call);
+ debugfs_remove_recursive(call->dir);
remove_subsystem_dir(call->class->system);
}
@@ -1450,15 +1462,43 @@
}
__setup("trace_event=", setup_trace_event);
+static __init int event_trace_enable(void)
+{
+ struct ftrace_event_call **iter, *call;
+ char *buf = bootup_event_buf;
+ char *token;
+ int ret;
+
+ for_each_event(iter, __start_ftrace_events, __stop_ftrace_events) {
+
+ call = *iter;
+ ret = event_init(call);
+ if (!ret)
+ list_add(&call->list, &ftrace_events);
+ }
+
+ while (true) {
+ token = strsep(&buf, ",");
+
+ if (!token)
+ break;
+ if (!*token)
+ continue;
+
+ ret = ftrace_set_clr_event(token, 1);
+ if (ret)
+ pr_warn("Failed to enable trace event: %s\n", token);
+ }
+ return 0;
+}
+
static __init int event_trace_init(void)
{
- struct ftrace_event_call **call;
+ struct ftrace_event_call *call;
struct dentry *d_tracer;
struct dentry *entry;
struct dentry *d_events;
int ret;
- char *buf = bootup_event_buf;
- char *token;
d_tracer = tracing_init_dentry();
if (!d_tracer)
@@ -1497,24 +1537,19 @@
if (trace_define_common_fields())
pr_warning("tracing: Failed to allocate common fields");
- for_each_event(call, __start_ftrace_events, __stop_ftrace_events) {
- __trace_add_event_call(*call, NULL, &ftrace_event_id_fops,
+ /*
+ * Early initialization already enabled ftrace event.
+ * Now it's only necessary to create the event directory.
+ */
+ list_for_each_entry(call, &ftrace_events, list) {
+
+ ret = event_create_dir(call, d_events,
+ &ftrace_event_id_fops,
&ftrace_enable_fops,
&ftrace_event_filter_fops,
&ftrace_event_format_fops);
- }
-
- while (true) {
- token = strsep(&buf, ",");
-
- if (!token)
- break;
- if (!*token)
- continue;
-
- ret = ftrace_set_clr_event(token, 1);
- if (ret)
- pr_warning("Failed to enable trace event: %s\n", token);
+ if (ret < 0)
+ event_remove(call);
}
ret = register_module_notifier(&trace_module_nb);
@@ -1523,6 +1558,7 @@
return 0;
}
+core_initcall(event_trace_enable);
fs_initcall(event_trace_init);
#ifdef CONFIG_FTRACE_STARTUP_TEST
@@ -1646,9 +1682,11 @@
event_test_stuff();
ret = __ftrace_set_clr_event(NULL, system->name, NULL, 0);
- if (WARN_ON_ONCE(ret))
+ if (WARN_ON_ONCE(ret)) {
pr_warning("error disabling system %s\n",
system->name);
+ continue;
+ }
pr_cont("OK\n");
}
@@ -1681,7 +1719,8 @@
static DEFINE_PER_CPU(atomic_t, ftrace_test_event_disable);
static void
-function_test_events_call(unsigned long ip, unsigned long parent_ip)
+function_test_events_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
struct ring_buffer_event *event;
struct ring_buffer *buffer;
@@ -1720,6 +1759,7 @@
static struct ftrace_ops trace_ops __initdata =
{
.func = function_test_events_call,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static __init void event_trace_self_test_with_function(void)
diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c
index 431dba8..c154797 100644
--- a/kernel/trace/trace_events_filter.c
+++ b/kernel/trace/trace_events_filter.c
@@ -2002,7 +2002,7 @@
static int __ftrace_function_set_filter(int filter, char *buf, int len,
struct function_filter_data *data)
{
- int i, re_cnt, ret;
+ int i, re_cnt, ret = -EINVAL;
int *reset;
char **re;
diff --git a/kernel/trace/trace_functions.c b/kernel/trace/trace_functions.c
index a426f41..483162a 100644
--- a/kernel/trace/trace_functions.c
+++ b/kernel/trace/trace_functions.c
@@ -49,7 +49,8 @@
}
static void
-function_trace_call_preempt_only(unsigned long ip, unsigned long parent_ip)
+function_trace_call_preempt_only(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
struct trace_array *tr = func_trace;
struct trace_array_cpu *data;
@@ -84,7 +85,9 @@
static struct tracer_flags func_flags;
static void
-function_trace_call(unsigned long ip, unsigned long parent_ip)
+function_trace_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
+
{
struct trace_array *tr = func_trace;
struct trace_array_cpu *data;
@@ -121,7 +124,8 @@
}
static void
-function_stack_trace_call(unsigned long ip, unsigned long parent_ip)
+function_stack_trace_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
struct trace_array *tr = func_trace;
struct trace_array_cpu *data;
@@ -164,13 +168,13 @@
static struct ftrace_ops trace_ops __read_mostly =
{
.func = function_trace_call,
- .flags = FTRACE_OPS_FL_GLOBAL,
+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
};
static struct ftrace_ops trace_stack_ops __read_mostly =
{
.func = function_stack_trace_call,
- .flags = FTRACE_OPS_FL_GLOBAL,
+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
};
static struct tracer_opt func_opts[] = {
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index ce27c8b..99b4378 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -143,7 +143,7 @@
return;
}
-#ifdef CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST
+#if defined(CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST) && !defined(CC_USING_FENTRY)
/*
* The arch may choose to record the frame pointer used
* and check it here to make sure that it is what we expect it
@@ -154,6 +154,9 @@
*
* Currently, x86_32 with optimize for size (-Os) makes the latest
* gcc do the above.
+ *
+ * Note, -mfentry does not use frame pointers, and this test
+ * is not needed if CC_USING_FENTRY is set.
*/
if (unlikely(current->ret_stack[index].fp != frame_pointer)) {
ftrace_graph_stop();
diff --git a/kernel/trace/trace_irqsoff.c b/kernel/trace/trace_irqsoff.c
index 99d20e9..d98ee82 100644
--- a/kernel/trace/trace_irqsoff.c
+++ b/kernel/trace/trace_irqsoff.c
@@ -136,7 +136,8 @@
* irqsoff uses its own tracer function to keep the overhead down:
*/
static void
-irqsoff_tracer_call(unsigned long ip, unsigned long parent_ip)
+irqsoff_tracer_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
struct trace_array *tr = irqsoff_trace;
struct trace_array_cpu *data;
@@ -153,7 +154,7 @@
static struct ftrace_ops trace_ops __read_mostly =
{
.func = irqsoff_tracer_call,
- .flags = FTRACE_OPS_FL_GLOBAL,
+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
};
#endif /* CONFIG_FUNCTION_TRACER */
diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_wakeup.c
index ff791ea..02170c0 100644
--- a/kernel/trace/trace_sched_wakeup.c
+++ b/kernel/trace/trace_sched_wakeup.c
@@ -108,7 +108,8 @@
* wakeup uses its own tracer function to keep the overhead down:
*/
static void
-wakeup_tracer_call(unsigned long ip, unsigned long parent_ip)
+wakeup_tracer_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
struct trace_array *tr = wakeup_trace;
struct trace_array_cpu *data;
@@ -129,7 +130,7 @@
static struct ftrace_ops trace_ops __read_mostly =
{
.func = wakeup_tracer_call,
- .flags = FTRACE_OPS_FL_GLOBAL,
+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
};
#endif /* CONFIG_FUNCTION_TRACER */
diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c
index 288541f..2c00a69 100644
--- a/kernel/trace/trace_selftest.c
+++ b/kernel/trace/trace_selftest.c
@@ -103,54 +103,67 @@
static int trace_selftest_test_probe1_cnt;
static void trace_selftest_test_probe1_func(unsigned long ip,
- unsigned long pip)
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
{
trace_selftest_test_probe1_cnt++;
}
static int trace_selftest_test_probe2_cnt;
static void trace_selftest_test_probe2_func(unsigned long ip,
- unsigned long pip)
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
{
trace_selftest_test_probe2_cnt++;
}
static int trace_selftest_test_probe3_cnt;
static void trace_selftest_test_probe3_func(unsigned long ip,
- unsigned long pip)
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
{
trace_selftest_test_probe3_cnt++;
}
static int trace_selftest_test_global_cnt;
static void trace_selftest_test_global_func(unsigned long ip,
- unsigned long pip)
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
{
trace_selftest_test_global_cnt++;
}
static int trace_selftest_test_dyn_cnt;
static void trace_selftest_test_dyn_func(unsigned long ip,
- unsigned long pip)
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
{
trace_selftest_test_dyn_cnt++;
}
static struct ftrace_ops test_probe1 = {
.func = trace_selftest_test_probe1_func,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static struct ftrace_ops test_probe2 = {
.func = trace_selftest_test_probe2_func,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static struct ftrace_ops test_probe3 = {
.func = trace_selftest_test_probe3_func,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static struct ftrace_ops test_global = {
- .func = trace_selftest_test_global_func,
- .flags = FTRACE_OPS_FL_GLOBAL,
+ .func = trace_selftest_test_global_func,
+ .flags = FTRACE_OPS_FL_GLOBAL | FTRACE_OPS_FL_RECURSION_SAFE,
};
static void print_counts(void)
@@ -393,10 +406,253 @@
return ret;
}
+
+static int trace_selftest_recursion_cnt;
+static void trace_selftest_test_recursion_func(unsigned long ip,
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
+{
+ /*
+ * This function is registered without the recursion safe flag.
+ * The ftrace infrastructure should provide the recursion
+ * protection. If not, this will crash the kernel!
+ */
+ trace_selftest_recursion_cnt++;
+ DYN_FTRACE_TEST_NAME();
+}
+
+static void trace_selftest_test_recursion_safe_func(unsigned long ip,
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
+{
+ /*
+ * We said we would provide our own recursion. By calling
+ * this function again, we should recurse back into this function
+ * and count again. But this only happens if the arch supports
+ * all of ftrace features and nothing else is using the function
+ * tracing utility.
+ */
+ if (trace_selftest_recursion_cnt++)
+ return;
+ DYN_FTRACE_TEST_NAME();
+}
+
+static struct ftrace_ops test_rec_probe = {
+ .func = trace_selftest_test_recursion_func,
+};
+
+static struct ftrace_ops test_recsafe_probe = {
+ .func = trace_selftest_test_recursion_safe_func,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
+};
+
+static int
+trace_selftest_function_recursion(void)
+{
+ int save_ftrace_enabled = ftrace_enabled;
+ int save_tracer_enabled = tracer_enabled;
+ char *func_name;
+ int len;
+ int ret;
+ int cnt;
+
+ /* The previous test PASSED */
+ pr_cont("PASSED\n");
+ pr_info("Testing ftrace recursion: ");
+
+
+ /* enable tracing, and record the filter function */
+ ftrace_enabled = 1;
+ tracer_enabled = 1;
+
+ /* Handle PPC64 '.' name */
+ func_name = "*" __stringify(DYN_FTRACE_TEST_NAME);
+ len = strlen(func_name);
+
+ ret = ftrace_set_filter(&test_rec_probe, func_name, len, 1);
+ if (ret) {
+ pr_cont("*Could not set filter* ");
+ goto out;
+ }
+
+ ret = register_ftrace_function(&test_rec_probe);
+ if (ret) {
+ pr_cont("*could not register callback* ");
+ goto out;
+ }
+
+ DYN_FTRACE_TEST_NAME();
+
+ unregister_ftrace_function(&test_rec_probe);
+
+ ret = -1;
+ if (trace_selftest_recursion_cnt != 1) {
+ pr_cont("*callback not called once (%d)* ",
+ trace_selftest_recursion_cnt);
+ goto out;
+ }
+
+ trace_selftest_recursion_cnt = 1;
+
+ pr_cont("PASSED\n");
+ pr_info("Testing ftrace recursion safe: ");
+
+ ret = ftrace_set_filter(&test_recsafe_probe, func_name, len, 1);
+ if (ret) {
+ pr_cont("*Could not set filter* ");
+ goto out;
+ }
+
+ ret = register_ftrace_function(&test_recsafe_probe);
+ if (ret) {
+ pr_cont("*could not register callback* ");
+ goto out;
+ }
+
+ DYN_FTRACE_TEST_NAME();
+
+ unregister_ftrace_function(&test_recsafe_probe);
+
+ /*
+ * If arch supports all ftrace features, and no other task
+ * was on the list, we should be fine.
+ */
+ if (!ftrace_nr_registered_ops() && !FTRACE_FORCE_LIST_FUNC)
+ cnt = 2; /* Should have recursed */
+ else
+ cnt = 1;
+
+ ret = -1;
+ if (trace_selftest_recursion_cnt != cnt) {
+ pr_cont("*callback not called expected %d times (%d)* ",
+ cnt, trace_selftest_recursion_cnt);
+ goto out;
+ }
+
+ ret = 0;
+out:
+ ftrace_enabled = save_ftrace_enabled;
+ tracer_enabled = save_tracer_enabled;
+
+ return ret;
+}
#else
# define trace_selftest_startup_dynamic_tracing(trace, tr, func) ({ 0; })
+# define trace_selftest_function_recursion() ({ 0; })
#endif /* CONFIG_DYNAMIC_FTRACE */
+static enum {
+ TRACE_SELFTEST_REGS_START,
+ TRACE_SELFTEST_REGS_FOUND,
+ TRACE_SELFTEST_REGS_NOT_FOUND,
+} trace_selftest_regs_stat;
+
+static void trace_selftest_test_regs_func(unsigned long ip,
+ unsigned long pip,
+ struct ftrace_ops *op,
+ struct pt_regs *pt_regs)
+{
+ if (pt_regs)
+ trace_selftest_regs_stat = TRACE_SELFTEST_REGS_FOUND;
+ else
+ trace_selftest_regs_stat = TRACE_SELFTEST_REGS_NOT_FOUND;
+}
+
+static struct ftrace_ops test_regs_probe = {
+ .func = trace_selftest_test_regs_func,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE | FTRACE_OPS_FL_SAVE_REGS,
+};
+
+static int
+trace_selftest_function_regs(void)
+{
+ int save_ftrace_enabled = ftrace_enabled;
+ int save_tracer_enabled = tracer_enabled;
+ char *func_name;
+ int len;
+ int ret;
+ int supported = 0;
+
+#ifdef ARCH_SUPPORTS_FTRACE_SAVE_REGS
+ supported = 1;
+#endif
+
+ /* The previous test PASSED */
+ pr_cont("PASSED\n");
+ pr_info("Testing ftrace regs%s: ",
+ !supported ? "(no arch support)" : "");
+
+ /* enable tracing, and record the filter function */
+ ftrace_enabled = 1;
+ tracer_enabled = 1;
+
+ /* Handle PPC64 '.' name */
+ func_name = "*" __stringify(DYN_FTRACE_TEST_NAME);
+ len = strlen(func_name);
+
+ ret = ftrace_set_filter(&test_regs_probe, func_name, len, 1);
+ /*
+ * If DYNAMIC_FTRACE is not set, then we just trace all functions.
+ * This test really doesn't care.
+ */
+ if (ret && ret != -ENODEV) {
+ pr_cont("*Could not set filter* ");
+ goto out;
+ }
+
+ ret = register_ftrace_function(&test_regs_probe);
+ /*
+ * Now if the arch does not support passing regs, then this should
+ * have failed.
+ */
+ if (!supported) {
+ if (!ret) {
+ pr_cont("*registered save-regs without arch support* ");
+ goto out;
+ }
+ test_regs_probe.flags |= FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED;
+ ret = register_ftrace_function(&test_regs_probe);
+ }
+ if (ret) {
+ pr_cont("*could not register callback* ");
+ goto out;
+ }
+
+
+ DYN_FTRACE_TEST_NAME();
+
+ unregister_ftrace_function(&test_regs_probe);
+
+ ret = -1;
+
+ switch (trace_selftest_regs_stat) {
+ case TRACE_SELFTEST_REGS_START:
+ pr_cont("*callback never called* ");
+ goto out;
+
+ case TRACE_SELFTEST_REGS_FOUND:
+ if (supported)
+ break;
+ pr_cont("*callback received regs without arch support* ");
+ goto out;
+
+ case TRACE_SELFTEST_REGS_NOT_FOUND:
+ if (!supported)
+ break;
+ pr_cont("*callback received NULL regs* ");
+ goto out;
+ }
+
+ ret = 0;
+out:
+ ftrace_enabled = save_ftrace_enabled;
+ tracer_enabled = save_tracer_enabled;
+
+ return ret;
+}
+
/*
* Simple verification test of ftrace function tracer.
* Enable ftrace, sleep 1/10 second, and then read the trace
@@ -442,7 +698,14 @@
ret = trace_selftest_startup_dynamic_tracing(trace, tr,
DYN_FTRACE_TEST_NAME);
+ if (ret)
+ goto out;
+ ret = trace_selftest_function_recursion();
+ if (ret)
+ goto out;
+
+ ret = trace_selftest_function_regs();
out:
ftrace_enabled = save_ftrace_enabled;
tracer_enabled = save_tracer_enabled;
@@ -778,6 +1041,8 @@
set_current_state(TASK_INTERRUPTIBLE);
schedule();
+ complete(x);
+
/* we are awake, now wait to disappear */
while (!kthread_should_stop()) {
/*
@@ -821,24 +1086,21 @@
/* reset the max latency */
tracing_max_latency = 0;
- /* sleep to let the RT thread sleep too */
- msleep(100);
+ while (p->on_rq) {
+ /*
+ * Sleep to make sure the RT thread is asleep too.
+ * On virtual machines we can't rely on timings,
+ * but we want to make sure this test still works.
+ */
+ msleep(100);
+ }
- /*
- * Yes this is slightly racy. It is possible that for some
- * strange reason that the RT thread we created, did not
- * call schedule for 100ms after doing the completion,
- * and we do a wakeup on a task that already is awake.
- * But that is extremely unlikely, and the worst thing that
- * happens in such a case, is that we disable tracing.
- * Honestly, if this race does happen something is horrible
- * wrong with the system.
- */
+ init_completion(&isrt);
wake_up_process(p);
- /* give a little time to let the thread wake up */
- msleep(100);
+ /* Wait for the task to wake up */
+ wait_for_completion(&isrt);
/* stop the tracing. */
tracing_stop();
diff --git a/kernel/trace/trace_stack.c b/kernel/trace/trace_stack.c
index d4545f4..0c1b1657 100644
--- a/kernel/trace/trace_stack.c
+++ b/kernel/trace/trace_stack.c
@@ -111,7 +111,8 @@
}
static void
-stack_trace_call(unsigned long ip, unsigned long parent_ip)
+stack_trace_call(unsigned long ip, unsigned long parent_ip,
+ struct ftrace_ops *op, struct pt_regs *pt_regs)
{
int cpu;
@@ -136,6 +137,7 @@
static struct ftrace_ops trace_ops __read_mostly =
{
.func = stack_trace_call,
+ .flags = FTRACE_OPS_FL_RECURSION_SAFE,
};
static ssize_t
diff --git a/kernel/trace/trace_syscalls.c b/kernel/trace/trace_syscalls.c
index 60e4d78..2485a7d 100644
--- a/kernel/trace/trace_syscalls.c
+++ b/kernel/trace/trace_syscalls.c
@@ -487,7 +487,7 @@
return 0;
}
-core_initcall(init_ftrace_syscalls);
+early_initcall(init_ftrace_syscalls);
#ifdef CONFIG_PERF_EVENTS
@@ -506,6 +506,8 @@
int size;
syscall_nr = syscall_get_nr(current, regs);
+ if (syscall_nr < 0)
+ return;
if (!test_bit(syscall_nr, enabled_perf_enter_syscalls))
return;
@@ -580,6 +582,8 @@
int size;
syscall_nr = syscall_get_nr(current, regs);
+ if (syscall_nr < 0)
+ return;
if (!test_bit(syscall_nr, enabled_perf_exit_syscalls))
return;
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 4b1dfba..9d4c8d5 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -22,6 +22,7 @@
#include <linux/notifier.h>
#include <linux/module.h>
#include <linux/sysctl.h>
+#include <linux/smpboot.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
@@ -29,16 +30,18 @@
int watchdog_enabled = 1;
int __read_mostly watchdog_thresh = 10;
+static int __read_mostly watchdog_disabled;
static DEFINE_PER_CPU(unsigned long, watchdog_touch_ts);
static DEFINE_PER_CPU(struct task_struct *, softlockup_watchdog);
static DEFINE_PER_CPU(struct hrtimer, watchdog_hrtimer);
static DEFINE_PER_CPU(bool, softlockup_touch_sync);
static DEFINE_PER_CPU(bool, soft_watchdog_warn);
+static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts);
+static DEFINE_PER_CPU(unsigned long, soft_lockup_hrtimer_cnt);
#ifdef CONFIG_HARDLOCKUP_DETECTOR
static DEFINE_PER_CPU(bool, hard_watchdog_warn);
static DEFINE_PER_CPU(bool, watchdog_nmi_touch);
-static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts);
static DEFINE_PER_CPU(unsigned long, hrtimer_interrupts_saved);
static DEFINE_PER_CPU(struct perf_event *, watchdog_ev);
#endif
@@ -248,13 +251,15 @@
__this_cpu_write(hard_watchdog_warn, false);
return;
}
+#endif /* CONFIG_HARDLOCKUP_DETECTOR */
+
static void watchdog_interrupt_count(void)
{
__this_cpu_inc(hrtimer_interrupts);
}
-#else
-static inline void watchdog_interrupt_count(void) { return; }
-#endif /* CONFIG_HARDLOCKUP_DETECTOR */
+
+static int watchdog_nmi_enable(unsigned int cpu);
+static void watchdog_nmi_disable(unsigned int cpu);
/* watchdog kicker functions */
static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
@@ -327,49 +332,68 @@
return HRTIMER_RESTART;
}
-
-/*
- * The watchdog thread - touches the timestamp.
- */
-static int watchdog(void *unused)
+static void watchdog_set_prio(unsigned int policy, unsigned int prio)
{
- struct sched_param param = { .sched_priority = 0 };
+ struct sched_param param = { .sched_priority = prio };
+
+ sched_setscheduler(current, policy, ¶m);
+}
+
+static void watchdog_enable(unsigned int cpu)
+{
struct hrtimer *hrtimer = &__raw_get_cpu_var(watchdog_hrtimer);
- /* initialize timestamp */
- __touch_watchdog();
+ if (!watchdog_enabled) {
+ kthread_park(current);
+ return;
+ }
+
+ /* Enable the perf event */
+ watchdog_nmi_enable(cpu);
/* kick off the timer for the hardlockup detector */
+ hrtimer_init(hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ hrtimer->function = watchdog_timer_fn;
+
/* done here because hrtimer_start can only pin to smp_processor_id() */
hrtimer_start(hrtimer, ns_to_ktime(get_sample_period()),
HRTIMER_MODE_REL_PINNED);
- set_current_state(TASK_INTERRUPTIBLE);
- /*
- * Run briefly (kicked by the hrtimer callback function) once every
- * get_sample_period() seconds (4 seconds by default) to reset the
- * softlockup timestamp. If this gets delayed for more than
- * 2*watchdog_thresh seconds then the debug-printout triggers in
- * watchdog_timer_fn().
- */
- while (!kthread_should_stop()) {
- __touch_watchdog();
- schedule();
-
- if (kthread_should_stop())
- break;
-
- set_current_state(TASK_INTERRUPTIBLE);
- }
- /*
- * Drop the policy/priority elevation during thread exit to avoid a
- * scheduling latency spike.
- */
- __set_current_state(TASK_RUNNING);
- sched_setscheduler(current, SCHED_NORMAL, ¶m);
- return 0;
+ /* initialize timestamp */
+ watchdog_set_prio(SCHED_FIFO, MAX_RT_PRIO - 1);
+ __touch_watchdog();
}
+static void watchdog_disable(unsigned int cpu)
+{
+ struct hrtimer *hrtimer = &__raw_get_cpu_var(watchdog_hrtimer);
+
+ watchdog_set_prio(SCHED_NORMAL, 0);
+ hrtimer_cancel(hrtimer);
+ /* disable the perf event */
+ watchdog_nmi_disable(cpu);
+}
+
+static int watchdog_should_run(unsigned int cpu)
+{
+ return __this_cpu_read(hrtimer_interrupts) !=
+ __this_cpu_read(soft_lockup_hrtimer_cnt);
+}
+
+/*
+ * The watchdog thread function - touches the timestamp.
+ *
+ * It only runs once every get_sample_period() seconds (4 seconds by
+ * default) to reset the softlockup timestamp. If this gets delayed
+ * for more than 2*watchdog_thresh seconds then the debug-printout
+ * triggers in watchdog_timer_fn().
+ */
+static void watchdog(unsigned int cpu)
+{
+ __this_cpu_write(soft_lockup_hrtimer_cnt,
+ __this_cpu_read(hrtimer_interrupts));
+ __touch_watchdog();
+}
#ifdef CONFIG_HARDLOCKUP_DETECTOR
/*
@@ -379,7 +403,7 @@
*/
static unsigned long cpu0_err;
-static int watchdog_nmi_enable(int cpu)
+static int watchdog_nmi_enable(unsigned int cpu)
{
struct perf_event_attr *wd_attr;
struct perf_event *event = per_cpu(watchdog_ev, cpu);
@@ -433,7 +457,7 @@
return 0;
}
-static void watchdog_nmi_disable(int cpu)
+static void watchdog_nmi_disable(unsigned int cpu)
{
struct perf_event *event = per_cpu(watchdog_ev, cpu);
@@ -447,107 +471,35 @@
return;
}
#else
-static int watchdog_nmi_enable(int cpu) { return 0; }
-static void watchdog_nmi_disable(int cpu) { return; }
+static int watchdog_nmi_enable(unsigned int cpu) { return 0; }
+static void watchdog_nmi_disable(unsigned int cpu) { return; }
#endif /* CONFIG_HARDLOCKUP_DETECTOR */
/* prepare/enable/disable routines */
-static void watchdog_prepare_cpu(int cpu)
-{
- struct hrtimer *hrtimer = &per_cpu(watchdog_hrtimer, cpu);
-
- WARN_ON(per_cpu(softlockup_watchdog, cpu));
- hrtimer_init(hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
- hrtimer->function = watchdog_timer_fn;
-}
-
-static int watchdog_enable(int cpu)
-{
- struct task_struct *p = per_cpu(softlockup_watchdog, cpu);
- int err = 0;
-
- /* enable the perf event */
- err = watchdog_nmi_enable(cpu);
-
- /* Regardless of err above, fall through and start softlockup */
-
- /* create the watchdog thread */
- if (!p) {
- struct sched_param param = { .sched_priority = MAX_RT_PRIO-1 };
- p = kthread_create_on_node(watchdog, NULL, cpu_to_node(cpu), "watchdog/%d", cpu);
- if (IS_ERR(p)) {
- pr_err("softlockup watchdog for %i failed\n", cpu);
- if (!err) {
- /* if hardlockup hasn't already set this */
- err = PTR_ERR(p);
- /* and disable the perf event */
- watchdog_nmi_disable(cpu);
- }
- goto out;
- }
- sched_setscheduler(p, SCHED_FIFO, ¶m);
- kthread_bind(p, cpu);
- per_cpu(watchdog_touch_ts, cpu) = 0;
- per_cpu(softlockup_watchdog, cpu) = p;
- wake_up_process(p);
- }
-
-out:
- return err;
-}
-
-static void watchdog_disable(int cpu)
-{
- struct task_struct *p = per_cpu(softlockup_watchdog, cpu);
- struct hrtimer *hrtimer = &per_cpu(watchdog_hrtimer, cpu);
-
- /*
- * cancel the timer first to stop incrementing the stats
- * and waking up the kthread
- */
- hrtimer_cancel(hrtimer);
-
- /* disable the perf event */
- watchdog_nmi_disable(cpu);
-
- /* stop the watchdog thread */
- if (p) {
- per_cpu(softlockup_watchdog, cpu) = NULL;
- kthread_stop(p);
- }
-}
-
/* sysctl functions */
#ifdef CONFIG_SYSCTL
static void watchdog_enable_all_cpus(void)
{
- int cpu;
+ unsigned int cpu;
- watchdog_enabled = 0;
-
- for_each_online_cpu(cpu)
- if (!watchdog_enable(cpu))
- /* if any cpu succeeds, watchdog is considered
- enabled for the system */
- watchdog_enabled = 1;
-
- if (!watchdog_enabled)
- pr_err("failed to be enabled on some cpus\n");
-
+ if (watchdog_disabled) {
+ watchdog_disabled = 0;
+ for_each_online_cpu(cpu)
+ kthread_unpark(per_cpu(softlockup_watchdog, cpu));
+ }
}
static void watchdog_disable_all_cpus(void)
{
- int cpu;
+ unsigned int cpu;
- for_each_online_cpu(cpu)
- watchdog_disable(cpu);
-
- /* if all watchdogs are disabled, then they are disabled for the system */
- watchdog_enabled = 0;
+ if (!watchdog_disabled) {
+ watchdog_disabled = 1;
+ for_each_online_cpu(cpu)
+ kthread_park(per_cpu(softlockup_watchdog, cpu));
+ }
}
-
/*
* proc handler for /proc/sys/kernel/nmi_watchdog,watchdog_thresh
*/
@@ -557,73 +509,36 @@
{
int ret;
+ if (watchdog_disabled < 0)
+ return -ENODEV;
+
ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
if (ret || !write)
- goto out;
+ return ret;
if (watchdog_enabled && watchdog_thresh)
watchdog_enable_all_cpus();
else
watchdog_disable_all_cpus();
-out:
return ret;
}
#endif /* CONFIG_SYSCTL */
-
-/*
- * Create/destroy watchdog threads as CPUs come and go:
- */
-static int __cpuinit
-cpu_callback(struct notifier_block *nfb, unsigned long action, void *hcpu)
-{
- int hotcpu = (unsigned long)hcpu;
-
- switch (action) {
- case CPU_UP_PREPARE:
- case CPU_UP_PREPARE_FROZEN:
- watchdog_prepare_cpu(hotcpu);
- break;
- case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
- if (watchdog_enabled)
- watchdog_enable(hotcpu);
- break;
-#ifdef CONFIG_HOTPLUG_CPU
- case CPU_UP_CANCELED:
- case CPU_UP_CANCELED_FROZEN:
- watchdog_disable(hotcpu);
- break;
- case CPU_DEAD:
- case CPU_DEAD_FROZEN:
- watchdog_disable(hotcpu);
- break;
-#endif /* CONFIG_HOTPLUG_CPU */
- }
-
- /*
- * hardlockup and softlockup are not important enough
- * to block cpu bring up. Just always succeed and
- * rely on printk output to flag problems.
- */
- return NOTIFY_OK;
-}
-
-static struct notifier_block __cpuinitdata cpu_nfb = {
- .notifier_call = cpu_callback
+static struct smp_hotplug_thread watchdog_threads = {
+ .store = &softlockup_watchdog,
+ .thread_should_run = watchdog_should_run,
+ .thread_fn = watchdog,
+ .thread_comm = "watchdog/%u",
+ .setup = watchdog_enable,
+ .park = watchdog_disable,
+ .unpark = watchdog_enable,
};
void __init lockup_detector_init(void)
{
- void *cpu = (void *)(long)smp_processor_id();
- int err;
-
- err = cpu_callback(&cpu_nfb, CPU_UP_PREPARE, cpu);
- WARN_ON(notifier_to_errno(err));
-
- cpu_callback(&cpu_nfb, CPU_ONLINE, cpu);
- register_cpu_notifier(&cpu_nfb);
-
- return;
+ if (smpboot_register_percpu_thread(&watchdog_threads)) {
+ pr_err("Failed to create watchdog threads, disabled\n");
+ watchdog_disabled = -ENODEV;
+ }
}
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 692d976..3c5a79e 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -66,6 +66,7 @@
/* pool flags */
POOL_MANAGE_WORKERS = 1 << 0, /* need to manage workers */
+ POOL_MANAGING_WORKERS = 1 << 1, /* managing workers */
/* worker flags */
WORKER_STARTED = 1 << 0, /* started */
@@ -652,7 +653,7 @@
/* Do we have too many workers and should some go away? */
static bool too_many_workers(struct worker_pool *pool)
{
- bool managing = mutex_is_locked(&pool->manager_mutex);
+ bool managing = pool->flags & POOL_MANAGING_WORKERS;
int nr_idle = pool->nr_idle + managing; /* manager is considered idle */
int nr_busy = pool->nr_workers - nr_idle;
@@ -1326,6 +1327,15 @@
/* we did our part, wait for rebind_workers() to finish up */
wait_event(gcwq->rebind_hold, !(worker->flags & WORKER_REBIND));
+
+ /*
+ * rebind_workers() shouldn't finish until all workers passed the
+ * above WORKER_REBIND wait. Tell it when done.
+ */
+ spin_lock_irq(&worker->pool->gcwq->lock);
+ if (!--worker->idle_rebind->cnt)
+ complete(&worker->idle_rebind->done);
+ spin_unlock_irq(&worker->pool->gcwq->lock);
}
/*
@@ -1339,8 +1349,16 @@
struct worker *worker = container_of(work, struct worker, rebind_work);
struct global_cwq *gcwq = worker->pool->gcwq;
- if (worker_maybe_bind_and_lock(worker))
- worker_clr_flags(worker, WORKER_REBIND);
+ worker_maybe_bind_and_lock(worker);
+
+ /*
+ * %WORKER_REBIND must be cleared even if the above binding failed;
+ * otherwise, we may confuse the next CPU_UP cycle or oops / get
+ * stuck by calling idle_worker_rebind() prematurely. If CPU went
+ * down again inbetween, %WORKER_UNBOUND would be set, so clearing
+ * %WORKER_REBIND is always safe.
+ */
+ worker_clr_flags(worker, WORKER_REBIND);
spin_unlock_irq(&gcwq->lock);
}
@@ -1396,12 +1414,15 @@
/* set REBIND and kick idle ones, we'll wait for these later */
for_each_worker_pool(pool, gcwq) {
list_for_each_entry(worker, &pool->idle_list, entry) {
+ unsigned long worker_flags = worker->flags;
+
if (worker->flags & WORKER_REBIND)
continue;
- /* morph UNBOUND to REBIND */
- worker->flags &= ~WORKER_UNBOUND;
- worker->flags |= WORKER_REBIND;
+ /* morph UNBOUND to REBIND atomically */
+ worker_flags &= ~WORKER_UNBOUND;
+ worker_flags |= WORKER_REBIND;
+ ACCESS_ONCE(worker->flags) = worker_flags;
idle_rebind.cnt++;
worker->idle_rebind = &idle_rebind;
@@ -1419,25 +1440,15 @@
goto retry;
}
- /*
- * All idle workers are rebound and waiting for %WORKER_REBIND to
- * be cleared inside idle_worker_rebind(). Clear and release.
- * Clearing %WORKER_REBIND from this foreign context is safe
- * because these workers are still guaranteed to be idle.
- */
- for_each_worker_pool(pool, gcwq)
- list_for_each_entry(worker, &pool->idle_list, entry)
- worker->flags &= ~WORKER_REBIND;
-
- wake_up_all(&gcwq->rebind_hold);
-
- /* rebind busy workers */
+ /* all idle workers are rebound, rebind busy workers */
for_each_busy_worker(worker, i, pos, gcwq) {
struct work_struct *rebind_work = &worker->rebind_work;
+ unsigned long worker_flags = worker->flags;
- /* morph UNBOUND to REBIND */
- worker->flags &= ~WORKER_UNBOUND;
- worker->flags |= WORKER_REBIND;
+ /* morph UNBOUND to REBIND atomically */
+ worker_flags &= ~WORKER_UNBOUND;
+ worker_flags |= WORKER_REBIND;
+ ACCESS_ONCE(worker->flags) = worker_flags;
if (test_and_set_bit(WORK_STRUCT_PENDING_BIT,
work_data_bits(rebind_work)))
@@ -1449,6 +1460,34 @@
worker->scheduled.next,
work_color_to_flags(WORK_NO_COLOR));
}
+
+ /*
+ * All idle workers are rebound and waiting for %WORKER_REBIND to
+ * be cleared inside idle_worker_rebind(). Clear and release.
+ * Clearing %WORKER_REBIND from this foreign context is safe
+ * because these workers are still guaranteed to be idle.
+ *
+ * We need to make sure all idle workers passed WORKER_REBIND wait
+ * in idle_worker_rebind() before returning; otherwise, workers can
+ * get stuck at the wait if hotplug cycle repeats.
+ */
+ idle_rebind.cnt = 1;
+ INIT_COMPLETION(idle_rebind.done);
+
+ for_each_worker_pool(pool, gcwq) {
+ list_for_each_entry(worker, &pool->idle_list, entry) {
+ worker->flags &= ~WORKER_REBIND;
+ idle_rebind.cnt++;
+ }
+ }
+
+ wake_up_all(&gcwq->rebind_hold);
+
+ if (--idle_rebind.cnt) {
+ spin_unlock_irq(&gcwq->lock);
+ wait_for_completion(&idle_rebind.done);
+ spin_lock_irq(&gcwq->lock);
+ }
}
static struct worker *alloc_worker(void)
@@ -1794,9 +1833,45 @@
struct worker_pool *pool = worker->pool;
bool ret = false;
- if (!mutex_trylock(&pool->manager_mutex))
+ if (pool->flags & POOL_MANAGING_WORKERS)
return ret;
+ pool->flags |= POOL_MANAGING_WORKERS;
+
+ /*
+ * To simplify both worker management and CPU hotplug, hold off
+ * management while hotplug is in progress. CPU hotplug path can't
+ * grab %POOL_MANAGING_WORKERS to achieve this because that can
+ * lead to idle worker depletion (all become busy thinking someone
+ * else is managing) which in turn can result in deadlock under
+ * extreme circumstances. Use @pool->manager_mutex to synchronize
+ * manager against CPU hotplug.
+ *
+ * manager_mutex would always be free unless CPU hotplug is in
+ * progress. trylock first without dropping @gcwq->lock.
+ */
+ if (unlikely(!mutex_trylock(&pool->manager_mutex))) {
+ spin_unlock_irq(&pool->gcwq->lock);
+ mutex_lock(&pool->manager_mutex);
+ /*
+ * CPU hotplug could have happened while we were waiting
+ * for manager_mutex. Hotplug itself can't handle us
+ * because manager isn't either on idle or busy list, and
+ * @gcwq's state and ours could have deviated.
+ *
+ * As hotplug is now excluded via manager_mutex, we can
+ * simply try to bind. It will succeed or fail depending
+ * on @gcwq's current state. Try it and adjust
+ * %WORKER_UNBOUND accordingly.
+ */
+ if (worker_maybe_bind_and_lock(worker))
+ worker->flags &= ~WORKER_UNBOUND;
+ else
+ worker->flags |= WORKER_UNBOUND;
+
+ ret = true;
+ }
+
pool->flags &= ~POOL_MANAGE_WORKERS;
/*
@@ -1806,6 +1881,7 @@
ret |= maybe_destroy_workers(pool);
ret |= maybe_create_worker(pool);
+ pool->flags &= ~POOL_MANAGING_WORKERS;
mutex_unlock(&pool->manager_mutex);
return ret;
}
@@ -3500,18 +3576,17 @@
#ifdef CONFIG_SMP
struct work_for_cpu {
- struct completion completion;
+ struct work_struct work;
long (*fn)(void *);
void *arg;
long ret;
};
-static int do_work_for_cpu(void *_wfc)
+static void work_for_cpu_fn(struct work_struct *work)
{
- struct work_for_cpu *wfc = _wfc;
+ struct work_for_cpu *wfc = container_of(work, struct work_for_cpu, work);
+
wfc->ret = wfc->fn(wfc->arg);
- complete(&wfc->completion);
- return 0;
}
/**
@@ -3526,19 +3601,11 @@
*/
long work_on_cpu(unsigned int cpu, long (*fn)(void *), void *arg)
{
- struct task_struct *sub_thread;
- struct work_for_cpu wfc = {
- .completion = COMPLETION_INITIALIZER_ONSTACK(wfc.completion),
- .fn = fn,
- .arg = arg,
- };
+ struct work_for_cpu wfc = { .fn = fn, .arg = arg };
- sub_thread = kthread_create(do_work_for_cpu, &wfc, "work_for_cpu");
- if (IS_ERR(sub_thread))
- return PTR_ERR(sub_thread);
- kthread_bind(sub_thread, cpu);
- wake_up_process(sub_thread);
- wait_for_completion(&wfc.completion);
+ INIT_WORK_ONSTACK(&wfc.work, work_for_cpu_fn);
+ schedule_work_on(cpu, &wfc.work);
+ flush_work(&wfc.work);
return wfc.ret;
}
EXPORT_SYMBOL_GPL(work_on_cpu);
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 2403a63..35c4565 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -452,7 +452,8 @@
config DEBUG_KMEMLEAK
bool "Kernel memory leak detector"
depends on DEBUG_KERNEL && EXPERIMENTAL && \
- (X86 || ARM || PPC || MIPS || S390 || SPARC64 || SUPERH || MICROBLAZE || TILE)
+ (X86 || ARM || PPC || MIPS || S390 || SPARC64 || SUPERH || \
+ MICROBLAZE || TILE || ARM64)
select DEBUG_FS
select STACKTRACE if STACKTRACE_SUPPORT
@@ -629,6 +630,20 @@
Say N if you are unsure.
+config PROVE_RCU_DELAY
+ bool "RCU debugging: preemptible RCU race provocation"
+ depends on DEBUG_KERNEL && PREEMPT_RCU
+ default n
+ help
+ There is a class of races that involve an unlikely preemption
+ of __rcu_read_unlock() just after ->rcu_read_lock_nesting has
+ been set to INT_MIN. This feature inserts a delay at that
+ point to increase the probability of these races.
+
+ Say Y to increase probability of preemption of __rcu_read_unlock().
+
+ Say N if you are unsure.
+
config SPARSE_RCU_POINTER
bool "RCU debugging: sparse-based checks for pointer usage"
default n
@@ -739,7 +754,8 @@
bool "Verbose BUG() reporting (adds 70K)" if DEBUG_KERNEL && EXPERT
depends on BUG
depends on ARM || AVR32 || M32R || M68K || SPARC32 || SPARC64 || \
- FRV || SUPERH || GENERIC_BUG || BLACKFIN || MN10300 || TILE
+ FRV || SUPERH || GENERIC_BUG || BLACKFIN || MN10300 || \
+ TILE || ARM64
default y
help
Say Y here to make BUG() panics output the file name and line number
diff --git a/lib/digsig.c b/lib/digsig.c
index 286d558..8c0e629 100644
--- a/lib/digsig.c
+++ b/lib/digsig.c
@@ -163,9 +163,11 @@
memcpy(out1 + head, p, l);
err = pkcs_1_v1_5_decode_emsa(out1, len, mblen, out2, &len);
+ if (err)
+ goto err;
- if (!err && len == hlen)
- err = memcmp(out2, h, hlen);
+ if (len != hlen || memcmp(out2, h, hlen))
+ err = -EINVAL;
err:
mpi_free(in);
diff --git a/lib/flex_proportions.c b/lib/flex_proportions.c
index c785554..ebf3bac 100644
--- a/lib/flex_proportions.c
+++ b/lib/flex_proportions.c
@@ -62,7 +62,7 @@
*/
bool fprop_new_period(struct fprop_global *p, int periods)
{
- u64 events;
+ s64 events;
unsigned long flags;
local_irq_save(flags);
diff --git a/mm/bootmem.c b/mm/bootmem.c
index bcb63ac..f468185 100644
--- a/mm/bootmem.c
+++ b/mm/bootmem.c
@@ -419,7 +419,7 @@
}
/**
- * reserve_bootmem - mark a page range as usable
+ * reserve_bootmem - mark a page range as reserved
* @addr: starting address of the range
* @size: size of the range in bytes
* @flags: reservation flags (see linux/bootmem.h)
diff --git a/mm/filemap.c b/mm/filemap.c
index fa5ca30..3843445 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1412,12 +1412,8 @@
retval = filemap_write_and_wait_range(mapping, pos,
pos + iov_length(iov, nr_segs) - 1);
if (!retval) {
- struct blk_plug plug;
-
- blk_start_plug(&plug);
retval = mapping->a_ops->direct_IO(READ, iocb,
iov, pos, nr_segs);
- blk_finish_plug(&plug);
}
if (retval > 0) {
*ppos = pos + retval;
@@ -2527,14 +2523,12 @@
{
struct file *file = iocb->ki_filp;
struct inode *inode = file->f_mapping->host;
- struct blk_plug plug;
ssize_t ret;
BUG_ON(iocb->ki_pos != pos);
sb_start_write(inode->i_sb);
mutex_lock(&inode->i_mutex);
- blk_start_plug(&plug);
ret = __generic_file_aio_write(iocb, iov, nr_segs, &iocb->ki_pos);
mutex_unlock(&inode->i_mutex);
@@ -2545,7 +2539,6 @@
if (err < 0 && ret > 0)
ret = err;
}
- blk_finish_plug(&plug);
sb_end_write(inode->i_sb);
return ret;
}
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 57c4b93..141dbb6 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1811,7 +1811,6 @@
src_page = pte_page(pteval);
copy_user_highpage(page, src_page, address, vma);
VM_BUG_ON(page_mapcount(src_page) != 1);
- VM_BUG_ON(page_count(src_page) != 2);
release_pte_page(src_page);
/*
* ptl mostly unnecessary, but preempt has to
diff --git a/mm/kmemleak.c b/mm/kmemleak.c
index 45eb621..0de83b4 100644
--- a/mm/kmemleak.c
+++ b/mm/kmemleak.c
@@ -1483,13 +1483,11 @@
{
struct kmemleak_object *prev_obj = v;
struct kmemleak_object *next_obj = NULL;
- struct list_head *n = &prev_obj->object_list;
+ struct kmemleak_object *obj = prev_obj;
++(*pos);
- list_for_each_continue_rcu(n, &object_list) {
- struct kmemleak_object *obj =
- list_entry(n, struct kmemleak_object, object_list);
+ list_for_each_entry_continue_rcu(obj, &object_list, object_list) {
if (get_object(obj)) {
next_obj = obj;
break;
diff --git a/mm/memblock.c b/mm/memblock.c
index 4d9393c..82aa349 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -246,7 +246,7 @@
min(new_area_start, memblock.current_limit),
new_alloc_size, PAGE_SIZE);
- new_array = addr ? __va(addr) : 0;
+ new_array = addr ? __va(addr) : NULL;
}
if (!addr) {
pr_err("memblock: Failed to double %s array from %ld to %ld entries !\n",
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 3ad25f9..6a5b90d 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -126,9 +126,6 @@
struct mem_section *ms;
struct page *page, *memmap;
- if (!pfn_valid(start_pfn))
- return;
-
section_nr = pfn_to_section_nr(start_pfn);
ms = __nr_to_section(section_nr);
@@ -187,9 +184,16 @@
end_pfn = pfn + pgdat->node_spanned_pages;
/* register_section info */
- for (; pfn < end_pfn; pfn += PAGES_PER_SECTION)
- register_page_bootmem_info_section(pfn);
-
+ for (; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+ /*
+ * Some platforms can assign the same pfn to multiple nodes - on
+ * node0 as well as nodeN. To avoid registering a pfn against
+ * multiple nodes we check that this pfn does not already
+ * reside in some other node.
+ */
+ if (pfn_valid(pfn) && (pfn_to_nid(pfn) == node))
+ register_page_bootmem_info_section(pfn);
+ }
}
#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index bd92431..4ada3be 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2562,7 +2562,7 @@
break;
default:
- BUG();
+ return -EINVAL;
}
l = strlen(policy_modes[mode]);
diff --git a/mm/mmap.c b/mm/mmap.c
index 9adee9f..ae18a48 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1356,9 +1356,8 @@
} else if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK))
make_pages_present(addr, addr + len);
- if (file && uprobe_mmap(vma))
- /* matching probes but cannot insert */
- goto unmap_and_free_vma;
+ if (file)
+ uprobe_mmap(vma);
return addr;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index c66fb87..c13ea75 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -584,7 +584,7 @@
combined_idx = buddy_idx & page_idx;
higher_page = page + (combined_idx - page_idx);
buddy_idx = __find_buddy_index(combined_idx, order + 1);
- higher_buddy = page + (buddy_idx - combined_idx);
+ higher_buddy = higher_page + (buddy_idx - combined_idx);
if (page_is_buddy(higher_page, higher_buddy, order + 1)) {
list_add_tail(&page->lru,
&zone->free_area[order].free_list[migratetype]);
diff --git a/mm/slab.c b/mm/slab.c
index f8b0d53..c685475 100644
--- a/mm/slab.c
+++ b/mm/slab.c
@@ -983,7 +983,7 @@
}
/* The caller cannot use PFMEMALLOC objects, find another one */
- for (i = 1; i < ac->avail; i++) {
+ for (i = 0; i < ac->avail; i++) {
/* If a !PFMEMALLOC object is found, swap them */
if (!is_obj_pfmemalloc(ac->entry[i])) {
objp = ac->entry[i];
@@ -1000,7 +1000,7 @@
l3 = cachep->nodelists[numa_mem_id()];
if (!list_empty(&l3->slabs_free) && force_refill) {
struct slab *slabp = virt_to_slab(objp);
- ClearPageSlabPfmemalloc(virt_to_page(slabp->s_mem));
+ ClearPageSlabPfmemalloc(virt_to_head_page(slabp->s_mem));
clear_obj_pfmemalloc(&objp);
recheck_pfmemalloc_active(cachep, ac);
return objp;
@@ -1032,7 +1032,7 @@
{
if (unlikely(pfmemalloc_active)) {
/* Some pfmemalloc slabs exist, check if this is one */
- struct page *page = virt_to_page(objp);
+ struct page *page = virt_to_head_page(objp);
if (PageSlabPfmemalloc(page))
set_obj_pfmemalloc(&objp);
}
@@ -3260,6 +3260,7 @@
/* cache_grow can reenable interrupts, then ac could change. */
ac = cpu_cache_get(cachep);
+ node = numa_mem_id();
/* no objects in sight? abort */
if (!x && (ac->avail == 0 || force_refill))
diff --git a/mm/slub.c b/mm/slub.c
index 8f78e25..2fdd96f 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1524,12 +1524,13 @@
}
static int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain);
+static inline bool pfmemalloc_match(struct page *page, gfp_t gfpflags);
/*
* Try to allocate a partial slab from a specific node.
*/
-static void *get_partial_node(struct kmem_cache *s,
- struct kmem_cache_node *n, struct kmem_cache_cpu *c)
+static void *get_partial_node(struct kmem_cache *s, struct kmem_cache_node *n,
+ struct kmem_cache_cpu *c, gfp_t flags)
{
struct page *page, *page2;
void *object = NULL;
@@ -1545,9 +1546,13 @@
spin_lock(&n->list_lock);
list_for_each_entry_safe(page, page2, &n->partial, lru) {
- void *t = acquire_slab(s, n, page, object == NULL);
+ void *t;
int available;
+ if (!pfmemalloc_match(page, flags))
+ continue;
+
+ t = acquire_slab(s, n, page, object == NULL);
if (!t)
break;
@@ -1614,7 +1619,7 @@
if (n && cpuset_zone_allowed_hardwall(zone, flags) &&
n->nr_partial > s->min_partial) {
- object = get_partial_node(s, n, c);
+ object = get_partial_node(s, n, c, flags);
if (object) {
/*
* Return the object even if
@@ -1643,7 +1648,7 @@
void *object;
int searchnode = (node == NUMA_NO_NODE) ? numa_node_id() : node;
- object = get_partial_node(s, get_node(s, searchnode), c);
+ object = get_partial_node(s, get_node(s, searchnode), c, flags);
if (object || node != NUMA_NO_NODE)
return object;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 8d01243..99b434b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3102,6 +3102,7 @@
/* failure at boot is fatal */
BUG_ON(system_state == SYSTEM_BOOTING);
printk("Failed to start kswapd on node %d\n",nid);
+ pgdat->kswapd = NULL;
ret = -1;
}
return ret;
diff --git a/net/8021q/vlanproc.c b/net/8021q/vlanproc.c
index c718fd3..4de77ea 100644
--- a/net/8021q/vlanproc.c
+++ b/net/8021q/vlanproc.c
@@ -105,7 +105,7 @@
};
/*
- * Proc filesystem derectory entries.
+ * Proc filesystem directory entries.
*/
/* Strings */
diff --git a/net/batman-adv/bat_iv_ogm.c b/net/batman-adv/bat_iv_ogm.c
index e877af8..469daab 100644
--- a/net/batman-adv/bat_iv_ogm.c
+++ b/net/batman-adv/bat_iv_ogm.c
@@ -642,7 +642,8 @@
struct batadv_neigh_node *router = NULL;
struct batadv_orig_node *orig_node_tmp;
struct hlist_node *node;
- uint8_t bcast_own_sum_orig, bcast_own_sum_neigh;
+ int if_num;
+ uint8_t sum_orig, sum_neigh;
uint8_t *neigh_addr;
batadv_dbg(BATADV_DBG_BATMAN, bat_priv,
@@ -727,17 +728,17 @@
if (router && (neigh_node->tq_avg == router->tq_avg)) {
orig_node_tmp = router->orig_node;
spin_lock_bh(&orig_node_tmp->ogm_cnt_lock);
- bcast_own_sum_orig =
- orig_node_tmp->bcast_own_sum[if_incoming->if_num];
+ if_num = router->if_incoming->if_num;
+ sum_orig = orig_node_tmp->bcast_own_sum[if_num];
spin_unlock_bh(&orig_node_tmp->ogm_cnt_lock);
orig_node_tmp = neigh_node->orig_node;
spin_lock_bh(&orig_node_tmp->ogm_cnt_lock);
- bcast_own_sum_neigh =
- orig_node_tmp->bcast_own_sum[if_incoming->if_num];
+ if_num = neigh_node->if_incoming->if_num;
+ sum_neigh = orig_node_tmp->bcast_own_sum[if_num];
spin_unlock_bh(&orig_node_tmp->ogm_cnt_lock);
- if (bcast_own_sum_orig >= bcast_own_sum_neigh)
+ if (sum_orig >= sum_neigh)
goto update_tt;
}
diff --git a/net/batman-adv/bitarray.h b/net/batman-adv/bitarray.h
index a081ce1..cebaae7 100644
--- a/net/batman-adv/bitarray.h
+++ b/net/batman-adv/bitarray.h
@@ -20,8 +20,8 @@
#ifndef _NET_BATMAN_ADV_BITARRAY_H_
#define _NET_BATMAN_ADV_BITARRAY_H_
-/* returns true if the corresponding bit in the given seq_bits indicates true
- * and curr_seqno is within range of last_seqno
+/* Returns 1 if the corresponding bit in the given seq_bits indicates true
+ * and curr_seqno is within range of last_seqno. Otherwise returns 0.
*/
static inline int batadv_test_bit(const unsigned long *seq_bits,
uint32_t last_seqno, uint32_t curr_seqno)
@@ -32,7 +32,7 @@
if (diff < 0 || diff >= BATADV_TQ_LOCAL_WINDOW_SIZE)
return 0;
else
- return test_bit(diff, seq_bits);
+ return test_bit(diff, seq_bits) != 0;
}
/* turn corresponding bit on, so we can remember that we got the packet */
diff --git a/net/batman-adv/soft-interface.c b/net/batman-adv/soft-interface.c
index 109ea2a..21c5357 100644
--- a/net/batman-adv/soft-interface.c
+++ b/net/batman-adv/soft-interface.c
@@ -100,18 +100,21 @@
{
struct batadv_priv *bat_priv = netdev_priv(dev);
struct sockaddr *addr = p;
+ uint8_t old_addr[ETH_ALEN];
if (!is_valid_ether_addr(addr->sa_data))
return -EADDRNOTAVAIL;
+ memcpy(old_addr, dev->dev_addr, ETH_ALEN);
+ memcpy(dev->dev_addr, addr->sa_data, ETH_ALEN);
+
/* only modify transtable if it has been initialized before */
if (atomic_read(&bat_priv->mesh_state) == BATADV_MESH_ACTIVE) {
- batadv_tt_local_remove(bat_priv, dev->dev_addr,
+ batadv_tt_local_remove(bat_priv, old_addr,
"mac address changed", false);
batadv_tt_local_add(dev, addr->sa_data, BATADV_NULL_IFINDEX);
}
- memcpy(dev->dev_addr, addr->sa_data, ETH_ALEN);
dev->addr_assign_type &= ~NET_ADDR_RANDOM;
return 0;
}
diff --git a/net/bluetooth/bnep/sock.c b/net/bluetooth/bnep/sock.c
index 5e5f5b4..1eaacf1 100644
--- a/net/bluetooth/bnep/sock.c
+++ b/net/bluetooth/bnep/sock.c
@@ -58,7 +58,7 @@
switch (cmd) {
case BNEPCONNADD:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (copy_from_user(&ca, argp, sizeof(ca)))
return -EFAULT;
@@ -84,7 +84,7 @@
case BNEPCONNDEL:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (copy_from_user(&cd, argp, sizeof(cd)))
return -EFAULT;
diff --git a/net/bluetooth/cmtp/sock.c b/net/bluetooth/cmtp/sock.c
index 311668d..32dc83d 100644
--- a/net/bluetooth/cmtp/sock.c
+++ b/net/bluetooth/cmtp/sock.c
@@ -72,7 +72,7 @@
switch (cmd) {
case CMTPCONNADD:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (copy_from_user(&ca, argp, sizeof(ca)))
return -EFAULT;
@@ -97,7 +97,7 @@
case CMTPCONNDEL:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (copy_from_user(&cd, argp, sizeof(cd)))
return -EFAULT;
diff --git a/net/bluetooth/hci_conn.c b/net/bluetooth/hci_conn.c
index 5ad7da2..3c094e7 100644
--- a/net/bluetooth/hci_conn.c
+++ b/net/bluetooth/hci_conn.c
@@ -29,6 +29,7 @@
#include <net/bluetooth/bluetooth.h>
#include <net/bluetooth/hci_core.h>
#include <net/bluetooth/a2mp.h>
+#include <net/bluetooth/smp.h>
static void hci_le_connect(struct hci_conn *conn)
{
@@ -619,6 +620,9 @@
{
BT_DBG("hcon %p", conn);
+ if (conn->type == LE_LINK)
+ return smp_conn_security(conn, sec_level);
+
/* For sdp we don't need the link key. */
if (sec_level == BT_SECURITY_SDP)
return 1;
diff --git a/net/bluetooth/hci_core.c b/net/bluetooth/hci_core.c
index d4de5db..0b997c8f 100644
--- a/net/bluetooth/hci_core.c
+++ b/net/bluetooth/hci_core.c
@@ -734,6 +734,8 @@
cancel_work_sync(&hdev->le_scan);
+ cancel_delayed_work(&hdev->power_off);
+
hci_req_cancel(hdev, ENODEV);
hci_req_lock(hdev);
diff --git a/net/bluetooth/hci_sock.c b/net/bluetooth/hci_sock.c
index 19fdac7..d5ace1e 100644
--- a/net/bluetooth/hci_sock.c
+++ b/net/bluetooth/hci_sock.c
@@ -490,7 +490,7 @@
switch (cmd) {
case HCISETRAW:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (test_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks))
return -EPERM;
@@ -510,12 +510,12 @@
case HCIBLOCKADDR:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_sock_blacklist_add(hdev, (void __user *) arg);
case HCIUNBLOCKADDR:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_sock_blacklist_del(hdev, (void __user *) arg);
default:
@@ -546,22 +546,22 @@
case HCIDEVUP:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_dev_open(arg);
case HCIDEVDOWN:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_dev_close(arg);
case HCIDEVRESET:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_dev_reset(arg);
case HCIDEVRESTAT:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_dev_reset_stat(arg);
case HCISETSCAN:
@@ -573,7 +573,7 @@
case HCISETACLMTU:
case HCISETSCOMTU:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
return hci_dev_cmd(cmd, argp);
case HCIINQUIRY:
diff --git a/net/bluetooth/hidp/sock.c b/net/bluetooth/hidp/sock.c
index 18b3f68..b24fb3b 100644
--- a/net/bluetooth/hidp/sock.c
+++ b/net/bluetooth/hidp/sock.c
@@ -56,7 +56,7 @@
switch (cmd) {
case HIDPCONNADD:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (copy_from_user(&ca, argp, sizeof(ca)))
return -EFAULT;
@@ -91,7 +91,7 @@
case HIDPCONNDEL:
if (!capable(CAP_NET_ADMIN))
- return -EACCES;
+ return -EPERM;
if (copy_from_user(&cd, argp, sizeof(cd)))
return -EFAULT;
diff --git a/net/bluetooth/l2cap_core.c b/net/bluetooth/l2cap_core.c
index daa149b..38c00f1 100644
--- a/net/bluetooth/l2cap_core.c
+++ b/net/bluetooth/l2cap_core.c
@@ -1008,7 +1008,7 @@
if (!conn)
return;
- if (chan->mode == L2CAP_MODE_ERTM) {
+ if (chan->mode == L2CAP_MODE_ERTM && chan->state == BT_CONNECTED) {
__clear_retrans_timer(chan);
__clear_monitor_timer(chan);
__clear_ack_timer(chan);
@@ -1199,14 +1199,15 @@
static void l2cap_conn_ready(struct l2cap_conn *conn)
{
struct l2cap_chan *chan;
+ struct hci_conn *hcon = conn->hcon;
BT_DBG("conn %p", conn);
- if (!conn->hcon->out && conn->hcon->type == LE_LINK)
+ if (!hcon->out && hcon->type == LE_LINK)
l2cap_le_conn_ready(conn);
- if (conn->hcon->out && conn->hcon->type == LE_LINK)
- smp_conn_security(conn, conn->hcon->pending_sec_level);
+ if (hcon->out && hcon->type == LE_LINK)
+ smp_conn_security(hcon, hcon->pending_sec_level);
mutex_lock(&conn->chan_lock);
@@ -1219,8 +1220,8 @@
continue;
}
- if (conn->hcon->type == LE_LINK) {
- if (smp_conn_security(conn, chan->sec_level))
+ if (hcon->type == LE_LINK) {
+ if (smp_conn_security(hcon, chan->sec_level))
l2cap_chan_ready(chan);
} else if (chan->chan_type != L2CAP_CHAN_CONN_ORIENTED) {
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index 1497edd..34bbe1c 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -616,7 +616,7 @@
break;
}
- if (smp_conn_security(conn, sec.level))
+ if (smp_conn_security(conn->hcon, sec.level))
break;
sk->sk_state = BT_CONFIG;
chan->state = BT_CONFIG;
diff --git a/net/bluetooth/mgmt.c b/net/bluetooth/mgmt.c
index ad6613d..eba022d 100644
--- a/net/bluetooth/mgmt.c
+++ b/net/bluetooth/mgmt.c
@@ -2875,6 +2875,22 @@
if (scan)
hci_send_cmd(hdev, HCI_OP_WRITE_SCAN_ENABLE, 1, &scan);
+ if (test_bit(HCI_SSP_ENABLED, &hdev->dev_flags)) {
+ u8 ssp = 1;
+
+ hci_send_cmd(hdev, HCI_OP_WRITE_SSP_MODE, 1, &ssp);
+ }
+
+ if (test_bit(HCI_LE_ENABLED, &hdev->dev_flags)) {
+ struct hci_cp_write_le_host_supported cp;
+
+ cp.le = 1;
+ cp.simul = !!(hdev->features[6] & LMP_SIMUL_LE_BR);
+
+ hci_send_cmd(hdev, HCI_OP_WRITE_LE_HOST_SUPPORTED,
+ sizeof(cp), &cp);
+ }
+
update_class(hdev);
update_name(hdev, hdev->dev_name);
update_eir(hdev);
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index 901a616..8c225ef 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -267,10 +267,10 @@
mgmt_auth_failed(conn->hcon->hdev, conn->dst, hcon->type,
hcon->dst_type, reason);
- if (test_and_clear_bit(HCI_CONN_LE_SMP_PEND, &conn->hcon->flags)) {
- cancel_delayed_work_sync(&conn->security_timer);
+ cancel_delayed_work_sync(&conn->security_timer);
+
+ if (test_and_clear_bit(HCI_CONN_LE_SMP_PEND, &conn->hcon->flags))
smp_chan_destroy(conn);
- }
}
#define JUST_WORKS 0x00
@@ -760,9 +760,9 @@
return 0;
}
-int smp_conn_security(struct l2cap_conn *conn, __u8 sec_level)
+int smp_conn_security(struct hci_conn *hcon, __u8 sec_level)
{
- struct hci_conn *hcon = conn->hcon;
+ struct l2cap_conn *conn = hcon->l2cap_data;
struct smp_chan *smp = conn->smp_chan;
__u8 authreq;
diff --git a/net/bridge/netfilter/ebt_log.c b/net/bridge/netfilter/ebt_log.c
index f88ee53..92de5e5 100644
--- a/net/bridge/netfilter/ebt_log.c
+++ b/net/bridge/netfilter/ebt_log.c
@@ -80,7 +80,7 @@
unsigned int bitmask;
spin_lock_bh(&ebt_log_lock);
- printk("<%c>%s IN=%s OUT=%s MAC source = %pM MAC dest = %pM proto = 0x%04x",
+ printk(KERN_SOH "%c%s IN=%s OUT=%s MAC source = %pM MAC dest = %pM proto = 0x%04x",
'0' + loginfo->u.log.level, prefix,
in ? in->name : "", out ? out->name : "",
eth_hdr(skb)->h_source, eth_hdr(skb)->h_dest,
diff --git a/net/caif/cfsrvl.c b/net/caif/cfsrvl.c
index dd485f6..ba217e9 100644
--- a/net/caif/cfsrvl.c
+++ b/net/caif/cfsrvl.c
@@ -211,9 +211,10 @@
void (*put)(struct cflayer *lyr))
{
struct cfsrvl *service;
- service = container_of(adapt_layer->dn, struct cfsrvl, layer);
- WARN_ON(adapt_layer == NULL || adapt_layer->dn == NULL);
+ if (WARN_ON(adapt_layer == NULL || adapt_layer->dn == NULL))
+ return;
+ service = container_of(adapt_layer->dn, struct cfsrvl, layer);
service->hold = hold;
service->put = put;
}
diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c
index 24c5eea..159aa8b 100644
--- a/net/ceph/messenger.c
+++ b/net/ceph/messenger.c
@@ -1073,16 +1073,13 @@
BUG_ON(kaddr == NULL);
base = kaddr + con->out_msg_pos.page_pos + bio_offset;
crc = crc32c(crc, base, len);
+ kunmap(page);
msg->footer.data_crc = cpu_to_le32(crc);
con->out_msg_pos.did_page_crc = true;
}
ret = ceph_tcp_sendpage(con->sock, page,
con->out_msg_pos.page_pos + bio_offset,
len, 1);
-
- if (do_datacrc)
- kunmap(page);
-
if (ret <= 0)
goto out;
diff --git a/net/core/dev.c b/net/core/dev.c
index 8398836..89e33a5 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2134,7 +2134,8 @@
static netdev_features_t harmonize_features(struct sk_buff *skb,
__be16 protocol, netdev_features_t features)
{
- if (!can_checksum_protocol(features, protocol)) {
+ if (skb->ip_summed != CHECKSUM_NONE &&
+ !can_checksum_protocol(features, protocol)) {
features &= ~NETIF_F_ALL_CSUM;
features &= ~NETIF_F_SG;
} else if (illegal_highdma(skb->dev, skb)) {
@@ -2647,15 +2648,16 @@
if (!skb_flow_dissect(skb, &keys))
return;
- if (keys.ports) {
- if ((__force u16)keys.port16[1] < (__force u16)keys.port16[0])
- swap(keys.port16[0], keys.port16[1]);
+ if (keys.ports)
skb->l4_rxhash = 1;
- }
/* get a consistent hash (same value on both flow directions) */
- if ((__force u32)keys.dst < (__force u32)keys.src)
+ if (((__force u32)keys.dst < (__force u32)keys.src) ||
+ (((__force u32)keys.dst == (__force u32)keys.src) &&
+ ((__force u16)keys.port16[1] < (__force u16)keys.port16[0]))) {
swap(keys.dst, keys.src);
+ swap(keys.port16[0], keys.port16[1]);
+ }
hash = jhash_3words((__force u32)keys.dst,
(__force u32)keys.src,
@@ -3321,7 +3323,7 @@
if (pt_prev) {
if (unlikely(skb_orphan_frags(skb, GFP_ATOMIC)))
- ret = -ENOMEM;
+ goto drop;
else
ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev);
} else {
diff --git a/net/core/netpoll.c b/net/core/netpoll.c
index 346b1eb..e4ba3e7 100644
--- a/net/core/netpoll.c
+++ b/net/core/netpoll.c
@@ -168,24 +168,16 @@
struct napi_struct *napi;
int budget = 16;
- WARN_ON_ONCE(!irqs_disabled());
-
list_for_each_entry(napi, &dev->napi_list, dev_list) {
- local_irq_enable();
if (napi->poll_owner != smp_processor_id() &&
spin_trylock(&napi->poll_lock)) {
- rcu_read_lock_bh();
budget = poll_one_napi(rcu_dereference_bh(dev->npinfo),
napi, budget);
- rcu_read_unlock_bh();
spin_unlock(&napi->poll_lock);
- if (!budget) {
- local_irq_disable();
+ if (!budget)
break;
- }
}
- local_irq_disable();
}
}
diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index cce9e53..148e73d 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -2721,7 +2721,7 @@
/* Eth + IPh + UDPh + mpls */
datalen = pkt_dev->cur_pkt_size - 14 - 20 - 8 -
pkt_dev->pkt_overhead;
- if (datalen < sizeof(struct pktgen_hdr))
+ if (datalen < 0 || datalen < sizeof(struct pktgen_hdr))
datalen = sizeof(struct pktgen_hdr);
udph->source = htons(pkt_dev->cur_udp_src);
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index fe00d12..e33ebae 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -3502,7 +3502,9 @@
if (!skb_cloned(from))
skb_shinfo(from)->nr_frags = 0;
- /* if the skb is cloned this does nothing since we set nr_frags to 0 */
+ /* if the skb is not cloned this does nothing
+ * since we set nr_frags to 0.
+ */
for (i = 0; i < skb_shinfo(from)->nr_frags; i++)
skb_frag_ref(from, i);
diff --git a/net/core/sock.c b/net/core/sock.c
index 8f67ced..a6000fb 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -691,7 +691,8 @@
case SO_KEEPALIVE:
#ifdef CONFIG_INET
- if (sk->sk_protocol == IPPROTO_TCP)
+ if (sk->sk_protocol == IPPROTO_TCP &&
+ sk->sk_type == SOCK_STREAM)
tcp_set_keepalive(sk, valbool);
#endif
sock_valbool_flag(sk, SOCK_KEEPOPEN, valbool);
@@ -1523,7 +1524,14 @@
void sock_edemux(struct sk_buff *skb)
{
- sock_put(skb->sk);
+ struct sock *sk = skb->sk;
+
+#ifdef CONFIG_INET
+ if (sk->sk_state == TCP_TIME_WAIT)
+ inet_twsk_put(inet_twsk(sk));
+ else
+#endif
+ sock_put(sk);
}
EXPORT_SYMBOL(sock_edemux);
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 77e87af..4780045 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -1225,7 +1225,7 @@
switch (event) {
case NETDEV_CHANGEADDR:
neigh_changeaddr(&arp_tbl, dev);
- rt_cache_flush(dev_net(dev), 0);
+ rt_cache_flush(dev_net(dev));
break;
default:
break;
diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c
index 44bf82e..e12fad7 100644
--- a/net/ipv4/devinet.c
+++ b/net/ipv4/devinet.c
@@ -725,7 +725,7 @@
break;
case SIOCSIFFLAGS:
- ret = -EACCES;
+ ret = -EPERM;
if (!capable(CAP_NET_ADMIN))
goto out;
break;
@@ -733,7 +733,7 @@
case SIOCSIFBRDADDR: /* Set the broadcast address */
case SIOCSIFDSTADDR: /* Set the destination address */
case SIOCSIFNETMASK: /* Set the netmask for the interface */
- ret = -EACCES;
+ ret = -EPERM;
if (!capable(CAP_NET_ADMIN))
goto out;
ret = -EINVAL;
@@ -1503,7 +1503,7 @@
if (i == IPV4_DEVCONF_ACCEPT_LOCAL - 1 ||
i == IPV4_DEVCONF_ROUTE_LOCALNET - 1)
if ((new_value == 0) && (old_value != 0))
- rt_cache_flush(net, 0);
+ rt_cache_flush(net);
}
return ret;
@@ -1537,7 +1537,7 @@
dev_disable_lro(idev->dev);
}
rtnl_unlock();
- rt_cache_flush(net, 0);
+ rt_cache_flush(net);
}
}
@@ -1554,7 +1554,7 @@
struct net *net = ctl->extra2;
if (write && *valp != val)
- rt_cache_flush(net, 0);
+ rt_cache_flush(net);
return ret;
}
diff --git a/net/ipv4/fib_frontend.c b/net/ipv4/fib_frontend.c
index c43ae3f..8e2b475 100644
--- a/net/ipv4/fib_frontend.c
+++ b/net/ipv4/fib_frontend.c
@@ -148,7 +148,7 @@
}
if (flushed)
- rt_cache_flush(net, -1);
+ rt_cache_flush(net);
}
/*
@@ -999,11 +999,11 @@
net->ipv4.fibnl = NULL;
}
-static void fib_disable_ip(struct net_device *dev, int force, int delay)
+static void fib_disable_ip(struct net_device *dev, int force)
{
if (fib_sync_down_dev(dev, force))
fib_flush(dev_net(dev));
- rt_cache_flush(dev_net(dev), delay);
+ rt_cache_flush(dev_net(dev));
arp_ifdown(dev);
}
@@ -1020,7 +1020,7 @@
fib_sync_up(dev);
#endif
atomic_inc(&net->ipv4.dev_addr_genid);
- rt_cache_flush(dev_net(dev), -1);
+ rt_cache_flush(dev_net(dev));
break;
case NETDEV_DOWN:
fib_del_ifaddr(ifa, NULL);
@@ -1029,9 +1029,9 @@
/* Last address was deleted from this interface.
* Disable IP.
*/
- fib_disable_ip(dev, 1, 0);
+ fib_disable_ip(dev, 1);
} else {
- rt_cache_flush(dev_net(dev), -1);
+ rt_cache_flush(dev_net(dev));
}
break;
}
@@ -1045,7 +1045,7 @@
struct net *net = dev_net(dev);
if (event == NETDEV_UNREGISTER) {
- fib_disable_ip(dev, 2, -1);
+ fib_disable_ip(dev, 2);
rt_flush_dev(dev);
return NOTIFY_DONE;
}
@@ -1062,14 +1062,14 @@
fib_sync_up(dev);
#endif
atomic_inc(&net->ipv4.dev_addr_genid);
- rt_cache_flush(dev_net(dev), -1);
+ rt_cache_flush(dev_net(dev));
break;
case NETDEV_DOWN:
- fib_disable_ip(dev, 0, 0);
+ fib_disable_ip(dev, 0);
break;
case NETDEV_CHANGEMTU:
case NETDEV_CHANGE:
- rt_cache_flush(dev_net(dev), 0);
+ rt_cache_flush(dev_net(dev));
break;
case NETDEV_UNREGISTER_BATCH:
break;
diff --git a/net/ipv4/fib_rules.c b/net/ipv4/fib_rules.c
index a83d74e..274309d 100644
--- a/net/ipv4/fib_rules.c
+++ b/net/ipv4/fib_rules.c
@@ -259,7 +259,7 @@
static void fib4_rule_flush_cache(struct fib_rules_ops *ops)
{
- rt_cache_flush(ops->fro_net, -1);
+ rt_cache_flush(ops->fro_net);
}
static const struct fib_rules_ops __net_initdata fib4_rules_ops_template = {
diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c
index 57bd978..d1b9359 100644
--- a/net/ipv4/fib_trie.c
+++ b/net/ipv4/fib_trie.c
@@ -1286,7 +1286,7 @@
fib_release_info(fi_drop);
if (state & FA_S_ACCESSED)
- rt_cache_flush(cfg->fc_nlinfo.nl_net, -1);
+ rt_cache_flush(cfg->fc_nlinfo.nl_net);
rtmsg_fib(RTM_NEWROUTE, htonl(key), new_fa, plen,
tb->tb_id, &cfg->fc_nlinfo, NLM_F_REPLACE);
@@ -1333,7 +1333,7 @@
list_add_tail_rcu(&new_fa->fa_list,
(fa ? &fa->fa_list : fa_head));
- rt_cache_flush(cfg->fc_nlinfo.nl_net, -1);
+ rt_cache_flush(cfg->fc_nlinfo.nl_net);
rtmsg_fib(RTM_NEWROUTE, htonl(key), new_fa, plen, tb->tb_id,
&cfg->fc_nlinfo, 0);
succeeded:
@@ -1708,7 +1708,7 @@
trie_leaf_remove(t, l);
if (fa->fa_state & FA_S_ACCESSED)
- rt_cache_flush(cfg->fc_nlinfo.nl_net, -1);
+ rt_cache_flush(cfg->fc_nlinfo.nl_net);
fib_release_info(fa->fa_info);
alias_free_mem_rcu(fa);
diff --git a/net/ipv4/inetpeer.c b/net/ipv4/inetpeer.c
index e1e0a4e..c7527f6 100644
--- a/net/ipv4/inetpeer.c
+++ b/net/ipv4/inetpeer.c
@@ -510,7 +510,10 @@
secure_ipv6_id(daddr->addr.a6));
p->metrics[RTAX_LOCK-1] = INETPEER_METRICS_NEW;
p->rate_tokens = 0;
- p->rate_last = 0;
+ /* 60*HZ is arbitrary, but chosen enough high so that the first
+ * calculation of tokens is at its maximum.
+ */
+ p->rate_last = jiffies - 60*HZ;
INIT_LIST_HEAD(&p->gc_list);
/* Link the node. */
diff --git a/net/ipv4/ipmr.c b/net/ipv4/ipmr.c
index 8eec8f4..ebdf06f 100644
--- a/net/ipv4/ipmr.c
+++ b/net/ipv4/ipmr.c
@@ -124,6 +124,8 @@
static struct kmem_cache *mrt_cachep __read_mostly;
static struct mr_table *ipmr_new_table(struct net *net, u32 id);
+static void ipmr_free_table(struct mr_table *mrt);
+
static int ip_mr_forward(struct net *net, struct mr_table *mrt,
struct sk_buff *skb, struct mfc_cache *cache,
int local);
@@ -131,6 +133,7 @@
struct sk_buff *pkt, vifi_t vifi, int assert);
static int __ipmr_fill_mroute(struct mr_table *mrt, struct sk_buff *skb,
struct mfc_cache *c, struct rtmsg *rtm);
+static void mroute_clean_tables(struct mr_table *mrt);
static void ipmr_expire_process(unsigned long arg);
#ifdef CONFIG_IP_MROUTE_MULTIPLE_TABLES
@@ -271,7 +274,7 @@
list_for_each_entry_safe(mrt, next, &net->ipv4.mr_tables, list) {
list_del(&mrt->list);
- kfree(mrt);
+ ipmr_free_table(mrt);
}
fib_rules_unregister(net->ipv4.mr_rules_ops);
}
@@ -299,7 +302,7 @@
static void __net_exit ipmr_rules_exit(struct net *net)
{
- kfree(net->ipv4.mrt);
+ ipmr_free_table(net->ipv4.mrt);
}
#endif
@@ -336,6 +339,13 @@
return mrt;
}
+static void ipmr_free_table(struct mr_table *mrt)
+{
+ del_timer_sync(&mrt->ipmr_expire_timer);
+ mroute_clean_tables(mrt);
+ kfree(mrt);
+}
+
/* Service routines creating virtual interfaces: DVMRP tunnels and PIMREG */
static void ipmr_del_tunnel(struct net_device *dev, struct vifctl *v)
diff --git a/net/ipv4/netfilter/nf_nat_sip.c b/net/ipv4/netfilter/nf_nat_sip.c
index 4ad9cf1..9c87cde 100644
--- a/net/ipv4/netfilter/nf_nat_sip.c
+++ b/net/ipv4/netfilter/nf_nat_sip.c
@@ -502,7 +502,10 @@
ret = nf_ct_expect_related(rtcp_exp);
if (ret == 0)
break;
- else if (ret != -EBUSY) {
+ else if (ret == -EBUSY) {
+ nf_ct_unexpect_related(rtp_exp);
+ continue;
+ } else if (ret < 0) {
nf_ct_unexpect_related(rtp_exp);
port = 0;
break;
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index ff0f071..d23c657 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -131,18 +131,20 @@
* 0 - deliver
* 1 - block
*/
-static __inline__ int icmp_filter(struct sock *sk, struct sk_buff *skb)
+static int icmp_filter(const struct sock *sk, const struct sk_buff *skb)
{
- int type;
+ struct icmphdr _hdr;
+ const struct icmphdr *hdr;
- if (!pskb_may_pull(skb, sizeof(struct icmphdr)))
+ hdr = skb_header_pointer(skb, skb_transport_offset(skb),
+ sizeof(_hdr), &_hdr);
+ if (!hdr)
return 1;
- type = icmp_hdr(skb)->type;
- if (type < 32) {
+ if (hdr->type < 32) {
__u32 data = raw_sk(sk)->filter.data;
- return ((1 << type) & data) != 0;
+ return ((1U << hdr->type) & data) != 0;
}
/* Do not block unknown ICMP types */
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index fd9ecb5..fd9af60 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -202,11 +202,6 @@
static DEFINE_PER_CPU(struct rt_cache_stat, rt_cache_stat);
#define RT_CACHE_STAT_INC(field) __this_cpu_inc(rt_cache_stat.field)
-static inline int rt_genid(struct net *net)
-{
- return atomic_read(&net->ipv4.rt_genid);
-}
-
#ifdef CONFIG_PROC_FS
static void *rt_cache_seq_start(struct seq_file *seq, loff_t *pos)
{
@@ -447,27 +442,9 @@
return rth->rt_genid != rt_genid(dev_net(rth->dst.dev));
}
-/*
- * Perturbation of rt_genid by a small quantity [1..256]
- * Using 8 bits of shuffling ensure we can call rt_cache_invalidate()
- * many times (2^24) without giving recent rt_genid.
- * Jenkins hash is strong enough that litle changes of rt_genid are OK.
- */
-static void rt_cache_invalidate(struct net *net)
+void rt_cache_flush(struct net *net)
{
- unsigned char shuffle;
-
- get_random_bytes(&shuffle, sizeof(shuffle));
- atomic_add(shuffle + 1U, &net->ipv4.rt_genid);
-}
-
-/*
- * delay < 0 : invalidate cache (fast : entries will be deleted later)
- * delay >= 0 : invalidate & flush cache (can be long)
- */
-void rt_cache_flush(struct net *net, int delay)
-{
- rt_cache_invalidate(net);
+ rt_genid_bump(net);
}
static struct neighbour *ipv4_neigh_lookup(const struct dst_entry *dst,
@@ -934,12 +911,14 @@
if (mtu < ip_rt_min_pmtu)
mtu = ip_rt_min_pmtu;
+ rcu_read_lock();
if (fib_lookup(dev_net(rt->dst.dev), fl4, &res) == 0) {
struct fib_nh *nh = &FIB_RES_NH(res);
update_or_create_fnhe(nh, fl4->daddr, 0, mtu,
jiffies + ip_rt_mtu_expires);
}
+ rcu_read_unlock();
return mtu;
}
@@ -956,7 +935,7 @@
dst->obsolete = DST_OBSOLETE_KILL;
} else {
rt->rt_pmtu = mtu;
- dst_set_expires(&rt->dst, ip_rt_mtu_expires);
+ rt->dst.expires = max(1UL, jiffies + ip_rt_mtu_expires);
}
}
@@ -1263,7 +1242,7 @@
{
struct rtable *rt = (struct rtable *) dst;
- if (dst->flags & DST_NOCACHE) {
+ if (!list_empty(&rt->rt_uncached)) {
spin_lock_bh(&rt_uncached_lock);
list_del(&rt->rt_uncached);
spin_unlock_bh(&rt_uncached_lock);
@@ -2343,7 +2322,7 @@
void ip_rt_multicast_event(struct in_device *in_dev)
{
- rt_cache_flush(dev_net(in_dev->dev), 0);
+ rt_cache_flush(dev_net(in_dev->dev));
}
#ifdef CONFIG_SYSCTL
@@ -2352,16 +2331,7 @@
size_t *lenp, loff_t *ppos)
{
if (write) {
- int flush_delay;
- ctl_table ctl;
- struct net *net;
-
- memcpy(&ctl, __ctl, sizeof(ctl));
- ctl.data = &flush_delay;
- proc_dointvec(&ctl, write, buffer, lenp, ppos);
-
- net = (struct net *)__ctl->extra1;
- rt_cache_flush(net, flush_delay);
+ rt_cache_flush((struct net *)__ctl->extra1);
return 0;
}
@@ -2531,8 +2501,7 @@
static __net_init int rt_genid_init(struct net *net)
{
- get_random_bytes(&net->ipv4.rt_genid,
- sizeof(net->ipv4.rt_genid));
+ atomic_set(&net->rt_genid, 0);
get_random_bytes(&net->ipv4.dev_addr_genid,
sizeof(net->ipv4.dev_addr_genid));
return 0;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 2109ff4..5f64193 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -1762,8 +1762,14 @@
}
#ifdef CONFIG_NET_DMA
- if (tp->ucopy.dma_chan)
- dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
+ if (tp->ucopy.dma_chan) {
+ if (tp->rcv_wnd == 0 &&
+ !skb_queue_empty(&sk->sk_async_wait_queue)) {
+ tcp_service_net_dma(sk, true);
+ tcp_cleanup_rbuf(sk, copied);
+ } else
+ dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
+ }
#endif
if (copied >= target) {
/* Do not sleep, just process backlog. */
@@ -2325,10 +2331,17 @@
tp->rx_opt.mss_clamp = opt.opt_val;
break;
case TCPOPT_WINDOW:
- if (opt.opt_val > 14)
- return -EFBIG;
+ {
+ u16 snd_wscale = opt.opt_val & 0xFFFF;
+ u16 rcv_wscale = opt.opt_val >> 16;
- tp->rx_opt.snd_wscale = opt.opt_val;
+ if (snd_wscale > 14 || rcv_wscale > 14)
+ return -EFBIG;
+
+ tp->rx_opt.snd_wscale = snd_wscale;
+ tp->rx_opt.rcv_wscale = rcv_wscale;
+ tp->rx_opt.wscale_ok = 1;
+ }
break;
case TCPOPT_SACK_PERM:
if (opt.opt_val != 0)
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 85308b9..d377f48 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2926,13 +2926,14 @@
* tcp_xmit_retransmit_queue().
*/
static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked,
- int newly_acked_sacked, bool is_dupack,
+ int prior_sacked, bool is_dupack,
int flag)
{
struct inet_connection_sock *icsk = inet_csk(sk);
struct tcp_sock *tp = tcp_sk(sk);
int do_lost = is_dupack || ((flag & FLAG_DATA_SACKED) &&
(tcp_fackets_out(tp) > tp->reordering));
+ int newly_acked_sacked = 0;
int fast_rexmit = 0;
if (WARN_ON(!tp->packets_out && tp->sacked_out))
@@ -2992,6 +2993,7 @@
tcp_add_reno_sack(sk);
} else
do_lost = tcp_try_undo_partial(sk, pkts_acked);
+ newly_acked_sacked = pkts_acked + tp->sacked_out - prior_sacked;
break;
case TCP_CA_Loss:
if (flag & FLAG_DATA_ACKED)
@@ -3013,6 +3015,7 @@
if (is_dupack)
tcp_add_reno_sack(sk);
}
+ newly_acked_sacked = pkts_acked + tp->sacked_out - prior_sacked;
if (icsk->icsk_ca_state <= TCP_CA_Disorder)
tcp_try_undo_dsack(sk);
@@ -3590,7 +3593,6 @@
int prior_packets;
int prior_sacked = tp->sacked_out;
int pkts_acked = 0;
- int newly_acked_sacked = 0;
bool frto_cwnd = false;
/* If the ack is older than previous acks
@@ -3666,8 +3668,6 @@
flag |= tcp_clean_rtx_queue(sk, prior_fackets, prior_snd_una);
pkts_acked = prior_packets - tp->packets_out;
- newly_acked_sacked = (prior_packets - prior_sacked) -
- (tp->packets_out - tp->sacked_out);
if (tp->frto_counter)
frto_cwnd = tcp_process_frto(sk, flag);
@@ -3681,7 +3681,7 @@
tcp_may_raise_cwnd(sk, flag))
tcp_cong_avoid(sk, ack, prior_in_flight);
is_dupack = !(flag & (FLAG_SND_UNA_ADVANCED | FLAG_NOT_DUP));
- tcp_fastretrans_alert(sk, pkts_acked, newly_acked_sacked,
+ tcp_fastretrans_alert(sk, pkts_acked, prior_sacked,
is_dupack, flag);
} else {
if ((flag & FLAG_DATA_ACKED) && !frto_cwnd)
@@ -3698,7 +3698,7 @@
no_queue:
/* If data was DSACKed, see if we can undo a cwnd reduction. */
if (flag & FLAG_DSACKING_ACK)
- tcp_fastretrans_alert(sk, pkts_acked, newly_acked_sacked,
+ tcp_fastretrans_alert(sk, pkts_acked, prior_sacked,
is_dupack, flag);
/* If this ack opens up a zero window, clear backoff. It was
* being used to time the probes, and is probably far higher than
@@ -3718,8 +3718,7 @@
*/
if (TCP_SKB_CB(skb)->sacked) {
flag |= tcp_sacktag_write_queue(sk, skb, prior_snd_una);
- newly_acked_sacked = tp->sacked_out - prior_sacked;
- tcp_fastretrans_alert(sk, pkts_acked, newly_acked_sacked,
+ tcp_fastretrans_alert(sk, pkts_acked, prior_sacked,
is_dupack, flag);
}
@@ -4662,7 +4661,7 @@
if (eaten > 0)
kfree_skb_partial(skb, fragstolen);
- else if (!sock_flag(sk, SOCK_DEAD))
+ if (!sock_flag(sk, SOCK_DEAD))
sk->sk_data_ready(sk, 0);
return;
}
@@ -5557,8 +5556,7 @@
#endif
if (eaten)
kfree_skb_partial(skb, fragstolen);
- else
- sk->sk_data_ready(sk, 0);
+ sk->sk_data_ready(sk, 0);
return 0;
}
}
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 6f6d1ac..2814f66 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -1226,6 +1226,11 @@
if (unlikely(err)) {
trace_kfree_skb(skb, udp_recvmsg);
+ if (!peeked) {
+ atomic_inc(&sk->sk_drops);
+ UDP_INC_STATS_USER(sock_net(sk),
+ UDP_MIB_INERRORS, is_udplite);
+ }
goto out_free;
}
diff --git a/net/ipv6/esp6.c b/net/ipv6/esp6.c
index 6dc7fd3..282f372 100644
--- a/net/ipv6/esp6.c
+++ b/net/ipv6/esp6.c
@@ -167,8 +167,6 @@
struct esp_data *esp = x->data;
/* skb is pure payload to encrypt */
- err = -ENOMEM;
-
aead = esp->aead;
alen = crypto_aead_authsize(aead);
@@ -203,8 +201,10 @@
}
tmp = esp_alloc_tmp(aead, nfrags + sglists, seqhilen);
- if (!tmp)
+ if (!tmp) {
+ err = -ENOMEM;
goto error;
+ }
seqhi = esp_tmp_seqhi(tmp);
iv = esp_tmp_iv(aead, tmp, seqhilen);
diff --git a/net/ipv6/inet6_connection_sock.c b/net/ipv6/inet6_connection_sock.c
index 0251a60..c4f9341 100644
--- a/net/ipv6/inet6_connection_sock.c
+++ b/net/ipv6/inet6_connection_sock.c
@@ -175,33 +175,12 @@
const struct in6_addr *saddr)
{
__ip6_dst_store(sk, dst, daddr, saddr);
-
-#ifdef CONFIG_XFRM
- {
- struct rt6_info *rt = (struct rt6_info *)dst;
- rt->rt6i_flow_cache_genid = atomic_read(&flow_cache_genid);
- }
-#endif
}
static inline
struct dst_entry *__inet6_csk_dst_check(struct sock *sk, u32 cookie)
{
- struct dst_entry *dst;
-
- dst = __sk_dst_check(sk, cookie);
-
-#ifdef CONFIG_XFRM
- if (dst) {
- struct rt6_info *rt = (struct rt6_info *)dst;
- if (rt->rt6i_flow_cache_genid != atomic_read(&flow_cache_genid)) {
- __sk_dst_reset(sk);
- dst = NULL;
- }
- }
-#endif
-
- return dst;
+ return __sk_dst_check(sk, cookie);
}
static struct dst_entry *inet6_csk_route_socket(struct sock *sk,
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 13690d6..286acfc 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -819,6 +819,10 @@
offsetof(struct rt6_info, rt6i_src),
allow_create, replace_required);
+ if (IS_ERR(sn)) {
+ err = PTR_ERR(sn);
+ sn = NULL;
+ }
if (!sn) {
/* If it is failed, discard just allocated
root, and then (in st_failure) stale node
diff --git a/net/ipv6/mip6.c b/net/ipv6/mip6.c
index 5b087c3..0f9bdc5 100644
--- a/net/ipv6/mip6.c
+++ b/net/ipv6/mip6.c
@@ -86,28 +86,30 @@
static int mip6_mh_filter(struct sock *sk, struct sk_buff *skb)
{
- struct ip6_mh *mh;
+ struct ip6_mh _hdr;
+ const struct ip6_mh *mh;
- if (!pskb_may_pull(skb, (skb_transport_offset(skb)) + 8) ||
- !pskb_may_pull(skb, (skb_transport_offset(skb) +
- ((skb_transport_header(skb)[1] + 1) << 3))))
+ mh = skb_header_pointer(skb, skb_transport_offset(skb),
+ sizeof(_hdr), &_hdr);
+ if (!mh)
return -1;
- mh = (struct ip6_mh *)skb_transport_header(skb);
+ if (((mh->ip6mh_hdrlen + 1) << 3) > skb->len)
+ return -1;
if (mh->ip6mh_hdrlen < mip6_mh_len(mh->ip6mh_type)) {
LIMIT_NETDEBUG(KERN_DEBUG "mip6: MH message too short: %d vs >=%d\n",
mh->ip6mh_hdrlen, mip6_mh_len(mh->ip6mh_type));
- mip6_param_prob(skb, 0, ((&mh->ip6mh_hdrlen) -
- skb_network_header(skb)));
+ mip6_param_prob(skb, 0, offsetof(struct ip6_mh, ip6mh_hdrlen) +
+ skb_network_header_len(skb));
return -1;
}
if (mh->ip6mh_proto != IPPROTO_NONE) {
LIMIT_NETDEBUG(KERN_DEBUG "mip6: MH invalid payload proto = %d\n",
mh->ip6mh_proto);
- mip6_param_prob(skb, 0, ((&mh->ip6mh_proto) -
- skb_network_header(skb)));
+ mip6_param_prob(skb, 0, offsetof(struct ip6_mh, ip6mh_proto) +
+ skb_network_header_len(skb));
return -1;
}
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index ef0579d..4a5f78b 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -107,21 +107,20 @@
* 0 - deliver
* 1 - block
*/
-static __inline__ int icmpv6_filter(struct sock *sk, struct sk_buff *skb)
+static int icmpv6_filter(const struct sock *sk, const struct sk_buff *skb)
{
- struct icmp6hdr *icmph;
- struct raw6_sock *rp = raw6_sk(sk);
+ struct icmp6hdr *_hdr;
+ const struct icmp6hdr *hdr;
- if (pskb_may_pull(skb, sizeof(struct icmp6hdr))) {
- __u32 *data = &rp->filter.data[0];
- int bit_nr;
+ hdr = skb_header_pointer(skb, skb_transport_offset(skb),
+ sizeof(_hdr), &_hdr);
+ if (hdr) {
+ const __u32 *data = &raw6_sk(sk)->filter.data[0];
+ unsigned int type = hdr->icmp6_type;
- icmph = (struct icmp6hdr *) skb->data;
- bit_nr = icmph->icmp6_type;
-
- return (data[bit_nr >> 5] & (1 << (bit_nr & 31))) != 0;
+ return (data[type >> 5] & (1U << (type & 31))) != 0;
}
- return 0;
+ return 1;
}
#if defined(CONFIG_IPV6_MIP6) || defined(CONFIG_IPV6_MIP6_MODULE)
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 8e80fd2..854e401 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -226,7 +226,7 @@
.dst = {
.__refcnt = ATOMIC_INIT(1),
.__use = 1,
- .obsolete = -1,
+ .obsolete = DST_OBSOLETE_FORCE_CHK,
.error = -ENETUNREACH,
.input = ip6_pkt_discard,
.output = ip6_pkt_discard_out,
@@ -246,7 +246,7 @@
.dst = {
.__refcnt = ATOMIC_INIT(1),
.__use = 1,
- .obsolete = -1,
+ .obsolete = DST_OBSOLETE_FORCE_CHK,
.error = -EACCES,
.input = ip6_pkt_prohibit,
.output = ip6_pkt_prohibit_out,
@@ -261,7 +261,7 @@
.dst = {
.__refcnt = ATOMIC_INIT(1),
.__use = 1,
- .obsolete = -1,
+ .obsolete = DST_OBSOLETE_FORCE_CHK,
.error = -EINVAL,
.input = dst_discard,
.output = dst_discard,
@@ -281,13 +281,14 @@
struct fib6_table *table)
{
struct rt6_info *rt = dst_alloc(&net->ipv6.ip6_dst_ops, dev,
- 0, DST_OBSOLETE_NONE, flags);
+ 0, DST_OBSOLETE_FORCE_CHK, flags);
if (rt) {
struct dst_entry *dst = &rt->dst;
memset(dst + 1, 0, sizeof(*rt) - sizeof(*dst));
rt6_init_peer(rt, table ? &table->tb6_peers : net->ipv6.peers);
+ rt->rt6i_genid = rt_genid(net);
}
return rt;
}
@@ -1031,6 +1032,13 @@
rt = (struct rt6_info *) dst;
+ /* All IPV6 dsts are created with ->obsolete set to the value
+ * DST_OBSOLETE_FORCE_CHK which forces validation calls down
+ * into this function always.
+ */
+ if (rt->rt6i_genid != rt_genid(dev_net(rt->dst.dev)))
+ return NULL;
+
if (rt->rt6i_node && (rt->rt6i_node->fn_sernum == cookie)) {
if (rt->rt6i_peer_genid != rt6_peer_genid()) {
if (!rt6_has_peer(rt))
@@ -1397,8 +1405,6 @@
goto out;
}
- rt->dst.obsolete = -1;
-
if (cfg->fc_flags & RTF_EXPIRES)
rt6_set_expires(rt, jiffies +
clock_t_to_jiffies(cfg->fc_expires));
@@ -2080,7 +2086,6 @@
rt->dst.input = ip6_input;
rt->dst.output = ip6_output;
rt->rt6i_idev = idev;
- rt->dst.obsolete = -1;
rt->rt6i_flags = RTF_UP | RTF_NONEXTHOP;
if (anycast)
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index a3e60cc..acd32e3 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -403,8 +403,9 @@
tp->mtu_info = ntohl(info);
if (!sock_owned_by_user(sk))
tcp_v6_mtu_reduced(sk);
- else
- set_bit(TCP_MTU_REDUCED_DEFERRED, &tp->tsq_flags);
+ else if (!test_and_set_bit(TCP_MTU_REDUCED_DEFERRED,
+ &tp->tsq_flags))
+ sock_hold(sk);
goto out;
}
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 99d0077..07e2bfe 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -394,6 +394,17 @@
}
if (unlikely(err)) {
trace_kfree_skb(skb, udpv6_recvmsg);
+ if (!peeked) {
+ atomic_inc(&sk->sk_drops);
+ if (is_udp4)
+ UDP_INC_STATS_USER(sock_net(sk),
+ UDP_MIB_INERRORS,
+ is_udplite);
+ else
+ UDP6_INC_STATS_USER(sock_net(sk),
+ UDP_MIB_INERRORS,
+ is_udplite);
+ }
goto out_free;
}
if (!peeked) {
diff --git a/net/l2tp/l2tp_core.c b/net/l2tp/l2tp_core.c
index 393355d..1a9f372 100644
--- a/net/l2tp/l2tp_core.c
+++ b/net/l2tp/l2tp_core.c
@@ -1347,11 +1347,10 @@
/* Remove from tunnel list */
spin_lock_bh(&pn->l2tp_tunnel_list_lock);
list_del_rcu(&tunnel->list);
+ kfree_rcu(tunnel, rcu);
spin_unlock_bh(&pn->l2tp_tunnel_list_lock);
- synchronize_rcu();
atomic_dec(&l2tp_tunnel_count);
- kfree(tunnel);
}
/* Create a socket for the tunnel, if one isn't set up by
@@ -1502,6 +1501,8 @@
return err;
}
+static struct lock_class_key l2tp_socket_class;
+
int l2tp_tunnel_create(struct net *net, int fd, int version, u32 tunnel_id, u32 peer_tunnel_id, struct l2tp_tunnel_cfg *cfg, struct l2tp_tunnel **tunnelp)
{
struct l2tp_tunnel *tunnel = NULL;
@@ -1606,6 +1607,8 @@
tunnel->old_sk_destruct = sk->sk_destruct;
sk->sk_destruct = &l2tp_tunnel_destruct;
tunnel->sock = sk;
+ lockdep_set_class_and_name(&sk->sk_lock.slock, &l2tp_socket_class, "l2tp_sock");
+
sk->sk_allocation = GFP_ATOMIC;
/* Add tunnel to our list */
diff --git a/net/l2tp/l2tp_core.h b/net/l2tp/l2tp_core.h
index a38ec6c..56d583e 100644
--- a/net/l2tp/l2tp_core.h
+++ b/net/l2tp/l2tp_core.h
@@ -163,6 +163,7 @@
struct l2tp_tunnel {
int magic; /* Should be L2TP_TUNNEL_MAGIC */
+ struct rcu_head rcu;
rwlock_t hlist_lock; /* protect session_hlist */
struct hlist_head session_hlist[L2TP_HASH_SIZE];
/* hashed list of sessions,
diff --git a/net/l2tp/l2tp_eth.c b/net/l2tp/l2tp_eth.c
index f9ee74d..3bfb34a 100644
--- a/net/l2tp/l2tp_eth.c
+++ b/net/l2tp/l2tp_eth.c
@@ -153,7 +153,7 @@
print_hex_dump_bytes("", DUMP_PREFIX_OFFSET, skb->data, length);
}
- if (!pskb_may_pull(skb, sizeof(ETH_HLEN)))
+ if (!pskb_may_pull(skb, ETH_HLEN))
goto error;
secpath_reset(skb);
diff --git a/net/l2tp/l2tp_netlink.c b/net/l2tp/l2tp_netlink.c
index d71cd92..6f93635 100644
--- a/net/l2tp/l2tp_netlink.c
+++ b/net/l2tp/l2tp_netlink.c
@@ -80,8 +80,8 @@
hdr = genlmsg_put(msg, info->snd_pid, info->snd_seq,
&l2tp_nl_family, 0, L2TP_CMD_NOOP);
- if (IS_ERR(hdr)) {
- ret = PTR_ERR(hdr);
+ if (!hdr) {
+ ret = -EMSGSIZE;
goto err_out;
}
@@ -250,8 +250,8 @@
hdr = genlmsg_put(skb, pid, seq, &l2tp_nl_family, flags,
L2TP_CMD_TUNNEL_GET);
- if (IS_ERR(hdr))
- return PTR_ERR(hdr);
+ if (!hdr)
+ return -EMSGSIZE;
if (nla_put_u8(skb, L2TP_ATTR_PROTO_VERSION, tunnel->version) ||
nla_put_u32(skb, L2TP_ATTR_CONN_ID, tunnel->tunnel_id) ||
@@ -617,8 +617,8 @@
sk = tunnel->sock;
hdr = genlmsg_put(skb, pid, seq, &l2tp_nl_family, flags, L2TP_CMD_SESSION_GET);
- if (IS_ERR(hdr))
- return PTR_ERR(hdr);
+ if (!hdr)
+ return -EMSGSIZE;
if (nla_put_u32(skb, L2TP_ATTR_CONN_ID, tunnel->tunnel_id) ||
nla_put_u32(skb, L2TP_ATTR_SESSION_ID, session->session_id) ||
diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c
index d41974a..a58c0b6 100644
--- a/net/mac80211/cfg.c
+++ b/net/mac80211/cfg.c
@@ -1378,6 +1378,8 @@
else
memset(next_hop, 0, ETH_ALEN);
+ memset(pinfo, 0, sizeof(*pinfo));
+
pinfo->generation = mesh_paths_generation;
pinfo->filled = MPATH_INFO_FRAME_QLEN |
@@ -1396,7 +1398,6 @@
pinfo->discovery_timeout =
jiffies_to_msecs(mpath->discovery_timeout);
pinfo->discovery_retries = mpath->discovery_retries;
- pinfo->flags = 0;
if (mpath->flags & MESH_PATH_ACTIVE)
pinfo->flags |= NL80211_MPATH_FLAG_ACTIVE;
if (mpath->flags & MESH_PATH_RESOLVING)
@@ -1405,10 +1406,8 @@
pinfo->flags |= NL80211_MPATH_FLAG_SN_VALID;
if (mpath->flags & MESH_PATH_FIXED)
pinfo->flags |= NL80211_MPATH_FLAG_FIXED;
- if (mpath->flags & MESH_PATH_RESOLVING)
- pinfo->flags |= NL80211_MPATH_FLAG_RESOLVING;
-
- pinfo->flags = mpath->flags;
+ if (mpath->flags & MESH_PATH_RESOLVED)
+ pinfo->flags |= NL80211_MPATH_FLAG_RESOLVED;
}
static int ieee80211_get_mpath(struct wiphy *wiphy, struct net_device *dev,
diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c
index a4a5acd..f76b833 100644
--- a/net/mac80211/mlme.c
+++ b/net/mac80211/mlme.c
@@ -3248,6 +3248,8 @@
goto out_unlock;
err_clear:
+ memset(ifmgd->bssid, 0, ETH_ALEN);
+ ieee80211_bss_info_change_notify(sdata, BSS_CHANGED_BSSID);
ifmgd->auth_data = NULL;
err_free:
kfree(auth_data);
@@ -3439,6 +3441,8 @@
err = 0;
goto out;
err_clear:
+ memset(ifmgd->bssid, 0, ETH_ALEN);
+ ieee80211_bss_info_change_notify(sdata, BSS_CHANGED_BSSID);
ifmgd->assoc_data = NULL;
err_free:
kfree(assoc_data);
diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c
index acf712f..c5e8c9c 100644
--- a/net/mac80211/tx.c
+++ b/net/mac80211/tx.c
@@ -1811,37 +1811,31 @@
meshhdrlen = ieee80211_new_mesh_header(&mesh_hdr,
sdata, NULL, NULL);
} else {
- int is_mesh_mcast = 1;
- const u8 *mesh_da;
+ /* DS -> MBSS (802.11-2012 13.11.3.3).
+ * For unicast with unknown forwarding information,
+ * destination might be in the MBSS or if that fails
+ * forwarded to another mesh gate. In either case
+ * resolution will be handled in ieee80211_xmit(), so
+ * leave the original DA. This also works for mcast */
+ const u8 *mesh_da = skb->data;
- if (is_multicast_ether_addr(skb->data))
- /* DA TA mSA AE:SA */
- mesh_da = skb->data;
- else {
- static const u8 bcast[ETH_ALEN] =
- { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
- if (mppath) {
- /* RA TA mDA mSA AE:DA SA */
- mesh_da = mppath->mpp;
- is_mesh_mcast = 0;
- } else if (mpath) {
- mesh_da = mpath->dst;
- is_mesh_mcast = 0;
- } else {
- /* DA TA mSA AE:SA */
- mesh_da = bcast;
- }
- }
+ if (mppath)
+ mesh_da = mppath->mpp;
+ else if (mpath)
+ mesh_da = mpath->dst;
+ rcu_read_unlock();
+
hdrlen = ieee80211_fill_mesh_addresses(&hdr, &fc,
mesh_da, sdata->vif.addr);
- rcu_read_unlock();
- if (is_mesh_mcast)
+ if (is_multicast_ether_addr(mesh_da))
+ /* DA TA mSA AE:SA */
meshhdrlen =
ieee80211_new_mesh_header(&mesh_hdr,
sdata,
skb->data + ETH_ALEN,
NULL);
else
+ /* RA TA mDA mSA AE:DA SA */
meshhdrlen =
ieee80211_new_mesh_header(&mesh_hdr,
sdata,
diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c
index 72bf32a..f51013c 100644
--- a/net/netfilter/ipvs/ip_vs_ctl.c
+++ b/net/netfilter/ipvs/ip_vs_ctl.c
@@ -1171,8 +1171,10 @@
goto out_err;
}
svc->stats.cpustats = alloc_percpu(struct ip_vs_cpu_stats);
- if (!svc->stats.cpustats)
+ if (!svc->stats.cpustats) {
+ ret = -ENOMEM;
goto out_err;
+ }
/* I'm the first user of the service */
atomic_set(&svc->usecnt, 0);
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index cf48755..2ceec64 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -249,12 +249,15 @@
{
struct nf_conn *ct = (void *)ul_conntrack;
struct net *net = nf_ct_net(ct);
+ struct nf_conntrack_ecache *ecache = nf_ct_ecache_find(ct);
+
+ BUG_ON(ecache == NULL);
if (nf_conntrack_event(IPCT_DESTROY, ct) < 0) {
/* bad luck, let's retry again */
- ct->timeout.expires = jiffies +
+ ecache->timeout.expires = jiffies +
(random32() % net->ct.sysctl_events_retry_timeout);
- add_timer(&ct->timeout);
+ add_timer(&ecache->timeout);
return;
}
/* we've got the event delivered, now it's dying */
@@ -268,6 +271,9 @@
void nf_ct_insert_dying_list(struct nf_conn *ct)
{
struct net *net = nf_ct_net(ct);
+ struct nf_conntrack_ecache *ecache = nf_ct_ecache_find(ct);
+
+ BUG_ON(ecache == NULL);
/* add this conntrack to the dying list */
spin_lock_bh(&nf_conntrack_lock);
@@ -275,10 +281,10 @@
&net->ct.dying);
spin_unlock_bh(&nf_conntrack_lock);
/* set a new timer to retry event delivery */
- setup_timer(&ct->timeout, death_by_event, (unsigned long)ct);
- ct->timeout.expires = jiffies +
+ setup_timer(&ecache->timeout, death_by_event, (unsigned long)ct);
+ ecache->timeout.expires = jiffies +
(random32() % net->ct.sysctl_events_retry_timeout);
- add_timer(&ct->timeout);
+ add_timer(&ecache->timeout);
}
EXPORT_SYMBOL_GPL(nf_ct_insert_dying_list);
diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index da4fc37..9807f32 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -2790,7 +2790,8 @@
goto err_unreg_subsys;
}
- if (register_pernet_subsys(&ctnetlink_net_ops)) {
+ ret = register_pernet_subsys(&ctnetlink_net_ops);
+ if (ret < 0) {
pr_err("ctnetlink_init: cannot register pernet operations\n");
goto err_unreg_exp_subsys;
}
diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
index a5ac11e..e046b37 100644
--- a/net/netfilter/nf_conntrack_proto_tcp.c
+++ b/net/netfilter/nf_conntrack_proto_tcp.c
@@ -158,21 +158,18 @@
* sCL -> sSS
*/
/* sNO, sSS, sSR, sES, sFW, sCW, sLA, sTW, sCL, sS2 */
-/*synack*/ { sIV, sIV, sIG, sIG, sIG, sIG, sIG, sIG, sIG, sSR },
+/*synack*/ { sIV, sIV, sSR, sIV, sIV, sIV, sIV, sIV, sIV, sSR },
/*
* sNO -> sIV Too late and no reason to do anything
* sSS -> sIV Client can't send SYN and then SYN/ACK
* sS2 -> sSR SYN/ACK sent to SYN2 in simultaneous open
- * sSR -> sIG
- * sES -> sIG Error: SYNs in window outside the SYN_SENT state
- * are errors. Receiver will reply with RST
- * and close the connection.
- * Or we are not in sync and hold a dead connection.
- * sFW -> sIG
- * sCW -> sIG
- * sLA -> sIG
- * sTW -> sIG
- * sCL -> sIG
+ * sSR -> sSR Late retransmitted SYN/ACK in simultaneous open
+ * sES -> sIV Invalid SYN/ACK packets sent by the client
+ * sFW -> sIV
+ * sCW -> sIV
+ * sLA -> sIV
+ * sTW -> sIV
+ * sCL -> sIV
*/
/* sNO, sSS, sSR, sES, sFW, sCW, sLA, sTW, sCL, sS2 */
/*fin*/ { sIV, sIV, sFW, sFW, sLA, sLA, sLA, sTW, sCL, sIV },
@@ -633,15 +630,9 @@
ack = sack = receiver->td_end;
}
- if (seq == end
- && (!tcph->rst
- || (seq == 0 && state->state == TCP_CONNTRACK_SYN_SENT)))
+ if (tcph->rst && seq == 0 && state->state == TCP_CONNTRACK_SYN_SENT)
/*
- * Packets contains no data: we assume it is valid
- * and check the ack value only.
- * However RST segments are always validated by their
- * SEQ number, except when seq == 0 (reset sent answering
- * SYN.
+ * RST sent answering SYN.
*/
seq = end = sender->td_end;
diff --git a/net/netfilter/nfnetlink_log.c b/net/netfilter/nfnetlink_log.c
index 169ab59..5cfb5be 100644
--- a/net/netfilter/nfnetlink_log.c
+++ b/net/netfilter/nfnetlink_log.c
@@ -381,6 +381,7 @@
struct nlmsghdr *nlh;
struct nfgenmsg *nfmsg;
sk_buff_data_t old_tail = inst->skb->tail;
+ struct sock *sk;
nlh = nlmsg_put(inst->skb, 0, 0,
NFNL_SUBSYS_ULOG << 8 | NFULNL_MSG_PACKET,
@@ -480,7 +481,7 @@
}
if (indev && skb_mac_header_was_set(skb)) {
- if (nla_put_be32(inst->skb, NFULA_HWTYPE, htons(skb->dev->type)) ||
+ if (nla_put_be16(inst->skb, NFULA_HWTYPE, htons(skb->dev->type)) ||
nla_put_be16(inst->skb, NFULA_HWLEN,
htons(skb->dev->hard_header_len)) ||
nla_put(inst->skb, NFULA_HWHEADER, skb->dev->hard_header_len,
@@ -499,18 +500,19 @@
}
/* UID */
- if (skb->sk) {
- read_lock_bh(&skb->sk->sk_callback_lock);
- if (skb->sk->sk_socket && skb->sk->sk_socket->file) {
- struct file *file = skb->sk->sk_socket->file;
+ sk = skb->sk;
+ if (sk && sk->sk_state != TCP_TIME_WAIT) {
+ read_lock_bh(&sk->sk_callback_lock);
+ if (sk->sk_socket && sk->sk_socket->file) {
+ struct file *file = sk->sk_socket->file;
__be32 uid = htonl(file->f_cred->fsuid);
__be32 gid = htonl(file->f_cred->fsgid);
- read_unlock_bh(&skb->sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
if (nla_put_be32(inst->skb, NFULA_UID, uid) ||
nla_put_be32(inst->skb, NFULA_GID, gid))
goto nla_put_failure;
} else
- read_unlock_bh(&skb->sk->sk_callback_lock);
+ read_unlock_bh(&sk->sk_callback_lock);
}
/* local sequence number */
@@ -996,8 +998,10 @@
#ifdef CONFIG_PROC_FS
if (!proc_create("nfnetlink_log", 0440,
- proc_net_netfilter, &nful_file_ops))
+ proc_net_netfilter, &nful_file_ops)) {
+ status = -ENOMEM;
goto cleanup_logger;
+ }
#endif
return status;
diff --git a/net/netfilter/xt_LOG.c b/net/netfilter/xt_LOG.c
index ff5f75f..91e9af4 100644
--- a/net/netfilter/xt_LOG.c
+++ b/net/netfilter/xt_LOG.c
@@ -145,6 +145,19 @@
return 0;
}
+static void dump_sk_uid_gid(struct sbuff *m, struct sock *sk)
+{
+ if (!sk || sk->sk_state == TCP_TIME_WAIT)
+ return;
+
+ read_lock_bh(&sk->sk_callback_lock);
+ if (sk->sk_socket && sk->sk_socket->file)
+ sb_add(m, "UID=%u GID=%u ",
+ sk->sk_socket->file->f_cred->fsuid,
+ sk->sk_socket->file->f_cred->fsgid);
+ read_unlock_bh(&sk->sk_callback_lock);
+}
+
/* One level of recursion won't kill us */
static void dump_ipv4_packet(struct sbuff *m,
const struct nf_loginfo *info,
@@ -361,14 +374,8 @@
}
/* Max length: 15 "UID=4294967295 " */
- if ((logflags & XT_LOG_UID) && !iphoff && skb->sk) {
- read_lock_bh(&skb->sk->sk_callback_lock);
- if (skb->sk->sk_socket && skb->sk->sk_socket->file)
- sb_add(m, "UID=%u GID=%u ",
- skb->sk->sk_socket->file->f_cred->fsuid,
- skb->sk->sk_socket->file->f_cred->fsgid);
- read_unlock_bh(&skb->sk->sk_callback_lock);
- }
+ if ((logflags & XT_LOG_UID) && !iphoff)
+ dump_sk_uid_gid(m, skb->sk);
/* Max length: 16 "MARK=0xFFFFFFFF " */
if (!iphoff && skb->mark)
@@ -436,8 +443,8 @@
const struct nf_loginfo *loginfo,
const char *prefix)
{
- sb_add(m, "<%d>%sIN=%s OUT=%s ", loginfo->u.log.level,
- prefix,
+ sb_add(m, KERN_SOH "%c%sIN=%s OUT=%s ",
+ '0' + loginfo->u.log.level, prefix,
in ? in->name : "",
out ? out->name : "");
#ifdef CONFIG_BRIDGE_NETFILTER
@@ -717,14 +724,8 @@
}
/* Max length: 15 "UID=4294967295 " */
- if ((logflags & XT_LOG_UID) && recurse && skb->sk) {
- read_lock_bh(&skb->sk->sk_callback_lock);
- if (skb->sk->sk_socket && skb->sk->sk_socket->file)
- sb_add(m, "UID=%u GID=%u ",
- skb->sk->sk_socket->file->f_cred->fsuid,
- skb->sk->sk_socket->file->f_cred->fsgid);
- read_unlock_bh(&skb->sk->sk_callback_lock);
- }
+ if ((logflags & XT_LOG_UID) && recurse)
+ dump_sk_uid_gid(m, skb->sk);
/* Max length: 16 "MARK=0xFFFFFFFF " */
if (!recurse && skb->mark)
diff --git a/net/netfilter/xt_limit.c b/net/netfilter/xt_limit.c
index 5c22ce8..a4c1e45 100644
--- a/net/netfilter/xt_limit.c
+++ b/net/netfilter/xt_limit.c
@@ -117,11 +117,11 @@
/* For SMP, we only want to use one set of state. */
r->master = priv;
+ /* User avg in seconds * XT_LIMIT_SCALE: convert to jiffies *
+ 128. */
+ priv->prev = jiffies;
+ priv->credit = user2credits(r->avg * r->burst); /* Credits full. */
if (r->cost == 0) {
- /* User avg in seconds * XT_LIMIT_SCALE: convert to jiffies *
- 128. */
- priv->prev = jiffies;
- priv->credit = user2credits(r->avg * r->burst); /* Credits full. */
r->credit_cap = priv->credit; /* Credits full. */
r->cost = user2credits(r->avg);
}
diff --git a/net/netlink/af_netlink.c b/net/netlink/af_netlink.c
index 1445d73..5270238 100644
--- a/net/netlink/af_netlink.c
+++ b/net/netlink/af_netlink.c
@@ -1373,7 +1373,8 @@
dst_pid = addr->nl_pid;
dst_group = ffs(addr->nl_groups);
err = -EPERM;
- if (dst_group && !netlink_capable(sock, NL_NONROOT_SEND))
+ if ((dst_group || dst_pid) &&
+ !netlink_capable(sock, NL_NONROOT_SEND))
goto out;
} else {
dst_pid = nlk->dst_pid;
@@ -2147,6 +2148,7 @@
rcu_assign_pointer(nl_table[NETLINK_USERSOCK].listeners, listeners);
nl_table[NETLINK_USERSOCK].module = THIS_MODULE;
nl_table[NETLINK_USERSOCK].registered = 1;
+ nl_table[NETLINK_USERSOCK].nl_nonroot = NL_NONROOT_SEND;
netlink_table_ungrab();
}
diff --git a/net/netrom/af_netrom.c b/net/netrom/af_netrom.c
index 06592d8..7261eb8 100644
--- a/net/netrom/af_netrom.c
+++ b/net/netrom/af_netrom.c
@@ -601,7 +601,7 @@
if (!capable(CAP_NET_BIND_SERVICE)) {
dev_put(dev);
release_sock(sk);
- return -EACCES;
+ return -EPERM;
}
nr->user_addr = addr->fsa_digipeater[0];
nr->source_addr = addr->fsa_ax25.sax25_call;
@@ -1169,7 +1169,12 @@
msg->msg_flags |= MSG_TRUNC;
}
- skb_copy_datagram_iovec(skb, 0, msg->msg_iov, copied);
+ er = skb_copy_datagram_iovec(skb, 0, msg->msg_iov, copied);
+ if (er < 0) {
+ skb_free_datagram(sk, skb);
+ release_sock(sk);
+ return er;
+ }
if (sax != NULL) {
sax->sax25_family = AF_NETROM;
diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c
index f3f96ba..954405c 100644
--- a/net/openvswitch/actions.c
+++ b/net/openvswitch/actions.c
@@ -45,7 +45,7 @@
return pskb_expand_head(skb, 0, 0, GFP_ATOMIC);
}
-/* remove VLAN header from packet and update csum accrodingly. */
+/* remove VLAN header from packet and update csum accordingly. */
static int __pop_vlan_tci(struct sk_buff *skb, __be16 *current_tci)
{
struct vlan_hdr *vhdr;
diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c
index d8277d2..cf58ced 100644
--- a/net/openvswitch/datapath.c
+++ b/net/openvswitch/datapath.c
@@ -425,10 +425,10 @@
static int validate_tp_port(const struct sw_flow_key *flow_key)
{
if (flow_key->eth.type == htons(ETH_P_IP)) {
- if (flow_key->ipv4.tp.src && flow_key->ipv4.tp.dst)
+ if (flow_key->ipv4.tp.src || flow_key->ipv4.tp.dst)
return 0;
} else if (flow_key->eth.type == htons(ETH_P_IPV6)) {
- if (flow_key->ipv6.tp.src && flow_key->ipv6.tp.dst)
+ if (flow_key->ipv6.tp.src || flow_key->ipv6.tp.dst)
return 0;
}
@@ -460,7 +460,7 @@
if (flow_key->eth.type != htons(ETH_P_IP))
return -EINVAL;
- if (!flow_key->ipv4.addr.src || !flow_key->ipv4.addr.dst)
+ if (!flow_key->ip.proto)
return -EINVAL;
ipv4_key = nla_data(ovs_key);
diff --git a/net/openvswitch/flow.h b/net/openvswitch/flow.h
index 9b75617..c30df1a 100644
--- a/net/openvswitch/flow.h
+++ b/net/openvswitch/flow.h
@@ -145,15 +145,17 @@
* OVS_KEY_ATTR_PRIORITY 4 -- 4 8
* OVS_KEY_ATTR_IN_PORT 4 -- 4 8
* OVS_KEY_ATTR_ETHERNET 12 -- 4 16
+ * OVS_KEY_ATTR_ETHERTYPE 2 2 4 8 (outer VLAN ethertype)
* OVS_KEY_ATTR_8021Q 4 -- 4 8
- * OVS_KEY_ATTR_ETHERTYPE 2 2 4 8
+ * OVS_KEY_ATTR_ENCAP 0 -- 4 4 (VLAN encapsulation)
+ * OVS_KEY_ATTR_ETHERTYPE 2 2 4 8 (inner VLAN ethertype)
* OVS_KEY_ATTR_IPV6 40 -- 4 44
* OVS_KEY_ATTR_ICMPV6 2 2 4 8
* OVS_KEY_ATTR_ND 28 -- 4 32
* -------------------------------------------------
- * total 132
+ * total 144
*/
-#define FLOW_BUFSIZE 132
+#define FLOW_BUFSIZE 144
int ovs_flow_to_nlattrs(const struct sw_flow_key *, struct sk_buff *);
int ovs_flow_from_nlattrs(struct sw_flow_key *swkey, int *key_lenp,
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index aee7196..c5c9e2a 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -1273,7 +1273,7 @@
spin_unlock(&f->lock);
}
-bool match_fanout_group(struct packet_type *ptype, struct sock * sk)
+static bool match_fanout_group(struct packet_type *ptype, struct sock * sk)
{
if (ptype->af_packet_priv == (void*)((struct packet_sock *)sk)->fanout)
return true;
diff --git a/net/sched/sch_cbq.c b/net/sched/sch_cbq.c
index 6aabd77..564b9fc 100644
--- a/net/sched/sch_cbq.c
+++ b/net/sched/sch_cbq.c
@@ -250,10 +250,11 @@
else if ((cl = defmap[res.classid & TC_PRIO_MAX]) == NULL)
cl = defmap[TC_PRIO_BESTEFFORT];
- if (cl == NULL || cl->level >= head->level)
+ if (cl == NULL)
goto fallback;
}
-
+ if (cl->level >= head->level)
+ goto fallback;
#ifdef CONFIG_NET_CLS_ACT
switch (result) {
case TC_ACT_QUEUED:
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index 9fc1c62..4e606fc 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -191,7 +191,6 @@
if (list_empty(&flow->flowchain)) {
list_add_tail(&flow->flowchain, &q->new_flows);
- codel_vars_init(&flow->cvars);
q->new_flow_count++;
flow->deficit = q->quantum;
flow->dropped = 0;
@@ -418,6 +417,7 @@
struct fq_codel_flow *flow = q->flows + i;
INIT_LIST_HEAD(&flow->flowchain);
+ codel_vars_init(&flow->cvars);
}
}
if (sch->limit >= 1)
diff --git a/net/sched/sch_gred.c b/net/sched/sch_gred.c
index e901583..d42234c 100644
--- a/net/sched/sch_gred.c
+++ b/net/sched/sch_gred.c
@@ -102,9 +102,8 @@
if (q == NULL)
continue;
- for (n = 0; n < table->DPs; n++)
- if (table->tab[n] && table->tab[n] != q &&
- table->tab[n]->prio == q->prio)
+ for (n = i + 1; n < table->DPs; n++)
+ if (table->tab[n] && table->tab[n]->prio == q->prio)
return 1;
}
@@ -137,6 +136,7 @@
struct gred_sched_data *q)
{
table->wred_set.qavg = q->vars.qavg;
+ table->wred_set.qidlestart = q->vars.qidlestart;
}
static inline int gred_use_ecn(struct gred_sched *t)
@@ -176,7 +176,7 @@
skb->tc_index = (skb->tc_index & ~GRED_VQ_MASK) | dp;
}
- /* sum up all the qaves of prios <= to ours to get the new qave */
+ /* sum up all the qaves of prios < ours to get the new qave */
if (!gred_wred_mode(t) && gred_rio_mode(t)) {
int i;
@@ -260,16 +260,18 @@
} else {
q->backlog -= qdisc_pkt_len(skb);
- if (!q->backlog && !gred_wred_mode(t))
- red_start_of_idle_period(&q->vars);
+ if (gred_wred_mode(t)) {
+ if (!sch->qstats.backlog)
+ red_start_of_idle_period(&t->wred_set);
+ } else {
+ if (!q->backlog)
+ red_start_of_idle_period(&q->vars);
+ }
}
return skb;
}
- if (gred_wred_mode(t) && !red_is_idling(&t->wred_set))
- red_start_of_idle_period(&t->wred_set);
-
return NULL;
}
@@ -291,19 +293,20 @@
q->backlog -= len;
q->stats.other++;
- if (!q->backlog && !gred_wred_mode(t))
- red_start_of_idle_period(&q->vars);
+ if (gred_wred_mode(t)) {
+ if (!sch->qstats.backlog)
+ red_start_of_idle_period(&t->wred_set);
+ } else {
+ if (!q->backlog)
+ red_start_of_idle_period(&q->vars);
+ }
}
qdisc_drop(skb, sch);
return len;
}
- if (gred_wred_mode(t) && !red_is_idling(&t->wred_set))
- red_start_of_idle_period(&t->wred_set);
-
return 0;
-
}
static void gred_reset(struct Qdisc *sch)
@@ -535,6 +538,7 @@
for (i = 0; i < MAX_DPs; i++) {
struct gred_sched_data *q = table->tab[i];
struct tc_gred_qopt opt;
+ unsigned long qavg;
memset(&opt, 0, sizeof(opt));
@@ -566,7 +570,9 @@
if (gred_wred_mode(table))
gred_load_wred_set(table, q);
- opt.qave = red_calc_qavg(&q->parms, &q->vars, q->vars.qavg);
+ qavg = red_calc_qavg(&q->parms, &q->vars,
+ q->vars.qavg >> q->parms.Wlog);
+ opt.qave = qavg >> q->parms.Wlog;
append_opt:
if (nla_append(skb, sizeof(opt), &opt) < 0)
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index e4723d3..211a212 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -865,7 +865,10 @@
if (mask) {
struct qfq_group *next = qfq_ffs(q, mask);
if (qfq_gt(roundedF, next->F)) {
- cl->S = next->F;
+ if (qfq_gt(limit, next->F))
+ cl->S = next->F;
+ else /* preserve timestamp correctness */
+ cl->S = limit;
return;
}
}
diff --git a/net/sctp/output.c b/net/sctp/output.c
index 838e18b..be50aa2 100644
--- a/net/sctp/output.c
+++ b/net/sctp/output.c
@@ -364,6 +364,25 @@
return retval;
}
+static void sctp_packet_release_owner(struct sk_buff *skb)
+{
+ sk_free(skb->sk);
+}
+
+static void sctp_packet_set_owner_w(struct sk_buff *skb, struct sock *sk)
+{
+ skb_orphan(skb);
+ skb->sk = sk;
+ skb->destructor = sctp_packet_release_owner;
+
+ /*
+ * The data chunks have already been accounted for in sctp_sendmsg(),
+ * therefore only reserve a single byte to keep socket around until
+ * the packet has been transmitted.
+ */
+ atomic_inc(&sk->sk_wmem_alloc);
+}
+
/* All packets are sent to the network through this function from
* sctp_outq_tail().
*
@@ -405,7 +424,7 @@
/* Set the owning socket so that we know where to get the
* destination IP address.
*/
- skb_set_owner_w(nskb, sk);
+ sctp_packet_set_owner_w(nskb, sk);
if (!sctp_transport_dst_check(tp)) {
sctp_transport_route(tp, NULL, sctp_sk(sk));
diff --git a/net/socket.c b/net/socket.c
index a5471f8..edc3c4a 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -2604,7 +2604,7 @@
err = sock_do_ioctl(net, sock, cmd, (unsigned long)&ktv);
set_fs(old_fs);
if (!err)
- err = compat_put_timeval(up, &ktv);
+ err = compat_put_timeval(&ktv, up);
return err;
}
@@ -2620,7 +2620,7 @@
err = sock_do_ioctl(net, sock, cmd, (unsigned long)&kts);
set_fs(old_fs);
if (!err)
- err = compat_put_timespec(up, &kts);
+ err = compat_put_timespec(&kts, up);
return err;
}
diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c
index 88f2bf6..bac973a 100644
--- a/net/sunrpc/svc_xprt.c
+++ b/net/sunrpc/svc_xprt.c
@@ -316,7 +316,6 @@
*/
void svc_xprt_enqueue(struct svc_xprt *xprt)
{
- struct svc_serv *serv = xprt->xpt_server;
struct svc_pool *pool;
struct svc_rqst *rqstp;
int cpu;
@@ -362,8 +361,6 @@
rqstp, rqstp->rq_xprt);
rqstp->rq_xprt = xprt;
svc_xprt_get(xprt);
- rqstp->rq_reserved = serv->sv_max_mesg;
- atomic_add(rqstp->rq_reserved, &xprt->xpt_reserved);
pool->sp_stats.threads_woken++;
wake_up(&rqstp->rq_wait);
} else {
@@ -640,8 +637,6 @@
if (xprt) {
rqstp->rq_xprt = xprt;
svc_xprt_get(xprt);
- rqstp->rq_reserved = serv->sv_max_mesg;
- atomic_add(rqstp->rq_reserved, &xprt->xpt_reserved);
/* As there is a shortage of threads and this request
* had to be queued, don't allow the thread to wait so
@@ -738,6 +733,8 @@
else
len = xprt->xpt_ops->xpo_recvfrom(rqstp);
dprintk("svc: got len=%d\n", len);
+ rqstp->rq_reserved = serv->sv_max_mesg;
+ atomic_add(rqstp->rq_reserved, &xprt->xpt_reserved);
}
svc_xprt_received(xprt);
@@ -794,7 +791,8 @@
/* Grab mutex to serialize outgoing data. */
mutex_lock(&xprt->xpt_mutex);
- if (test_bit(XPT_DEAD, &xprt->xpt_flags))
+ if (test_bit(XPT_DEAD, &xprt->xpt_flags)
+ || test_bit(XPT_CLOSE, &xprt->xpt_flags))
len = -ENOTCONN;
else
len = xprt->xpt_ops->xpo_sendto(rqstp);
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index 18bc130..998aa8c 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -1129,9 +1129,9 @@
if (len >= 0)
svsk->sk_tcplen += len;
if (len != want) {
+ svc_tcp_save_pages(svsk, rqstp);
if (len < 0 && len != -EAGAIN)
goto err_other;
- svc_tcp_save_pages(svsk, rqstp);
dprintk("svc: incomplete TCP record (%d of %d)\n",
svsk->sk_tcplen, svsk->sk_reclen);
goto err_noclose;
diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c
index a5a402a..5d7f61d 100644
--- a/net/sunrpc/xprt.c
+++ b/net/sunrpc/xprt.c
@@ -969,11 +969,11 @@
return false;
}
-static void xprt_alloc_slot(struct rpc_task *task)
+void xprt_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task)
{
- struct rpc_xprt *xprt = task->tk_xprt;
struct rpc_rqst *req;
+ spin_lock(&xprt->reserve_lock);
if (!list_empty(&xprt->free)) {
req = list_entry(xprt->free.next, struct rpc_rqst, rq_list);
list_del(&req->rq_list);
@@ -994,12 +994,29 @@
default:
task->tk_status = -EAGAIN;
}
+ spin_unlock(&xprt->reserve_lock);
return;
out_init_req:
task->tk_status = 0;
task->tk_rqstp = req;
xprt_request_init(task, xprt);
+ spin_unlock(&xprt->reserve_lock);
}
+EXPORT_SYMBOL_GPL(xprt_alloc_slot);
+
+void xprt_lock_and_alloc_slot(struct rpc_xprt *xprt, struct rpc_task *task)
+{
+ /* Note: grabbing the xprt_lock_write() ensures that we throttle
+ * new slot allocation if the transport is congested (i.e. when
+ * reconnecting a stream transport or when out of socket write
+ * buffer space).
+ */
+ if (xprt_lock_write(xprt, task)) {
+ xprt_alloc_slot(xprt, task);
+ xprt_release_write(xprt, task);
+ }
+}
+EXPORT_SYMBOL_GPL(xprt_lock_and_alloc_slot);
static void xprt_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *req)
{
@@ -1083,20 +1100,9 @@
if (task->tk_rqstp != NULL)
return;
- /* Note: grabbing the xprt_lock_write() here is not strictly needed,
- * but ensures that we throttle new slot allocation if the transport
- * is congested (e.g. if reconnecting or if we're out of socket
- * write buffer space).
- */
task->tk_timeout = 0;
task->tk_status = -EAGAIN;
- if (!xprt_lock_write(xprt, task))
- return;
-
- spin_lock(&xprt->reserve_lock);
- xprt_alloc_slot(task);
- spin_unlock(&xprt->reserve_lock);
- xprt_release_write(xprt, task);
+ xprt->ops->alloc_slot(xprt, task);
}
static inline __be32 xprt_alloc_xid(struct rpc_xprt *xprt)
diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c
index 06cdbff..5d9202d 100644
--- a/net/sunrpc/xprtrdma/transport.c
+++ b/net/sunrpc/xprtrdma/transport.c
@@ -713,6 +713,7 @@
static struct rpc_xprt_ops xprt_rdma_procs = {
.reserve_xprt = xprt_rdma_reserve_xprt,
.release_xprt = xprt_release_xprt_cong, /* sunrpc/xprt.c */
+ .alloc_slot = xprt_alloc_slot,
.release_request = xprt_release_rqst_cong, /* ditto */
.set_retrans_timeout = xprt_set_retrans_timeout_def, /* ditto */
.rpcbind = rpcb_getport_async, /* sunrpc/rpcb_clnt.c */
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 4005672..a35b8e5 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -2473,6 +2473,7 @@
static struct rpc_xprt_ops xs_local_ops = {
.reserve_xprt = xprt_reserve_xprt,
.release_xprt = xs_tcp_release_xprt,
+ .alloc_slot = xprt_alloc_slot,
.rpcbind = xs_local_rpcbind,
.set_port = xs_local_set_port,
.connect = xs_connect,
@@ -2489,6 +2490,7 @@
.set_buffer_size = xs_udp_set_buffer_size,
.reserve_xprt = xprt_reserve_xprt_cong,
.release_xprt = xprt_release_xprt_cong,
+ .alloc_slot = xprt_alloc_slot,
.rpcbind = rpcb_getport_async,
.set_port = xs_set_port,
.connect = xs_connect,
@@ -2506,6 +2508,7 @@
static struct rpc_xprt_ops xs_tcp_ops = {
.reserve_xprt = xprt_reserve_xprt,
.release_xprt = xs_tcp_release_xprt,
+ .alloc_slot = xprt_lock_and_alloc_slot,
.rpcbind = rpcb_getport_async,
.set_port = xs_set_port,
.connect = xs_connect,
diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c
index 97026f3..1e37dbf 100644
--- a/net/wireless/nl80211.c
+++ b/net/wireless/nl80211.c
@@ -5633,8 +5633,10 @@
sizeof(connect.ht_capa_mask));
if (info->attrs[NL80211_ATTR_HT_CAPABILITY]) {
- if (!info->attrs[NL80211_ATTR_HT_CAPABILITY_MASK])
+ if (!info->attrs[NL80211_ATTR_HT_CAPABILITY_MASK]) {
+ kfree(connkeys);
return -EINVAL;
+ }
memcpy(&connect.ht_capa,
nla_data(info->attrs[NL80211_ATTR_HT_CAPABILITY]),
sizeof(connect.ht_capa));
diff --git a/net/wireless/reg.c b/net/wireless/reg.c
index 2ded3c7..72d170c 100644
--- a/net/wireless/reg.c
+++ b/net/wireless/reg.c
@@ -350,6 +350,9 @@
struct reg_regdb_search_request *request;
const struct ieee80211_regdomain *curdom, *regdom;
int i, r;
+ bool set_reg = false;
+
+ mutex_lock(&cfg80211_mutex);
mutex_lock(®_regdb_search_mutex);
while (!list_empty(®_regdb_search_list)) {
@@ -365,9 +368,7 @@
r = reg_copy_regd(®dom, curdom);
if (r)
break;
- mutex_lock(&cfg80211_mutex);
- set_regdom(regdom);
- mutex_unlock(&cfg80211_mutex);
+ set_reg = true;
break;
}
}
@@ -375,6 +376,11 @@
kfree(request);
}
mutex_unlock(®_regdb_search_mutex);
+
+ if (set_reg)
+ set_regdom(regdom);
+
+ mutex_unlock(&cfg80211_mutex);
}
static DECLARE_WORK(reg_regdb_work, reg_regdb_search);
diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
index 54a0dc2e..ab2bb42 100644
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c
@@ -212,7 +212,7 @@
/* only the first xfrm gets the encap type */
encap_type = 0;
- if (async && x->repl->check(x, skb, seq)) {
+ if (async && x->repl->recheck(x, skb, seq)) {
XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATESEQERROR);
goto drop_unlock;
}
diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 5a2aa17..387848e 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -585,6 +585,7 @@
xfrm_pol_hold(policy);
net->xfrm.policy_count[dir]++;
atomic_inc(&flow_cache_genid);
+ rt_genid_bump(net);
if (delpol)
__xfrm_policy_unlink(delpol, dir);
policy->index = delpol ? delpol->index : xfrm_gen_index(net, dir);
@@ -1763,7 +1764,7 @@
if (!afinfo) {
dst_release(dst_orig);
- ret = ERR_PTR(-EINVAL);
+ return ERR_PTR(-EINVAL);
} else {
ret = afinfo->blackhole_route(net, dst_orig);
}
diff --git a/net/xfrm/xfrm_replay.c b/net/xfrm/xfrm_replay.c
index 2f6d11d..3efb07d 100644
--- a/net/xfrm/xfrm_replay.c
+++ b/net/xfrm/xfrm_replay.c
@@ -420,6 +420,18 @@
return -EINVAL;
}
+static int xfrm_replay_recheck_esn(struct xfrm_state *x,
+ struct sk_buff *skb, __be32 net_seq)
+{
+ if (unlikely(XFRM_SKB_CB(skb)->seq.input.hi !=
+ htonl(xfrm_replay_seqhi(x, net_seq)))) {
+ x->stats.replay_window++;
+ return -EINVAL;
+ }
+
+ return xfrm_replay_check_esn(x, skb, net_seq);
+}
+
static void xfrm_replay_advance_esn(struct xfrm_state *x, __be32 net_seq)
{
unsigned int bitnr, nr, i;
@@ -479,6 +491,7 @@
static struct xfrm_replay xfrm_replay_legacy = {
.advance = xfrm_replay_advance,
.check = xfrm_replay_check,
+ .recheck = xfrm_replay_check,
.notify = xfrm_replay_notify,
.overflow = xfrm_replay_overflow,
};
@@ -486,6 +499,7 @@
static struct xfrm_replay xfrm_replay_bmp = {
.advance = xfrm_replay_advance_bmp,
.check = xfrm_replay_check_bmp,
+ .recheck = xfrm_replay_check_bmp,
.notify = xfrm_replay_notify_bmp,
.overflow = xfrm_replay_overflow_bmp,
};
@@ -493,6 +507,7 @@
static struct xfrm_replay xfrm_replay_esn = {
.advance = xfrm_replay_advance_esn,
.check = xfrm_replay_check_esn,
+ .recheck = xfrm_replay_recheck_esn,
.notify = xfrm_replay_notify_bmp,
.overflow = xfrm_replay_overflow_esn,
};
diff --git a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
index 87cd0e4..210be48 100644
--- a/net/xfrm/xfrm_state.c
+++ b/net/xfrm/xfrm_state.c
@@ -1994,8 +1994,10 @@
goto error;
x->outer_mode = xfrm_get_mode(x->props.mode, family);
- if (x->outer_mode == NULL)
+ if (x->outer_mode == NULL) {
+ err = -EPROTONOSUPPORT;
goto error;
+ }
if (init_replay) {
err = xfrm_init_replay(x);
diff --git a/net/xfrm/xfrm_user.c b/net/xfrm/xfrm_user.c
index e75d8e4..289f4bf 100644
--- a/net/xfrm/xfrm_user.c
+++ b/net/xfrm/xfrm_user.c
@@ -123,9 +123,21 @@
struct nlattr **attrs)
{
struct nlattr *rt = attrs[XFRMA_REPLAY_ESN_VAL];
+ struct xfrm_replay_state_esn *rs;
- if ((p->flags & XFRM_STATE_ESN) && !rt)
- return -EINVAL;
+ if (p->flags & XFRM_STATE_ESN) {
+ if (!rt)
+ return -EINVAL;
+
+ rs = nla_data(rt);
+
+ if (rs->bmp_len > XFRMA_REPLAY_ESN_MAX / sizeof(rs->bmp[0]) / 8)
+ return -EINVAL;
+
+ if (nla_len(rt) < xfrm_replay_state_esn_len(rs) &&
+ nla_len(rt) != sizeof(*rs))
+ return -EINVAL;
+ }
if (!rt)
return 0;
@@ -370,14 +382,15 @@
struct nlattr *rp)
{
struct xfrm_replay_state_esn *up;
+ int ulen;
if (!replay_esn || !rp)
return 0;
up = nla_data(rp);
+ ulen = xfrm_replay_state_esn_len(up);
- if (xfrm_replay_state_esn_len(replay_esn) !=
- xfrm_replay_state_esn_len(up))
+ if (nla_len(rp) < ulen || xfrm_replay_state_esn_len(replay_esn) != ulen)
return -EINVAL;
return 0;
@@ -388,22 +401,28 @@
struct nlattr *rta)
{
struct xfrm_replay_state_esn *p, *pp, *up;
+ int klen, ulen;
if (!rta)
return 0;
up = nla_data(rta);
+ klen = xfrm_replay_state_esn_len(up);
+ ulen = nla_len(rta) >= klen ? klen : sizeof(*up);
- p = kmemdup(up, xfrm_replay_state_esn_len(up), GFP_KERNEL);
+ p = kzalloc(klen, GFP_KERNEL);
if (!p)
return -ENOMEM;
- pp = kmemdup(up, xfrm_replay_state_esn_len(up), GFP_KERNEL);
+ pp = kzalloc(klen, GFP_KERNEL);
if (!pp) {
kfree(p);
return -ENOMEM;
}
+ memcpy(p, up, ulen);
+ memcpy(pp, up, ulen);
+
*replay_esn = p;
*preplay_esn = pp;
@@ -442,10 +461,11 @@
* somehow made shareable and move it to xfrm_state.c - JHS
*
*/
-static void xfrm_update_ae_params(struct xfrm_state *x, struct nlattr **attrs)
+static void xfrm_update_ae_params(struct xfrm_state *x, struct nlattr **attrs,
+ int update_esn)
{
struct nlattr *rp = attrs[XFRMA_REPLAY_VAL];
- struct nlattr *re = attrs[XFRMA_REPLAY_ESN_VAL];
+ struct nlattr *re = update_esn ? attrs[XFRMA_REPLAY_ESN_VAL] : NULL;
struct nlattr *lt = attrs[XFRMA_LTIME_VAL];
struct nlattr *et = attrs[XFRMA_ETIMER_THRESH];
struct nlattr *rt = attrs[XFRMA_REPLAY_THRESH];
@@ -555,7 +575,7 @@
goto error;
/* override default values from above */
- xfrm_update_ae_params(x, attrs);
+ xfrm_update_ae_params(x, attrs, 0);
return x;
@@ -689,6 +709,7 @@
static void copy_to_user_state(struct xfrm_state *x, struct xfrm_usersa_info *p)
{
+ memset(p, 0, sizeof(*p));
memcpy(&p->id, &x->id, sizeof(p->id));
memcpy(&p->sel, &x->sel, sizeof(p->sel));
memcpy(&p->lft, &x->lft, sizeof(p->lft));
@@ -742,7 +763,7 @@
return -EMSGSIZE;
algo = nla_data(nla);
- strcpy(algo->alg_name, auth->alg_name);
+ strncpy(algo->alg_name, auth->alg_name, sizeof(algo->alg_name));
memcpy(algo->alg_key, auth->alg_key, (auth->alg_key_len + 7) / 8);
algo->alg_key_len = auth->alg_key_len;
@@ -878,6 +899,7 @@
{
struct xfrm_dump_info info;
struct sk_buff *skb;
+ int err;
skb = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_ATOMIC);
if (!skb)
@@ -888,9 +910,10 @@
info.nlmsg_seq = seq;
info.nlmsg_flags = 0;
- if (dump_one_state(x, 0, &info)) {
+ err = dump_one_state(x, 0, &info);
+ if (err) {
kfree_skb(skb);
- return NULL;
+ return ERR_PTR(err);
}
return skb;
@@ -1317,6 +1340,7 @@
static void copy_to_user_policy(struct xfrm_policy *xp, struct xfrm_userpolicy_info *p, int dir)
{
+ memset(p, 0, sizeof(*p));
memcpy(&p->sel, &xp->selector, sizeof(p->sel));
memcpy(&p->lft, &xp->lft, sizeof(p->lft));
memcpy(&p->curlft, &xp->curlft, sizeof(p->curlft));
@@ -1421,6 +1445,7 @@
struct xfrm_user_tmpl *up = &vec[i];
struct xfrm_tmpl *kp = &xp->xfrm_vec[i];
+ memset(up, 0, sizeof(*up));
memcpy(&up->id, &kp->id, sizeof(up->id));
up->family = kp->encap_family;
memcpy(&up->saddr, &kp->saddr, sizeof(up->saddr));
@@ -1546,6 +1571,7 @@
{
struct xfrm_dump_info info;
struct sk_buff *skb;
+ int err;
skb = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
if (!skb)
@@ -1556,9 +1582,10 @@
info.nlmsg_seq = seq;
info.nlmsg_flags = 0;
- if (dump_one_policy(xp, dir, 0, &info) < 0) {
+ err = dump_one_policy(xp, dir, 0, &info);
+ if (err) {
kfree_skb(skb);
- return NULL;
+ return ERR_PTR(err);
}
return skb;
@@ -1822,7 +1849,7 @@
goto out;
spin_lock_bh(&x->lock);
- xfrm_update_ae_params(x, attrs);
+ xfrm_update_ae_params(x, attrs, 1);
spin_unlock_bh(&x->lock);
c.event = nlh->nlmsg_type;
diff --git a/scripts/Makefile.fwinst b/scripts/Makefile.fwinst
index 6bf8e87..4d908d1 100644
--- a/scripts/Makefile.fwinst
+++ b/scripts/Makefile.fwinst
@@ -27,7 +27,7 @@
installed-mod-fw := $(addprefix $(INSTALL_FW_PATH)/,$(mod-fw))
installed-fw := $(addprefix $(INSTALL_FW_PATH)/,$(fw-shipped-all))
-installed-fw-dirs := $(sort $(dir $(installed-fw))) $(INSTALL_FW_PATH)/.
+installed-fw-dirs := $(sort $(dir $(installed-fw))) $(INSTALL_FW_PATH)/./
# Workaround for make < 3.81, where .SECONDEXPANSION doesn't work.
PHONY += $(INSTALL_FW_PATH)/$$(%) install-all-dirs
diff --git a/scripts/checksyscalls.sh b/scripts/checksyscalls.sh
index d24810f..fd8fa9a 100755
--- a/scripts/checksyscalls.sh
+++ b/scripts/checksyscalls.sh
@@ -200,7 +200,7 @@
syscall_list() {
grep '^[0-9]' "$1" | sort -n | (
while read nr abi name entry ; do
- echo <<EOF
+ cat <<EOF
#if !defined(__NR_${name}) && !defined(__IGNORE_${name})
#warning syscall ${name} not implemented
#endif
diff --git a/scripts/coccinelle/api/memdup_user.cocci b/scripts/coccinelle/api/memdup_user.cocci
index 2efac28..2b131a8 100644
--- a/scripts/coccinelle/api/memdup_user.cocci
+++ b/scripts/coccinelle/api/memdup_user.cocci
@@ -51,10 +51,10 @@
p << r.p;
@@
-coccilib.org.print_todo(p[0], "WARNING opportunity for memdep_user")
+coccilib.org.print_todo(p[0], "WARNING opportunity for memdup_user")
@script:python depends on report@
p << r.p;
@@
-coccilib.report.print_report(p[0], "WARNING opportunity for memdep_user")
+coccilib.report.print_report(p[0], "WARNING opportunity for memdup_user")
diff --git a/scripts/kconfig/streamline_config.pl b/scripts/kconfig/streamline_config.pl
index 2fbbbc1..3368939 100644
--- a/scripts/kconfig/streamline_config.pl
+++ b/scripts/kconfig/streamline_config.pl
@@ -100,7 +100,7 @@
},
);
-sub find_config {
+sub read_config {
foreach my $conf (@searchconfigs) {
my $file = $conf->{"file"};
@@ -115,17 +115,15 @@
print STDERR "using config: '$file'\n";
- open(CIN, "$exec $file |") || die "Failed to run $exec $file";
- return;
+ open(my $infile, '-|', "$exec $file") || die "Failed to run $exec $file";
+ my @x = <$infile>;
+ close $infile;
+ return @x;
}
die "No config file found";
}
-find_config;
-
-# Read in the entire config file into config_file
-my @config_file = <CIN>;
-close CIN;
+my @config_file = read_config;
# Parse options
my $localmodconfig = 0;
@@ -135,7 +133,7 @@
"localyesconfig" => \$localyesconfig);
# Get the build source and top level Kconfig file (passed in)
-my $ksource = $ARGV[0];
+my $ksource = ($ARGV[0] ? $ARGV[0] : '.');
my $kconfig = $ARGV[1];
my $lsmod_file = $ENV{'LSMOD'};
@@ -173,8 +171,8 @@
$source =~ s/\$$env/$ENV{$env}/;
}
- open(KIN, "$source") || die "Can't open $kconfig";
- while (<KIN>) {
+ open(my $kinfile, '<', $source) || die "Can't open $kconfig";
+ while (<$kinfile>) {
chomp;
# Make sure that lines ending with \ continue
@@ -251,10 +249,10 @@
$state = "NONE";
}
}
- close(KIN);
+ close($kinfile);
# read in any configs that were found.
- foreach $kconfig (@kconfigs) {
+ foreach my $kconfig (@kconfigs) {
if (!defined($read_kconfigs{$kconfig})) {
$read_kconfigs{$kconfig} = 1;
read_kconfig($kconfig);
@@ -295,8 +293,8 @@
my $line = "";
my %make_vars;
- open(MIN,$makefile) || die "Can't open $makefile";
- while (<MIN>) {
+ open(my $infile, '<', $makefile) || die "Can't open $makefile";
+ while (<$infile>) {
# if this line ends with a backslash, continue
chomp;
if (/^(.*)\\$/) {
@@ -343,10 +341,11 @@
}
}
}
- close(MIN);
+ close($infile);
}
my %modules;
+my $linfile;
if (defined($lsmod_file)) {
if ( ! -f $lsmod_file) {
@@ -356,13 +355,10 @@
die "$lsmod_file not found";
}
}
- if ( -x $lsmod_file) {
- # the file is executable, run it
- open(LIN, "$lsmod_file|");
- } else {
- # Just read the contents
- open(LIN, "$lsmod_file");
- }
+
+ my $otype = ( -x $lsmod_file) ? '-|' : '<';
+ open($linfile, $otype, $lsmod_file);
+
} else {
# see what modules are loaded on this system
@@ -379,16 +375,16 @@
$lsmod = "lsmod";
}
- open(LIN,"$lsmod|") || die "Can not call lsmod with $lsmod";
+ open($linfile, '-|', $lsmod) || die "Can not call lsmod with $lsmod";
}
-while (<LIN>) {
+while (<$linfile>) {
next if (/^Module/); # Skip the first line.
if (/^(\S+)/) {
$modules{$1} = 1;
}
}
-close (LIN);
+close ($linfile);
# add to the configs hash all configs that are needed to enable
# a loaded module. This is a direct obj-${CONFIG_FOO} += bar.o
@@ -605,6 +601,8 @@
if (defined($configs{$1})) {
if ($localyesconfig) {
$setconfigs{$1} = 'y';
+ print "$1=y\n";
+ next;
} else {
$setconfigs{$1} = $2;
}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 4629038c..b3d907e 100644
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -74,8 +74,13 @@
info KSYM ${2}
local kallsymopt;
+ if [ -n "${CONFIG_SYMBOL_PREFIX}" ]; then
+ kallsymopt="${kallsymopt} \
+ --symbol-prefix=${CONFIG_SYMBOL_PREFIX}"
+ fi
+
if [ -n "${CONFIG_KALLSYMS_ALL}" ]; then
- kallsymopt=--all-symbols
+ kallsymopt="${kallsymopt} --all-symbols"
fi
local aflags="${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
@@ -211,7 +216,7 @@
if ! cmp -s System.map .tmp_System.map; then
echo >&2 Inconsistent kallsyms data
- echo >&2 echo Try "make KALLSYMS_EXTRA_PASS=1" as a workaround
+ echo >&2 Try "make KALLSYMS_EXTRA_PASS=1" as a workaround
cleanup
exit 1
fi
diff --git a/scripts/recordmcount.h b/scripts/recordmcount.h
index 54e35c1..9d1421e 100644
--- a/scripts/recordmcount.h
+++ b/scripts/recordmcount.h
@@ -261,11 +261,13 @@
&sym0[Elf_r_sym(relp)];
char const *symname = &str0[w(symp->st_name)];
char const *mcount = gpfx == '_' ? "_mcount" : "mcount";
+ char const *fentry = "__fentry__";
if (symname[0] == '.')
++symname; /* ppc64 hack */
if (strcmp(mcount, symname) == 0 ||
- (altmcount && strcmp(altmcount, symname) == 0))
+ (altmcount && strcmp(altmcount, symname) == 0) ||
+ (strcmp(fentry, symname) == 0))
mcountsym = Elf_r_sym(relp);
return mcountsym;
diff --git a/scripts/sortextable.c b/scripts/sortextable.c
index 6acf834..f19ddc4 100644
--- a/scripts/sortextable.c
+++ b/scripts/sortextable.c
@@ -161,7 +161,7 @@
#define SORTEXTABLE_64
#include "sortextable.h"
-static int compare_x86_table(const void *a, const void *b)
+static int compare_relative_table(const void *a, const void *b)
{
int32_t av = (int32_t)r(a);
int32_t bv = (int32_t)r(b);
@@ -173,7 +173,7 @@
return 0;
}
-static void sort_x86_table(char *extab_image, int image_size)
+static void sort_relative_table(char *extab_image, int image_size)
{
int i;
@@ -188,7 +188,7 @@
i += 4;
}
- qsort(extab_image, image_size / 8, 8, compare_x86_table);
+ qsort(extab_image, image_size / 8, 8, compare_relative_table);
/* Now denormalize. */
i = 0;
@@ -245,9 +245,9 @@
break;
case EM_386:
case EM_X86_64:
- custom_sort = sort_x86_table;
- break;
case EM_S390:
+ custom_sort = sort_relative_table;
+ break;
case EM_MIPS:
break;
} /* end switch */
diff --git a/security/apparmor/.gitignore b/security/apparmor/.gitignore
index 4d995ae..9cdec70 100644
--- a/security/apparmor/.gitignore
+++ b/security/apparmor/.gitignore
@@ -1,6 +1,5 @@
#
# Generated include files
#
-af_names.h
capability_names.h
rlim_names.h
diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c
index 3364fbf..6cfc647 100644
--- a/security/keys/keyctl.c
+++ b/security/keys/keyctl.c
@@ -1486,7 +1486,6 @@
oldwork = NULL;
parent = me->real_parent;
- task_lock(parent);
/* the parent mustn't be init and mustn't be a kernel thread */
if (parent->pid <= 1 || !parent->mm)
goto unlock;
@@ -1530,7 +1529,6 @@
if (!ret)
newwork = NULL;
unlock:
- task_unlock(parent);
write_unlock_irq(&tasklist_lock);
rcu_read_unlock();
if (oldwork)
diff --git a/security/selinux/include/xfrm.h b/security/selinux/include/xfrm.h
index c220f31..65f67cb 100644
--- a/security/selinux/include/xfrm.h
+++ b/security/selinux/include/xfrm.h
@@ -51,6 +51,7 @@
static inline void selinux_xfrm_notify_policyload(void)
{
atomic_inc(&flow_cache_genid);
+ rt_genid_bump(&init_net);
}
#else
static inline int selinux_xfrm_enabled(void)
diff --git a/sound/core/compress_offload.c b/sound/core/compress_offload.c
index ec2118d..eb60cb8 100644
--- a/sound/core/compress_offload.c
+++ b/sound/core/compress_offload.c
@@ -80,14 +80,12 @@
int maj = imajor(inode);
int ret;
- if (f->f_flags & O_WRONLY)
+ if ((f->f_flags & O_ACCMODE) == O_WRONLY)
dirn = SND_COMPRESS_PLAYBACK;
- else if (f->f_flags & O_RDONLY)
+ else if ((f->f_flags & O_ACCMODE) == O_RDONLY)
dirn = SND_COMPRESS_CAPTURE;
- else {
- pr_err("invalid direction\n");
+ else
return -EINVAL;
- }
if (maj == snd_major)
compr = snd_lookup_minor_data(iminor(inode),
diff --git a/sound/oss/.gitignore b/sound/oss/.gitignore
index 7efb12b..12a3920 100644
--- a/sound/oss/.gitignore
+++ b/sound/oss/.gitignore
@@ -1,4 +1,3 @@
#Ignore generated files
-maui_boot.h
pss_boot.h
trix_boot.h
diff --git a/sound/pci/hda/hda_codec.c b/sound/pci/hda/hda_codec.c
index f560051..1c65cc5 100644
--- a/sound/pci/hda/hda_codec.c
+++ b/sound/pci/hda/hda_codec.c
@@ -1209,6 +1209,9 @@
kfree(codec);
}
+static bool snd_hda_codec_get_supported_ps(struct hda_codec *codec,
+ hda_nid_t fg, unsigned int power_state);
+
static void hda_set_power_state(struct hda_codec *codec, hda_nid_t fg,
unsigned int power_state);
@@ -1317,6 +1320,10 @@
AC_VERB_GET_SUBSYSTEM_ID, 0);
}
+ codec->epss = snd_hda_codec_get_supported_ps(codec,
+ codec->afg ? codec->afg : codec->mfg,
+ AC_PWRST_EPSS);
+
/* power-up all before initialization */
hda_set_power_state(codec,
codec->afg ? codec->afg : codec->mfg,
@@ -2346,6 +2353,7 @@
}
if (codec->patch_ops.free)
codec->patch_ops.free(codec);
+ memset(&codec->patch_ops, 0, sizeof(codec->patch_ops));
snd_hda_jack_tbl_clear(codec);
codec->proc_widget_hook = NULL;
codec->spec = NULL;
@@ -2361,7 +2369,6 @@
codec->num_pcms = 0;
codec->pcm_info = NULL;
codec->preset = NULL;
- memset(&codec->patch_ops, 0, sizeof(codec->patch_ops));
codec->slave_dig_outs = NULL;
codec->spdif_status_reset = 0;
module_put(codec->owner);
@@ -3543,8 +3550,7 @@
/* this delay seems necessary to avoid click noise at power-down */
if (power_state == AC_PWRST_D3) {
/* transition time less than 10ms for power down */
- bool epss = snd_hda_codec_get_supported_ps(codec, fg, AC_PWRST_EPSS);
- msleep(epss ? 10 : 100);
+ msleep(codec->epss ? 10 : 100);
}
/* repeat power states setting at most 10 times*/
diff --git a/sound/pci/hda/hda_codec.h b/sound/pci/hda/hda_codec.h
index 7fbc1bc..e5a7e19 100644
--- a/sound/pci/hda/hda_codec.h
+++ b/sound/pci/hda/hda_codec.h
@@ -862,6 +862,7 @@
unsigned int ignore_misc_bit:1; /* ignore MISC_NO_PRESENCE bit */
unsigned int no_jack_detect:1; /* Machine has no jack-detection */
unsigned int pcm_format_first:1; /* PCM format must be set first */
+ unsigned int epss:1; /* supporting EPSS? */
#ifdef CONFIG_SND_HDA_POWER_SAVE
unsigned int power_on :1; /* current (global) power-state */
int power_transition; /* power-state in transition */
diff --git a/sound/pci/hda/hda_intel.c b/sound/pci/hda/hda_intel.c
index 60882c6..c4763c5 100644
--- a/sound/pci/hda/hda_intel.c
+++ b/sound/pci/hda/hda_intel.c
@@ -2701,6 +2701,8 @@
SND_PCI_QUIRK(0x1043, 0x813d, "ASUS P5AD2", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1043, 0x81b3, "ASUS", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1043, 0x81e7, "ASUS M2V", POS_FIX_LPIB),
+ SND_PCI_QUIRK(0x1043, 0x1ac3, "ASUS X53S", POS_FIX_POSBUF),
+ SND_PCI_QUIRK(0x1043, 0x1b43, "ASUS K53E", POS_FIX_POSBUF),
SND_PCI_QUIRK(0x104d, 0x9069, "Sony VPCS11V9E", POS_FIX_LPIB),
SND_PCI_QUIRK(0x10de, 0xcb89, "Macbook Pro 7,1", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1297, 0x3166, "Shuttle", POS_FIX_LPIB),
diff --git a/sound/pci/hda/patch_sigmatel.c b/sound/pci/hda/patch_sigmatel.c
index ea5775a..3d4722f 100644
--- a/sound/pci/hda/patch_sigmatel.c
+++ b/sound/pci/hda/patch_sigmatel.c
@@ -1075,7 +1075,7 @@
static const char * const slave_pfxs[] = {
"Front", "Surround", "Center", "LFE", "Side",
- "Headphone", "Speaker", "IEC958",
+ "Headphone", "Speaker", "IEC958", "PCM",
NULL
};
@@ -4543,6 +4543,9 @@
struct auto_pin_cfg *cfg = &spec->autocfg;
int i;
+ if (cfg->speaker_outs == 0)
+ return;
+
for (i = 0; i < cfg->line_outs; i++) {
if (presence)
break;
@@ -5531,6 +5534,7 @@
snd_hda_codec_set_pincfg(codec, 0xf, 0x2181205e);
}
+ codec->epss = 0; /* longer delay needed for D3 */
codec->no_trigger_sense = 1;
codec->spec = spec;
diff --git a/sound/pci/ice1712/prodigy_hifi.c b/sound/pci/ice1712/prodigy_hifi.c
index 764cc93d..075d5aa 100644
--- a/sound/pci/ice1712/prodigy_hifi.c
+++ b/sound/pci/ice1712/prodigy_hifi.c
@@ -297,6 +297,7 @@
}
static const DECLARE_TLV_DB_SCALE(db_scale_wm_dac, -12700, 100, 1);
+static const DECLARE_TLV_DB_LINEAR(ak4396_db_scale, TLV_DB_GAIN_MUTE, 0);
static struct snd_kcontrol_new prodigy_hd2_controls[] __devinitdata = {
{
@@ -307,7 +308,7 @@
.info = ak4396_dac_vol_info,
.get = ak4396_dac_vol_get,
.put = ak4396_dac_vol_put,
- .tlv = { .p = db_scale_wm_dac },
+ .tlv = { .p = ak4396_db_scale },
},
};
diff --git a/sound/soc/codecs/arizona.c b/sound/soc/codecs/arizona.c
index 5c9caca..1cf7a32 100644
--- a/sound/soc/codecs/arizona.c
+++ b/sound/soc/codecs/arizona.c
@@ -426,7 +426,7 @@
940800,
1411200,
1881600,
- 2882400,
+ 2822400,
3763200,
5644800,
7526400,
diff --git a/sound/soc/codecs/mc13783.c b/sound/soc/codecs/mc13783.c
index 8f726c0..115a403 100644
--- a/sound/soc/codecs/mc13783.c
+++ b/sound/soc/codecs/mc13783.c
@@ -659,7 +659,7 @@
.id = MC13783_ID_STEREO_DAC,
.playback = {
.stream_name = "Playback",
- .channels_min = 1,
+ .channels_min = 2,
.channels_max = 2,
.rates = SNDRV_PCM_RATE_8000_96000,
.formats = MC13783_FORMATS,
@@ -670,7 +670,7 @@
.id = MC13783_ID_STEREO_CODEC,
.capture = {
.stream_name = "Capture",
- .channels_min = 1,
+ .channels_min = 2,
.channels_max = 2,
.rates = MC13783_RATES_RECORD,
.formats = MC13783_FORMATS,
@@ -692,14 +692,14 @@
.id = MC13783_ID_SYNC,
.playback = {
.stream_name = "Playback",
- .channels_min = 1,
+ .channels_min = 2,
.channels_max = 2,
.rates = SNDRV_PCM_RATE_8000_96000,
.formats = MC13783_FORMATS,
},
.capture = {
.stream_name = "Capture",
- .channels_min = 1,
+ .channels_min = 2,
.channels_max = 2,
.rates = MC13783_RATES_RECORD,
.formats = MC13783_FORMATS,
diff --git a/sound/soc/codecs/wm2000.c b/sound/soc/codecs/wm2000.c
index 3fd5b29..a3acb7a 100644
--- a/sound/soc/codecs/wm2000.c
+++ b/sound/soc/codecs/wm2000.c
@@ -702,7 +702,7 @@
}
static const struct regmap_config wm2000_regmap = {
- .reg_bits = 8,
+ .reg_bits = 16,
.val_bits = 8,
.max_register = WM2000_REG_IF_CTL,
diff --git a/sound/soc/codecs/wm8904.c b/sound/soc/codecs/wm8904.c
index 0013afe..dc4262e 100644
--- a/sound/soc/codecs/wm8904.c
+++ b/sound/soc/codecs/wm8904.c
@@ -100,7 +100,7 @@
{ 14, 0x0000 }, /* R14 - Power Management 2 */
{ 15, 0x0000 }, /* R15 - Power Management 3 */
{ 18, 0x0000 }, /* R18 - Power Management 6 */
- { 19, 0x945E }, /* R20 - Clock Rates 0 */
+ { 20, 0x945E }, /* R20 - Clock Rates 0 */
{ 21, 0x0C05 }, /* R21 - Clock Rates 1 */
{ 22, 0x0006 }, /* R22 - Clock Rates 2 */
{ 24, 0x0050 }, /* R24 - Audio Interface 0 */
diff --git a/sound/soc/fsl/imx-sgtl5000.c b/sound/soc/fsl/imx-sgtl5000.c
index fb21b17..199408e 100644
--- a/sound/soc/fsl/imx-sgtl5000.c
+++ b/sound/soc/fsl/imx-sgtl5000.c
@@ -94,7 +94,7 @@
dev_err(&pdev->dev, "audmux internal port setup failed\n");
return ret;
}
- imx_audmux_v2_configure_port(ext_port,
+ ret = imx_audmux_v2_configure_port(ext_port,
IMX_AUDMUX_V2_PTCR_SYN,
IMX_AUDMUX_V2_PDCR_RXDSEL(int_port));
if (ret) {
diff --git a/sound/soc/omap/am3517evm.c b/sound/soc/omap/am3517evm.c
index 009533a..df65f98 100644
--- a/sound/soc/omap/am3517evm.c
+++ b/sound/soc/omap/am3517evm.c
@@ -59,7 +59,7 @@
return ret;
}
- snd_soc_dai_set_sysclk(cpu_dai, OMAP_MCBSP_FSR_SRC_FSX, 0,
+ ret = snd_soc_dai_set_sysclk(cpu_dai, OMAP_MCBSP_FSR_SRC_FSX, 0,
SND_SOC_CLOCK_IN);
if (ret < 0) {
printk(KERN_ERR "can't set CPU system clock OMAP_MCBSP_FSR_SRC_FSX\n");
diff --git a/sound/soc/samsung/dma.c b/sound/soc/samsung/dma.c
index f3ebc38..b70964e 100644
--- a/sound/soc/samsung/dma.c
+++ b/sound/soc/samsung/dma.c
@@ -34,9 +34,7 @@
.info = SNDRV_PCM_INFO_INTERLEAVED |
SNDRV_PCM_INFO_BLOCK_TRANSFER |
SNDRV_PCM_INFO_MMAP |
- SNDRV_PCM_INFO_MMAP_VALID |
- SNDRV_PCM_INFO_PAUSE |
- SNDRV_PCM_INFO_RESUME,
+ SNDRV_PCM_INFO_MMAP_VALID,
.formats = SNDRV_PCM_FMTBIT_S16_LE |
SNDRV_PCM_FMTBIT_U16_LE |
SNDRV_PCM_FMTBIT_U8 |
@@ -248,15 +246,11 @@
switch (cmd) {
case SNDRV_PCM_TRIGGER_START:
- case SNDRV_PCM_TRIGGER_RESUME:
- case SNDRV_PCM_TRIGGER_PAUSE_RELEASE:
prtd->state |= ST_RUNNING;
prtd->params->ops->trigger(prtd->params->ch);
break;
case SNDRV_PCM_TRIGGER_STOP:
- case SNDRV_PCM_TRIGGER_SUSPEND:
- case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
prtd->state &= ~ST_RUNNING;
prtd->params->ops->stop(prtd->params->ch);
break;
diff --git a/sound/soc/soc-dapm.c b/sound/soc/soc-dapm.c
index dd7c49f..f90139b 100644
--- a/sound/soc/soc-dapm.c
+++ b/sound/soc/soc-dapm.c
@@ -291,8 +291,11 @@
if (dapm->codec->driver->set_bias_level)
ret = dapm->codec->driver->set_bias_level(dapm->codec,
level);
- } else
+ else
+ dapm->bias_level = level;
+ } else if (!card || dapm != &card->dapm) {
dapm->bias_level = level;
+ }
if (ret != 0)
goto out;
diff --git a/sound/soc/spear/spear_pcm.c b/sound/soc/spear/spear_pcm.c
index 97c2cac..8c7f237 100644
--- a/sound/soc/spear/spear_pcm.c
+++ b/sound/soc/spear/spear_pcm.c
@@ -138,7 +138,7 @@
continue;
buf = &substream->dma_buffer;
- if (!buf && !buf->area)
+ if (!buf || !buf->area)
continue;
dma_free_writecombine(pcm->card->dev, buf->bytes,
diff --git a/sound/soc/tegra/tegra_alc5632.c b/sound/soc/tegra/tegra_alc5632.c
index e463529..76cb1b3 100644
--- a/sound/soc/tegra/tegra_alc5632.c
+++ b/sound/soc/tegra/tegra_alc5632.c
@@ -89,7 +89,6 @@
.name = "Headset detection",
.report = SND_JACK_HEADSET,
.debounce_time = 150,
- .invert = 1,
};
static const struct snd_soc_dapm_widget tegra_alc5632_dapm_widgets[] = {
diff --git a/sound/soc/tegra/tegra_pcm.c b/sound/soc/tegra/tegra_pcm.c
index 5658bce..8d6900c 100644
--- a/sound/soc/tegra/tegra_pcm.c
+++ b/sound/soc/tegra/tegra_pcm.c
@@ -334,11 +334,11 @@
if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK) {
slave_config.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
slave_config.dst_addr = dmap->addr;
- slave_config.src_maxburst = 0;
+ slave_config.dst_maxburst = 4;
} else {
slave_config.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
slave_config.src_addr = dmap->addr;
- slave_config.dst_maxburst = 0;
+ slave_config.src_maxburst = 4;
}
slave_config.slave_id = dmap->req_sel;
diff --git a/sound/soc/ux500/ux500_msp_i2s.c b/sound/soc/ux500/ux500_msp_i2s.c
index 5c472f3..eb85113 100644
--- a/sound/soc/ux500/ux500_msp_i2s.c
+++ b/sound/soc/ux500/ux500_msp_i2s.c
@@ -663,7 +663,6 @@
struct ux500_msp **msp_p,
struct msp_i2s_platform_data *platform_data)
{
- int ret = 0;
struct resource *res = NULL;
struct i2s_controller *i2s_cont;
struct ux500_msp *msp;
@@ -685,15 +684,14 @@
if (res == NULL) {
dev_err(&pdev->dev, "%s: ERROR: Unable to get resource!\n",
__func__);
- ret = -ENOMEM;
- goto err_res;
+ return -ENOMEM;
}
- msp->registers = ioremap(res->start, (res->end - res->start + 1));
+ msp->registers = devm_ioremap(&pdev->dev, res->start,
+ resource_size(res));
if (msp->registers == NULL) {
dev_err(&pdev->dev, "%s: ERROR: ioremap failed!\n", __func__);
- ret = -ENOMEM;
- goto err_res;
+ return -ENOMEM;
}
msp->msp_state = MSP_STATE_IDLE;
@@ -705,7 +703,7 @@
dev_err(&pdev->dev,
"%s: ERROR: Failed to allocate I2S-controller!\n",
__func__);
- goto err_i2s_cont;
+ return -ENOMEM;
}
i2s_cont->dev.parent = &pdev->dev;
i2s_cont->data = (void *)msp;
@@ -716,14 +714,6 @@
msp->i2s_cont = i2s_cont;
return 0;
-
-err_i2s_cont:
- iounmap(msp->registers);
-
-err_res:
- devm_kfree(&pdev->dev, msp);
-
- return ret;
}
void ux500_msp_i2s_cleanup_msp(struct platform_device *pdev,
@@ -732,11 +722,6 @@
dev_dbg(msp->dev, "%s: Enter (id = %d).\n", __func__, msp->id);
device_unregister(&msp->i2s_cont->dev);
- devm_kfree(&pdev->dev, msp->i2s_cont);
-
- iounmap(msp->registers);
-
- devm_kfree(&pdev->dev, msp);
}
MODULE_LICENSE("GPL v2");
diff --git a/sound/usb/card.c b/sound/usb/card.c
index d5b5c33..4a469f0 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -553,7 +553,7 @@
struct snd_usb_audio *chip)
{
struct snd_card *card;
- struct list_head *p;
+ struct list_head *p, *n;
if (chip == (void *)-1L)
return;
@@ -570,7 +570,7 @@
snd_usb_stream_disconnect(p);
}
/* release the endpoint resources */
- list_for_each(p, &chip->ep_list) {
+ list_for_each_safe(p, n, &chip->ep_list) {
snd_usb_endpoint_free(p);
}
/* release the midi resources */
diff --git a/sound/usb/endpoint.c b/sound/usb/endpoint.c
index c411812..060dccb 100644
--- a/sound/usb/endpoint.c
+++ b/sound/usb/endpoint.c
@@ -141,7 +141,7 @@
*
* For implicit feedback, next_packet_size() is unused.
*/
-static int next_packet_size(struct snd_usb_endpoint *ep)
+int snd_usb_endpoint_next_packet_size(struct snd_usb_endpoint *ep)
{
unsigned long flags;
int ret;
@@ -177,15 +177,6 @@
ep->retire_data_urb(ep->data_subs, urb);
}
-static void prepare_outbound_urb_sizes(struct snd_usb_endpoint *ep,
- struct snd_urb_ctx *ctx)
-{
- int i;
-
- for (i = 0; i < ctx->packets; ++i)
- ctx->packet_size[i] = next_packet_size(ep);
-}
-
/*
* Prepare a PLAYBACK urb for submission to the bus.
*/
@@ -206,7 +197,13 @@
/* no data provider, so send silence */
unsigned int offs = 0;
for (i = 0; i < ctx->packets; ++i) {
- int counts = ctx->packet_size[i];
+ int counts;
+
+ if (ctx->packet_size[i])
+ counts = ctx->packet_size[i];
+ else
+ counts = snd_usb_endpoint_next_packet_size(ep);
+
urb->iso_frame_desc[i].offset = offs * ep->stride;
urb->iso_frame_desc[i].length = counts * ep->stride;
offs += counts;
@@ -370,7 +367,6 @@
goto exit_clear;
}
- prepare_outbound_urb_sizes(ep, ctx);
prepare_outbound_urb(ep, ctx);
} else {
retire_inbound_urb(ep, ctx);
@@ -799,7 +795,9 @@
/**
* snd_usb_endpoint_start: start an snd_usb_endpoint
*
- * @ep: the endpoint to start
+ * @ep: the endpoint to start
+ * @can_sleep: flag indicating whether the operation is executed in
+ * non-atomic context
*
* A call to this function will increment the use count of the endpoint.
* In case it is not already running, the URBs for this endpoint will be
@@ -809,7 +807,7 @@
*
* Returns an error if the URB submission failed, 0 in all other cases.
*/
-int snd_usb_endpoint_start(struct snd_usb_endpoint *ep)
+int snd_usb_endpoint_start(struct snd_usb_endpoint *ep, int can_sleep)
{
int err;
unsigned int i;
@@ -821,6 +819,11 @@
if (++ep->use_count != 1)
return 0;
+ /* just to be sure */
+ deactivate_urbs(ep, 0, can_sleep);
+ if (can_sleep)
+ wait_clear_urbs(ep);
+
ep->active_mask = 0;
ep->unlink_mask = 0;
ep->phase = 0;
@@ -850,7 +853,6 @@
goto __error;
if (usb_pipeout(ep->pipe)) {
- prepare_outbound_urb_sizes(ep, urb->context);
prepare_outbound_urb(ep, urb->context);
} else {
prepare_inbound_urb(ep, urb->context);
diff --git a/sound/usb/endpoint.h b/sound/usb/endpoint.h
index ee2723f..cbbbdf2 100644
--- a/sound/usb/endpoint.h
+++ b/sound/usb/endpoint.h
@@ -13,7 +13,7 @@
struct audioformat *fmt,
struct snd_usb_endpoint *sync_ep);
-int snd_usb_endpoint_start(struct snd_usb_endpoint *ep);
+int snd_usb_endpoint_start(struct snd_usb_endpoint *ep, int can_sleep);
void snd_usb_endpoint_stop(struct snd_usb_endpoint *ep,
int force, int can_sleep, int wait);
int snd_usb_endpoint_activate(struct snd_usb_endpoint *ep);
@@ -21,6 +21,7 @@
void snd_usb_endpoint_free(struct list_head *head);
int snd_usb_endpoint_implict_feedback_sink(struct snd_usb_endpoint *ep);
+int snd_usb_endpoint_next_packet_size(struct snd_usb_endpoint *ep);
void snd_usb_handle_sync_urb(struct snd_usb_endpoint *ep,
struct snd_usb_endpoint *sender,
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 62ec808..f782ce1 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -212,7 +212,7 @@
}
}
-static int start_endpoints(struct snd_usb_substream *subs)
+static int start_endpoints(struct snd_usb_substream *subs, int can_sleep)
{
int err;
@@ -225,7 +225,7 @@
snd_printdd(KERN_DEBUG "Starting data EP @%p\n", ep);
ep->data_subs = subs;
- err = snd_usb_endpoint_start(ep);
+ err = snd_usb_endpoint_start(ep, can_sleep);
if (err < 0) {
clear_bit(SUBSTREAM_FLAG_DATA_EP_STARTED, &subs->flags);
return err;
@@ -236,10 +236,25 @@
!test_and_set_bit(SUBSTREAM_FLAG_SYNC_EP_STARTED, &subs->flags)) {
struct snd_usb_endpoint *ep = subs->sync_endpoint;
+ if (subs->data_endpoint->iface != subs->sync_endpoint->iface ||
+ subs->data_endpoint->alt_idx != subs->sync_endpoint->alt_idx) {
+ err = usb_set_interface(subs->dev,
+ subs->sync_endpoint->iface,
+ subs->sync_endpoint->alt_idx);
+ if (err < 0) {
+ snd_printk(KERN_ERR
+ "%d:%d:%d: cannot set interface (%d)\n",
+ subs->dev->devnum,
+ subs->sync_endpoint->iface,
+ subs->sync_endpoint->alt_idx, err);
+ return -EIO;
+ }
+ }
+
snd_printdd(KERN_DEBUG "Starting sync EP @%p\n", ep);
ep->sync_slave = subs->data_endpoint;
- err = snd_usb_endpoint_start(ep);
+ err = snd_usb_endpoint_start(ep, can_sleep);
if (err < 0) {
clear_bit(SUBSTREAM_FLAG_SYNC_EP_STARTED, &subs->flags);
return err;
@@ -544,13 +559,10 @@
subs->last_frame_number = 0;
runtime->delay = 0;
- /* clear the pending deactivation on the target EPs */
- deactivate_endpoints(subs);
-
/* for playback, submit the URBs now; otherwise, the first hwptr_done
* updates for all URBs would happen at the same time when starting */
if (subs->direction == SNDRV_PCM_STREAM_PLAYBACK)
- return start_endpoints(subs);
+ return start_endpoints(subs, 1);
return 0;
}
@@ -1032,6 +1044,7 @@
struct urb *urb)
{
struct snd_pcm_runtime *runtime = subs->pcm_substream->runtime;
+ struct snd_usb_endpoint *ep = subs->data_endpoint;
struct snd_urb_ctx *ctx = urb->context;
unsigned int counts, frames, bytes;
int i, stride, period_elapsed = 0;
@@ -1043,7 +1056,11 @@
urb->number_of_packets = 0;
spin_lock_irqsave(&subs->lock, flags);
for (i = 0; i < ctx->packets; i++) {
- counts = ctx->packet_size[i];
+ if (ctx->packet_size[i])
+ counts = ctx->packet_size[i];
+ else
+ counts = snd_usb_endpoint_next_packet_size(ep);
+
/* set up descriptor */
urb->iso_frame_desc[i].offset = frames * stride;
urb->iso_frame_desc[i].length = counts * stride;
@@ -1094,7 +1111,16 @@
subs->hwptr_done += bytes;
if (subs->hwptr_done >= runtime->buffer_size * stride)
subs->hwptr_done -= runtime->buffer_size * stride;
+
+ /* update delay with exact number of samples queued */
+ runtime->delay = subs->last_delay;
runtime->delay += frames;
+ subs->last_delay = runtime->delay;
+
+ /* realign last_frame_number */
+ subs->last_frame_number = usb_get_current_frame_number(subs->dev);
+ subs->last_frame_number &= 0xFF; /* keep 8 LSBs */
+
spin_unlock_irqrestore(&subs->lock, flags);
urb->transfer_buffer_length = bytes;
if (period_elapsed)
@@ -1112,12 +1138,32 @@
struct snd_pcm_runtime *runtime = subs->pcm_substream->runtime;
int stride = runtime->frame_bits >> 3;
int processed = urb->transfer_buffer_length / stride;
+ int est_delay;
+
+ /* ignore the delay accounting when procssed=0 is given, i.e.
+ * silent payloads are procssed before handling the actual data
+ */
+ if (!processed)
+ return;
spin_lock_irqsave(&subs->lock, flags);
- if (processed > runtime->delay)
- runtime->delay = 0;
+ est_delay = snd_usb_pcm_delay(subs, runtime->rate);
+ /* update delay with exact number of samples played */
+ if (processed > subs->last_delay)
+ subs->last_delay = 0;
else
- runtime->delay -= processed;
+ subs->last_delay -= processed;
+ runtime->delay = subs->last_delay;
+
+ /*
+ * Report when delay estimate is off by more than 2ms.
+ * The error should be lower than 2ms since the estimate relies
+ * on two reads of a counter updated every ms.
+ */
+ if (abs(est_delay - subs->last_delay) * 1000 > runtime->rate * 2)
+ snd_printk(KERN_DEBUG "delay: estimated %d, actual %d\n",
+ est_delay, subs->last_delay);
+
spin_unlock_irqrestore(&subs->lock, flags);
}
@@ -1175,7 +1221,7 @@
switch (cmd) {
case SNDRV_PCM_TRIGGER_START:
- err = start_endpoints(subs);
+ err = start_endpoints(subs, 0);
if (err < 0)
return err;
diff --git a/tools/lib/traceevent/Makefile b/tools/lib/traceevent/Makefile
index 14131cb..04d959f 100644
--- a/tools/lib/traceevent/Makefile
+++ b/tools/lib/traceevent/Makefile
@@ -129,7 +129,7 @@
# Append required CFLAGS
override CFLAGS += $(CONFIG_FLAGS) $(INCLUDES) $(PLUGIN_DIR_SQ)
-override CFLAGS += $(udis86-flags)
+override CFLAGS += $(udis86-flags) -D_GNU_SOURCE
ifeq ($(VERBOSE),1)
Q =
diff --git a/tools/lib/traceevent/event-parse.c b/tools/lib/traceevent/event-parse.c
index 5f34aa3..47264b4 100644
--- a/tools/lib/traceevent/event-parse.c
+++ b/tools/lib/traceevent/event-parse.c
@@ -24,13 +24,14 @@
* Frederic Weisbecker gave his permission to relicense the code to
* the Lesser General Public License.
*/
-#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <ctype.h>
#include <errno.h>
+#include <stdint.h>
+#include <limits.h>
#include "event-parse.h"
#include "event-utils.h"
@@ -117,14 +118,7 @@
struct print_arg *alloc_arg(void)
{
- struct print_arg *arg;
-
- arg = malloc_or_die(sizeof(*arg));
- if (!arg)
- return NULL;
- memset(arg, 0, sizeof(*arg));
-
- return arg;
+ return calloc(1, sizeof(struct print_arg));
}
struct cmdline {
@@ -158,7 +152,9 @@
struct cmdline *cmdlines;
int i;
- cmdlines = malloc_or_die(sizeof(*cmdlines) * pevent->cmdline_count);
+ cmdlines = malloc(sizeof(*cmdlines) * pevent->cmdline_count);
+ if (!cmdlines)
+ return -1;
i = 0;
while (cmdlist) {
@@ -186,8 +182,8 @@
if (!pid)
return "<idle>";
- if (!pevent->cmdlines)
- cmdline_init(pevent);
+ if (!pevent->cmdlines && cmdline_init(pevent))
+ return "<not enough memory for cmdlines!>";
key.pid = pid;
@@ -215,8 +211,8 @@
if (!pid)
return 1;
- if (!pevent->cmdlines)
- cmdline_init(pevent);
+ if (!pevent->cmdlines && cmdline_init(pevent))
+ return 0;
key.pid = pid;
@@ -258,10 +254,14 @@
return -1;
}
- cmdlines[pevent->cmdline_count].pid = pid;
cmdlines[pevent->cmdline_count].comm = strdup(comm);
- if (!cmdlines[pevent->cmdline_count].comm)
- die("malloc comm");
+ if (!cmdlines[pevent->cmdline_count].comm) {
+ free(cmdlines);
+ errno = ENOMEM;
+ return -1;
+ }
+
+ cmdlines[pevent->cmdline_count].pid = pid;
if (cmdlines[pevent->cmdline_count].comm)
pevent->cmdline_count++;
@@ -288,10 +288,15 @@
if (pevent->cmdlines)
return add_new_comm(pevent, comm, pid);
- item = malloc_or_die(sizeof(*item));
+ item = malloc(sizeof(*item));
+ if (!item)
+ return -1;
+
item->comm = strdup(comm);
- if (!item->comm)
- die("malloc comm");
+ if (!item->comm) {
+ free(item);
+ return -1;
+ }
item->pid = pid;
item->next = pevent->cmdlist;
@@ -355,7 +360,10 @@
struct func_map *func_map;
int i;
- func_map = malloc_or_die(sizeof(*func_map) * (pevent->func_count + 1));
+ func_map = malloc(sizeof(*func_map) * (pevent->func_count + 1));
+ if (!func_map)
+ return -1;
+
funclist = pevent->funclist;
i = 0;
@@ -455,25 +463,36 @@
int pevent_register_function(struct pevent *pevent, char *func,
unsigned long long addr, char *mod)
{
- struct func_list *item;
+ struct func_list *item = malloc(sizeof(*item));
- item = malloc_or_die(sizeof(*item));
+ if (!item)
+ return -1;
item->next = pevent->funclist;
item->func = strdup(func);
- if (mod)
+ if (!item->func)
+ goto out_free;
+
+ if (mod) {
item->mod = strdup(mod);
- else
+ if (!item->mod)
+ goto out_free_func;
+ } else
item->mod = NULL;
item->addr = addr;
- if (!item->func || (mod && !item->mod))
- die("malloc func");
-
pevent->funclist = item;
pevent->func_count++;
return 0;
+
+out_free_func:
+ free(item->func);
+ item->func = NULL;
+out_free:
+ free(item);
+ errno = ENOMEM;
+ return -1;
}
/**
@@ -524,14 +543,16 @@
return 0;
}
-static void printk_map_init(struct pevent *pevent)
+static int printk_map_init(struct pevent *pevent)
{
struct printk_list *printklist;
struct printk_list *item;
struct printk_map *printk_map;
int i;
- printk_map = malloc_or_die(sizeof(*printk_map) * (pevent->printk_count + 1));
+ printk_map = malloc(sizeof(*printk_map) * (pevent->printk_count + 1));
+ if (!printk_map)
+ return -1;
printklist = pevent->printklist;
@@ -549,6 +570,8 @@
pevent->printk_map = printk_map;
pevent->printklist = NULL;
+
+ return 0;
}
static struct printk_map *
@@ -557,8 +580,8 @@
struct printk_map *printk;
struct printk_map key;
- if (!pevent->printk_map)
- printk_map_init(pevent);
+ if (!pevent->printk_map && printk_map_init(pevent))
+ return NULL;
key.addr = addr;
@@ -580,21 +603,27 @@
int pevent_register_print_string(struct pevent *pevent, char *fmt,
unsigned long long addr)
{
- struct printk_list *item;
+ struct printk_list *item = malloc(sizeof(*item));
- item = malloc_or_die(sizeof(*item));
+ if (!item)
+ return -1;
item->next = pevent->printklist;
- item->printk = strdup(fmt);
item->addr = addr;
+ item->printk = strdup(fmt);
if (!item->printk)
- die("malloc fmt");
+ goto out_free;
pevent->printklist = item;
pevent->printk_count++;
return 0;
+
+out_free:
+ free(item);
+ errno = ENOMEM;
+ return -1;
}
/**
@@ -619,24 +648,18 @@
static struct event_format *alloc_event(void)
{
- struct event_format *event;
-
- event = malloc(sizeof(*event));
- if (!event)
- return NULL;
- memset(event, 0, sizeof(*event));
-
- return event;
+ return calloc(1, sizeof(struct event_format));
}
-static void add_event(struct pevent *pevent, struct event_format *event)
+static int add_event(struct pevent *pevent, struct event_format *event)
{
int i;
+ struct event_format **events = realloc(pevent->events, sizeof(event) *
+ (pevent->nr_events + 1));
+ if (!events)
+ return -1;
- pevent->events = realloc(pevent->events, sizeof(event) *
- (pevent->nr_events + 1));
- if (!pevent->events)
- die("Can not allocate events");
+ pevent->events = events;
for (i = 0; i < pevent->nr_events; i++) {
if (pevent->events[i]->id > event->id)
@@ -651,6 +674,8 @@
pevent->nr_events++;
event->pevent = pevent;
+
+ return 0;
}
static int event_item_type(enum event_type type)
@@ -827,9 +852,9 @@
switch (type) {
case EVENT_NEWLINE:
case EVENT_DELIM:
- *tok = malloc_or_die(2);
- (*tok)[0] = ch;
- (*tok)[1] = 0;
+ if (asprintf(tok, "%c", ch) < 0)
+ return EVENT_ERROR;
+
return type;
case EVENT_OP:
@@ -1240,8 +1265,10 @@
last_token = token;
- field = malloc_or_die(sizeof(*field));
- memset(field, 0, sizeof(*field));
+ field = calloc(1, sizeof(*field));
+ if (!field)
+ goto fail;
+
field->event = event;
/* read the rest of the type */
@@ -1282,7 +1309,7 @@
}
if (!field->type) {
- die("no type found");
+ do_warning("%s: no type found", __func__);
goto fail;
}
field->name = last_token;
@@ -1329,7 +1356,7 @@
free_token(token);
type = read_token(&token);
if (type == EVENT_NONE) {
- die("failed to find token");
+ do_warning("failed to find token");
goto fail;
}
}
@@ -1538,6 +1565,14 @@
left = alloc_arg();
right = alloc_arg();
+ if (!arg || !left || !right) {
+ do_warning("%s: not enough memory!", __func__);
+ /* arg will be freed at out_free */
+ free_arg(left);
+ free_arg(right);
+ goto out_free;
+ }
+
arg->type = PRINT_OP;
arg->op.left = left;
arg->op.right = right;
@@ -1580,6 +1615,12 @@
char *token = NULL;
arg = alloc_arg();
+ if (!arg) {
+ do_warning("%s: not enough memory!", __func__);
+ /* '*tok' is set to top->op.op. No need to free. */
+ *tok = NULL;
+ return EVENT_ERROR;
+ }
*tok = NULL;
type = process_arg(event, arg, &token);
@@ -1595,8 +1636,7 @@
return type;
out_free:
- free_token(*tok);
- *tok = NULL;
+ free_token(token);
free_arg(arg);
return EVENT_ERROR;
}
@@ -1682,7 +1722,7 @@
if (arg->type == PRINT_OP && !arg->op.left) {
/* handle single op */
if (token[1]) {
- die("bad op token %s", token);
+ do_warning("bad op token %s", token);
goto out_free;
}
switch (token[0]) {
@@ -1699,10 +1739,16 @@
/* make an empty left */
left = alloc_arg();
+ if (!left)
+ goto out_warn_free;
+
left->type = PRINT_NULL;
arg->op.left = left;
right = alloc_arg();
+ if (!right)
+ goto out_warn_free;
+
arg->op.right = right;
/* do not free the token, it belongs to an op */
@@ -1712,6 +1758,9 @@
} else if (strcmp(token, "?") == 0) {
left = alloc_arg();
+ if (!left)
+ goto out_warn_free;
+
/* copy the top arg to the left */
*left = *arg;
@@ -1720,6 +1769,7 @@
arg->op.left = left;
arg->op.prio = 0;
+ /* it will set arg->op.right */
type = process_cond(event, arg, tok);
} else if (strcmp(token, ">>") == 0 ||
@@ -1739,6 +1789,8 @@
strcmp(token, "!=") == 0) {
left = alloc_arg();
+ if (!left)
+ goto out_warn_free;
/* copy the top arg to the left */
*left = *arg;
@@ -1746,6 +1798,7 @@
arg->type = PRINT_OP;
arg->op.op = token;
arg->op.left = left;
+ arg->op.right = NULL;
if (set_op_prio(arg) == -1) {
event->flags |= EVENT_FL_FAILED;
@@ -1762,12 +1815,14 @@
type == EVENT_DELIM && (strcmp(token, ")") == 0)) {
char *new_atom;
- if (left->type != PRINT_ATOM)
- die("bad pointer type");
+ if (left->type != PRINT_ATOM) {
+ do_warning("bad pointer type");
+ goto out_free;
+ }
new_atom = realloc(left->atom.atom,
strlen(left->atom.atom) + 3);
if (!new_atom)
- goto out_free;
+ goto out_warn_free;
left->atom.atom = new_atom;
strcat(left->atom.atom, " *");
@@ -1779,12 +1834,18 @@
}
right = alloc_arg();
+ if (!right)
+ goto out_warn_free;
+
type = process_arg_token(event, right, tok, type);
arg->op.right = right;
} else if (strcmp(token, "[") == 0) {
left = alloc_arg();
+ if (!left)
+ goto out_warn_free;
+
*left = *arg;
arg->type = PRINT_OP;
@@ -1793,6 +1854,7 @@
arg->op.prio = 0;
+ /* it will set arg->op.right */
type = process_array(event, arg, tok);
} else {
@@ -1816,14 +1878,16 @@
return type;
- out_free:
+out_warn_free:
+ do_warning("%s: not enough memory!", __func__);
+out_free:
free_token(token);
*tok = NULL;
return EVENT_ERROR;
}
static enum event_type
-process_entry(struct event_format *event __unused, struct print_arg *arg,
+process_entry(struct event_format *event __maybe_unused, struct print_arg *arg,
char **tok)
{
enum event_type type;
@@ -1880,7 +1944,11 @@
return val;
}
- ref = malloc_or_die(len);
+ ref = malloc(len);
+ if (!ref) {
+ do_warning("%s: not enough memory!", __func__);
+ return val;
+ }
memcpy(ref, type, len);
/* chop off the " *" */
@@ -1957,8 +2025,10 @@
static unsigned long long
eval_type(unsigned long long val, struct print_arg *arg, int pointer)
{
- if (arg->type != PRINT_TYPE)
- die("expected type argument");
+ if (arg->type != PRINT_TYPE) {
+ do_warning("expected type argument");
+ return 0;
+ }
return eval_type_str(val, arg->typecast.type, pointer);
}
@@ -2143,7 +2213,7 @@
case PRINT_STRING:
case PRINT_BSTRING:
default:
- die("invalid eval type %d", arg->type);
+ do_warning("invalid eval type %d", arg->type);
break;
}
@@ -2166,6 +2236,8 @@
break;
arg = alloc_arg();
+ if (!arg)
+ goto out_free;
free_token(token);
type = process_arg(event, arg, &token);
@@ -2179,30 +2251,33 @@
if (test_type_token(type, token, EVENT_DELIM, ","))
goto out_free;
- field = malloc_or_die(sizeof(*field));
- memset(field, 0, sizeof(*field));
+ field = calloc(1, sizeof(*field));
+ if (!field)
+ goto out_free;
value = arg_eval(arg);
if (value == NULL)
- goto out_free;
+ goto out_free_field;
field->value = strdup(value);
if (field->value == NULL)
- goto out_free;
+ goto out_free_field;
free_arg(arg);
arg = alloc_arg();
+ if (!arg)
+ goto out_free;
free_token(token);
type = process_arg(event, arg, &token);
if (test_type_token(type, token, EVENT_OP, "}"))
- goto out_free;
+ goto out_free_field;
value = arg_eval(arg);
if (value == NULL)
- goto out_free;
+ goto out_free_field;
field->str = strdup(value);
if (field->str == NULL)
- goto out_free;
+ goto out_free_field;
free_arg(arg);
arg = NULL;
@@ -2216,6 +2291,8 @@
*tok = token;
return type;
+out_free_field:
+ free_flag_sym(field);
out_free:
free_arg(arg);
free_token(token);
@@ -2235,6 +2312,10 @@
arg->type = PRINT_FLAGS;
field = alloc_arg();
+ if (!field) {
+ do_warning("%s: not enough memory!", __func__);
+ goto out_free;
+ }
type = process_arg(event, field, &token);
@@ -2243,7 +2324,7 @@
type = process_op(event, field, &token);
if (test_type_token(type, token, EVENT_DELIM, ","))
- goto out_free;
+ goto out_free_field;
free_token(token);
arg->flags.field = field;
@@ -2265,7 +2346,9 @@
type = read_token_item(tok);
return type;
- out_free:
+out_free_field:
+ free_arg(field);
+out_free:
free_token(token);
*tok = NULL;
return EVENT_ERROR;
@@ -2282,10 +2365,14 @@
arg->type = PRINT_SYMBOL;
field = alloc_arg();
+ if (!field) {
+ do_warning("%s: not enough memory!", __func__);
+ goto out_free;
+ }
type = process_arg(event, field, &token);
if (test_type_token(type, token, EVENT_DELIM, ","))
- goto out_free;
+ goto out_free_field;
arg->symbol.field = field;
@@ -2297,7 +2384,9 @@
type = read_token_item(tok);
return type;
- out_free:
+out_free_field:
+ free_arg(field);
+out_free:
free_token(token);
*tok = NULL;
return EVENT_ERROR;
@@ -2314,6 +2403,11 @@
arg->type = PRINT_HEX;
field = alloc_arg();
+ if (!field) {
+ do_warning("%s: not enough memory!", __func__);
+ goto out_free;
+ }
+
type = process_arg(event, field, &token);
if (test_type_token(type, token, EVENT_DELIM, ","))
@@ -2324,6 +2418,12 @@
free_token(token);
field = alloc_arg();
+ if (!field) {
+ do_warning("%s: not enough memory!", __func__);
+ *tok = NULL;
+ return EVENT_ERROR;
+ }
+
type = process_arg(event, field, &token);
if (test_type_token(type, token, EVENT_DELIM, ")"))
@@ -2381,6 +2481,12 @@
free_token(token);
arg = alloc_arg();
+ if (!field) {
+ do_warning("%s: not enough memory!", __func__);
+ *tok = NULL;
+ return EVENT_ERROR;
+ }
+
type = process_arg(event, arg, &token);
if (type == EVENT_ERROR)
goto out_free_arg;
@@ -2434,10 +2540,16 @@
/* make this a typecast and contine */
/* prevous must be an atom */
- if (arg->type != PRINT_ATOM)
- die("previous needed to be PRINT_ATOM");
+ if (arg->type != PRINT_ATOM) {
+ do_warning("previous needed to be PRINT_ATOM");
+ goto out_free;
+ }
item_arg = alloc_arg();
+ if (!item_arg) {
+ do_warning("%s: not enough memory!", __func__);
+ goto out_free;
+ }
arg->type = PRINT_TYPE;
arg->typecast.type = arg->atom.atom;
@@ -2457,7 +2569,8 @@
static enum event_type
-process_str(struct event_format *event __unused, struct print_arg *arg, char **tok)
+process_str(struct event_format *event __maybe_unused, struct print_arg *arg,
+ char **tok)
{
enum event_type type;
char *token;
@@ -2532,6 +2645,11 @@
next_arg = &(arg->func.args);
for (i = 0; i < func->nr_args; i++) {
farg = alloc_arg();
+ if (!farg) {
+ do_warning("%s: not enough memory!", __func__);
+ return EVENT_ERROR;
+ }
+
type = process_arg(event, farg, &token);
if (i < (func->nr_args - 1))
test = ",";
@@ -2676,7 +2794,8 @@
case EVENT_ERROR ... EVENT_NEWLINE:
default:
- die("unexpected type %d", type);
+ do_warning("unexpected type %d", type);
+ return EVENT_ERROR;
}
*tok = token;
@@ -2697,6 +2816,10 @@
}
arg = alloc_arg();
+ if (!arg) {
+ do_warning("%s: not enough memory!", __func__);
+ return -1;
+ }
type = process_arg(event, arg, &token);
@@ -2768,10 +2891,8 @@
if (type == EVENT_DQUOTE) {
char *cat;
- cat = malloc_or_die(strlen(event->print_fmt.format) +
- strlen(token) + 1);
- strcpy(cat, event->print_fmt.format);
- strcat(cat, token);
+ if (asprintf(&cat, "%s%s", event->print_fmt.format, token) < 0)
+ goto fail;
free_token(token);
free_token(event->print_fmt.format);
event->print_fmt.format = NULL;
@@ -2925,8 +3046,10 @@
* All events should have the same common elements.
* Pick any event to find where the type is;
*/
- if (!pevent->events)
- die("no event_list!");
+ if (!pevent->events) {
+ do_warning("no event_list!");
+ return -1;
+ }
event = pevent->events[0];
field = pevent_find_common_field(event, type);
@@ -3084,7 +3207,8 @@
if (!arg->field.field) {
arg->field.field = pevent_find_any_field(event, arg->field.name);
if (!arg->field.field)
- die("field %s not found", arg->field.name);
+ goto out_warning_field;
+
}
/* must be a number */
val = pevent_read_number(pevent, data + arg->field.field->offset,
@@ -3145,8 +3269,10 @@
if (!larg->field.field) {
larg->field.field =
pevent_find_any_field(event, larg->field.name);
- if (!larg->field.field)
- die("field %s not found", larg->field.name);
+ if (!larg->field.field) {
+ arg = larg;
+ goto out_warning_field;
+ }
}
field_size = larg->field.field->elementsize;
offset = larg->field.field->offset +
@@ -3182,7 +3308,7 @@
val = left != right;
break;
default:
- die("unknown op '%s'", arg->op.op);
+ goto out_warning_op;
}
break;
case '~':
@@ -3212,7 +3338,7 @@
val = left <= right;
break;
default:
- die("unknown op '%s'", arg->op.op);
+ goto out_warning_op;
}
break;
case '>':
@@ -3227,12 +3353,13 @@
val = left >= right;
break;
default:
- die("unknown op '%s'", arg->op.op);
+ goto out_warning_op;
}
break;
case '=':
if (arg->op.op[1] != '=')
- die("unknown op '%s'", arg->op.op);
+ goto out_warning_op;
+
val = left == right;
break;
case '-':
@@ -3248,13 +3375,21 @@
val = left * right;
break;
default:
- die("unknown op '%s'", arg->op.op);
+ goto out_warning_op;
}
break;
default: /* not sure what to do there */
return 0;
}
return val;
+
+out_warning_op:
+ do_warning("%s: unknown op '%s'", __func__, arg->op.op);
+ return 0;
+
+out_warning_field:
+ do_warning("%s: field %s not found", __func__, arg->field.name);
+ return 0;
}
struct flag {
@@ -3331,8 +3466,10 @@
field = arg->field.field;
if (!field) {
field = pevent_find_any_field(event, arg->field.name);
- if (!field)
- die("field %s not found", arg->field.name);
+ if (!field) {
+ str = arg->field.name;
+ goto out_warning_field;
+ }
arg->field.field = field;
}
/* Zero sized fields, mean the rest of the data */
@@ -3349,7 +3486,11 @@
trace_seq_printf(s, "%lx", addr);
break;
}
- str = malloc_or_die(len + 1);
+ str = malloc(len + 1);
+ if (!str) {
+ do_warning("%s: not enough memory!", __func__);
+ return;
+ }
memcpy(str, data + field->offset, len);
str[len] = 0;
print_str_to_seq(s, format, len_arg, str);
@@ -3389,7 +3530,7 @@
str = arg->hex.field->field.name;
field = pevent_find_any_field(event, str);
if (!field)
- die("field %s not found", str);
+ goto out_warning_field;
arg->hex.field->field.field = field;
}
hex = data + field->offset;
@@ -3441,6 +3582,11 @@
/* well... */
break;
}
+
+ return;
+
+out_warning_field:
+ do_warning("%s: field %s not found", __func__, arg->field.name);
}
static unsigned long long
@@ -3467,7 +3613,11 @@
farg = arg->func.args;
param = func_handle->params;
- args = malloc_or_die(sizeof(*args) * func_handle->nr_args);
+ ret = ULLONG_MAX;
+ args = malloc(sizeof(*args) * func_handle->nr_args);
+ if (!args)
+ goto out;
+
for (i = 0; i < func_handle->nr_args; i++) {
switch (param->type) {
case PEVENT_FUNC_ARG_INT:
@@ -3479,13 +3629,19 @@
trace_seq_init(&str);
print_str_arg(&str, data, size, event, "%s", -1, farg);
trace_seq_terminate(&str);
- string = malloc_or_die(sizeof(*string));
+ string = malloc(sizeof(*string));
+ if (!string) {
+ do_warning("%s(%d): malloc str", __func__, __LINE__);
+ goto out_free;
+ }
string->next = strings;
string->str = strdup(str.buffer);
- if (!string->str)
- die("malloc str");
-
- args[i] = (unsigned long long)string->str;
+ if (!string->str) {
+ free(string);
+ do_warning("%s(%d): malloc str", __func__, __LINE__);
+ goto out_free;
+ }
+ args[i] = (uintptr_t)string->str;
strings = string;
trace_seq_destroy(&str);
break;
@@ -3494,14 +3650,15 @@
* Something went totally wrong, this is not
* an input error, something in this code broke.
*/
- die("Unexpected end of arguments\n");
- break;
+ do_warning("Unexpected end of arguments\n");
+ goto out_free;
}
farg = farg->next;
param = param->next;
}
ret = (*func_handle->func)(s, args);
+out_free:
free(args);
while (strings) {
string = strings;
@@ -3515,6 +3672,18 @@
return ret;
}
+static void free_args(struct print_arg *args)
+{
+ struct print_arg *next;
+
+ while (args) {
+ next = args->next;
+
+ free_arg(args);
+ args = next;
+ }
+}
+
static struct print_arg *make_bprint_args(char *fmt, void *data, int size, struct event_format *event)
{
struct pevent *pevent = event->pevent;
@@ -3530,11 +3699,15 @@
if (!field) {
field = pevent_find_field(event, "buf");
- if (!field)
- die("can't find buffer field for binary printk");
+ if (!field) {
+ do_warning("can't find buffer field for binary printk");
+ return NULL;
+ }
ip_field = pevent_find_field(event, "ip");
- if (!ip_field)
- die("can't find ip field for binary printk");
+ if (!ip_field) {
+ do_warning("can't find ip field for binary printk");
+ return NULL;
+ }
pevent->bprint_buf_field = field;
pevent->bprint_ip_field = ip_field;
}
@@ -3545,13 +3718,18 @@
* The first arg is the IP pointer.
*/
args = alloc_arg();
+ if (!args) {
+ do_warning("%s(%d): not enough memory!", __func__, __LINE__);
+ return NULL;
+ }
arg = args;
arg->next = NULL;
next = &arg->next;
arg->type = PRINT_ATOM;
- arg->atom.atom = malloc_or_die(32);
- sprintf(arg->atom.atom, "%lld", ip);
+
+ if (asprintf(&arg->atom.atom, "%lld", ip) < 0)
+ goto out_free;
/* skip the first "%pf : " */
for (ptr = fmt + 6, bptr = data + field->offset;
@@ -3606,10 +3784,17 @@
val = pevent_read_number(pevent, bptr, vsize);
bptr += vsize;
arg = alloc_arg();
+ if (!arg) {
+ do_warning("%s(%d): not enough memory!",
+ __func__, __LINE__);
+ goto out_free;
+ }
arg->next = NULL;
arg->type = PRINT_ATOM;
- arg->atom.atom = malloc_or_die(32);
- sprintf(arg->atom.atom, "%lld", val);
+ if (asprintf(&arg->atom.atom, "%lld", val) < 0) {
+ free(arg);
+ goto out_free;
+ }
*next = arg;
next = &arg->next;
/*
@@ -3622,11 +3807,16 @@
break;
case 's':
arg = alloc_arg();
+ if (!arg) {
+ do_warning("%s(%d): not enough memory!",
+ __func__, __LINE__);
+ goto out_free;
+ }
arg->next = NULL;
arg->type = PRINT_BSTRING;
arg->string.string = strdup(bptr);
if (!arg->string.string)
- break;
+ goto out_free;
bptr += strlen(bptr) + 1;
*next = arg;
next = &arg->next;
@@ -3637,22 +3827,15 @@
}
return args;
-}
-static void free_args(struct print_arg *args)
-{
- struct print_arg *next;
-
- while (args) {
- next = args->next;
-
- free_arg(args);
- args = next;
- }
+out_free:
+ free_args(args);
+ return NULL;
}
static char *
-get_bprint_format(void *data, int size __unused, struct event_format *event)
+get_bprint_format(void *data, int size __maybe_unused,
+ struct event_format *event)
{
struct pevent *pevent = event->pevent;
unsigned long long addr;
@@ -3665,8 +3848,10 @@
if (!field) {
field = pevent_find_field(event, "fmt");
- if (!field)
- die("can't find format field for binary printk");
+ if (!field) {
+ do_warning("can't find format field for binary printk");
+ return NULL;
+ }
pevent->bprint_fmt_field = field;
}
@@ -3674,9 +3859,8 @@
printk = find_printk(pevent, addr);
if (!printk) {
- format = malloc_or_die(45);
- sprintf(format, "%%pf : (NO FORMAT FOUND at %llx)\n",
- addr);
+ if (asprintf(&format, "%%pf : (NO FORMAT FOUND at %llx)\n", addr) < 0)
+ return NULL;
return format;
}
@@ -3684,8 +3868,8 @@
/* Remove any quotes. */
if (*p == '"')
p++;
- format = malloc_or_die(strlen(p) + 10);
- sprintf(format, "%s : %s", "%pf", p);
+ if (asprintf(&format, "%s : %s", "%pf", p) < 0)
+ return NULL;
/* remove ending quotes and new line since we will add one too */
p = format + strlen(format) - 1;
if (*p == '"')
@@ -3720,8 +3904,11 @@
if (!arg->field.field) {
arg->field.field =
pevent_find_any_field(event, arg->field.name);
- if (!arg->field.field)
- die("field %s not found", arg->field.name);
+ if (!arg->field.field) {
+ do_warning("%s: field %s not found",
+ __func__, arg->field.name);
+ return;
+ }
}
if (arg->field.field->size != 6) {
trace_seq_printf(s, "INVALIDMAC");
@@ -3888,8 +4075,11 @@
goto cont_process;
case '*':
/* The argument is the length. */
- if (!arg)
- die("no argument match");
+ if (!arg) {
+ do_warning("no argument match");
+ event->flags |= EVENT_FL_FAILED;
+ goto out_failed;
+ }
len_arg = eval_num_arg(data, size, event, arg);
len_as_arg = 1;
arg = arg->next;
@@ -3922,15 +4112,21 @@
case 'x':
case 'X':
case 'u':
- if (!arg)
- die("no argument match");
+ if (!arg) {
+ do_warning("no argument match");
+ event->flags |= EVENT_FL_FAILED;
+ goto out_failed;
+ }
len = ((unsigned long)ptr + 1) -
(unsigned long)saveptr;
/* should never happen */
- if (len > 31)
- die("bad format!");
+ if (len > 31) {
+ do_warning("bad format!");
+ event->flags |= EVENT_FL_FAILED;
+ len = 31;
+ }
memcpy(format, saveptr, len);
format[len] = 0;
@@ -3994,19 +4190,26 @@
trace_seq_printf(s, format, (long long)val);
break;
default:
- die("bad count (%d)", ls);
+ do_warning("bad count (%d)", ls);
+ event->flags |= EVENT_FL_FAILED;
}
break;
case 's':
- if (!arg)
- die("no matching argument");
+ if (!arg) {
+ do_warning("no matching argument");
+ event->flags |= EVENT_FL_FAILED;
+ goto out_failed;
+ }
len = ((unsigned long)ptr + 1) -
(unsigned long)saveptr;
/* should never happen */
- if (len > 31)
- die("bad format!");
+ if (len > 31) {
+ do_warning("bad format!");
+ event->flags |= EVENT_FL_FAILED;
+ len = 31;
+ }
memcpy(format, saveptr, len);
format[len] = 0;
@@ -4024,6 +4227,11 @@
trace_seq_putc(s, *ptr);
}
+ if (event->flags & EVENT_FL_FAILED) {
+out_failed:
+ trace_seq_printf(s, "[FAILED TO PARSE]");
+ }
+
if (args) {
free_args(args);
free(bprint_fmt);
@@ -4356,7 +4564,10 @@
struct format_field *field;
int i = 0;
- fields = malloc_or_die(sizeof(*fields) * (count + 1));
+ fields = malloc(sizeof(*fields) * (count + 1));
+ if (!fields)
+ return NULL;
+
for (field = list; field; field = field->next) {
fields[i++] = field;
if (i == count + 1) {
@@ -4672,6 +4883,142 @@
}
/**
+ * __pevent_parse_format - parse the event format
+ * @buf: the buffer storing the event format string
+ * @size: the size of @buf
+ * @sys: the system the event belongs to
+ *
+ * This parses the event format and creates an event structure
+ * to quickly parse raw data for a given event.
+ *
+ * These files currently come from:
+ *
+ * /sys/kernel/debug/tracing/events/.../.../format
+ */
+enum pevent_errno __pevent_parse_format(struct event_format **eventp,
+ struct pevent *pevent, const char *buf,
+ unsigned long size, const char *sys)
+{
+ struct event_format *event;
+ int ret;
+
+ init_input_buf(buf, size);
+
+ *eventp = event = alloc_event();
+ if (!event)
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+
+ event->name = event_read_name();
+ if (!event->name) {
+ /* Bad event? */
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ goto event_alloc_failed;
+ }
+
+ if (strcmp(sys, "ftrace") == 0) {
+ event->flags |= EVENT_FL_ISFTRACE;
+
+ if (strcmp(event->name, "bprint") == 0)
+ event->flags |= EVENT_FL_ISBPRINT;
+ }
+
+ event->id = event_read_id();
+ if (event->id < 0) {
+ ret = PEVENT_ERRNO__READ_ID_FAILED;
+ /*
+ * This isn't an allocation error actually.
+ * But as the ID is critical, just bail out.
+ */
+ goto event_alloc_failed;
+ }
+
+ event->system = strdup(sys);
+ if (!event->system) {
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ goto event_alloc_failed;
+ }
+
+ ret = event_read_format(event);
+ if (ret < 0) {
+ ret = PEVENT_ERRNO__READ_FORMAT_FAILED;
+ goto event_parse_failed;
+ }
+
+ /*
+ * If the event has an override, don't print warnings if the event
+ * print format fails to parse.
+ */
+ if (pevent && find_event_handle(pevent, event))
+ show_warning = 0;
+
+ ret = event_read_print(event);
+ show_warning = 1;
+
+ if (ret < 0) {
+ ret = PEVENT_ERRNO__READ_PRINT_FAILED;
+ goto event_parse_failed;
+ }
+
+ if (!ret && (event->flags & EVENT_FL_ISFTRACE)) {
+ struct format_field *field;
+ struct print_arg *arg, **list;
+
+ /* old ftrace had no args */
+ list = &event->print_fmt.args;
+ for (field = event->format.fields; field; field = field->next) {
+ arg = alloc_arg();
+ if (!arg) {
+ event->flags |= EVENT_FL_FAILED;
+ return PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED;
+ }
+ arg->type = PRINT_FIELD;
+ arg->field.name = strdup(field->name);
+ if (!arg->field.name) {
+ event->flags |= EVENT_FL_FAILED;
+ free_arg(arg);
+ return PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED;
+ }
+ arg->field.field = field;
+ *list = arg;
+ list = &arg->next;
+ }
+ return 0;
+ }
+
+ return 0;
+
+ event_parse_failed:
+ event->flags |= EVENT_FL_FAILED;
+ return ret;
+
+ event_alloc_failed:
+ free(event->system);
+ free(event->name);
+ free(event);
+ *eventp = NULL;
+ return ret;
+}
+
+/**
+ * pevent_parse_format - parse the event format
+ * @buf: the buffer storing the event format string
+ * @size: the size of @buf
+ * @sys: the system the event belongs to
+ *
+ * This parses the event format and creates an event structure
+ * to quickly parse raw data for a given event.
+ *
+ * These files currently come from:
+ *
+ * /sys/kernel/debug/tracing/events/.../.../format
+ */
+enum pevent_errno pevent_parse_format(struct event_format **eventp, const char *buf,
+ unsigned long size, const char *sys)
+{
+ return __pevent_parse_format(eventp, NULL, buf, size, sys);
+}
+
+/**
* pevent_parse_event - parse the event format
* @pevent: the handle to the pevent
* @buf: the buffer storing the event format string
@@ -4685,90 +5032,21 @@
*
* /sys/kernel/debug/tracing/events/.../.../format
*/
-int pevent_parse_event(struct pevent *pevent,
- const char *buf, unsigned long size,
- const char *sys)
+enum pevent_errno pevent_parse_event(struct pevent *pevent, const char *buf,
+ unsigned long size, const char *sys)
{
- struct event_format *event;
- int ret;
+ struct event_format *event = NULL;
+ int ret = __pevent_parse_format(&event, pevent, buf, size, sys);
- init_input_buf(buf, size);
-
- event = alloc_event();
- if (!event)
- return -ENOMEM;
-
- event->name = event_read_name();
- if (!event->name) {
- /* Bad event? */
- free(event);
- return -1;
- }
-
- if (strcmp(sys, "ftrace") == 0) {
-
- event->flags |= EVENT_FL_ISFTRACE;
-
- if (strcmp(event->name, "bprint") == 0)
- event->flags |= EVENT_FL_ISBPRINT;
- }
-
- event->id = event_read_id();
- if (event->id < 0)
- die("failed to read event id");
-
- event->system = strdup(sys);
- if (!event->system)
- die("failed to allocate system");
+ if (event == NULL)
+ return ret;
/* Add pevent to event so that it can be referenced */
event->pevent = pevent;
- ret = event_read_format(event);
- if (ret < 0) {
- do_warning("failed to read event format for %s", event->name);
- goto event_failed;
- }
-
- /*
- * If the event has an override, don't print warnings if the event
- * print format fails to parse.
- */
- if (find_event_handle(pevent, event))
- show_warning = 0;
-
- ret = event_read_print(event);
- if (ret < 0) {
- do_warning("failed to read event print fmt for %s",
- event->name);
- show_warning = 1;
- goto event_failed;
- }
- show_warning = 1;
-
- add_event(pevent, event);
-
- if (!ret && (event->flags & EVENT_FL_ISFTRACE)) {
- struct format_field *field;
- struct print_arg *arg, **list;
-
- /* old ftrace had no args */
-
- list = &event->print_fmt.args;
- for (field = event->format.fields; field; field = field->next) {
- arg = alloc_arg();
- *list = arg;
- list = &arg->next;
- arg->type = PRINT_FIELD;
- arg->field.name = strdup(field->name);
- if (!arg->field.name) {
- do_warning("failed to allocate field name");
- event->flags |= EVENT_FL_FAILED;
- return -1;
- }
- arg->field.field = field;
- }
- return 0;
+ if (add_event(pevent, event)) {
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ goto event_add_failed;
}
#define PRINT_ARGS 0
@@ -4777,11 +5055,57 @@
return 0;
- event_failed:
- event->flags |= EVENT_FL_FAILED;
- /* still add it even if it failed */
- add_event(pevent, event);
- return -1;
+event_add_failed:
+ pevent_free_format(event);
+ return ret;
+}
+
+#undef _PE
+#define _PE(code, str) str
+static const char * const pevent_error_str[] = {
+ PEVENT_ERRORS
+};
+#undef _PE
+
+int pevent_strerror(struct pevent *pevent, enum pevent_errno errnum,
+ char *buf, size_t buflen)
+{
+ int idx;
+ const char *msg;
+
+ if (errnum >= 0) {
+ msg = strerror_r(errnum, buf, buflen);
+ if (msg != buf) {
+ size_t len = strlen(msg);
+ memcpy(buf, msg, min(buflen - 1, len));
+ *(buf + min(buflen - 1, len)) = '\0';
+ }
+ return 0;
+ }
+
+ if (errnum <= __PEVENT_ERRNO__START ||
+ errnum >= __PEVENT_ERRNO__END)
+ return -1;
+
+ idx = errnum - __PEVENT_ERRNO__START - 1;
+ msg = pevent_error_str[idx];
+
+ switch (errnum) {
+ case PEVENT_ERRNO__MEM_ALLOC_FAILED:
+ case PEVENT_ERRNO__PARSE_EVENT_FAILED:
+ case PEVENT_ERRNO__READ_ID_FAILED:
+ case PEVENT_ERRNO__READ_FORMAT_FAILED:
+ case PEVENT_ERRNO__READ_PRINT_FAILED:
+ case PEVENT_ERRNO__OLD_FTRACE_ARG_FAILED:
+ snprintf(buf, buflen, "%s", msg);
+ break;
+
+ default:
+ /* cannot reach here */
+ break;
+ }
+
+ return 0;
}
int get_field_val(struct trace_seq *s, struct format_field *field,
@@ -5000,6 +5324,7 @@
struct pevent_func_params *param;
enum pevent_func_arg_type type;
va_list ap;
+ int ret;
func_handle = find_func_handler(pevent, name);
if (func_handle) {
@@ -5012,14 +5337,20 @@
remove_func_handler(pevent, name);
}
- func_handle = malloc_or_die(sizeof(*func_handle));
- memset(func_handle, 0, sizeof(*func_handle));
+ func_handle = calloc(1, sizeof(*func_handle));
+ if (!func_handle) {
+ do_warning("Failed to allocate function handler");
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
func_handle->ret_type = ret_type;
func_handle->name = strdup(name);
func_handle->func = func;
- if (!func_handle->name)
- die("Failed to allocate function name");
+ if (!func_handle->name) {
+ do_warning("Failed to allocate function name");
+ free(func_handle);
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
next_param = &(func_handle->params);
va_start(ap, name);
@@ -5029,11 +5360,17 @@
break;
if (type < 0 || type >= PEVENT_FUNC_ARG_MAX_TYPES) {
- warning("Invalid argument type %d", type);
+ do_warning("Invalid argument type %d", type);
+ ret = PEVENT_ERRNO__INVALID_ARG_TYPE;
goto out_free;
}
- param = malloc_or_die(sizeof(*param));
+ param = malloc(sizeof(*param));
+ if (!param) {
+ do_warning("Failed to allocate function param");
+ ret = PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ goto out_free;
+ }
param->type = type;
param->next = NULL;
@@ -5051,7 +5388,7 @@
out_free:
va_end(ap);
free_func_handle(func_handle);
- return -1;
+ return ret;
}
/**
@@ -5103,8 +5440,12 @@
not_found:
/* Save for later use. */
- handle = malloc_or_die(sizeof(*handle));
- memset(handle, 0, sizeof(*handle));
+ handle = calloc(1, sizeof(*handle));
+ if (!handle) {
+ do_warning("Failed to allocate event handler");
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
+ }
+
handle->id = id;
if (event_name)
handle->event_name = strdup(event_name);
@@ -5113,7 +5454,11 @@
if ((event_name && !handle->event_name) ||
(sys_name && !handle->sys_name)) {
- die("Failed to allocate event/sys name");
+ do_warning("Failed to allocate event/sys name");
+ free((void *)handle->event_name);
+ free((void *)handle->sys_name);
+ free(handle);
+ return PEVENT_ERRNO__MEM_ALLOC_FAILED;
}
handle->func = func;
@@ -5129,13 +5474,10 @@
*/
struct pevent *pevent_alloc(void)
{
- struct pevent *pevent;
+ struct pevent *pevent = calloc(1, sizeof(*pevent));
- pevent = malloc(sizeof(*pevent));
- if (!pevent)
- return NULL;
- memset(pevent, 0, sizeof(*pevent));
- pevent->ref_count = 1;
+ if (pevent)
+ pevent->ref_count = 1;
return pevent;
}
@@ -5164,7 +5506,7 @@
free_format_fields(format->fields);
}
-static void free_event(struct event_format *event)
+void pevent_free_format(struct event_format *event)
{
free(event->name);
free(event->system);
@@ -5250,7 +5592,7 @@
}
for (i = 0; i < pevent->nr_events; i++)
- free_event(pevent->events[i]);
+ pevent_free_format(pevent->events[i]);
while (pevent->handlers) {
handle = pevent->handlers;
diff --git a/tools/lib/traceevent/event-parse.h b/tools/lib/traceevent/event-parse.h
index 5772ad8..24a4bba 100644
--- a/tools/lib/traceevent/event-parse.h
+++ b/tools/lib/traceevent/event-parse.h
@@ -24,8 +24,8 @@
#include <stdarg.h>
#include <regex.h>
-#ifndef __unused
-#define __unused __attribute__ ((unused))
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((unused))
#endif
/* ----------------------- trace_seq ----------------------- */
@@ -49,7 +49,7 @@
int cpu;
int ref_count;
int locked; /* Do not free, even if ref_count is zero */
- void *private;
+ void *priv;
#if DEBUG_RECORD
struct pevent_record *prev;
struct pevent_record *next;
@@ -106,7 +106,7 @@
char *plugin_alias;
char *description;
char *value;
- void *private;
+ void *priv;
int set;
};
@@ -345,6 +345,35 @@
PEVENT_NSEC_OUTPUT = 1, /* output in NSECS */
};
+#define PEVENT_ERRORS \
+ _PE(MEM_ALLOC_FAILED, "failed to allocate memory"), \
+ _PE(PARSE_EVENT_FAILED, "failed to parse event"), \
+ _PE(READ_ID_FAILED, "failed to read event id"), \
+ _PE(READ_FORMAT_FAILED, "failed to read event format"), \
+ _PE(READ_PRINT_FAILED, "failed to read event print fmt"), \
+ _PE(OLD_FTRACE_ARG_FAILED,"failed to allocate field name for ftrace"),\
+ _PE(INVALID_ARG_TYPE, "invalid argument type")
+
+#undef _PE
+#define _PE(__code, __str) PEVENT_ERRNO__ ## __code
+enum pevent_errno {
+ PEVENT_ERRNO__SUCCESS = 0,
+
+ /*
+ * Choose an arbitrary negative big number not to clash with standard
+ * errno since SUS requires the errno has distinct positive values.
+ * See 'Issue 6' in the link below.
+ *
+ * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html
+ */
+ __PEVENT_ERRNO__START = -100000,
+
+ PEVENT_ERRORS,
+
+ __PEVENT_ERRNO__END,
+};
+#undef _PE
+
struct cmdline;
struct cmdline_list;
struct func_map;
@@ -509,8 +538,11 @@
int pevent_parse_header_page(struct pevent *pevent, char *buf, unsigned long size,
int long_size);
-int pevent_parse_event(struct pevent *pevent, const char *buf,
- unsigned long size, const char *sys);
+enum pevent_errno pevent_parse_event(struct pevent *pevent, const char *buf,
+ unsigned long size, const char *sys);
+enum pevent_errno pevent_parse_format(struct event_format **eventp, const char *buf,
+ unsigned long size, const char *sys);
+void pevent_free_format(struct event_format *event);
void *pevent_get_field_raw(struct trace_seq *s, struct event_format *event,
const char *name, struct pevent_record *record,
@@ -561,6 +593,8 @@
const char *pevent_data_comm_from_pid(struct pevent *pevent, int pid);
void pevent_event_info(struct trace_seq *s, struct event_format *event,
struct pevent_record *record);
+int pevent_strerror(struct pevent *pevent, enum pevent_errno errnum,
+ char *buf, size_t buflen);
struct event_format **pevent_list_events(struct pevent *pevent, enum event_sort_type);
struct format_field **pevent_event_common_fields(struct event_format *event);
diff --git a/tools/lib/traceevent/event-utils.h b/tools/lib/traceevent/event-utils.h
index 0829638..bc07500 100644
--- a/tools/lib/traceevent/event-utils.h
+++ b/tools/lib/traceevent/event-utils.h
@@ -39,6 +39,12 @@
void __vwarning(const char *fmt, ...);
void __vpr_stat(const char *fmt, ...);
+#define min(x, y) ({ \
+ typeof(x) _min1 = (x); \
+ typeof(y) _min2 = (y); \
+ (void) (&_min1 == &_min2); \
+ _min1 < _min2 ? _min1 : _min2; })
+
static inline char *strim(char *string)
{
char *ret;
diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 26b823b..8f8fbc2 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -21,3 +21,5 @@
config.mak.autogen
*-bison.*
*-flex.*
+*.pyc
+*.pyo
diff --git a/tools/perf/Documentation/Makefile b/tools/perf/Documentation/Makefile
index ca600e0..9f2e44f 100644
--- a/tools/perf/Documentation/Makefile
+++ b/tools/perf/Documentation/Makefile
@@ -195,10 +195,10 @@
#install-html: html
# '$(SHELL_PATH_SQ)' ./install-webdoc.sh $(DESTDIR)$(htmldir)
-../PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
- $(QUIET_SUBDIR0)../ $(QUIET_SUBDIR1) PERF-VERSION-FILE
+$(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
+ $(QUIET_SUBDIR0)../ $(QUIET_SUBDIR1) $(OUTPUT)PERF-VERSION-FILE
--include ../PERF-VERSION-FILE
+-include $(OUTPUT)PERF-VERSION-FILE
#
# Determine "include::" file references in asciidoc files.
diff --git a/tools/perf/Documentation/jit-interface.txt b/tools/perf/Documentation/jit-interface.txt
new file mode 100644
index 0000000..a8656f5
--- /dev/null
+++ b/tools/perf/Documentation/jit-interface.txt
@@ -0,0 +1,15 @@
+perf supports a simple JIT interface to resolve symbols for dynamic code generated
+by a JIT.
+
+The JIT has to write a /tmp/perf-%d.map (%d = pid of process) file
+
+This is a text file.
+
+Each line has the following format, fields separated with spaces:
+
+START SIZE symbolname
+
+START and SIZE are hex numbers without 0x.
+symbolname is the rest of the line, so it could contain special characters.
+
+The ownership of the file has to match the process.
diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt
index c89f9e1..c8ffd9f 100644
--- a/tools/perf/Documentation/perf-annotate.txt
+++ b/tools/perf/Documentation/perf-annotate.txt
@@ -85,6 +85,9 @@
-M::
--disassembler-style=:: Set disassembler style for objdump.
+--objdump=<path>::
+ Path to objdump binary.
+
SEE ALSO
--------
linkperf:perf-record[1], linkperf:perf-report[1]
diff --git a/tools/perf/Documentation/perf-diff.txt b/tools/perf/Documentation/perf-diff.txt
index 74d7481..ab7f667 100644
--- a/tools/perf/Documentation/perf-diff.txt
+++ b/tools/perf/Documentation/perf-diff.txt
@@ -17,6 +17,9 @@
If no parameters are passed it will assume perf.data.old and perf.data.
+The differential profile is displayed only for events matching both
+specified perf.data files.
+
OPTIONS
-------
-M::
diff --git a/tools/perf/Documentation/perf-kvm.txt b/tools/perf/Documentation/perf-kvm.txt
index dd84cb2..326f2cb 100644
--- a/tools/perf/Documentation/perf-kvm.txt
+++ b/tools/perf/Documentation/perf-kvm.txt
@@ -12,7 +12,7 @@
[--guestkallsyms=<path> --guestmodules=<path> | --guestvmlinux=<path>]]
{top|record|report|diff|buildid-list}
'perf kvm' [--host] [--guest] [--guestkallsyms=<path> --guestmodules=<path>
- | --guestvmlinux=<path>] {top|record|report|diff|buildid-list}
+ | --guestvmlinux=<path>] {top|record|report|diff|buildid-list|stat}
DESCRIPTION
-----------
@@ -38,6 +38,18 @@
so that other tools can be used to fetch packages with matching symbol tables
for use by perf report.
+ 'perf kvm stat <command>' to run a command and gather performance counter
+ statistics.
+ Especially, perf 'kvm stat record/report' generates a statistical analysis
+ of KVM events. Currently, vmexit, mmio and ioport events are supported.
+ 'perf kvm stat record <command>' records kvm events and the events between
+ start and end <command>.
+ And this command produces a file which contains tracing results of kvm
+ events.
+
+ 'perf kvm stat report' reports statistical data which includes events
+ handled time, samples, and so on.
+
OPTIONS
-------
-i::
@@ -68,7 +80,21 @@
--guestvmlinux=<path>::
Guest os kernel vmlinux.
+STAT REPORT OPTIONS
+-------------------
+--vcpu=<value>::
+ analyze events which occures on this vcpu. (default: all vcpus)
+
+--events=<value>::
+ events to be analyzed. Possible values: vmexit, mmio, ioport.
+ (default: vmexit)
+-k::
+--key=<value>::
+ Sorting key. Possible values: sample (default, sort by samples
+ number), time (sort by average time).
+
SEE ALSO
--------
linkperf:perf-top[1], linkperf:perf-record[1], linkperf:perf-report[1],
-linkperf:perf-diff[1], linkperf:perf-buildid-list[1]
+linkperf:perf-diff[1], linkperf:perf-buildid-list[1],
+linkperf:perf-stat[1]
diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt
index ddc2252..d1e39dc 100644
--- a/tools/perf/Documentation/perf-list.txt
+++ b/tools/perf/Documentation/perf-list.txt
@@ -15,24 +15,43 @@
This command displays the symbolic event types which can be selected in the
various perf commands with the -e option.
+[[EVENT_MODIFIERS]]
EVENT MODIFIERS
---------------
Events can optionally have a modifer by appending a colon and one or
-more modifiers. Modifiers allow the user to restrict when events are
-counted with 'u' for user-space, 'k' for kernel, 'h' for hypervisor.
-Additional modifiers are 'G' for guest counting (in KVM guests) and 'H'
-for host counting (not in KVM guests).
+more modifiers. Modifiers allow the user to restrict the events to be
+counted. The following modifiers exist:
+
+ u - user-space counting
+ k - kernel counting
+ h - hypervisor counting
+ G - guest counting (in KVM guests)
+ H - host counting (not in KVM guests)
+ p - precise level
The 'p' modifier can be used for specifying how precise the instruction
-address should be. The 'p' modifier is currently only implemented for
-Intel PEBS and can be specified multiple times:
- 0 - SAMPLE_IP can have arbitrary skid
- 1 - SAMPLE_IP must have constant skid
- 2 - SAMPLE_IP requested to have 0 skid
- 3 - SAMPLE_IP must have 0 skid
+address should be. The 'p' modifier can be specified multiple times:
-The PEBS implementation now supports up to 2.
+ 0 - SAMPLE_IP can have arbitrary skid
+ 1 - SAMPLE_IP must have constant skid
+ 2 - SAMPLE_IP requested to have 0 skid
+ 3 - SAMPLE_IP must have 0 skid
+
+For Intel systems precise event sampling is implemented with PEBS
+which supports up to precise-level 2.
+
+On AMD systems it is implemented using IBS (up to precise-level 2).
+The precise modifier works with event types 0x76 (cpu-cycles, CPU
+clocks not halted) and 0xC1 (micro-ops retired). Both events map to
+IBS execution sampling (IBS op) with the IBS Op Counter Control bit
+(IbsOpCntCtl) set respectively (see AMD64 Architecture Programmer’s
+Manual Volume 2: System Programming, 13.3 Instruction-Based
+Sampling). Examples to use IBS:
+
+ perf record -a -e cpu-cycles:p ... # use ibs op counting cycles
+ perf record -a -e r076:p ... # same as -e cpu-cycles:p
+ perf record -a -e r0C1:p ... # use ibs op counting micro-ops
RAW HARDWARE EVENT DESCRIPTOR
-----------------------------
@@ -44,6 +63,11 @@
of IA32_PERFEVTSELx MSRs) or AMD's PerfEvtSeln (see [AMD64 Architecture Programmer’s Manual Volume 2: System Programming], Page 344,
Figure 13-7 Performance Event-Select Register (PerfEvtSeln)).
+Note: Only the following bit fields can be set in x86 counter
+registers: event, umask, edge, inv, cmask. Esp. guest/host only and
+OS/user mode flags must be setup using <<EVENT_MODIFIERS, EVENT
+MODIFIERS>>.
+
Example:
If the Intel docs for a QM720 Core i7 describe an event as:
@@ -91,4 +115,4 @@
linkperf:perf-stat[1], linkperf:perf-top[1],
linkperf:perf-record[1],
http://www.intel.com/Assets/PDF/manual/253669.pdf[Intel® 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide],
-http://support.amd.com/us/Processor_TechDocs/24593.pdf[AMD64 Architecture Programmer’s Manual Volume 2: System Programming]
+http://support.amd.com/us/Processor_TechDocs/24593_APM_v2.pdf[AMD64 Architecture Programmer’s Manual Volume 2: System Programming]
diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt
index 495210a..f4d91be 100644
--- a/tools/perf/Documentation/perf-report.txt
+++ b/tools/perf/Documentation/perf-report.txt
@@ -168,6 +168,9 @@
branch stacks and it will automatically switch to the branch view mode,
unless --no-branch-stack is used.
+--objdump=<path>::
+ Path to objdump binary.
+
SEE ALSO
--------
linkperf:perf-stat[1], linkperf:perf-annotate[1]
diff --git a/tools/perf/Documentation/perf-script-perl.txt b/tools/perf/Documentation/perf-script-perl.txt
index 3152cca..d00bef2 100644
--- a/tools/perf/Documentation/perf-script-perl.txt
+++ b/tools/perf/Documentation/perf-script-perl.txt
@@ -116,8 +116,8 @@
descriptions below):
----
- use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/perf-script-Util/lib";
- use lib "./perf-script-Util/lib";
+ use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
+ use lib "./Perf-Trace-Util/lib";
use Perf::Trace::Core;
use Perf::Trace::Context;
use Perf::Trace::Util;
diff --git a/tools/perf/Documentation/perf-script-python.txt b/tools/perf/Documentation/perf-script-python.txt
index 4710220..a4027f2 100644
--- a/tools/perf/Documentation/perf-script-python.txt
+++ b/tools/perf/Documentation/perf-script-python.txt
@@ -129,7 +129,7 @@
import sys
sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/perf-script-Util/lib/Perf/Trace')
+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
from perf_trace_context import *
from Core import *
@@ -216,7 +216,7 @@
import sys
sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/perf-script-Util/lib/Perf/Trace')
+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
from perf_trace_context import *
from Core import *
@@ -279,7 +279,7 @@
import sys
sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/perf-script-Util/lib/Perf/Trace')
+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
from perf_trace_context import *
from Core import *
@@ -391,7 +391,7 @@
drwxr-xr-x 4 trz trz 4096 2010-01-26 22:29 ..
drwxr-xr-x 2 trz trz 4096 2010-01-26 22:29 bin
-rw-r--r-- 1 trz trz 2548 2010-01-26 22:29 check-perf-script.py
-drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 perf-script-Util
+drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 Perf-Trace-Util
-rw-r--r-- 1 trz trz 1462 2010-01-26 22:30 syscall-counts.py
----
@@ -518,7 +518,7 @@
import sys
sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/perf-script-Util/lib/Perf/Trace')
+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
from perf_trace_context import *
from Core import *
diff --git a/tools/perf/Documentation/perf-trace.txt b/tools/perf/Documentation/perf-trace.txt
new file mode 100644
index 0000000..3a2ae37
--- /dev/null
+++ b/tools/perf/Documentation/perf-trace.txt
@@ -0,0 +1,53 @@
+perf-trace(1)
+=============
+
+NAME
+----
+perf-trace - strace inspired tool
+
+SYNOPSIS
+--------
+[verse]
+'perf trace'
+
+DESCRIPTION
+-----------
+This command will show the events associated with the target, initially
+syscalls, but other system events like pagefaults, task lifetime events,
+scheduling events, etc.
+
+Initially this is a live mode only tool, but eventually will work with
+perf.data files like the other tools, allowing a detached 'record' from
+analysis phases.
+
+OPTIONS
+-------
+
+--all-cpus::
+ System-wide collection from all CPUs.
+
+-p::
+--pid=::
+ Record events on existing process ID (comma separated list).
+
+--tid=::
+ Record events on existing thread ID (comma separated list).
+
+--uid=::
+ Record events in threads owned by uid. Name or number.
+
+--no-inherit::
+ Child tasks do not inherit counters.
+
+--mmap-pages=::
+ Number of mmap data pages. Must be a power of two.
+
+--cpu::
+Collect samples only on the list of CPUs provided. Multiple CPUs can be provided as a
+comma-separated list with no space: 0,1. Ranges of CPUs are specified with -: 0-2.
+In per-thread mode with inheritance mode on (default), Events are captured only when
+the thread executes on the designated CPUs. Default is to monitor all CPUs.
+
+SEE ALSO
+--------
+linkperf:perf-record[1], linkperf:perf-script[1]
diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index b4b572e..80db3f4 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -10,8 +10,12 @@
lib/rbtree.c
include/linux/swab.h
arch/*/include/asm/unistd*.h
+arch/*/include/asm/perf_regs.h
arch/*/lib/memcpy*.S
arch/*/lib/memset*.S
include/linux/poison.h
include/linux/magic.h
include/linux/hw_breakpoint.h
+arch/x86/include/asm/svm.h
+arch/x86/include/asm/vmx.h
+arch/x86/include/asm/kvm_host.h
diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index 35655c3..e5e71e7 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -37,7 +37,14 @@
#
# Define NO_NEWT if you do not want TUI support.
#
+# Define NO_GTK2 if you do not want GTK+ GUI support.
+#
# Define NO_DEMANGLE if you do not want C++ symbol demangling.
+#
+# Define NO_LIBELF if you do not want libelf dependency (e.g. cross-builds)
+#
+# Define NO_LIBUNWIND if you do not want libunwind dependency for dwarf
+# backtrace post unwind.
$(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
@$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
@@ -50,16 +57,19 @@
-e s/s390x/s390/ -e s/parisc64/parisc/ \
-e s/ppc.*/powerpc/ -e s/mips.*/mips/ \
-e s/sh[234].*/sh/ )
+NO_PERF_REGS := 1
CC = $(CROSS_COMPILE)gcc
AR = $(CROSS_COMPILE)ar
# Additional ARCH settings for x86
ifeq ($(ARCH),i386)
- ARCH := x86
+ override ARCH := x86
+ NO_PERF_REGS := 0
+ LIBUNWIND_LIBS = -lunwind -lunwind-x86
endif
ifeq ($(ARCH),x86_64)
- ARCH := x86
+ override ARCH := x86
IS_X86_64 := 0
ifeq (, $(findstring m32,$(EXTRA_CFLAGS)))
IS_X86_64 := $(shell echo __x86_64__ | ${CC} -E -xc - | tail -n 1)
@@ -69,6 +79,8 @@
ARCH_CFLAGS := -DARCH_X86_64
ARCH_INCLUDE = ../../arch/x86/lib/memcpy_64.S ../../arch/x86/lib/memset_64.S
endif
+ NO_PERF_REGS := 0
+ LIBUNWIND_LIBS = -lunwind -lunwind-x86_64
endif
# Treat warnings as errors unless directed not to
@@ -89,7 +101,7 @@
PARSER_DEBUG_CFLAGS := -DPARSER_DEBUG
endif
-CFLAGS = -fno-omit-frame-pointer -ggdb3 -Wall -Wextra -std=gnu99 $(CFLAGS_WERROR) $(CFLAGS_OPTIMIZE) $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) $(PARSER_DEBUG_CFLAGS)
+CFLAGS = -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu99 $(CFLAGS_WERROR) $(CFLAGS_OPTIMIZE) $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) $(PARSER_DEBUG_CFLAGS)
EXTLIBS = -lpthread -lrt -lelf -lm
ALL_CFLAGS = $(CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
ALL_LDFLAGS = $(LDFLAGS)
@@ -186,10 +198,10 @@
TRACE_EVENT_DIR = ../lib/traceevent/
-ifeq ("$(origin O)", "command line")
- TE_PATH=$(OUTPUT)/
+ifneq ($(OUTPUT),)
+ TE_PATH=$(OUTPUT)
else
- TE_PATH=$(TRACE_EVENT_DIR)/
+ TE_PATH=$(TRACE_EVENT_DIR)
endif
LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
@@ -221,13 +233,13 @@
FLEX = flex
BISON= bison
-$(OUTPUT)util/parse-events-flex.c: util/parse-events.l
+$(OUTPUT)util/parse-events-flex.c: util/parse-events.l $(OUTPUT)util/parse-events-bison.c
$(QUIET_FLEX)$(FLEX) --header-file=$(OUTPUT)util/parse-events-flex.h $(PARSER_DEBUG_FLEX) -t util/parse-events.l > $(OUTPUT)util/parse-events-flex.c
$(OUTPUT)util/parse-events-bison.c: util/parse-events.y
$(QUIET_BISON)$(BISON) -v util/parse-events.y -d $(PARSER_DEBUG_BISON) -o $(OUTPUT)util/parse-events-bison.c
-$(OUTPUT)util/pmu-flex.c: util/pmu.l
+$(OUTPUT)util/pmu-flex.c: util/pmu.l $(OUTPUT)util/pmu-bison.c
$(QUIET_FLEX)$(FLEX) --header-file=$(OUTPUT)util/pmu-flex.h -t util/pmu.l > $(OUTPUT)util/pmu-flex.c
$(OUTPUT)util/pmu-bison.c: util/pmu.y
@@ -252,6 +264,7 @@
LIB_H += util/include/linux/kernel.h
LIB_H += util/include/linux/list.h
LIB_H += util/include/linux/export.h
+LIB_H += util/include/linux/magic.h
LIB_H += util/include/linux/poison.h
LIB_H += util/include/linux/prefetch.h
LIB_H += util/include/linux/rbtree.h
@@ -321,6 +334,10 @@
LIB_H += util/target.h
LIB_H += util/rblist.h
LIB_H += util/intlist.h
+LIB_H += util/perf_regs.h
+LIB_H += util/unwind.h
+LIB_H += ui/helpline.h
+LIB_H += util/vdso.h
LIB_OBJS += $(OUTPUT)util/abspath.o
LIB_OBJS += $(OUTPUT)util/alias.o
@@ -356,6 +373,7 @@
LIB_OBJS += $(OUTPUT)util/wrapper.o
LIB_OBJS += $(OUTPUT)util/sigchain.o
LIB_OBJS += $(OUTPUT)util/symbol.o
+LIB_OBJS += $(OUTPUT)util/symbol-elf.o
LIB_OBJS += $(OUTPUT)util/dso-test-data.o
LIB_OBJS += $(OUTPUT)util/color.o
LIB_OBJS += $(OUTPUT)util/pager.o
@@ -387,11 +405,15 @@
LIB_OBJS += $(OUTPUT)util/target.o
LIB_OBJS += $(OUTPUT)util/rblist.o
LIB_OBJS += $(OUTPUT)util/intlist.o
+LIB_OBJS += $(OUTPUT)util/vdso.o
+LIB_OBJS += $(OUTPUT)util/stat.o
+
+LIB_OBJS += $(OUTPUT)ui/helpline.o
+LIB_OBJS += $(OUTPUT)ui/hist.o
+LIB_OBJS += $(OUTPUT)ui/stdio/hist.o
BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o
-
BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
-
# Benchmark modules
BUILTIN_OBJS += $(OUTPUT)bench/sched-messaging.o
BUILTIN_OBJS += $(OUTPUT)bench/sched-pipe.o
@@ -449,13 +471,38 @@
-include config.mak.autogen
-include config.mak
-ifndef NO_DWARF
-FLAGS_DWARF=$(ALL_CFLAGS) -ldw -lelf $(ALL_LDFLAGS) $(EXTLIBS)
-ifneq ($(call try-cc,$(SOURCE_DWARF),$(FLAGS_DWARF)),y)
- msg := $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev);
+ifdef NO_LIBELF
NO_DWARF := 1
-endif # Dwarf support
-endif # NO_DWARF
+ NO_DEMANGLE := 1
+ NO_LIBUNWIND := 1
+else
+FLAGS_LIBELF=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS)
+ifneq ($(call try-cc,$(SOURCE_LIBELF),$(FLAGS_LIBELF)),y)
+ FLAGS_GLIBC=$(ALL_CFLAGS) $(ALL_LDFLAGS)
+ ifneq ($(call try-cc,$(SOURCE_GLIBC),$(FLAGS_GLIBC)),y)
+ msg := $(error No gnu/libc-version.h found, please install glibc-dev[el]/glibc-static);
+ else
+ NO_LIBELF := 1
+ NO_DWARF := 1
+ NO_DEMANGLE := 1
+ endif
+endif
+endif # NO_LIBELF
+
+ifndef NO_LIBUNWIND
+# for linking with debug library, run like:
+# make DEBUG=1 LIBUNWIND_DIR=/opt/libunwind/
+ifdef LIBUNWIND_DIR
+ LIBUNWIND_CFLAGS := -I$(LIBUNWIND_DIR)/include
+ LIBUNWIND_LDFLAGS := -L$(LIBUNWIND_DIR)/lib
+endif
+
+FLAGS_UNWIND=$(LIBUNWIND_CFLAGS) $(ALL_CFLAGS) $(LIBUNWIND_LDFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) $(LIBUNWIND_LIBS)
+ifneq ($(call try-cc,$(SOURCE_LIBUNWIND),$(FLAGS_UNWIND)),y)
+ msg := $(warning No libunwind found, disabling post unwind support. Please install libunwind-dev[el] >= 0.99);
+ NO_LIBUNWIND := 1
+endif # Libunwind support
+endif # NO_LIBUNWIND
-include arch/$(ARCH)/Makefile
@@ -463,20 +510,34 @@
BASIC_CFLAGS += -I$(OUTPUT)
endif
-FLAGS_LIBELF=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS)
-ifneq ($(call try-cc,$(SOURCE_LIBELF),$(FLAGS_LIBELF)),y)
- FLAGS_GLIBC=$(ALL_CFLAGS) $(ALL_LDFLAGS)
- ifneq ($(call try-cc,$(SOURCE_GLIBC),$(FLAGS_GLIBC)),y)
- msg := $(error No gnu/libc-version.h found, please install glibc-dev[el]/glibc-static);
- else
- msg := $(error No libelf.h/libelf found, please install libelf-dev/elfutils-libelf-devel);
- endif
-endif
+ifdef NO_LIBELF
+BASIC_CFLAGS += -DNO_LIBELF_SUPPORT
+
+EXTLIBS := $(filter-out -lelf,$(EXTLIBS))
+
+# Remove ELF/DWARF dependent codes
+LIB_OBJS := $(filter-out $(OUTPUT)util/symbol-elf.o,$(LIB_OBJS))
+LIB_OBJS := $(filter-out $(OUTPUT)util/dwarf-aux.o,$(LIB_OBJS))
+LIB_OBJS := $(filter-out $(OUTPUT)util/probe-event.o,$(LIB_OBJS))
+LIB_OBJS := $(filter-out $(OUTPUT)util/probe-finder.o,$(LIB_OBJS))
+
+BUILTIN_OBJS := $(filter-out $(OUTPUT)builtin-probe.o,$(BUILTIN_OBJS))
+
+# Use minimal symbol handling
+LIB_OBJS += $(OUTPUT)util/symbol-minimal.o
+
+else # NO_LIBELF
ifneq ($(call try-cc,$(SOURCE_ELF_MMAP),$(FLAGS_COMMON)),y)
BASIC_CFLAGS += -DLIBELF_NO_MMAP
endif
+FLAGS_DWARF=$(ALL_CFLAGS) -ldw -lelf $(ALL_LDFLAGS) $(EXTLIBS)
+ifneq ($(call try-cc,$(SOURCE_DWARF),$(FLAGS_DWARF)),y)
+ msg := $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev);
+ NO_DWARF := 1
+endif # Dwarf support
+
ifndef NO_DWARF
ifeq ($(origin PERF_HAVE_DWARF_REGS), undefined)
msg := $(warning DWARF register mappings have not been defined for architecture $(ARCH), DWARF support disabled);
@@ -487,6 +548,29 @@
LIB_OBJS += $(OUTPUT)util/dwarf-aux.o
endif # PERF_HAVE_DWARF_REGS
endif # NO_DWARF
+endif # NO_LIBELF
+
+ifdef NO_LIBUNWIND
+ BASIC_CFLAGS += -DNO_LIBUNWIND_SUPPORT
+else
+ EXTLIBS += $(LIBUNWIND_LIBS)
+ BASIC_CFLAGS := $(LIBUNWIND_CFLAGS) $(BASIC_CFLAGS)
+ BASIC_LDFLAGS := $(LIBUNWIND_LDFLAGS) $(BASIC_LDFLAGS)
+ LIB_OBJS += $(OUTPUT)util/unwind.o
+endif
+
+ifdef NO_LIBAUDIT
+ BASIC_CFLAGS += -DNO_LIBAUDIT_SUPPORT
+else
+ FLAGS_LIBAUDIT = $(ALL_CFLAGS) $(ALL_LDFLAGS) -laudit
+ ifneq ($(call try-cc,$(SOURCE_LIBAUDIT),$(FLAGS_LIBAUDIT)),y)
+ msg := $(warning No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev);
+ BASIC_CFLAGS += -DNO_LIBAUDIT_SUPPORT
+ else
+ BUILTIN_OBJS += $(OUTPUT)builtin-trace.o
+ EXTLIBS += -laudit
+ endif
+endif
ifdef NO_NEWT
BASIC_CFLAGS += -DNO_NEWT_SUPPORT
@@ -504,14 +588,13 @@
LIB_OBJS += $(OUTPUT)ui/browsers/annotate.o
LIB_OBJS += $(OUTPUT)ui/browsers/hists.o
LIB_OBJS += $(OUTPUT)ui/browsers/map.o
- LIB_OBJS += $(OUTPUT)ui/helpline.o
LIB_OBJS += $(OUTPUT)ui/progress.o
LIB_OBJS += $(OUTPUT)ui/util.o
LIB_OBJS += $(OUTPUT)ui/tui/setup.o
LIB_OBJS += $(OUTPUT)ui/tui/util.o
+ LIB_OBJS += $(OUTPUT)ui/tui/helpline.o
LIB_H += ui/browser.h
LIB_H += ui/browsers/map.h
- LIB_H += ui/helpline.h
LIB_H += ui/keysyms.h
LIB_H += ui/libslang.h
LIB_H += ui/progress.h
@@ -523,7 +606,7 @@
ifdef NO_GTK2
BASIC_CFLAGS += -DNO_GTK2_SUPPORT
else
- FLAGS_GTK2=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) $(shell pkg-config --libs --cflags gtk+-2.0)
+ FLAGS_GTK2=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) $(shell pkg-config --libs --cflags gtk+-2.0 2>/dev/null)
ifneq ($(call try-cc,$(SOURCE_GTK2),$(FLAGS_GTK2)),y)
msg := $(warning GTK2 not found, disables GTK2 support. Please install gtk2-devel or libgtk2.0-dev);
BASIC_CFLAGS += -DNO_GTK2_SUPPORT
@@ -531,11 +614,12 @@
ifeq ($(call try-cc,$(SOURCE_GTK2_INFOBAR),$(FLAGS_GTK2)),y)
BASIC_CFLAGS += -DHAVE_GTK_INFO_BAR
endif
- BASIC_CFLAGS += $(shell pkg-config --cflags gtk+-2.0)
- EXTLIBS += $(shell pkg-config --libs gtk+-2.0)
+ BASIC_CFLAGS += $(shell pkg-config --cflags gtk+-2.0 2>/dev/null)
+ EXTLIBS += $(shell pkg-config --libs gtk+-2.0 2>/dev/null)
LIB_OBJS += $(OUTPUT)ui/gtk/browser.o
LIB_OBJS += $(OUTPUT)ui/gtk/setup.o
LIB_OBJS += $(OUTPUT)ui/gtk/util.o
+ LIB_OBJS += $(OUTPUT)ui/gtk/helpline.o
# Make sure that it'd be included only once.
ifneq ($(findstring -DNO_NEWT_SUPPORT,$(BASIC_CFLAGS)),)
LIB_OBJS += $(OUTPUT)ui/setup.o
@@ -644,7 +728,7 @@
EXTLIBS += -liberty
BASIC_CFLAGS += -DHAVE_CPLUS_DEMANGLE
else
- FLAGS_BFD=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) -lbfd
+ FLAGS_BFD=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) -DPACKAGE='perf' -lbfd
has_bfd := $(call try-cc,$(SOURCE_BFD),$(FLAGS_BFD))
ifeq ($(has_bfd),y)
EXTLIBS += -lbfd
@@ -674,6 +758,13 @@
endif
endif
+ifeq ($(NO_PERF_REGS),0)
+ ifeq ($(ARCH),x86)
+ LIB_H += arch/x86/include/perf_regs.h
+ endif
+else
+ BASIC_CFLAGS += -DNO_PERF_REGS
+endif
ifdef NO_STRLCPY
BASIC_CFLAGS += -DNO_STRLCPY
@@ -683,6 +774,14 @@
endif
endif
+ifdef NO_BACKTRACE
+ BASIC_CFLAGS += -DNO_BACKTRACE
+else
+ ifneq ($(call try-cc,$(SOURCE_BACKTRACE),),y)
+ BASIC_CFLAGS += -DNO_BACKTRACE
+ endif
+endif
+
ifdef ASCIIDOC8
export ASCIIDOC8
endif
@@ -700,6 +799,7 @@
template_dir_SQ = $(subst ','\'',$(template_dir))
htmldir_SQ = $(subst ','\'',$(htmldir))
prefix_SQ = $(subst ','\'',$(prefix))
+sysconfdir_SQ = $(subst ','\'',$(sysconfdir))
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
@@ -767,10 +867,10 @@
# over the general rule for .o
$(OUTPUT)util/%-flex.o: $(OUTPUT)util/%-flex.c $(OUTPUT)PERF-CFLAGS
- $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) -Iutil/ -w $<
+ $(QUIET_CC)$(CC) -o $@ -c -Iutil/ $(ALL_CFLAGS) -w $<
$(OUTPUT)util/%-bison.o: $(OUTPUT)util/%-bison.c $(OUTPUT)PERF-CFLAGS
- $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -Iutil/ -w $<
+ $(QUIET_CC)$(CC) -o $@ -c -Iutil/ $(ALL_CFLAGS) -DYYENABLE_NLS=0 -DYYLTYPE_IS_TRIVIAL=0 -w $<
$(OUTPUT)%.o: %.c $(OUTPUT)PERF-CFLAGS
$(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
@@ -842,7 +942,10 @@
# libtraceevent.a
$(LIBTRACEEVENT):
- $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) $(COMMAND_O) libtraceevent.a
+ $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) libtraceevent.a
+
+$(LIBTRACEEVENT)-clean:
+ $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) clean
help:
@echo 'Perf make targets:'
@@ -951,6 +1054,8 @@
$(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'
$(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'
$(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d'
+ $(INSTALL) bash_completion '$(DESTDIR_SQ)$(sysconfdir_SQ)/bash_completion.d/perf'
install-python_ext:
$(PYTHON_WORD) util/setup.py --quiet install --root='/$(DESTDIR_SQ)'
@@ -981,7 +1086,7 @@
### Cleaning rules
-clean:
+clean: $(LIBTRACEEVENT)-clean
$(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS)
$(RM) $(ALL_PROGRAMS) perf
$(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope*
diff --git a/tools/perf/arch/x86/Makefile b/tools/perf/arch/x86/Makefile
index 744e629..815841c 100644
--- a/tools/perf/arch/x86/Makefile
+++ b/tools/perf/arch/x86/Makefile
@@ -2,4 +2,7 @@
PERF_HAVE_DWARF_REGS := 1
LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/dwarf-regs.o
endif
+ifndef NO_LIBUNWIND
+LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/unwind.o
+endif
LIB_OBJS += $(OUTPUT)arch/$(ARCH)/util/header.o
diff --git a/tools/perf/arch/x86/include/perf_regs.h b/tools/perf/arch/x86/include/perf_regs.h
new file mode 100644
index 0000000..46fc9f1
--- /dev/null
+++ b/tools/perf/arch/x86/include/perf_regs.h
@@ -0,0 +1,80 @@
+#ifndef ARCH_PERF_REGS_H
+#define ARCH_PERF_REGS_H
+
+#include <stdlib.h>
+#include "../../util/types.h"
+#include "../../../../../arch/x86/include/asm/perf_regs.h"
+
+#ifndef ARCH_X86_64
+#define PERF_REGS_MASK ((1ULL << PERF_REG_X86_32_MAX) - 1)
+#else
+#define REG_NOSUPPORT ((1ULL << PERF_REG_X86_DS) | \
+ (1ULL << PERF_REG_X86_ES) | \
+ (1ULL << PERF_REG_X86_FS) | \
+ (1ULL << PERF_REG_X86_GS))
+#define PERF_REGS_MASK (((1ULL << PERF_REG_X86_64_MAX) - 1) & ~REG_NOSUPPORT)
+#endif
+#define PERF_REG_IP PERF_REG_X86_IP
+#define PERF_REG_SP PERF_REG_X86_SP
+
+static inline const char *perf_reg_name(int id)
+{
+ switch (id) {
+ case PERF_REG_X86_AX:
+ return "AX";
+ case PERF_REG_X86_BX:
+ return "BX";
+ case PERF_REG_X86_CX:
+ return "CX";
+ case PERF_REG_X86_DX:
+ return "DX";
+ case PERF_REG_X86_SI:
+ return "SI";
+ case PERF_REG_X86_DI:
+ return "DI";
+ case PERF_REG_X86_BP:
+ return "BP";
+ case PERF_REG_X86_SP:
+ return "SP";
+ case PERF_REG_X86_IP:
+ return "IP";
+ case PERF_REG_X86_FLAGS:
+ return "FLAGS";
+ case PERF_REG_X86_CS:
+ return "CS";
+ case PERF_REG_X86_SS:
+ return "SS";
+ case PERF_REG_X86_DS:
+ return "DS";
+ case PERF_REG_X86_ES:
+ return "ES";
+ case PERF_REG_X86_FS:
+ return "FS";
+ case PERF_REG_X86_GS:
+ return "GS";
+#ifdef ARCH_X86_64
+ case PERF_REG_X86_R8:
+ return "R8";
+ case PERF_REG_X86_R9:
+ return "R9";
+ case PERF_REG_X86_R10:
+ return "R10";
+ case PERF_REG_X86_R11:
+ return "R11";
+ case PERF_REG_X86_R12:
+ return "R12";
+ case PERF_REG_X86_R13:
+ return "R13";
+ case PERF_REG_X86_R14:
+ return "R14";
+ case PERF_REG_X86_R15:
+ return "R15";
+#endif /* ARCH_X86_64 */
+ default:
+ return NULL;
+ }
+
+ return NULL;
+}
+
+#endif /* ARCH_PERF_REGS_H */
diff --git a/tools/perf/arch/x86/util/unwind.c b/tools/perf/arch/x86/util/unwind.c
new file mode 100644
index 0000000..78d956e
--- /dev/null
+++ b/tools/perf/arch/x86/util/unwind.c
@@ -0,0 +1,111 @@
+
+#include <errno.h>
+#include <libunwind.h>
+#include "perf_regs.h"
+#include "../../util/unwind.h"
+
+#ifdef ARCH_X86_64
+int unwind__arch_reg_id(int regnum)
+{
+ int id;
+
+ switch (regnum) {
+ case UNW_X86_64_RAX:
+ id = PERF_REG_X86_AX;
+ break;
+ case UNW_X86_64_RDX:
+ id = PERF_REG_X86_DX;
+ break;
+ case UNW_X86_64_RCX:
+ id = PERF_REG_X86_CX;
+ break;
+ case UNW_X86_64_RBX:
+ id = PERF_REG_X86_BX;
+ break;
+ case UNW_X86_64_RSI:
+ id = PERF_REG_X86_SI;
+ break;
+ case UNW_X86_64_RDI:
+ id = PERF_REG_X86_DI;
+ break;
+ case UNW_X86_64_RBP:
+ id = PERF_REG_X86_BP;
+ break;
+ case UNW_X86_64_RSP:
+ id = PERF_REG_X86_SP;
+ break;
+ case UNW_X86_64_R8:
+ id = PERF_REG_X86_R8;
+ break;
+ case UNW_X86_64_R9:
+ id = PERF_REG_X86_R9;
+ break;
+ case UNW_X86_64_R10:
+ id = PERF_REG_X86_R10;
+ break;
+ case UNW_X86_64_R11:
+ id = PERF_REG_X86_R11;
+ break;
+ case UNW_X86_64_R12:
+ id = PERF_REG_X86_R12;
+ break;
+ case UNW_X86_64_R13:
+ id = PERF_REG_X86_R13;
+ break;
+ case UNW_X86_64_R14:
+ id = PERF_REG_X86_R14;
+ break;
+ case UNW_X86_64_R15:
+ id = PERF_REG_X86_R15;
+ break;
+ case UNW_X86_64_RIP:
+ id = PERF_REG_X86_IP;
+ break;
+ default:
+ pr_err("unwind: invalid reg id %d\n", regnum);
+ return -EINVAL;
+ }
+
+ return id;
+}
+#else
+int unwind__arch_reg_id(int regnum)
+{
+ int id;
+
+ switch (regnum) {
+ case UNW_X86_EAX:
+ id = PERF_REG_X86_AX;
+ break;
+ case UNW_X86_EDX:
+ id = PERF_REG_X86_DX;
+ break;
+ case UNW_X86_ECX:
+ id = PERF_REG_X86_CX;
+ break;
+ case UNW_X86_EBX:
+ id = PERF_REG_X86_BX;
+ break;
+ case UNW_X86_ESI:
+ id = PERF_REG_X86_SI;
+ break;
+ case UNW_X86_EDI:
+ id = PERF_REG_X86_DI;
+ break;
+ case UNW_X86_EBP:
+ id = PERF_REG_X86_BP;
+ break;
+ case UNW_X86_ESP:
+ id = PERF_REG_X86_SP;
+ break;
+ case UNW_X86_EIP:
+ id = PERF_REG_X86_IP;
+ break;
+ default:
+ pr_err("unwind: invalid reg id %d\n", regnum);
+ return -EINVAL;
+ }
+
+ return id;
+}
+#endif /* ARCH_X86_64 */
diff --git a/tools/perf/bash_completion b/tools/perf/bash_completion
new file mode 100644
index 0000000..1958fa5
--- /dev/null
+++ b/tools/perf/bash_completion
@@ -0,0 +1,26 @@
+# perf completion
+
+have perf &&
+_perf()
+{
+ local cur cmd
+
+ COMPREPLY=()
+ _get_comp_words_by_ref cur prev
+
+ cmd=${COMP_WORDS[0]}
+
+ # List perf subcommands
+ if [ $COMP_CWORD -eq 1 ]; then
+ cmds=$($cmd --list-cmds)
+ COMPREPLY=( $( compgen -W '$cmds' -- "$cur" ) )
+ # List possible events for -e option
+ elif [[ $prev == "-e" && "${COMP_WORDS[1]}" == @(record|stat|top) ]]; then
+ cmds=$($cmd list --raw-dump)
+ COMPREPLY=( $( compgen -W '$cmds' -- "$cur" ) )
+ # Fall down to list regular files
+ else
+ _filedir
+ fi
+} &&
+complete -F _perf perf
diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
index a09bece..8f89998 100644
--- a/tools/perf/bench/bench.h
+++ b/tools/perf/bench/bench.h
@@ -3,7 +3,8 @@
extern int bench_sched_messaging(int argc, const char **argv, const char *prefix);
extern int bench_sched_pipe(int argc, const char **argv, const char *prefix);
-extern int bench_mem_memcpy(int argc, const char **argv, const char *prefix __used);
+extern int bench_mem_memcpy(int argc, const char **argv,
+ const char *prefix __maybe_unused);
extern int bench_mem_memset(int argc, const char **argv, const char *prefix);
#define BENCH_FORMAT_DEFAULT_STR "default"
diff --git a/tools/perf/bench/mem-memcpy.c b/tools/perf/bench/mem-memcpy.c
index 02dad5d..93c83e3 100644
--- a/tools/perf/bench/mem-memcpy.c
+++ b/tools/perf/bench/mem-memcpy.c
@@ -177,7 +177,7 @@
} while (0)
int bench_mem_memcpy(int argc, const char **argv,
- const char *prefix __used)
+ const char *prefix __maybe_unused)
{
int i;
size_t len;
diff --git a/tools/perf/bench/mem-memset.c b/tools/perf/bench/mem-memset.c
index 350cc95..c6e4bc5 100644
--- a/tools/perf/bench/mem-memset.c
+++ b/tools/perf/bench/mem-memset.c
@@ -171,7 +171,7 @@
} while (0)
int bench_mem_memset(int argc, const char **argv,
- const char *prefix __used)
+ const char *prefix __maybe_unused)
{
int i;
size_t len;
diff --git a/tools/perf/bench/sched-messaging.c b/tools/perf/bench/sched-messaging.c
index d1d1b30..cc1190a 100644
--- a/tools/perf/bench/sched-messaging.c
+++ b/tools/perf/bench/sched-messaging.c
@@ -267,7 +267,7 @@
};
int bench_sched_messaging(int argc, const char **argv,
- const char *prefix __used)
+ const char *prefix __maybe_unused)
{
unsigned int i, total_children;
struct timeval start, stop, diff;
diff --git a/tools/perf/bench/sched-pipe.c b/tools/perf/bench/sched-pipe.c
index 0c7454f..69cfba8 100644
--- a/tools/perf/bench/sched-pipe.c
+++ b/tools/perf/bench/sched-pipe.c
@@ -43,7 +43,7 @@
};
int bench_sched_pipe(int argc, const char **argv,
- const char *prefix __used)
+ const char *prefix __maybe_unused)
{
int pipe_1[2], pipe_2[2];
int m = 0, i;
@@ -55,14 +55,14 @@
* discarding returned value of read(), write()
* causes error in building environment for perf
*/
- int __used ret, wait_stat;
- pid_t pid, retpid;
+ int __maybe_unused ret, wait_stat;
+ pid_t pid, retpid __maybe_unused;
argc = parse_options(argc, argv, options,
bench_sched_pipe_usage, 0);
- assert(!pipe(pipe_1));
- assert(!pipe(pipe_2));
+ BUG_ON(pipe(pipe_1));
+ BUG_ON(pipe(pipe_2));
pid = fork();
assert(pid >= 0);
diff --git a/tools/perf/builtin-annotate.c b/tools/perf/builtin-annotate.c
index 67522cf..9ea3854 100644
--- a/tools/perf/builtin-annotate.c
+++ b/tools/perf/builtin-annotate.c
@@ -239,7 +239,7 @@
NULL
};
-int cmd_annotate(int argc, const char **argv, const char *prefix __used)
+int cmd_annotate(int argc, const char **argv, const char *prefix __maybe_unused)
{
struct perf_annotate annotate = {
.tool = {
@@ -282,6 +282,8 @@
"Display raw encoding of assembly instructions (default)"),
OPT_STRING('M', "disassembler-style", &disassembler_style, "disassembler style",
"Specify disassembler style (e.g. -M intel for intel syntax)"),
+ OPT_STRING(0, "objdump", &objdump_path, "path",
+ "objdump binary to use for disassembly and annotations"),
OPT_END()
};
diff --git a/tools/perf/builtin-bench.c b/tools/perf/builtin-bench.c
index 1f31002..cae9a5f 100644
--- a/tools/perf/builtin-bench.c
+++ b/tools/perf/builtin-bench.c
@@ -173,7 +173,7 @@
all_suite(&subsystems[i]);
}
-int cmd_bench(int argc, const char **argv, const char *prefix __used)
+int cmd_bench(int argc, const char **argv, const char *prefix __maybe_unused)
{
int i, j, status = 0;
diff --git a/tools/perf/builtin-buildid-cache.c b/tools/perf/builtin-buildid-cache.c
index 29ad20e..8365455 100644
--- a/tools/perf/builtin-buildid-cache.c
+++ b/tools/perf/builtin-buildid-cache.c
@@ -43,15 +43,16 @@
}
build_id__sprintf(build_id, sizeof(build_id), sbuild_id);
- err = build_id_cache__add_s(sbuild_id, debugdir, filename, false);
+ err = build_id_cache__add_s(sbuild_id, debugdir, filename,
+ false, false);
if (verbose)
pr_info("Adding %s %s: %s\n", sbuild_id, filename,
err ? "FAIL" : "Ok");
return err;
}
-static int build_id_cache__remove_file(const char *filename __used,
- const char *debugdir __used)
+static int build_id_cache__remove_file(const char *filename __maybe_unused,
+ const char *debugdir __maybe_unused)
{
u8 build_id[BUILD_ID_SIZE];
char sbuild_id[BUILD_ID_SIZE * 2 + 1];
@@ -119,7 +120,8 @@
return 0;
}
-int cmd_buildid_cache(int argc, const char **argv, const char *prefix __used)
+int cmd_buildid_cache(int argc, const char **argv,
+ const char *prefix __maybe_unused)
{
argc = parse_options(argc, argv, buildid_cache_options,
buildid_cache_usage, 0);
diff --git a/tools/perf/builtin-buildid-list.c b/tools/perf/builtin-buildid-list.c
index 6b2bcfb..1159fee 100644
--- a/tools/perf/builtin-buildid-list.c
+++ b/tools/perf/builtin-buildid-list.c
@@ -16,8 +16,6 @@
#include "util/session.h"
#include "util/symbol.h"
-#include <libelf.h>
-
static const char *input_name;
static bool force;
static bool show_kernel;
@@ -71,7 +69,7 @@
{
struct perf_session *session;
- elf_version(EV_CURRENT);
+ symbol__elf_init();
session = perf_session__new(input_name, O_RDONLY, force, false,
&build_id__mark_dso_hit_ops);
@@ -105,7 +103,8 @@
return perf_session__list_build_ids();
}
-int cmd_buildid_list(int argc, const char **argv, const char *prefix __used)
+int cmd_buildid_list(int argc, const char **argv,
+ const char *prefix __maybe_unused)
{
argc = parse_options(argc, argv, options, buildid_list_usage, 0);
setup_pager();
diff --git a/tools/perf/builtin-diff.c b/tools/perf/builtin-diff.c
index d29d350..761f419 100644
--- a/tools/perf/builtin-diff.c
+++ b/tools/perf/builtin-diff.c
@@ -10,6 +10,7 @@
#include "util/event.h"
#include "util/hist.h"
#include "util/evsel.h"
+#include "util/evlist.h"
#include "util/session.h"
#include "util/tool.h"
#include "util/sort.h"
@@ -24,11 +25,6 @@
static bool force;
static bool show_displacement;
-struct perf_diff {
- struct perf_tool tool;
- struct perf_session *session;
-};
-
static int hists__add_entry(struct hists *self,
struct addr_location *al, u64 period)
{
@@ -37,14 +33,12 @@
return -ENOMEM;
}
-static int diff__process_sample_event(struct perf_tool *tool,
+static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel __used,
+ struct perf_evsel *evsel,
struct machine *machine)
{
- struct perf_diff *_diff = container_of(tool, struct perf_diff, tool);
- struct perf_session *session = _diff->session;
struct addr_location al;
if (perf_event__preprocess_sample(event, machine, &al, sample, NULL) < 0) {
@@ -56,26 +50,24 @@
if (al.filtered || al.sym == NULL)
return 0;
- if (hists__add_entry(&session->hists, &al, sample->period)) {
+ if (hists__add_entry(&evsel->hists, &al, sample->period)) {
pr_warning("problem incrementing symbol period, skipping event\n");
return -1;
}
- session->hists.stats.total_period += sample->period;
+ evsel->hists.stats.total_period += sample->period;
return 0;
}
-static struct perf_diff diff = {
- .tool = {
- .sample = diff__process_sample_event,
- .mmap = perf_event__process_mmap,
- .comm = perf_event__process_comm,
- .exit = perf_event__process_task,
- .fork = perf_event__process_task,
- .lost = perf_event__process_lost,
- .ordered_samples = true,
- .ordering_requires_timestamps = true,
- },
+static struct perf_tool tool = {
+ .sample = diff__process_sample_event,
+ .mmap = perf_event__process_mmap,
+ .comm = perf_event__process_comm,
+ .exit = perf_event__process_task,
+ .fork = perf_event__process_task,
+ .lost = perf_event__process_lost,
+ .ordered_samples = true,
+ .ordering_requires_timestamps = true,
};
static void perf_session__insert_hist_entry_by_name(struct rb_root *root,
@@ -146,34 +138,71 @@
}
}
+static struct perf_evsel *evsel_match(struct perf_evsel *evsel,
+ struct perf_evlist *evlist)
+{
+ struct perf_evsel *e;
+
+ list_for_each_entry(e, &evlist->entries, node)
+ if (perf_evsel__match2(evsel, e))
+ return e;
+
+ return NULL;
+}
+
static int __cmd_diff(void)
{
int ret, i;
#define older (session[0])
#define newer (session[1])
struct perf_session *session[2];
+ struct perf_evlist *evlist_new, *evlist_old;
+ struct perf_evsel *evsel;
+ bool first = true;
older = perf_session__new(input_old, O_RDONLY, force, false,
- &diff.tool);
+ &tool);
newer = perf_session__new(input_new, O_RDONLY, force, false,
- &diff.tool);
+ &tool);
if (session[0] == NULL || session[1] == NULL)
return -ENOMEM;
for (i = 0; i < 2; ++i) {
- diff.session = session[i];
- ret = perf_session__process_events(session[i], &diff.tool);
+ ret = perf_session__process_events(session[i], &tool);
if (ret)
goto out_delete;
- hists__output_resort(&session[i]->hists);
}
- if (show_displacement)
- hists__resort_entries(&older->hists);
+ evlist_old = older->evlist;
+ evlist_new = newer->evlist;
- hists__match(&older->hists, &newer->hists);
- hists__fprintf(&newer->hists, &older->hists,
- show_displacement, true, 0, 0, stdout);
+ list_for_each_entry(evsel, &evlist_new->entries, node)
+ hists__output_resort(&evsel->hists);
+
+ list_for_each_entry(evsel, &evlist_old->entries, node) {
+ hists__output_resort(&evsel->hists);
+
+ if (show_displacement)
+ hists__resort_entries(&evsel->hists);
+ }
+
+ list_for_each_entry(evsel, &evlist_new->entries, node) {
+ struct perf_evsel *evsel_old;
+
+ evsel_old = evsel_match(evsel, evlist_old);
+ if (!evsel_old)
+ continue;
+
+ fprintf(stdout, "%s# Event '%s'\n#\n", first ? "" : "\n",
+ perf_evsel__name(evsel));
+
+ first = false;
+
+ hists__match(&evsel_old->hists, &evsel->hists);
+ hists__fprintf(&evsel->hists, &evsel_old->hists,
+ show_displacement, true, 0, 0, stdout);
+ }
+
out_delete:
for (i = 0; i < 2; ++i)
perf_session__delete(session[i]);
@@ -213,7 +242,7 @@
OPT_END()
};
-int cmd_diff(int argc, const char **argv, const char *prefix __used)
+int cmd_diff(int argc, const char **argv, const char *prefix __maybe_unused)
{
sort_order = diff__default_sort_order;
argc = parse_options(argc, argv, options, diff_usage, 0);
@@ -235,6 +264,7 @@
if (symbol__init() < 0)
return -1;
+ perf_hpp__init(true, show_displacement);
setup_sorting(diff_usage, options);
setup_pager();
diff --git a/tools/perf/builtin-evlist.c b/tools/perf/builtin-evlist.c
index 0dd5a05..1fb1641 100644
--- a/tools/perf/builtin-evlist.c
+++ b/tools/perf/builtin-evlist.c
@@ -113,7 +113,7 @@
NULL
};
-int cmd_evlist(int argc, const char **argv, const char *prefix __used)
+int cmd_evlist(int argc, const char **argv, const char *prefix __maybe_unused)
{
struct perf_attr_details details = { .verbose = false, };
const char *input_name = NULL;
diff --git a/tools/perf/builtin-help.c b/tools/perf/builtin-help.c
index 6d5a8a7..25c8b94 100644
--- a/tools/perf/builtin-help.c
+++ b/tools/perf/builtin-help.c
@@ -24,13 +24,14 @@
} *man_viewer_info_list;
enum help_format {
+ HELP_FORMAT_NONE,
HELP_FORMAT_MAN,
HELP_FORMAT_INFO,
HELP_FORMAT_WEB,
};
static bool show_all = false;
-static enum help_format help_format = HELP_FORMAT_MAN;
+static enum help_format help_format = HELP_FORMAT_NONE;
static struct option builtin_help_options[] = {
OPT_BOOLEAN('a', "all", &show_all, "print all available commands"),
OPT_SET_UINT('m', "man", &help_format, "show man page", HELP_FORMAT_MAN),
@@ -54,7 +55,9 @@
return HELP_FORMAT_INFO;
if (!strcmp(format, "web") || !strcmp(format, "html"))
return HELP_FORMAT_WEB;
- die("unrecognized help format '%s'", format);
+
+ pr_err("unrecognized help format '%s'", format);
+ return HELP_FORMAT_NONE;
}
static const char *get_man_viewer_info(const char *name)
@@ -259,6 +262,8 @@
if (!value)
return config_error_nonbool(var);
help_format = parse_help_format(value);
+ if (help_format == HELP_FORMAT_NONE)
+ return -1;
return 0;
}
if (!strcmp(var, "man.viewer")) {
@@ -352,7 +357,7 @@
warning("'%s': unknown man viewer.", name);
}
-static void show_man_page(const char *perf_cmd)
+static int show_man_page(const char *perf_cmd)
{
struct man_viewer_list *viewer;
const char *page = cmd_to_page(perf_cmd);
@@ -365,28 +370,35 @@
if (fallback)
exec_viewer(fallback, page);
exec_viewer("man", page);
- die("no man viewer handled the request");
+
+ pr_err("no man viewer handled the request");
+ return -1;
}
-static void show_info_page(const char *perf_cmd)
+static int show_info_page(const char *perf_cmd)
{
const char *page = cmd_to_page(perf_cmd);
setenv("INFOPATH", system_path(PERF_INFO_PATH), 1);
execlp("info", "info", "perfman", page, NULL);
+ return -1;
}
-static void get_html_page_path(struct strbuf *page_path, const char *page)
+static int get_html_page_path(struct strbuf *page_path, const char *page)
{
struct stat st;
const char *html_path = system_path(PERF_HTML_PATH);
/* Check that we have a perf documentation directory. */
if (stat(mkpath("%s/perf.html", html_path), &st)
- || !S_ISREG(st.st_mode))
- die("'%s': not a documentation directory.", html_path);
+ || !S_ISREG(st.st_mode)) {
+ pr_err("'%s': not a documentation directory.", html_path);
+ return -1;
+ }
strbuf_init(page_path, 0);
strbuf_addf(page_path, "%s/%s.html", html_path, page);
+
+ return 0;
}
/*
@@ -401,19 +413,23 @@
}
#endif
-static void show_html_page(const char *perf_cmd)
+static int show_html_page(const char *perf_cmd)
{
const char *page = cmd_to_page(perf_cmd);
struct strbuf page_path; /* it leaks but we exec bellow */
- get_html_page_path(&page_path, page);
+ if (get_html_page_path(&page_path, page) != 0)
+ return -1;
open_html(page_path.buf);
+
+ return 0;
}
-int cmd_help(int argc, const char **argv, const char *prefix __used)
+int cmd_help(int argc, const char **argv, const char *prefix __maybe_unused)
{
const char *alias;
+ int rc = 0;
load_command_list("perf-", &main_cmds, &other_cmds);
@@ -444,16 +460,20 @@
switch (help_format) {
case HELP_FORMAT_MAN:
- show_man_page(argv[0]);
+ rc = show_man_page(argv[0]);
break;
case HELP_FORMAT_INFO:
- show_info_page(argv[0]);
+ rc = show_info_page(argv[0]);
break;
case HELP_FORMAT_WEB:
- show_html_page(argv[0]);
+ rc = show_html_page(argv[0]);
+ break;
+ case HELP_FORMAT_NONE:
+ /* fall-through */
default:
+ rc = -1;
break;
}
- return 0;
+ return rc;
}
diff --git a/tools/perf/builtin-inject.c b/tools/perf/builtin-inject.c
index 3beab48..1eaa661 100644
--- a/tools/perf/builtin-inject.c
+++ b/tools/perf/builtin-inject.c
@@ -17,9 +17,9 @@
static char const *input_name = "-";
static bool inject_build_ids;
-static int perf_event__repipe_synth(struct perf_tool *tool __used,
+static int perf_event__repipe_synth(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct machine *machine __used)
+ struct machine *machine __maybe_unused)
{
uint32_t size;
void *buf = event;
@@ -40,7 +40,8 @@
static int perf_event__repipe_op2_synth(struct perf_tool *tool,
union perf_event *event,
- struct perf_session *session __used)
+ struct perf_session *session
+ __maybe_unused)
{
return perf_event__repipe_synth(tool, event, NULL);
}
@@ -52,13 +53,14 @@
}
static int perf_event__repipe_tracing_data_synth(union perf_event *event,
- struct perf_session *session __used)
+ struct perf_session *session
+ __maybe_unused)
{
return perf_event__repipe_synth(NULL, event, NULL);
}
static int perf_event__repipe_attr(union perf_event *event,
- struct perf_evlist **pevlist __used)
+ struct perf_evlist **pevlist __maybe_unused)
{
int ret;
ret = perf_event__process_attr(event, pevlist);
@@ -70,7 +72,7 @@
static int perf_event__repipe(struct perf_tool *tool,
union perf_event *event,
- struct perf_sample *sample __used,
+ struct perf_sample *sample __maybe_unused,
struct machine *machine)
{
return perf_event__repipe_synth(tool, event, machine);
@@ -78,8 +80,8 @@
static int perf_event__repipe_sample(struct perf_tool *tool,
union perf_event *event,
- struct perf_sample *sample __used,
- struct perf_evsel *evsel __used,
+ struct perf_sample *sample __maybe_unused,
+ struct perf_evsel *evsel __maybe_unused,
struct machine *machine)
{
return perf_event__repipe_synth(tool, event, machine);
@@ -163,7 +165,7 @@
static int perf_event__inject_buildid(struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel __used,
+ struct perf_evsel *evsel __maybe_unused,
struct machine *machine)
{
struct addr_location al;
@@ -191,10 +193,13 @@
* If this fails, too bad, let the other side
* account this as unresolved.
*/
- } else
+ } else {
+#ifndef NO_LIBELF_SUPPORT
pr_warning("no symbols found in %s, maybe "
"install a debug package?\n",
al.map->dso->long_name);
+#endif
+ }
}
}
@@ -221,7 +226,7 @@
extern volatile int session_done;
-static void sig_handler(int sig __attribute__((__unused__)))
+static void sig_handler(int sig __maybe_unused)
{
session_done = 1;
}
@@ -264,7 +269,7 @@
OPT_END()
};
-int cmd_inject(int argc, const char **argv, const char *prefix __used)
+int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
{
argc = parse_options(argc, argv, options, report_usage, 0);
diff --git a/tools/perf/builtin-kmem.c b/tools/perf/builtin-kmem.c
index ce35015..bc912c6 100644
--- a/tools/perf/builtin-kmem.c
+++ b/tools/perf/builtin-kmem.c
@@ -1,6 +1,8 @@
#include "builtin.h"
#include "perf.h"
+#include "util/evlist.h"
+#include "util/evsel.h"
#include "util/util.h"
#include "util/cache.h"
#include "util/symbol.h"
@@ -57,46 +59,52 @@
#define PATH_SYS_NODE "/sys/devices/system/node"
-struct perf_kmem {
- struct perf_tool tool;
- struct perf_session *session;
-};
-
-static void init_cpunode_map(void)
+static int init_cpunode_map(void)
{
FILE *fp;
- int i;
+ int i, err = -1;
fp = fopen("/sys/devices/system/cpu/kernel_max", "r");
if (!fp) {
max_cpu_num = 4096;
- return;
+ return 0;
}
- if (fscanf(fp, "%d", &max_cpu_num) < 1)
- die("Failed to read 'kernel_max' from sysfs");
+ if (fscanf(fp, "%d", &max_cpu_num) < 1) {
+ pr_err("Failed to read 'kernel_max' from sysfs");
+ goto out_close;
+ }
+
max_cpu_num++;
cpunode_map = calloc(max_cpu_num, sizeof(int));
- if (!cpunode_map)
- die("calloc");
+ if (!cpunode_map) {
+ pr_err("%s: calloc failed\n", __func__);
+ goto out_close;
+ }
+
for (i = 0; i < max_cpu_num; i++)
cpunode_map[i] = -1;
+
+ err = 0;
+out_close:
fclose(fp);
+ return err;
}
-static void setup_cpunode_map(void)
+static int setup_cpunode_map(void)
{
struct dirent *dent1, *dent2;
DIR *dir1, *dir2;
unsigned int cpu, mem;
char buf[PATH_MAX];
- init_cpunode_map();
+ if (init_cpunode_map())
+ return -1;
dir1 = opendir(PATH_SYS_NODE);
if (!dir1)
- return;
+ return -1;
while ((dent1 = readdir(dir1)) != NULL) {
if (dent1->d_type != DT_DIR ||
@@ -116,10 +124,11 @@
closedir(dir2);
}
closedir(dir1);
+ return 0;
}
-static void insert_alloc_stat(unsigned long call_site, unsigned long ptr,
- int bytes_req, int bytes_alloc, int cpu)
+static int insert_alloc_stat(unsigned long call_site, unsigned long ptr,
+ int bytes_req, int bytes_alloc, int cpu)
{
struct rb_node **node = &root_alloc_stat.rb_node;
struct rb_node *parent = NULL;
@@ -143,8 +152,10 @@
data->bytes_alloc += bytes_alloc;
} else {
data = malloc(sizeof(*data));
- if (!data)
- die("malloc");
+ if (!data) {
+ pr_err("%s: malloc failed\n", __func__);
+ return -1;
+ }
data->ptr = ptr;
data->pingpong = 0;
data->hit = 1;
@@ -156,9 +167,10 @@
}
data->call_site = call_site;
data->alloc_cpu = cpu;
+ return 0;
}
-static void insert_caller_stat(unsigned long call_site,
+static int insert_caller_stat(unsigned long call_site,
int bytes_req, int bytes_alloc)
{
struct rb_node **node = &root_caller_stat.rb_node;
@@ -183,8 +195,10 @@
data->bytes_alloc += bytes_alloc;
} else {
data = malloc(sizeof(*data));
- if (!data)
- die("malloc");
+ if (!data) {
+ pr_err("%s: malloc failed\n", __func__);
+ return -1;
+ }
data->call_site = call_site;
data->pingpong = 0;
data->hit = 1;
@@ -194,39 +208,43 @@
rb_link_node(&data->node, parent, node);
rb_insert_color(&data->node, &root_caller_stat);
}
+
+ return 0;
}
-static void process_alloc_event(void *data,
- struct event_format *event,
- int cpu,
- u64 timestamp __used,
- struct thread *thread __used,
- int node)
+static int perf_evsel__process_alloc_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- unsigned long call_site;
- unsigned long ptr;
- int bytes_req;
- int bytes_alloc;
- int node1, node2;
+ unsigned long ptr = perf_evsel__intval(evsel, sample, "ptr"),
+ call_site = perf_evsel__intval(evsel, sample, "call_site");
+ int bytes_req = perf_evsel__intval(evsel, sample, "bytes_req"),
+ bytes_alloc = perf_evsel__intval(evsel, sample, "bytes_alloc");
- ptr = raw_field_value(event, "ptr", data);
- call_site = raw_field_value(event, "call_site", data);
- bytes_req = raw_field_value(event, "bytes_req", data);
- bytes_alloc = raw_field_value(event, "bytes_alloc", data);
-
- insert_alloc_stat(call_site, ptr, bytes_req, bytes_alloc, cpu);
- insert_caller_stat(call_site, bytes_req, bytes_alloc);
+ if (insert_alloc_stat(call_site, ptr, bytes_req, bytes_alloc, sample->cpu) ||
+ insert_caller_stat(call_site, bytes_req, bytes_alloc))
+ return -1;
total_requested += bytes_req;
total_allocated += bytes_alloc;
- if (node) {
- node1 = cpunode_map[cpu];
- node2 = raw_field_value(event, "node", data);
+ nr_allocs++;
+ return 0;
+}
+
+static int perf_evsel__process_alloc_node_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
+{
+ int ret = perf_evsel__process_alloc_event(evsel, sample);
+
+ if (!ret) {
+ int node1 = cpunode_map[sample->cpu],
+ node2 = perf_evsel__intval(evsel, sample, "node");
+
if (node1 != node2)
nr_cross_allocs++;
}
- nr_allocs++;
+
+ return ret;
}
static int ptr_cmp(struct alloc_stat *, struct alloc_stat *);
@@ -257,66 +275,37 @@
return NULL;
}
-static void process_free_event(void *data,
- struct event_format *event,
- int cpu,
- u64 timestamp __used,
- struct thread *thread __used)
+static int perf_evsel__process_free_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- unsigned long ptr;
+ unsigned long ptr = perf_evsel__intval(evsel, sample, "ptr");
struct alloc_stat *s_alloc, *s_caller;
- ptr = raw_field_value(event, "ptr", data);
-
s_alloc = search_alloc_stat(ptr, 0, &root_alloc_stat, ptr_cmp);
if (!s_alloc)
- return;
+ return 0;
- if (cpu != s_alloc->alloc_cpu) {
+ if ((short)sample->cpu != s_alloc->alloc_cpu) {
s_alloc->pingpong++;
s_caller = search_alloc_stat(0, s_alloc->call_site,
&root_caller_stat, callsite_cmp);
- assert(s_caller);
+ if (!s_caller)
+ return -1;
s_caller->pingpong++;
}
s_alloc->alloc_cpu = -1;
+
+ return 0;
}
-static void process_raw_event(struct perf_tool *tool,
- union perf_event *raw_event __used, void *data,
- int cpu, u64 timestamp, struct thread *thread)
-{
- struct perf_kmem *kmem = container_of(tool, struct perf_kmem, tool);
- struct event_format *event;
- int type;
+typedef int (*tracepoint_handler)(struct perf_evsel *evsel,
+ struct perf_sample *sample);
- type = trace_parse_common_type(kmem->session->pevent, data);
- event = pevent_find_event(kmem->session->pevent, type);
-
- if (!strcmp(event->name, "kmalloc") ||
- !strcmp(event->name, "kmem_cache_alloc")) {
- process_alloc_event(data, event, cpu, timestamp, thread, 0);
- return;
- }
-
- if (!strcmp(event->name, "kmalloc_node") ||
- !strcmp(event->name, "kmem_cache_alloc_node")) {
- process_alloc_event(data, event, cpu, timestamp, thread, 1);
- return;
- }
-
- if (!strcmp(event->name, "kfree") ||
- !strcmp(event->name, "kmem_cache_free")) {
- process_free_event(data, event, cpu, timestamp, thread);
- return;
- }
-}
-
-static int process_sample_event(struct perf_tool *tool,
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel __used,
+ struct perf_evsel *evsel,
struct machine *machine)
{
struct thread *thread = machine__findnew_thread(machine, event->ip.pid);
@@ -329,18 +318,18 @@
dump_printf(" ... thread: %s:%d\n", thread->comm, thread->pid);
- process_raw_event(tool, event, sample->raw_data, sample->cpu,
- sample->time, thread);
+ if (evsel->handler.func != NULL) {
+ tracepoint_handler f = evsel->handler.func;
+ return f(evsel, sample);
+ }
return 0;
}
-static struct perf_kmem perf_kmem = {
- .tool = {
- .sample = process_sample_event,
- .comm = perf_event__process_comm,
- .ordered_samples = true,
- },
+static struct perf_tool perf_kmem = {
+ .sample = process_sample_event,
+ .comm = perf_event__process_comm,
+ .ordered_samples = true,
};
static double fragmentation(unsigned long n_req, unsigned long n_alloc)
@@ -496,22 +485,32 @@
{
int err = -EINVAL;
struct perf_session *session;
+ const struct perf_evsel_str_handler kmem_tracepoints[] = {
+ { "kmem:kmalloc", perf_evsel__process_alloc_event, },
+ { "kmem:kmem_cache_alloc", perf_evsel__process_alloc_event, },
+ { "kmem:kmalloc_node", perf_evsel__process_alloc_node_event, },
+ { "kmem:kmem_cache_alloc_node", perf_evsel__process_alloc_node_event, },
+ { "kmem:kfree", perf_evsel__process_free_event, },
+ { "kmem:kmem_cache_free", perf_evsel__process_free_event, },
+ };
- session = perf_session__new(input_name, O_RDONLY, 0, false,
- &perf_kmem.tool);
+ session = perf_session__new(input_name, O_RDONLY, 0, false, &perf_kmem);
if (session == NULL)
return -ENOMEM;
- perf_kmem.session = session;
-
if (perf_session__create_kernel_maps(session) < 0)
goto out_delete;
if (!perf_session__has_traces(session, "kmem record"))
goto out_delete;
+ if (perf_session__set_tracepoints_handlers(session, kmem_tracepoints)) {
+ pr_err("Initializing perf session tracepoint handlers failed\n");
+ return -1;
+ }
+
setup_pager();
- err = perf_session__process_events(session, &perf_kmem.tool);
+ err = perf_session__process_events(session, &perf_kmem);
if (err != 0)
goto out_delete;
sort_result();
@@ -635,8 +634,10 @@
for (i = 0; i < NUM_AVAIL_SORTS; i++) {
if (!strcmp(avail_sorts[i]->name, tok)) {
sort = malloc(sizeof(*sort));
- if (!sort)
- die("malloc");
+ if (!sort) {
+ pr_err("%s: malloc failed\n", __func__);
+ return -1;
+ }
memcpy(sort, avail_sorts[i], sizeof(*sort));
list_add_tail(&sort->list, list);
return 0;
@@ -651,8 +652,10 @@
char *tok;
char *str = strdup(arg);
- if (!str)
- die("strdup");
+ if (!str) {
+ pr_err("%s: strdup failed\n", __func__);
+ return -1;
+ }
while (true) {
tok = strsep(&str, ",");
@@ -669,8 +672,8 @@
return 0;
}
-static int parse_sort_opt(const struct option *opt __used,
- const char *arg, int unset __used)
+static int parse_sort_opt(const struct option *opt __maybe_unused,
+ const char *arg, int unset __maybe_unused)
{
if (!arg)
return -1;
@@ -683,22 +686,24 @@
return 0;
}
-static int parse_caller_opt(const struct option *opt __used,
- const char *arg __used, int unset __used)
+static int parse_caller_opt(const struct option *opt __maybe_unused,
+ const char *arg __maybe_unused,
+ int unset __maybe_unused)
{
caller_flag = (alloc_flag + 1);
return 0;
}
-static int parse_alloc_opt(const struct option *opt __used,
- const char *arg __used, int unset __used)
+static int parse_alloc_opt(const struct option *opt __maybe_unused,
+ const char *arg __maybe_unused,
+ int unset __maybe_unused)
{
alloc_flag = (caller_flag + 1);
return 0;
}
-static int parse_line_opt(const struct option *opt __used,
- const char *arg, int unset __used)
+static int parse_line_opt(const struct option *opt __maybe_unused,
+ const char *arg, int unset __maybe_unused)
{
int lines;
@@ -768,7 +773,7 @@
return cmd_record(i, rec_argv, NULL);
}
-int cmd_kmem(int argc, const char **argv, const char *prefix __used)
+int cmd_kmem(int argc, const char **argv, const char *prefix __maybe_unused)
{
argc = parse_options(argc, argv, kmem_options, kmem_usage, 0);
@@ -780,7 +785,8 @@
if (!strncmp(argv[0], "rec", 3)) {
return __cmd_record(argc, argv);
} else if (!strcmp(argv[0], "stat")) {
- setup_cpunode_map();
+ if (setup_cpunode_map())
+ return -1;
if (list_empty(&caller_sort))
setup_sorting(&caller_sort, default_sort_order);
diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
index 9fc6e0f..a28c9ca 100644
--- a/tools/perf/builtin-kvm.c
+++ b/tools/perf/builtin-kvm.c
@@ -1,6 +1,7 @@
#include "builtin.h"
#include "perf.h"
+#include "util/evsel.h"
#include "util/util.h"
#include "util/cache.h"
#include "util/symbol.h"
@@ -10,8 +11,10 @@
#include "util/parse-options.h"
#include "util/trace-event.h"
-
#include "util/debug.h"
+#include "util/debugfs.h"
+#include "util/tool.h"
+#include "util/stat.h"
#include <sys/prctl.h>
@@ -19,11 +22,836 @@
#include <pthread.h>
#include <math.h>
-static const char *file_name;
+#include "../../arch/x86/include/asm/svm.h"
+#include "../../arch/x86/include/asm/vmx.h"
+#include "../../arch/x86/include/asm/kvm.h"
+
+struct event_key {
+ #define INVALID_KEY (~0ULL)
+ u64 key;
+ int info;
+};
+
+struct kvm_events_ops {
+ bool (*is_begin_event)(struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct event_key *key);
+ bool (*is_end_event)(struct perf_evsel *evsel,
+ struct perf_sample *sample, struct event_key *key);
+ void (*decode_key)(struct event_key *key, char decode[20]);
+ const char *name;
+};
+
+static void exit_event_get_key(struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct event_key *key)
+{
+ key->info = 0;
+ key->key = perf_evsel__intval(evsel, sample, "exit_reason");
+}
+
+static bool kvm_exit_event(struct perf_evsel *evsel)
+{
+ return !strcmp(evsel->name, "kvm:kvm_exit");
+}
+
+static bool exit_event_begin(struct perf_evsel *evsel,
+ struct perf_sample *sample, struct event_key *key)
+{
+ if (kvm_exit_event(evsel)) {
+ exit_event_get_key(evsel, sample, key);
+ return true;
+ }
+
+ return false;
+}
+
+static bool kvm_entry_event(struct perf_evsel *evsel)
+{
+ return !strcmp(evsel->name, "kvm:kvm_entry");
+}
+
+static bool exit_event_end(struct perf_evsel *evsel,
+ struct perf_sample *sample __maybe_unused,
+ struct event_key *key __maybe_unused)
+{
+ return kvm_entry_event(evsel);
+}
+
+struct exit_reasons_table {
+ unsigned long exit_code;
+ const char *reason;
+};
+
+struct exit_reasons_table vmx_exit_reasons[] = {
+ VMX_EXIT_REASONS
+};
+
+struct exit_reasons_table svm_exit_reasons[] = {
+ SVM_EXIT_REASONS
+};
+
+static int cpu_isa;
+
+static const char *get_exit_reason(u64 exit_code)
+{
+ int table_size = ARRAY_SIZE(svm_exit_reasons);
+ struct exit_reasons_table *table = svm_exit_reasons;
+
+ if (cpu_isa == 1) {
+ table = vmx_exit_reasons;
+ table_size = ARRAY_SIZE(vmx_exit_reasons);
+ }
+
+ while (table_size--) {
+ if (table->exit_code == exit_code)
+ return table->reason;
+ table++;
+ }
+
+ pr_err("unknown kvm exit code:%lld on %s\n",
+ (unsigned long long)exit_code, cpu_isa ? "VMX" : "SVM");
+ return "UNKNOWN";
+}
+
+static void exit_event_decode_key(struct event_key *key, char decode[20])
+{
+ const char *exit_reason = get_exit_reason(key->key);
+
+ scnprintf(decode, 20, "%s", exit_reason);
+}
+
+static struct kvm_events_ops exit_events = {
+ .is_begin_event = exit_event_begin,
+ .is_end_event = exit_event_end,
+ .decode_key = exit_event_decode_key,
+ .name = "VM-EXIT"
+};
+
+ /*
+ * For the mmio events, we treat:
+ * the time of MMIO write: kvm_mmio(KVM_TRACE_MMIO_WRITE...) -> kvm_entry
+ * the time of MMIO read: kvm_exit -> kvm_mmio(KVM_TRACE_MMIO_READ...).
+ */
+static void mmio_event_get_key(struct perf_evsel *evsel, struct perf_sample *sample,
+ struct event_key *key)
+{
+ key->key = perf_evsel__intval(evsel, sample, "gpa");
+ key->info = perf_evsel__intval(evsel, sample, "type");
+}
+
+#define KVM_TRACE_MMIO_READ_UNSATISFIED 0
+#define KVM_TRACE_MMIO_READ 1
+#define KVM_TRACE_MMIO_WRITE 2
+
+static bool mmio_event_begin(struct perf_evsel *evsel,
+ struct perf_sample *sample, struct event_key *key)
+{
+ /* MMIO read begin event in kernel. */
+ if (kvm_exit_event(evsel))
+ return true;
+
+ /* MMIO write begin event in kernel. */
+ if (!strcmp(evsel->name, "kvm:kvm_mmio") &&
+ perf_evsel__intval(evsel, sample, "type") == KVM_TRACE_MMIO_WRITE) {
+ mmio_event_get_key(evsel, sample, key);
+ return true;
+ }
+
+ return false;
+}
+
+static bool mmio_event_end(struct perf_evsel *evsel, struct perf_sample *sample,
+ struct event_key *key)
+{
+ /* MMIO write end event in kernel. */
+ if (kvm_entry_event(evsel))
+ return true;
+
+ /* MMIO read end event in kernel.*/
+ if (!strcmp(evsel->name, "kvm:kvm_mmio") &&
+ perf_evsel__intval(evsel, sample, "type") == KVM_TRACE_MMIO_READ) {
+ mmio_event_get_key(evsel, sample, key);
+ return true;
+ }
+
+ return false;
+}
+
+static void mmio_event_decode_key(struct event_key *key, char decode[20])
+{
+ scnprintf(decode, 20, "%#lx:%s", (unsigned long)key->key,
+ key->info == KVM_TRACE_MMIO_WRITE ? "W" : "R");
+}
+
+static struct kvm_events_ops mmio_events = {
+ .is_begin_event = mmio_event_begin,
+ .is_end_event = mmio_event_end,
+ .decode_key = mmio_event_decode_key,
+ .name = "MMIO Access"
+};
+
+ /* The time of emulation pio access is from kvm_pio to kvm_entry. */
+static void ioport_event_get_key(struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct event_key *key)
+{
+ key->key = perf_evsel__intval(evsel, sample, "port");
+ key->info = perf_evsel__intval(evsel, sample, "rw");
+}
+
+static bool ioport_event_begin(struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct event_key *key)
+{
+ if (!strcmp(evsel->name, "kvm:kvm_pio")) {
+ ioport_event_get_key(evsel, sample, key);
+ return true;
+ }
+
+ return false;
+}
+
+static bool ioport_event_end(struct perf_evsel *evsel,
+ struct perf_sample *sample __maybe_unused,
+ struct event_key *key __maybe_unused)
+{
+ return kvm_entry_event(evsel);
+}
+
+static void ioport_event_decode_key(struct event_key *key, char decode[20])
+{
+ scnprintf(decode, 20, "%#llx:%s", (unsigned long long)key->key,
+ key->info ? "POUT" : "PIN");
+}
+
+static struct kvm_events_ops ioport_events = {
+ .is_begin_event = ioport_event_begin,
+ .is_end_event = ioport_event_end,
+ .decode_key = ioport_event_decode_key,
+ .name = "IO Port Access"
+};
+
+static const char *report_event = "vmexit";
+struct kvm_events_ops *events_ops;
+
+static bool register_kvm_events_ops(void)
+{
+ bool ret = true;
+
+ if (!strcmp(report_event, "vmexit"))
+ events_ops = &exit_events;
+ else if (!strcmp(report_event, "mmio"))
+ events_ops = &mmio_events;
+ else if (!strcmp(report_event, "ioport"))
+ events_ops = &ioport_events;
+ else {
+ pr_err("Unknown report event:%s\n", report_event);
+ ret = false;
+ }
+
+ return ret;
+}
+
+struct kvm_event_stats {
+ u64 time;
+ struct stats stats;
+};
+
+struct kvm_event {
+ struct list_head hash_entry;
+ struct rb_node rb;
+
+ struct event_key key;
+
+ struct kvm_event_stats total;
+
+ #define DEFAULT_VCPU_NUM 8
+ int max_vcpu;
+ struct kvm_event_stats *vcpu;
+};
+
+struct vcpu_event_record {
+ int vcpu_id;
+ u64 start_time;
+ struct kvm_event *last_event;
+};
+
+#define EVENTS_BITS 12
+#define EVENTS_CACHE_SIZE (1UL << EVENTS_BITS)
+
+static u64 total_time;
+static u64 total_count;
+static struct list_head kvm_events_cache[EVENTS_CACHE_SIZE];
+
+static void init_kvm_event_record(void)
+{
+ int i;
+
+ for (i = 0; i < (int)EVENTS_CACHE_SIZE; i++)
+ INIT_LIST_HEAD(&kvm_events_cache[i]);
+}
+
+static int kvm_events_hash_fn(u64 key)
+{
+ return key & (EVENTS_CACHE_SIZE - 1);
+}
+
+static bool kvm_event_expand(struct kvm_event *event, int vcpu_id)
+{
+ int old_max_vcpu = event->max_vcpu;
+
+ if (vcpu_id < event->max_vcpu)
+ return true;
+
+ while (event->max_vcpu <= vcpu_id)
+ event->max_vcpu += DEFAULT_VCPU_NUM;
+
+ event->vcpu = realloc(event->vcpu,
+ event->max_vcpu * sizeof(*event->vcpu));
+ if (!event->vcpu) {
+ pr_err("Not enough memory\n");
+ return false;
+ }
+
+ memset(event->vcpu + old_max_vcpu, 0,
+ (event->max_vcpu - old_max_vcpu) * sizeof(*event->vcpu));
+ return true;
+}
+
+static struct kvm_event *kvm_alloc_init_event(struct event_key *key)
+{
+ struct kvm_event *event;
+
+ event = zalloc(sizeof(*event));
+ if (!event) {
+ pr_err("Not enough memory\n");
+ return NULL;
+ }
+
+ event->key = *key;
+ return event;
+}
+
+static struct kvm_event *find_create_kvm_event(struct event_key *key)
+{
+ struct kvm_event *event;
+ struct list_head *head;
+
+ BUG_ON(key->key == INVALID_KEY);
+
+ head = &kvm_events_cache[kvm_events_hash_fn(key->key)];
+ list_for_each_entry(event, head, hash_entry)
+ if (event->key.key == key->key && event->key.info == key->info)
+ return event;
+
+ event = kvm_alloc_init_event(key);
+ if (!event)
+ return NULL;
+
+ list_add(&event->hash_entry, head);
+ return event;
+}
+
+static bool handle_begin_event(struct vcpu_event_record *vcpu_record,
+ struct event_key *key, u64 timestamp)
+{
+ struct kvm_event *event = NULL;
+
+ if (key->key != INVALID_KEY)
+ event = find_create_kvm_event(key);
+
+ vcpu_record->last_event = event;
+ vcpu_record->start_time = timestamp;
+ return true;
+}
+
+static void
+kvm_update_event_stats(struct kvm_event_stats *kvm_stats, u64 time_diff)
+{
+ kvm_stats->time += time_diff;
+ update_stats(&kvm_stats->stats, time_diff);
+}
+
+static double kvm_event_rel_stddev(int vcpu_id, struct kvm_event *event)
+{
+ struct kvm_event_stats *kvm_stats = &event->total;
+
+ if (vcpu_id != -1)
+ kvm_stats = &event->vcpu[vcpu_id];
+
+ return rel_stddev_stats(stddev_stats(&kvm_stats->stats),
+ avg_stats(&kvm_stats->stats));
+}
+
+static bool update_kvm_event(struct kvm_event *event, int vcpu_id,
+ u64 time_diff)
+{
+ kvm_update_event_stats(&event->total, time_diff);
+
+ if (!kvm_event_expand(event, vcpu_id))
+ return false;
+
+ kvm_update_event_stats(&event->vcpu[vcpu_id], time_diff);
+ return true;
+}
+
+static bool handle_end_event(struct vcpu_event_record *vcpu_record,
+ struct event_key *key, u64 timestamp)
+{
+ struct kvm_event *event;
+ u64 time_begin, time_diff;
+
+ event = vcpu_record->last_event;
+ time_begin = vcpu_record->start_time;
+
+ /* The begin event is not caught. */
+ if (!time_begin)
+ return true;
+
+ /*
+ * In some case, the 'begin event' only records the start timestamp,
+ * the actual event is recognized in the 'end event' (e.g. mmio-event).
+ */
+
+ /* Both begin and end events did not get the key. */
+ if (!event && key->key == INVALID_KEY)
+ return true;
+
+ if (!event)
+ event = find_create_kvm_event(key);
+
+ if (!event)
+ return false;
+
+ vcpu_record->last_event = NULL;
+ vcpu_record->start_time = 0;
+
+ BUG_ON(timestamp < time_begin);
+
+ time_diff = timestamp - time_begin;
+ return update_kvm_event(event, vcpu_record->vcpu_id, time_diff);
+}
+
+static
+struct vcpu_event_record *per_vcpu_record(struct thread *thread,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample)
+{
+ /* Only kvm_entry records vcpu id. */
+ if (!thread->priv && kvm_entry_event(evsel)) {
+ struct vcpu_event_record *vcpu_record;
+
+ vcpu_record = zalloc(sizeof(*vcpu_record));
+ if (!vcpu_record) {
+ pr_err("%s: Not enough memory\n", __func__);
+ return NULL;
+ }
+
+ vcpu_record->vcpu_id = perf_evsel__intval(evsel, sample, "vcpu_id");
+ thread->priv = vcpu_record;
+ }
+
+ return thread->priv;
+}
+
+static bool handle_kvm_event(struct thread *thread, struct perf_evsel *evsel,
+ struct perf_sample *sample)
+{
+ struct vcpu_event_record *vcpu_record;
+ struct event_key key = {.key = INVALID_KEY};
+
+ vcpu_record = per_vcpu_record(thread, evsel, sample);
+ if (!vcpu_record)
+ return true;
+
+ if (events_ops->is_begin_event(evsel, sample, &key))
+ return handle_begin_event(vcpu_record, &key, sample->time);
+
+ if (events_ops->is_end_event(evsel, sample, &key))
+ return handle_end_event(vcpu_record, &key, sample->time);
+
+ return true;
+}
+
+typedef int (*key_cmp_fun)(struct kvm_event*, struct kvm_event*, int);
+struct kvm_event_key {
+ const char *name;
+ key_cmp_fun key;
+};
+
+static int trace_vcpu = -1;
+#define GET_EVENT_KEY(func, field) \
+static u64 get_event_ ##func(struct kvm_event *event, int vcpu) \
+{ \
+ if (vcpu == -1) \
+ return event->total.field; \
+ \
+ if (vcpu >= event->max_vcpu) \
+ return 0; \
+ \
+ return event->vcpu[vcpu].field; \
+}
+
+#define COMPARE_EVENT_KEY(func, field) \
+GET_EVENT_KEY(func, field) \
+static int compare_kvm_event_ ## func(struct kvm_event *one, \
+ struct kvm_event *two, int vcpu)\
+{ \
+ return get_event_ ##func(one, vcpu) > \
+ get_event_ ##func(two, vcpu); \
+}
+
+GET_EVENT_KEY(time, time);
+COMPARE_EVENT_KEY(count, stats.n);
+COMPARE_EVENT_KEY(mean, stats.mean);
+
+#define DEF_SORT_NAME_KEY(name, compare_key) \
+ { #name, compare_kvm_event_ ## compare_key }
+
+static struct kvm_event_key keys[] = {
+ DEF_SORT_NAME_KEY(sample, count),
+ DEF_SORT_NAME_KEY(time, mean),
+ { NULL, NULL }
+};
+
+static const char *sort_key = "sample";
+static key_cmp_fun compare;
+
+static bool select_key(void)
+{
+ int i;
+
+ for (i = 0; keys[i].name; i++) {
+ if (!strcmp(keys[i].name, sort_key)) {
+ compare = keys[i].key;
+ return true;
+ }
+ }
+
+ pr_err("Unknown compare key:%s\n", sort_key);
+ return false;
+}
+
+static struct rb_root result;
+static void insert_to_result(struct kvm_event *event, key_cmp_fun bigger,
+ int vcpu)
+{
+ struct rb_node **rb = &result.rb_node;
+ struct rb_node *parent = NULL;
+ struct kvm_event *p;
+
+ while (*rb) {
+ p = container_of(*rb, struct kvm_event, rb);
+ parent = *rb;
+
+ if (bigger(event, p, vcpu))
+ rb = &(*rb)->rb_left;
+ else
+ rb = &(*rb)->rb_right;
+ }
+
+ rb_link_node(&event->rb, parent, rb);
+ rb_insert_color(&event->rb, &result);
+}
+
+static void update_total_count(struct kvm_event *event, int vcpu)
+{
+ total_count += get_event_count(event, vcpu);
+ total_time += get_event_time(event, vcpu);
+}
+
+static bool event_is_valid(struct kvm_event *event, int vcpu)
+{
+ return !!get_event_count(event, vcpu);
+}
+
+static void sort_result(int vcpu)
+{
+ unsigned int i;
+ struct kvm_event *event;
+
+ for (i = 0; i < EVENTS_CACHE_SIZE; i++)
+ list_for_each_entry(event, &kvm_events_cache[i], hash_entry)
+ if (event_is_valid(event, vcpu)) {
+ update_total_count(event, vcpu);
+ insert_to_result(event, compare, vcpu);
+ }
+}
+
+/* returns left most element of result, and erase it */
+static struct kvm_event *pop_from_result(void)
+{
+ struct rb_node *node = rb_first(&result);
+
+ if (!node)
+ return NULL;
+
+ rb_erase(node, &result);
+ return container_of(node, struct kvm_event, rb);
+}
+
+static void print_vcpu_info(int vcpu)
+{
+ pr_info("Analyze events for ");
+
+ if (vcpu == -1)
+ pr_info("all VCPUs:\n\n");
+ else
+ pr_info("VCPU %d:\n\n", vcpu);
+}
+
+static void print_result(int vcpu)
+{
+ char decode[20];
+ struct kvm_event *event;
+
+ pr_info("\n\n");
+ print_vcpu_info(vcpu);
+ pr_info("%20s ", events_ops->name);
+ pr_info("%10s ", "Samples");
+ pr_info("%9s ", "Samples%");
+
+ pr_info("%9s ", "Time%");
+ pr_info("%16s ", "Avg time");
+ pr_info("\n\n");
+
+ while ((event = pop_from_result())) {
+ u64 ecount, etime;
+
+ ecount = get_event_count(event, vcpu);
+ etime = get_event_time(event, vcpu);
+
+ events_ops->decode_key(&event->key, decode);
+ pr_info("%20s ", decode);
+ pr_info("%10llu ", (unsigned long long)ecount);
+ pr_info("%8.2f%% ", (double)ecount / total_count * 100);
+ pr_info("%8.2f%% ", (double)etime / total_time * 100);
+ pr_info("%9.2fus ( +-%7.2f%% )", (double)etime / ecount/1e3,
+ kvm_event_rel_stddev(vcpu, event));
+ pr_info("\n");
+ }
+
+ pr_info("\nTotal Samples:%lld, Total events handled time:%.2fus.\n\n",
+ (unsigned long long)total_count, total_time / 1e3);
+}
+
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct perf_evsel *evsel,
+ struct machine *machine)
+{
+ struct thread *thread = machine__findnew_thread(machine, sample->tid);
+
+ if (thread == NULL) {
+ pr_debug("problem processing %d event, skipping it.\n",
+ event->header.type);
+ return -1;
+ }
+
+ if (!handle_kvm_event(thread, evsel, sample))
+ return -1;
+
+ return 0;
+}
+
+static struct perf_tool eops = {
+ .sample = process_sample_event,
+ .comm = perf_event__process_comm,
+ .ordered_samples = true,
+};
+
+static int get_cpu_isa(struct perf_session *session)
+{
+ char *cpuid = session->header.env.cpuid;
+ int isa;
+
+ if (strstr(cpuid, "Intel"))
+ isa = 1;
+ else if (strstr(cpuid, "AMD"))
+ isa = 0;
+ else {
+ pr_err("CPU %s is not supported.\n", cpuid);
+ isa = -ENOTSUP;
+ }
+
+ return isa;
+}
+
+static const char *file_name;
+
+static int read_events(void)
+{
+ struct perf_session *kvm_session;
+ int ret;
+
+ kvm_session = perf_session__new(file_name, O_RDONLY, 0, false, &eops);
+ if (!kvm_session) {
+ pr_err("Initializing perf session failed\n");
+ return -EINVAL;
+ }
+
+ if (!perf_session__has_traces(kvm_session, "kvm record"))
+ return -EINVAL;
+
+ /*
+ * Do not use 'isa' recorded in kvm_exit tracepoint since it is not
+ * traced in the old kernel.
+ */
+ ret = get_cpu_isa(kvm_session);
+
+ if (ret < 0)
+ return ret;
+
+ cpu_isa = ret;
+
+ return perf_session__process_events(kvm_session, &eops);
+}
+
+static bool verify_vcpu(int vcpu)
+{
+ if (vcpu != -1 && vcpu < 0) {
+ pr_err("Invalid vcpu:%d.\n", vcpu);
+ return false;
+ }
+
+ return true;
+}
+
+static int kvm_events_report_vcpu(int vcpu)
+{
+ int ret = -EINVAL;
+
+ if (!verify_vcpu(vcpu))
+ goto exit;
+
+ if (!select_key())
+ goto exit;
+
+ if (!register_kvm_events_ops())
+ goto exit;
+
+ init_kvm_event_record();
+ setup_pager();
+
+ ret = read_events();
+ if (ret)
+ goto exit;
+
+ sort_result(vcpu);
+ print_result(vcpu);
+exit:
+ return ret;
+}
+
+static const char * const record_args[] = {
+ "record",
+ "-R",
+ "-f",
+ "-m", "1024",
+ "-c", "1",
+ "-e", "kvm:kvm_entry",
+ "-e", "kvm:kvm_exit",
+ "-e", "kvm:kvm_mmio",
+ "-e", "kvm:kvm_pio",
+};
+
+#define STRDUP_FAIL_EXIT(s) \
+ ({ char *_p; \
+ _p = strdup(s); \
+ if (!_p) \
+ return -ENOMEM; \
+ _p; \
+ })
+
+static int kvm_events_record(int argc, const char **argv)
+{
+ unsigned int rec_argc, i, j;
+ const char **rec_argv;
+
+ rec_argc = ARRAY_SIZE(record_args) + argc + 2;
+ rec_argv = calloc(rec_argc + 1, sizeof(char *));
+
+ if (rec_argv == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < ARRAY_SIZE(record_args); i++)
+ rec_argv[i] = STRDUP_FAIL_EXIT(record_args[i]);
+
+ rec_argv[i++] = STRDUP_FAIL_EXIT("-o");
+ rec_argv[i++] = STRDUP_FAIL_EXIT(file_name);
+
+ for (j = 1; j < (unsigned int)argc; j++, i++)
+ rec_argv[i] = argv[j];
+
+ return cmd_record(i, rec_argv, NULL);
+}
+
+static const char * const kvm_events_report_usage[] = {
+ "perf kvm stat report [<options>]",
+ NULL
+};
+
+static const struct option kvm_events_report_options[] = {
+ OPT_STRING(0, "event", &report_event, "report event",
+ "event for reporting: vmexit, mmio, ioport"),
+ OPT_INTEGER(0, "vcpu", &trace_vcpu,
+ "vcpu id to report"),
+ OPT_STRING('k', "key", &sort_key, "sort-key",
+ "key for sorting: sample(sort by samples number)"
+ " time (sort by avg time)"),
+ OPT_END()
+};
+
+static int kvm_events_report(int argc, const char **argv)
+{
+ symbol__init();
+
+ if (argc) {
+ argc = parse_options(argc, argv,
+ kvm_events_report_options,
+ kvm_events_report_usage, 0);
+ if (argc)
+ usage_with_options(kvm_events_report_usage,
+ kvm_events_report_options);
+ }
+
+ return kvm_events_report_vcpu(trace_vcpu);
+}
+
+static void print_kvm_stat_usage(void)
+{
+ printf("Usage: perf kvm stat <command>\n\n");
+
+ printf("# Available commands:\n");
+ printf("\trecord: record kvm events\n");
+ printf("\treport: report statistical data of kvm events\n");
+
+ printf("\nOtherwise, it is the alias of 'perf stat':\n");
+}
+
+static int kvm_cmd_stat(int argc, const char **argv)
+{
+ if (argc == 1) {
+ print_kvm_stat_usage();
+ goto perf_stat;
+ }
+
+ if (!strncmp(argv[1], "rec", 3))
+ return kvm_events_record(argc - 1, argv + 1);
+
+ if (!strncmp(argv[1], "rep", 3))
+ return kvm_events_report(argc - 1 , argv + 1);
+
+perf_stat:
+ return cmd_stat(argc, argv, NULL);
+}
+
static char name_buffer[256];
static const char * const kvm_usage[] = {
- "perf kvm [<options>] {top|record|report|diff|buildid-list}",
+ "perf kvm [<options>] {top|record|report|diff|buildid-list|stat}",
NULL
};
@@ -102,7 +930,7 @@
return cmd_buildid_list(i, rec_argv, NULL);
}
-int cmd_kvm(int argc, const char **argv, const char *prefix __used)
+int cmd_kvm(int argc, const char **argv, const char *prefix __maybe_unused)
{
perf_host = 0;
perf_guest = 1;
@@ -135,6 +963,8 @@
return cmd_top(argc, argv, NULL);
else if (!strncmp(argv[0], "buildid-list", 12))
return __cmd_buildid_list(argc, argv);
+ else if (!strncmp(argv[0], "stat", 4))
+ return kvm_cmd_stat(argc, argv);
else
usage_with_options(kvm_usage, kvm_options);
diff --git a/tools/perf/builtin-list.c b/tools/perf/builtin-list.c
index 6313b6e..1948ece 100644
--- a/tools/perf/builtin-list.c
+++ b/tools/perf/builtin-list.c
@@ -14,20 +14,20 @@
#include "util/parse-events.h"
#include "util/cache.h"
-int cmd_list(int argc, const char **argv, const char *prefix __used)
+int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
{
setup_pager();
if (argc == 1)
- print_events(NULL);
+ print_events(NULL, false);
else {
int i;
for (i = 1; i < argc; ++i) {
- if (i > 1)
+ if (i > 2)
putchar('\n');
if (strncmp(argv[i], "tracepoint", 10) == 0)
- print_tracepoint_events(NULL, NULL);
+ print_tracepoint_events(NULL, NULL, false);
else if (strcmp(argv[i], "hw") == 0 ||
strcmp(argv[i], "hardware") == 0)
print_events_type(PERF_TYPE_HARDWARE);
@@ -36,13 +36,15 @@
print_events_type(PERF_TYPE_SOFTWARE);
else if (strcmp(argv[i], "cache") == 0 ||
strcmp(argv[i], "hwcache") == 0)
- print_hwcache_events(NULL);
+ print_hwcache_events(NULL, false);
+ else if (strcmp(argv[i], "--raw-dump") == 0)
+ print_events(NULL, true);
else {
char *sep = strchr(argv[i], ':'), *s;
int sep_idx;
if (sep == NULL) {
- print_events(argv[i]);
+ print_events(argv[i], false);
continue;
}
sep_idx = sep - argv[i];
@@ -51,7 +53,7 @@
return -1;
s[sep_idx] = '\0';
- print_tracepoint_events(s, s + sep_idx + 1);
+ print_tracepoint_events(s, s + sep_idx + 1, false);
free(s);
}
}
diff --git a/tools/perf/builtin-lock.c b/tools/perf/builtin-lock.c
index b3c4285..7d6e099 100644
--- a/tools/perf/builtin-lock.c
+++ b/tools/perf/builtin-lock.c
@@ -1,6 +1,8 @@
#include "builtin.h"
#include "perf.h"
+#include "util/evlist.h"
+#include "util/evsel.h"
#include "util/util.h"
#include "util/cache.h"
#include "util/symbol.h"
@@ -40,7 +42,7 @@
struct rb_node rb; /* used for sorting */
/*
- * FIXME: raw_field_value() returns unsigned long long,
+ * FIXME: perf_evsel__intval() returns u64,
* so address of lockdep_map should be dealed as 64bit.
* Is there more better solution?
*/
@@ -160,8 +162,10 @@
return st;
st = zalloc(sizeof(struct thread_stat));
- if (!st)
- die("memory allocation failed\n");
+ if (!st) {
+ pr_err("memory allocation failed\n");
+ return NULL;
+ }
st->tid = tid;
INIT_LIST_HEAD(&st->seq_list);
@@ -180,8 +184,10 @@
struct thread_stat *st;
st = zalloc(sizeof(struct thread_stat));
- if (!st)
- die("memory allocation failed\n");
+ if (!st) {
+ pr_err("memory allocation failed\n");
+ return NULL;
+ }
st->tid = tid;
INIT_LIST_HEAD(&st->seq_list);
@@ -247,18 +253,20 @@
{ NULL, NULL }
};
-static void select_key(void)
+static int select_key(void)
{
int i;
for (i = 0; keys[i].name; i++) {
if (!strcmp(keys[i].name, sort_key)) {
compare = keys[i].key;
- return;
+ return 0;
}
}
- die("Unknown compare key:%s\n", sort_key);
+ pr_err("Unknown compare key: %s\n", sort_key);
+
+ return -1;
}
static void insert_to_result(struct lock_stat *st,
@@ -323,61 +331,24 @@
return new;
alloc_failed:
- die("memory allocation failed\n");
+ pr_err("memory allocation failed\n");
+ return NULL;
}
static const char *input_name;
-struct raw_event_sample {
- u32 size;
- char data[0];
-};
-
-struct trace_acquire_event {
- void *addr;
- const char *name;
- int flag;
-};
-
-struct trace_acquired_event {
- void *addr;
- const char *name;
-};
-
-struct trace_contended_event {
- void *addr;
- const char *name;
-};
-
-struct trace_release_event {
- void *addr;
- const char *name;
-};
-
struct trace_lock_handler {
- void (*acquire_event)(struct trace_acquire_event *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
+ int (*acquire_event)(struct perf_evsel *evsel,
+ struct perf_sample *sample);
- void (*acquired_event)(struct trace_acquired_event *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
+ int (*acquired_event)(struct perf_evsel *evsel,
+ struct perf_sample *sample);
- void (*contended_event)(struct trace_contended_event *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
+ int (*contended_event)(struct perf_evsel *evsel,
+ struct perf_sample *sample);
- void (*release_event)(struct trace_release_event *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
+ int (*release_event)(struct perf_evsel *evsel,
+ struct perf_sample *sample);
};
static struct lock_seq_stat *get_seq(struct thread_stat *ts, void *addr)
@@ -390,8 +361,10 @@
}
seq = zalloc(sizeof(struct lock_seq_stat));
- if (!seq)
- die("Not enough memory\n");
+ if (!seq) {
+ pr_err("memory allocation failed\n");
+ return NULL;
+ }
seq->state = SEQ_STATE_UNINITIALIZED;
seq->addr = addr;
@@ -414,33 +387,42 @@
READ_LOCK = 2,
};
-static void
-report_lock_acquire_event(struct trace_acquire_event *acquire_event,
- struct event_format *__event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int report_lock_acquire_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
+ void *addr;
struct lock_stat *ls;
struct thread_stat *ts;
struct lock_seq_stat *seq;
+ const char *name = perf_evsel__strval(evsel, sample, "name");
+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
+ int flag = perf_evsel__intval(evsel, sample, "flag");
- ls = lock_stat_findnew(acquire_event->addr, acquire_event->name);
+ memcpy(&addr, &tmp, sizeof(void *));
+
+ ls = lock_stat_findnew(addr, name);
+ if (!ls)
+ return -1;
if (ls->discard)
- return;
+ return 0;
- ts = thread_stat_findnew(thread->pid);
- seq = get_seq(ts, acquire_event->addr);
+ ts = thread_stat_findnew(sample->tid);
+ if (!ts)
+ return -1;
+
+ seq = get_seq(ts, addr);
+ if (!seq)
+ return -1;
switch (seq->state) {
case SEQ_STATE_UNINITIALIZED:
case SEQ_STATE_RELEASED:
- if (!acquire_event->flag) {
+ if (!flag) {
seq->state = SEQ_STATE_ACQUIRING;
} else {
- if (acquire_event->flag & TRY_LOCK)
+ if (flag & TRY_LOCK)
ls->nr_trylock++;
- if (acquire_event->flag & READ_LOCK)
+ if (flag & READ_LOCK)
ls->nr_readlock++;
seq->state = SEQ_STATE_READ_ACQUIRED;
seq->read_count = 1;
@@ -448,7 +430,7 @@
}
break;
case SEQ_STATE_READ_ACQUIRED:
- if (acquire_event->flag & READ_LOCK) {
+ if (flag & READ_LOCK) {
seq->read_count++;
ls->nr_acquired++;
goto end;
@@ -473,38 +455,46 @@
}
ls->nr_acquire++;
- seq->prev_event_time = timestamp;
+ seq->prev_event_time = sample->time;
end:
- return;
+ return 0;
}
-static void
-report_lock_acquired_event(struct trace_acquired_event *acquired_event,
- struct event_format *__event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int report_lock_acquired_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
+ void *addr;
struct lock_stat *ls;
struct thread_stat *ts;
struct lock_seq_stat *seq;
u64 contended_term;
+ const char *name = perf_evsel__strval(evsel, sample, "name");
+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
- ls = lock_stat_findnew(acquired_event->addr, acquired_event->name);
+ memcpy(&addr, &tmp, sizeof(void *));
+
+ ls = lock_stat_findnew(addr, name);
+ if (!ls)
+ return -1;
if (ls->discard)
- return;
+ return 0;
- ts = thread_stat_findnew(thread->pid);
- seq = get_seq(ts, acquired_event->addr);
+ ts = thread_stat_findnew(sample->tid);
+ if (!ts)
+ return -1;
+
+ seq = get_seq(ts, addr);
+ if (!seq)
+ return -1;
switch (seq->state) {
case SEQ_STATE_UNINITIALIZED:
/* orphan event, do nothing */
- return;
+ return 0;
case SEQ_STATE_ACQUIRING:
break;
case SEQ_STATE_CONTENDED:
- contended_term = timestamp - seq->prev_event_time;
+ contended_term = sample->time - seq->prev_event_time;
ls->wait_time_total += contended_term;
if (contended_term < ls->wait_time_min)
ls->wait_time_min = contended_term;
@@ -529,33 +519,41 @@
seq->state = SEQ_STATE_ACQUIRED;
ls->nr_acquired++;
- seq->prev_event_time = timestamp;
+ seq->prev_event_time = sample->time;
end:
- return;
+ return 0;
}
-static void
-report_lock_contended_event(struct trace_contended_event *contended_event,
- struct event_format *__event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int report_lock_contended_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
+ void *addr;
struct lock_stat *ls;
struct thread_stat *ts;
struct lock_seq_stat *seq;
+ const char *name = perf_evsel__strval(evsel, sample, "name");
+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
- ls = lock_stat_findnew(contended_event->addr, contended_event->name);
+ memcpy(&addr, &tmp, sizeof(void *));
+
+ ls = lock_stat_findnew(addr, name);
+ if (!ls)
+ return -1;
if (ls->discard)
- return;
+ return 0;
- ts = thread_stat_findnew(thread->pid);
- seq = get_seq(ts, contended_event->addr);
+ ts = thread_stat_findnew(sample->tid);
+ if (!ts)
+ return -1;
+
+ seq = get_seq(ts, addr);
+ if (!seq)
+ return -1;
switch (seq->state) {
case SEQ_STATE_UNINITIALIZED:
/* orphan event, do nothing */
- return;
+ return 0;
case SEQ_STATE_ACQUIRING:
break;
case SEQ_STATE_RELEASED:
@@ -576,28 +574,36 @@
seq->state = SEQ_STATE_CONTENDED;
ls->nr_contended++;
- seq->prev_event_time = timestamp;
+ seq->prev_event_time = sample->time;
end:
- return;
+ return 0;
}
-static void
-report_lock_release_event(struct trace_release_event *release_event,
- struct event_format *__event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int report_lock_release_event(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
+ void *addr;
struct lock_stat *ls;
struct thread_stat *ts;
struct lock_seq_stat *seq;
+ const char *name = perf_evsel__strval(evsel, sample, "name");
+ u64 tmp = perf_evsel__intval(evsel, sample, "lockdep_addr");
- ls = lock_stat_findnew(release_event->addr, release_event->name);
+ memcpy(&addr, &tmp, sizeof(void *));
+
+ ls = lock_stat_findnew(addr, name);
+ if (!ls)
+ return -1;
if (ls->discard)
- return;
+ return 0;
- ts = thread_stat_findnew(thread->pid);
- seq = get_seq(ts, release_event->addr);
+ ts = thread_stat_findnew(sample->tid);
+ if (!ts)
+ return -1;
+
+ seq = get_seq(ts, addr);
+ if (!seq)
+ return -1;
switch (seq->state) {
case SEQ_STATE_UNINITIALIZED:
@@ -631,7 +637,7 @@
list_del(&seq->list);
free(seq);
end:
- return;
+ return 0;
}
/* lock oriented handlers */
@@ -645,96 +651,36 @@
static struct trace_lock_handler *trace_handler;
-static void
-process_lock_acquire_event(void *data,
- struct event_format *event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int perf_evsel__process_lock_acquire(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- struct trace_acquire_event acquire_event;
- u64 tmp; /* this is required for casting... */
-
- tmp = raw_field_value(event, "lockdep_addr", data);
- memcpy(&acquire_event.addr, &tmp, sizeof(void *));
- acquire_event.name = (char *)raw_field_ptr(event, "name", data);
- acquire_event.flag = (int)raw_field_value(event, "flag", data);
-
if (trace_handler->acquire_event)
- trace_handler->acquire_event(&acquire_event, event, cpu, timestamp, thread);
+ return trace_handler->acquire_event(evsel, sample);
+ return 0;
}
-static void
-process_lock_acquired_event(void *data,
- struct event_format *event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int perf_evsel__process_lock_acquired(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- struct trace_acquired_event acquired_event;
- u64 tmp; /* this is required for casting... */
-
- tmp = raw_field_value(event, "lockdep_addr", data);
- memcpy(&acquired_event.addr, &tmp, sizeof(void *));
- acquired_event.name = (char *)raw_field_ptr(event, "name", data);
-
- if (trace_handler->acquire_event)
- trace_handler->acquired_event(&acquired_event, event, cpu, timestamp, thread);
+ if (trace_handler->acquired_event)
+ return trace_handler->acquired_event(evsel, sample);
+ return 0;
}
-static void
-process_lock_contended_event(void *data,
- struct event_format *event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int perf_evsel__process_lock_contended(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- struct trace_contended_event contended_event;
- u64 tmp; /* this is required for casting... */
-
- tmp = raw_field_value(event, "lockdep_addr", data);
- memcpy(&contended_event.addr, &tmp, sizeof(void *));
- contended_event.name = (char *)raw_field_ptr(event, "name", data);
-
- if (trace_handler->acquire_event)
- trace_handler->contended_event(&contended_event, event, cpu, timestamp, thread);
+ if (trace_handler->contended_event)
+ return trace_handler->contended_event(evsel, sample);
+ return 0;
}
-static void
-process_lock_release_event(void *data,
- struct event_format *event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int perf_evsel__process_lock_release(struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- struct trace_release_event release_event;
- u64 tmp; /* this is required for casting... */
-
- tmp = raw_field_value(event, "lockdep_addr", data);
- memcpy(&release_event.addr, &tmp, sizeof(void *));
- release_event.name = (char *)raw_field_ptr(event, "name", data);
-
- if (trace_handler->acquire_event)
- trace_handler->release_event(&release_event, event, cpu, timestamp, thread);
-}
-
-static void
-process_raw_event(void *data, int cpu, u64 timestamp, struct thread *thread)
-{
- struct event_format *event;
- int type;
-
- type = trace_parse_common_type(session->pevent, data);
- event = pevent_find_event(session->pevent, type);
-
- if (!strcmp(event->name, "lock_acquire"))
- process_lock_acquire_event(data, event, cpu, timestamp, thread);
- if (!strcmp(event->name, "lock_acquired"))
- process_lock_acquired_event(data, event, cpu, timestamp, thread);
- if (!strcmp(event->name, "lock_contended"))
- process_lock_contended_event(data, event, cpu, timestamp, thread);
- if (!strcmp(event->name, "lock_release"))
- process_lock_release_event(data, event, cpu, timestamp, thread);
+ if (trace_handler->release_event)
+ return trace_handler->release_event(evsel, sample);
+ return 0;
}
static void print_bad_events(int bad, int total)
@@ -836,20 +782,29 @@
}
}
-static void dump_info(void)
+static int dump_info(void)
{
+ int rc = 0;
+
if (info_threads)
dump_threads();
else if (info_map)
dump_map();
- else
- die("Unknown type of information\n");
+ else {
+ rc = -1;
+ pr_err("Unknown type of information\n");
+ }
+
+ return rc;
}
-static int process_sample_event(struct perf_tool *tool __used,
+typedef int (*tracepoint_handler)(struct perf_evsel *evsel,
+ struct perf_sample *sample);
+
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel __used,
+ struct perf_evsel *evsel,
struct machine *machine)
{
struct thread *thread = machine__findnew_thread(machine, sample->tid);
@@ -860,7 +815,10 @@
return -1;
}
- process_raw_event(sample->raw_data, sample->cpu, sample->time, thread);
+ if (evsel->handler.func != NULL) {
+ tracepoint_handler f = evsel->handler.func;
+ return f(evsel, sample);
+ }
return 0;
}
@@ -871,11 +829,25 @@
.ordered_samples = true,
};
+static const struct perf_evsel_str_handler lock_tracepoints[] = {
+ { "lock:lock_acquire", perf_evsel__process_lock_acquire, }, /* CONFIG_LOCKDEP */
+ { "lock:lock_acquired", perf_evsel__process_lock_acquired, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
+ { "lock:lock_contended", perf_evsel__process_lock_contended, }, /* CONFIG_LOCKDEP, CONFIG_LOCK_STAT */
+ { "lock:lock_release", perf_evsel__process_lock_release, }, /* CONFIG_LOCKDEP */
+};
+
static int read_events(void)
{
session = perf_session__new(input_name, O_RDONLY, 0, false, &eops);
- if (!session)
- die("Initializing perf session failed\n");
+ if (!session) {
+ pr_err("Initializing perf session failed\n");
+ return -1;
+ }
+
+ if (perf_session__set_tracepoints_handlers(session, lock_tracepoints)) {
+ pr_err("Initializing perf session tracepoint handlers failed\n");
+ return -1;
+ }
return perf_session__process_events(session, &eops);
}
@@ -892,13 +864,18 @@
}
}
-static void __cmd_report(void)
+static int __cmd_report(void)
{
setup_pager();
- select_key();
- read_events();
+
+ if ((select_key() != 0) ||
+ (read_events() != 0))
+ return -1;
+
sort_result();
print_result();
+
+ return 0;
}
static const char * const report_usage[] = {
@@ -944,10 +921,6 @@
"-f",
"-m", "1024",
"-c", "1",
- "-e", "lock:lock_acquire",
- "-e", "lock:lock_acquired",
- "-e", "lock:lock_contended",
- "-e", "lock:lock_release",
};
static int __cmd_record(int argc, const char **argv)
@@ -955,15 +928,31 @@
unsigned int rec_argc, i, j;
const char **rec_argv;
- rec_argc = ARRAY_SIZE(record_args) + argc - 1;
- rec_argv = calloc(rec_argc + 1, sizeof(char *));
+ for (i = 0; i < ARRAY_SIZE(lock_tracepoints); i++) {
+ if (!is_valid_tracepoint(lock_tracepoints[i].name)) {
+ pr_err("tracepoint %s is not enabled. "
+ "Are CONFIG_LOCKDEP and CONFIG_LOCK_STAT enabled?\n",
+ lock_tracepoints[i].name);
+ return 1;
+ }
+ }
+ rec_argc = ARRAY_SIZE(record_args) + argc - 1;
+ /* factor of 2 is for -e in front of each tracepoint */
+ rec_argc += 2 * ARRAY_SIZE(lock_tracepoints);
+
+ rec_argv = calloc(rec_argc + 1, sizeof(char *));
if (rec_argv == NULL)
return -ENOMEM;
for (i = 0; i < ARRAY_SIZE(record_args); i++)
rec_argv[i] = strdup(record_args[i]);
+ for (j = 0; j < ARRAY_SIZE(lock_tracepoints); j++) {
+ rec_argv[i++] = "-e";
+ rec_argv[i++] = strdup(lock_tracepoints[j].name);
+ }
+
for (j = 1; j < (unsigned int)argc; j++, i++)
rec_argv[i] = argv[j];
@@ -972,9 +961,10 @@
return cmd_record(i, rec_argv, NULL);
}
-int cmd_lock(int argc, const char **argv, const char *prefix __used)
+int cmd_lock(int argc, const char **argv, const char *prefix __maybe_unused)
{
unsigned int i;
+ int rc = 0;
symbol__init();
for (i = 0; i < LOCKHASH_SIZE; i++)
@@ -1009,11 +999,13 @@
/* recycling report_lock_ops */
trace_handler = &report_lock_ops;
setup_pager();
- read_events();
- dump_info();
+ if (read_events() != 0)
+ rc = -1;
+ else
+ rc = dump_info();
} else {
usage_with_options(lock_usage, lock_options);
}
- return 0;
+ return rc;
}
diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index e215ae6..118aa89 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -143,8 +143,8 @@
return ret;
}
-static int opt_add_probe_event(const struct option *opt __used,
- const char *str, int unset __used)
+static int opt_add_probe_event(const struct option *opt __maybe_unused,
+ const char *str, int unset __maybe_unused)
{
if (str) {
params.mod_events = true;
@@ -153,8 +153,8 @@
return 0;
}
-static int opt_del_probe_event(const struct option *opt __used,
- const char *str, int unset __used)
+static int opt_del_probe_event(const struct option *opt __maybe_unused,
+ const char *str, int unset __maybe_unused)
{
if (str) {
params.mod_events = true;
@@ -166,7 +166,7 @@
}
static int opt_set_target(const struct option *opt, const char *str,
- int unset __used)
+ int unset __maybe_unused)
{
int ret = -ENOENT;
@@ -188,8 +188,8 @@
}
#ifdef DWARF_SUPPORT
-static int opt_show_lines(const struct option *opt __used,
- const char *str, int unset __used)
+static int opt_show_lines(const struct option *opt __maybe_unused,
+ const char *str, int unset __maybe_unused)
{
int ret = 0;
@@ -209,8 +209,8 @@
return ret;
}
-static int opt_show_vars(const struct option *opt __used,
- const char *str, int unset __used)
+static int opt_show_vars(const struct option *opt __maybe_unused,
+ const char *str, int unset __maybe_unused)
{
struct perf_probe_event *pev = ¶ms.events[params.nevents];
int ret;
@@ -229,8 +229,8 @@
}
#endif
-static int opt_set_filter(const struct option *opt __used,
- const char *str, int unset __used)
+static int opt_set_filter(const struct option *opt __maybe_unused,
+ const char *str, int unset __maybe_unused)
{
const char *err;
@@ -327,7 +327,7 @@
OPT_END()
};
-int cmd_probe(int argc, const char **argv, const char *prefix __used)
+int cmd_probe(int argc, const char **argv, const char *prefix __maybe_unused)
{
int ret;
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 4db6e1b..f14cb5f 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -31,6 +31,15 @@
#include <sched.h>
#include <sys/mman.h>
+#define CALLCHAIN_HELP "do call-graph (stack chain/backtrace) recording: "
+
+#ifdef NO_LIBUNWIND_SUPPORT
+static char callchain_help[] = CALLCHAIN_HELP "[fp]";
+#else
+static unsigned long default_stack_dump_size = 8192;
+static char callchain_help[] = CALLCHAIN_HELP "[fp] dwarf";
+#endif
+
enum write_mode_t {
WRITE_FORCE,
WRITE_APPEND
@@ -62,32 +71,38 @@
rec->bytes_written += size;
}
-static void write_output(struct perf_record *rec, void *buf, size_t size)
+static int write_output(struct perf_record *rec, void *buf, size_t size)
{
while (size) {
int ret = write(rec->output, buf, size);
- if (ret < 0)
- die("failed to write");
+ if (ret < 0) {
+ pr_err("failed to write\n");
+ return -1;
+ }
size -= ret;
buf += ret;
rec->bytes_written += ret;
}
+
+ return 0;
}
static int process_synthesized_event(struct perf_tool *tool,
union perf_event *event,
- struct perf_sample *sample __used,
- struct machine *machine __used)
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
struct perf_record *rec = container_of(tool, struct perf_record, tool);
- write_output(rec, event, event->header.size);
+ if (write_output(rec, event, event->header.size) < 0)
+ return -1;
+
return 0;
}
-static void perf_record__mmap_read(struct perf_record *rec,
+static int perf_record__mmap_read(struct perf_record *rec,
struct perf_mmap *md)
{
unsigned int head = perf_mmap__read_head(md);
@@ -95,9 +110,10 @@
unsigned char *data = md->base + rec->page_size;
unsigned long size;
void *buf;
+ int rc = 0;
if (old == head)
- return;
+ return 0;
rec->samples++;
@@ -108,17 +124,26 @@
size = md->mask + 1 - (old & md->mask);
old += size;
- write_output(rec, buf, size);
+ if (write_output(rec, buf, size) < 0) {
+ rc = -1;
+ goto out;
+ }
}
buf = &data[old & md->mask];
size = head - old;
old += size;
- write_output(rec, buf, size);
+ if (write_output(rec, buf, size) < 0) {
+ rc = -1;
+ goto out;
+ }
md->prev = old;
perf_mmap__write_tail(md, old);
+
+out:
+ return rc;
}
static volatile int done = 0;
@@ -134,7 +159,7 @@
signr = sig;
}
-static void perf_record__sig_exit(int exit_status __used, void *arg)
+static void perf_record__sig_exit(int exit_status __maybe_unused, void *arg)
{
struct perf_record *rec = arg;
int status;
@@ -163,31 +188,32 @@
if (evlist->nr_entries != other->nr_entries)
return false;
- pair = list_entry(other->entries.next, struct perf_evsel, node);
+ pair = perf_evlist__first(other);
list_for_each_entry(pos, &evlist->entries, node) {
if (memcmp(&pos->attr, &pair->attr, sizeof(pos->attr) != 0))
return false;
- pair = list_entry(pair->node.next, struct perf_evsel, node);
+ pair = perf_evsel__next(pair);
}
return true;
}
-static void perf_record__open(struct perf_record *rec)
+static int perf_record__open(struct perf_record *rec)
{
- struct perf_evsel *pos, *first;
+ struct perf_evsel *pos;
struct perf_evlist *evlist = rec->evlist;
struct perf_session *session = rec->session;
struct perf_record_opts *opts = &rec->opts;
-
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ int rc = 0;
perf_evlist__config_attrs(evlist, opts);
+ if (opts->group)
+ perf_evlist__set_leader(evlist);
+
list_for_each_entry(pos, &evlist->entries, node) {
struct perf_event_attr *attr = &pos->attr;
- struct xyarray *group_fd = NULL;
/*
* Check if parse_single_tracepoint_event has already asked for
* PERF_SAMPLE_TIME.
@@ -202,24 +228,24 @@
*/
bool time_needed = attr->sample_type & PERF_SAMPLE_TIME;
- if (opts->group && pos != first)
- group_fd = first->fd;
fallback_missing_features:
if (opts->exclude_guest_missing)
attr->exclude_guest = attr->exclude_host = 0;
retry_sample_id:
attr->sample_id_all = opts->sample_id_all_missing ? 0 : 1;
try_again:
- if (perf_evsel__open(pos, evlist->cpus, evlist->threads,
- opts->group, group_fd) < 0) {
+ if (perf_evsel__open(pos, evlist->cpus, evlist->threads) < 0) {
int err = errno;
if (err == EPERM || err == EACCES) {
ui__error_paranoid();
- exit(EXIT_FAILURE);
+ rc = -err;
+ goto out;
} else if (err == ENODEV && opts->target.cpu_list) {
- die("No such device - did you specify"
- " an out-of-range profile CPU?\n");
+ pr_err("No such device - did you specify"
+ " an out-of-range profile CPU?\n");
+ rc = -err;
+ goto out;
} else if (err == EINVAL) {
if (!opts->exclude_guest_missing &&
(attr->exclude_guest || attr->exclude_host)) {
@@ -266,42 +292,57 @@
if (err == ENOENT) {
ui__error("The %s event is not supported.\n",
perf_evsel__name(pos));
- exit(EXIT_FAILURE);
+ rc = -err;
+ goto out;
}
printf("\n");
- error("sys_perf_event_open() syscall returned with %d (%s). /bin/dmesg may provide additional information.\n",
- err, strerror(err));
+ error("sys_perf_event_open() syscall returned with %d "
+ "(%s) for event %s. /bin/dmesg may provide "
+ "additional information.\n",
+ err, strerror(err), perf_evsel__name(pos));
#if defined(__i386__) || defined(__x86_64__)
- if (attr->type == PERF_TYPE_HARDWARE && err == EOPNOTSUPP)
- die("No hardware sampling interrupt available."
- " No APIC? If so then you can boot the kernel"
- " with the \"lapic\" boot parameter to"
- " force-enable it.\n");
+ if (attr->type == PERF_TYPE_HARDWARE &&
+ err == EOPNOTSUPP) {
+ pr_err("No hardware sampling interrupt available."
+ " No APIC? If so then you can boot the kernel"
+ " with the \"lapic\" boot parameter to"
+ " force-enable it.\n");
+ rc = -err;
+ goto out;
+ }
#endif
- die("No CONFIG_PERF_EVENTS=y kernel support configured?\n");
+ pr_err("No CONFIG_PERF_EVENTS=y kernel support configured?\n");
+ rc = -err;
+ goto out;
}
}
- if (perf_evlist__set_filters(evlist)) {
+ if (perf_evlist__apply_filters(evlist)) {
error("failed to set filter with %d (%s)\n", errno,
strerror(errno));
- exit(-1);
+ rc = -1;
+ goto out;
}
if (perf_evlist__mmap(evlist, opts->mmap_pages, false) < 0) {
- if (errno == EPERM)
- die("Permission error mapping pages.\n"
- "Consider increasing "
- "/proc/sys/kernel/perf_event_mlock_kb,\n"
- "or try again with a smaller value of -m/--mmap_pages.\n"
- "(current value: %d)\n", opts->mmap_pages);
- else if (!is_power_of_2(opts->mmap_pages))
- die("--mmap_pages/-m value must be a power of two.");
-
- die("failed to mmap with %d (%s)\n", errno, strerror(errno));
+ if (errno == EPERM) {
+ pr_err("Permission error mapping pages.\n"
+ "Consider increasing "
+ "/proc/sys/kernel/perf_event_mlock_kb,\n"
+ "or try again with a smaller value of -m/--mmap_pages.\n"
+ "(current value: %d)\n", opts->mmap_pages);
+ rc = -errno;
+ } else if (!is_power_of_2(opts->mmap_pages)) {
+ pr_err("--mmap_pages/-m value must be a power of two.");
+ rc = -EINVAL;
+ } else {
+ pr_err("failed to mmap with %d (%s)\n", errno, strerror(errno));
+ rc = -errno;
+ }
+ goto out;
}
if (rec->file_new)
@@ -309,11 +350,14 @@
else {
if (!perf_evlist__equal(session->evlist, evlist)) {
fprintf(stderr, "incompatible append\n");
- exit(-1);
+ rc = -1;
+ goto out;
}
}
perf_session__set_id_hdr_size(session);
+out:
+ return rc;
}
static int process_buildids(struct perf_record *rec)
@@ -329,10 +373,13 @@
size, &build_id__mark_dso_hit_ops);
}
-static void perf_record__exit(int status __used, void *arg)
+static void perf_record__exit(int status, void *arg)
{
struct perf_record *rec = arg;
+ if (status != 0)
+ return;
+
if (!rec->opts.pipe_output) {
rec->session->header.data_size += rec->bytes_written;
@@ -387,17 +434,26 @@
.type = PERF_RECORD_FINISHED_ROUND,
};
-static void perf_record__mmap_read_all(struct perf_record *rec)
+static int perf_record__mmap_read_all(struct perf_record *rec)
{
int i;
+ int rc = 0;
for (i = 0; i < rec->evlist->nr_mmaps; i++) {
- if (rec->evlist->mmap[i].base)
- perf_record__mmap_read(rec, &rec->evlist->mmap[i]);
+ if (rec->evlist->mmap[i].base) {
+ if (perf_record__mmap_read(rec, &rec->evlist->mmap[i]) != 0) {
+ rc = -1;
+ goto out;
+ }
+ }
}
if (perf_header__has_feat(&rec->session->header, HEADER_TRACING_DATA))
- write_output(rec, &finished_round_event, sizeof(finished_round_event));
+ rc = write_output(rec, &finished_round_event,
+ sizeof(finished_round_event));
+
+out:
+ return rc;
}
static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
@@ -457,7 +513,7 @@
output = open(output_name, flags, S_IRUSR | S_IWUSR);
if (output < 0) {
perror("failed to create output file");
- exit(-1);
+ return -1;
}
rec->output = output;
@@ -497,7 +553,10 @@
}
}
- perf_record__open(rec);
+ if (perf_record__open(rec) != 0) {
+ err = -1;
+ goto out_delete_session;
+ }
/*
* perf_session__delete(session) will be called at perf_record__exit()
@@ -507,19 +566,20 @@
if (opts->pipe_output) {
err = perf_header__write_pipe(output);
if (err < 0)
- return err;
+ goto out_delete_session;
} else if (rec->file_new) {
err = perf_session__write_header(session, evsel_list,
output, false);
if (err < 0)
- return err;
+ goto out_delete_session;
}
if (!rec->no_buildid
&& !perf_header__has_feat(&session->header, HEADER_BUILD_ID)) {
pr_err("Couldn't generate buildids. "
"Use --no-buildid to profile anyway.\n");
- return -1;
+ err = -1;
+ goto out_delete_session;
}
rec->post_processing_offset = lseek(output, 0, SEEK_CUR);
@@ -527,7 +587,8 @@
machine = perf_session__find_host_machine(session);
if (!machine) {
pr_err("Couldn't find native kernel information.\n");
- return -1;
+ err = -1;
+ goto out_delete_session;
}
if (opts->pipe_output) {
@@ -535,14 +596,14 @@
process_synthesized_event);
if (err < 0) {
pr_err("Couldn't synthesize attrs.\n");
- return err;
+ goto out_delete_session;
}
err = perf_event__synthesize_event_types(tool, process_synthesized_event,
machine);
if (err < 0) {
pr_err("Couldn't synthesize event_types.\n");
- return err;
+ goto out_delete_session;
}
if (have_tracepoints(&evsel_list->entries)) {
@@ -558,7 +619,7 @@
process_synthesized_event);
if (err <= 0) {
pr_err("Couldn't record tracing data.\n");
- return err;
+ goto out_delete_session;
}
advance_output(rec, err);
}
@@ -586,20 +647,24 @@
perf_event__synthesize_guest_os);
if (!opts->target.system_wide)
- perf_event__synthesize_thread_map(tool, evsel_list->threads,
+ err = perf_event__synthesize_thread_map(tool, evsel_list->threads,
process_synthesized_event,
machine);
else
- perf_event__synthesize_threads(tool, process_synthesized_event,
+ err = perf_event__synthesize_threads(tool, process_synthesized_event,
machine);
+ if (err != 0)
+ goto out_delete_session;
+
if (rec->realtime_prio) {
struct sched_param param;
param.sched_priority = rec->realtime_prio;
if (sched_setscheduler(0, SCHED_FIFO, ¶m)) {
pr_err("Could not set realtime priority.\n");
- exit(-1);
+ err = -1;
+ goto out_delete_session;
}
}
@@ -614,7 +679,10 @@
for (;;) {
int hits = rec->samples;
- perf_record__mmap_read_all(rec);
+ if (perf_record__mmap_read_all(rec) < 0) {
+ err = -1;
+ goto out_delete_session;
+ }
if (hits == rec->samples) {
if (done)
@@ -732,6 +800,106 @@
return ret;
}
+#ifndef NO_LIBUNWIND_SUPPORT
+static int get_stack_size(char *str, unsigned long *_size)
+{
+ char *endptr;
+ unsigned long size;
+ unsigned long max_size = round_down(USHRT_MAX, sizeof(u64));
+
+ size = strtoul(str, &endptr, 0);
+
+ do {
+ if (*endptr)
+ break;
+
+ size = round_up(size, sizeof(u64));
+ if (!size || size > max_size)
+ break;
+
+ *_size = size;
+ return 0;
+
+ } while (0);
+
+ pr_err("callchain: Incorrect stack dump size (max %ld): %s\n",
+ max_size, str);
+ return -1;
+}
+#endif /* !NO_LIBUNWIND_SUPPORT */
+
+static int
+parse_callchain_opt(const struct option *opt __maybe_unused, const char *arg,
+ int unset)
+{
+ struct perf_record *rec = (struct perf_record *)opt->value;
+ char *tok, *name, *saveptr = NULL;
+ char *buf;
+ int ret = -1;
+
+ /* --no-call-graph */
+ if (unset)
+ return 0;
+
+ /* We specified default option if none is provided. */
+ BUG_ON(!arg);
+
+ /* We need buffer that we know we can write to. */
+ buf = malloc(strlen(arg) + 1);
+ if (!buf)
+ return -ENOMEM;
+
+ strcpy(buf, arg);
+
+ tok = strtok_r((char *)buf, ",", &saveptr);
+ name = tok ? : (char *)buf;
+
+ do {
+ /* Framepointer style */
+ if (!strncmp(name, "fp", sizeof("fp"))) {
+ if (!strtok_r(NULL, ",", &saveptr)) {
+ rec->opts.call_graph = CALLCHAIN_FP;
+ ret = 0;
+ } else
+ pr_err("callchain: No more arguments "
+ "needed for -g fp\n");
+ break;
+
+#ifndef NO_LIBUNWIND_SUPPORT
+ /* Dwarf style */
+ } else if (!strncmp(name, "dwarf", sizeof("dwarf"))) {
+ ret = 0;
+ rec->opts.call_graph = CALLCHAIN_DWARF;
+ rec->opts.stack_dump_size = default_stack_dump_size;
+
+ tok = strtok_r(NULL, ",", &saveptr);
+ if (tok) {
+ unsigned long size = 0;
+
+ ret = get_stack_size(tok, &size);
+ rec->opts.stack_dump_size = size;
+ }
+
+ if (!ret)
+ pr_debug("callchain: stack dump size %d\n",
+ rec->opts.stack_dump_size);
+#endif /* !NO_LIBUNWIND_SUPPORT */
+ } else {
+ pr_err("callchain: Unknown -g option "
+ "value: %s\n", arg);
+ break;
+ }
+
+ } while (0);
+
+ free(buf);
+
+ if (!ret)
+ pr_debug("callchain: type %d\n", rec->opts.call_graph);
+
+ return ret;
+}
+
static const char * const record_usage[] = {
"perf record [<options>] [<command>]",
"perf record [<options>] -- <command> [<options>]",
@@ -803,8 +971,9 @@
"number of mmap data pages"),
OPT_BOOLEAN(0, "group", &record.opts.group,
"put the counters into a counter group"),
- OPT_BOOLEAN('g', "call-graph", &record.opts.call_graph,
- "do call-graph (stack chain/backtrace) recording"),
+ OPT_CALLBACK_DEFAULT('g', "call-graph", &record, "mode[,dump_size]",
+ callchain_help, &parse_callchain_opt,
+ "fp"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show counter open errors, etc)"),
OPT_BOOLEAN('q', "quiet", &quiet, "don't print any message"),
@@ -836,7 +1005,7 @@
OPT_END()
};
-int cmd_record(int argc, const char **argv, const char *prefix __used)
+int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
{
int err = -ENOMEM;
struct perf_evsel *pos;
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 7c88a24..1da243d 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -69,8 +69,8 @@
if ((sort__has_parent || symbol_conf.use_callchain)
&& sample->callchain) {
- err = machine__resolve_callchain(machine, al->thread,
- sample->callchain, &parent);
+ err = machine__resolve_callchain(machine, evsel, al->thread,
+ sample, &parent);
if (err)
return err;
}
@@ -93,7 +93,7 @@
struct annotation *notes;
err = -ENOMEM;
bx = he->branch_info;
- if (bx->from.sym && use_browser > 0) {
+ if (bx->from.sym && use_browser == 1 && sort__has_sym) {
notes = symbol__annotation(bx->from.sym);
if (!notes->src
&& symbol__alloc_hist(bx->from.sym) < 0)
@@ -107,7 +107,7 @@
goto out;
}
- if (bx->to.sym && use_browser > 0) {
+ if (bx->to.sym && use_browser == 1 && sort__has_sym) {
notes = symbol__annotation(bx->to.sym);
if (!notes->src
&& symbol__alloc_hist(bx->to.sym) < 0)
@@ -140,8 +140,8 @@
struct hist_entry *he;
if ((sort__has_parent || symbol_conf.use_callchain) && sample->callchain) {
- err = machine__resolve_callchain(machine, al->thread,
- sample->callchain, &parent);
+ err = machine__resolve_callchain(machine, evsel, al->thread,
+ sample, &parent);
if (err)
return err;
}
@@ -162,7 +162,7 @@
* so we don't allocated the extra space needed because the stdio
* code will not use it.
*/
- if (he->ms.sym != NULL && use_browser > 0) {
+ if (he->ms.sym != NULL && use_browser == 1 && sort__has_sym) {
struct annotation *notes = symbol__annotation(he->ms.sym);
assert(evsel != NULL);
@@ -223,9 +223,9 @@
static int process_read_event(struct perf_tool *tool,
union perf_event *event,
- struct perf_sample *sample __used,
+ struct perf_sample *sample __maybe_unused,
struct perf_evsel *evsel,
- struct machine *machine __used)
+ struct machine *machine __maybe_unused)
{
struct perf_report *rep = container_of(tool, struct perf_report, tool);
@@ -287,7 +287,7 @@
extern volatile int session_done;
-static void sig_handler(int sig __used)
+static void sig_handler(int sig __maybe_unused)
{
session_done = 1;
}
@@ -397,17 +397,17 @@
desc);
}
- if (dump_trace) {
- perf_session__fprintf_nr_events(session, stdout);
- goto out_delete;
- }
-
if (verbose > 3)
perf_session__fprintf(session, stdout);
if (verbose > 2)
perf_session__fprintf_dsos(session, stdout);
+ if (dump_trace) {
+ perf_session__fprintf_nr_events(session, stdout);
+ goto out_delete;
+ }
+
nr_samples = 0;
list_for_each_entry(pos, &session->evlist->entries, node) {
struct hists *hists = &pos->hists;
@@ -533,13 +533,14 @@
}
static int
-parse_branch_mode(const struct option *opt __used, const char *str __used, int unset)
+parse_branch_mode(const struct option *opt __maybe_unused,
+ const char *str __maybe_unused, int unset)
{
sort__branch_mode = !unset;
return 0;
}
-int cmd_report(int argc, const char **argv, const char *prefix __used)
+int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
{
struct perf_session *session;
struct stat st;
@@ -638,6 +639,8 @@
"Show a column with the sum of periods"),
OPT_CALLBACK_NOOPT('b', "branch-stack", &sort__branch_mode, "",
"use branch records for histogram filling", parse_branch_mode),
+ OPT_STRING(0, "objdump", &objdump_path, "path",
+ "objdump binary to use for disassembly and annotations"),
OPT_END()
};
@@ -686,15 +689,19 @@
if (strcmp(report.input_name, "-") != 0)
setup_browser(true);
- else
+ else {
use_browser = 0;
+ perf_hpp__init(false, false);
+ }
+
+ setup_sorting(report_usage, options);
/*
* Only in the newt browser we are doing integrated annotation,
* so don't allocate extra space that won't be used in the stdio
* implementation.
*/
- if (use_browser > 0) {
+ if (use_browser == 1 && sort__has_sym) {
symbol_conf.priv_size = sizeof(struct annotation);
report.annotate_init = symbol__annotate_init;
/*
@@ -717,8 +724,6 @@
if (symbol__init() < 0)
goto error;
- setup_sorting(report_usage, options);
-
if (parent_pattern != default_parent_pattern) {
if (sort_dimension__add("parent") < 0)
goto error;
diff --git a/tools/perf/builtin-sched.c b/tools/perf/builtin-sched.c
index 7a9ad2b..9b9e32e 100644
--- a/tools/perf/builtin-sched.c
+++ b/tools/perf/builtin-sched.c
@@ -23,31 +23,12 @@
#include <pthread.h>
#include <math.h>
-static const char *input_name;
-
-static char default_sort_order[] = "avg, max, switch, runtime";
-static const char *sort_order = default_sort_order;
-
-static int profile_cpu = -1;
-
#define PR_SET_NAME 15 /* Set process name */
#define MAX_CPUS 4096
-
-static u64 run_measurement_overhead;
-static u64 sleep_measurement_overhead;
-
#define COMM_LEN 20
#define SYM_LEN 129
-
#define MAX_PID 65536
-static unsigned long nr_tasks;
-
-struct perf_sched {
- struct perf_tool tool;
- struct perf_session *session;
-};
-
struct sched_atom;
struct task_desc {
@@ -85,44 +66,6 @@
struct task_desc *wakee;
};
-static struct task_desc *pid_to_task[MAX_PID];
-
-static struct task_desc **tasks;
-
-static pthread_mutex_t start_work_mutex = PTHREAD_MUTEX_INITIALIZER;
-static u64 start_time;
-
-static pthread_mutex_t work_done_wait_mutex = PTHREAD_MUTEX_INITIALIZER;
-
-static unsigned long nr_run_events;
-static unsigned long nr_sleep_events;
-static unsigned long nr_wakeup_events;
-
-static unsigned long nr_sleep_corrections;
-static unsigned long nr_run_events_optimized;
-
-static unsigned long targetless_wakeups;
-static unsigned long multitarget_wakeups;
-
-static u64 cpu_usage;
-static u64 runavg_cpu_usage;
-static u64 parent_cpu_usage;
-static u64 runavg_parent_cpu_usage;
-
-static unsigned long nr_runs;
-static u64 sum_runtime;
-static u64 sum_fluct;
-static u64 run_avg;
-
-static unsigned int replay_repeat = 10;
-static unsigned long nr_timestamps;
-static unsigned long nr_unordered_timestamps;
-static unsigned long nr_state_machine_bugs;
-static unsigned long nr_context_switch_bugs;
-static unsigned long nr_events;
-static unsigned long nr_lost_chunks;
-static unsigned long nr_lost_events;
-
#define TASK_STATE_TO_CHAR_STR "RSDTtZX"
enum thread_state {
@@ -154,11 +97,79 @@
typedef int (*sort_fn_t)(struct work_atoms *, struct work_atoms *);
-static struct rb_root atom_root, sorted_atom_root;
+struct perf_sched;
-static u64 all_runtime;
-static u64 all_count;
+struct trace_sched_handler {
+ int (*switch_event)(struct perf_sched *sched, struct perf_evsel *evsel,
+ struct perf_sample *sample, struct machine *machine);
+ int (*runtime_event)(struct perf_sched *sched, struct perf_evsel *evsel,
+ struct perf_sample *sample, struct machine *machine);
+
+ int (*wakeup_event)(struct perf_sched *sched, struct perf_evsel *evsel,
+ struct perf_sample *sample, struct machine *machine);
+
+ int (*fork_event)(struct perf_sched *sched, struct perf_evsel *evsel,
+ struct perf_sample *sample);
+
+ int (*migrate_task_event)(struct perf_sched *sched,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine);
+};
+
+struct perf_sched {
+ struct perf_tool tool;
+ const char *input_name;
+ const char *sort_order;
+ unsigned long nr_tasks;
+ struct task_desc *pid_to_task[MAX_PID];
+ struct task_desc **tasks;
+ const struct trace_sched_handler *tp_handler;
+ pthread_mutex_t start_work_mutex;
+ pthread_mutex_t work_done_wait_mutex;
+ int profile_cpu;
+/*
+ * Track the current task - that way we can know whether there's any
+ * weird events, such as a task being switched away that is not current.
+ */
+ int max_cpu;
+ u32 curr_pid[MAX_CPUS];
+ struct thread *curr_thread[MAX_CPUS];
+ char next_shortname1;
+ char next_shortname2;
+ unsigned int replay_repeat;
+ unsigned long nr_run_events;
+ unsigned long nr_sleep_events;
+ unsigned long nr_wakeup_events;
+ unsigned long nr_sleep_corrections;
+ unsigned long nr_run_events_optimized;
+ unsigned long targetless_wakeups;
+ unsigned long multitarget_wakeups;
+ unsigned long nr_runs;
+ unsigned long nr_timestamps;
+ unsigned long nr_unordered_timestamps;
+ unsigned long nr_state_machine_bugs;
+ unsigned long nr_context_switch_bugs;
+ unsigned long nr_events;
+ unsigned long nr_lost_chunks;
+ unsigned long nr_lost_events;
+ u64 run_measurement_overhead;
+ u64 sleep_measurement_overhead;
+ u64 start_time;
+ u64 cpu_usage;
+ u64 runavg_cpu_usage;
+ u64 parent_cpu_usage;
+ u64 runavg_parent_cpu_usage;
+ u64 sum_runtime;
+ u64 sum_fluct;
+ u64 run_avg;
+ u64 all_runtime;
+ u64 all_count;
+ u64 cpu_last_switched[MAX_CPUS];
+ struct rb_root atom_root, sorted_atom_root;
+ struct list_head sort_list, cmp_pid;
+};
static u64 get_nsecs(void)
{
@@ -169,13 +180,13 @@
return ts.tv_sec * 1000000000ULL + ts.tv_nsec;
}
-static void burn_nsecs(u64 nsecs)
+static void burn_nsecs(struct perf_sched *sched, u64 nsecs)
{
u64 T0 = get_nsecs(), T1;
do {
T1 = get_nsecs();
- } while (T1 + run_measurement_overhead < T0 + nsecs);
+ } while (T1 + sched->run_measurement_overhead < T0 + nsecs);
}
static void sleep_nsecs(u64 nsecs)
@@ -188,24 +199,24 @@
nanosleep(&ts, NULL);
}
-static void calibrate_run_measurement_overhead(void)
+static void calibrate_run_measurement_overhead(struct perf_sched *sched)
{
u64 T0, T1, delta, min_delta = 1000000000ULL;
int i;
for (i = 0; i < 10; i++) {
T0 = get_nsecs();
- burn_nsecs(0);
+ burn_nsecs(sched, 0);
T1 = get_nsecs();
delta = T1-T0;
min_delta = min(min_delta, delta);
}
- run_measurement_overhead = min_delta;
+ sched->run_measurement_overhead = min_delta;
printf("run measurement overhead: %" PRIu64 " nsecs\n", min_delta);
}
-static void calibrate_sleep_measurement_overhead(void)
+static void calibrate_sleep_measurement_overhead(struct perf_sched *sched)
{
u64 T0, T1, delta, min_delta = 1000000000ULL;
int i;
@@ -218,7 +229,7 @@
min_delta = min(min_delta, delta);
}
min_delta -= 10000;
- sleep_measurement_overhead = min_delta;
+ sched->sleep_measurement_overhead = min_delta;
printf("sleep measurement overhead: %" PRIu64 " nsecs\n", min_delta);
}
@@ -251,8 +262,8 @@
return task->atoms[task->nr_events - 1];
}
-static void
-add_sched_event_run(struct task_desc *task, u64 timestamp, u64 duration)
+static void add_sched_event_run(struct perf_sched *sched, struct task_desc *task,
+ u64 timestamp, u64 duration)
{
struct sched_atom *event, *curr_event = last_event(task);
@@ -261,7 +272,7 @@
* to it:
*/
if (curr_event && curr_event->type == SCHED_EVENT_RUN) {
- nr_run_events_optimized++;
+ sched->nr_run_events_optimized++;
curr_event->duration += duration;
return;
}
@@ -271,12 +282,11 @@
event->type = SCHED_EVENT_RUN;
event->duration = duration;
- nr_run_events++;
+ sched->nr_run_events++;
}
-static void
-add_sched_event_wakeup(struct task_desc *task, u64 timestamp,
- struct task_desc *wakee)
+static void add_sched_event_wakeup(struct perf_sched *sched, struct task_desc *task,
+ u64 timestamp, struct task_desc *wakee)
{
struct sched_atom *event, *wakee_event;
@@ -286,11 +296,11 @@
wakee_event = last_event(wakee);
if (!wakee_event || wakee_event->type != SCHED_EVENT_SLEEP) {
- targetless_wakeups++;
+ sched->targetless_wakeups++;
return;
}
if (wakee_event->wait_sem) {
- multitarget_wakeups++;
+ sched->multitarget_wakeups++;
return;
}
@@ -299,89 +309,89 @@
wakee_event->specific_wait = 1;
event->wait_sem = wakee_event->wait_sem;
- nr_wakeup_events++;
+ sched->nr_wakeup_events++;
}
-static void
-add_sched_event_sleep(struct task_desc *task, u64 timestamp,
- u64 task_state __used)
+static void add_sched_event_sleep(struct perf_sched *sched, struct task_desc *task,
+ u64 timestamp, u64 task_state __maybe_unused)
{
struct sched_atom *event = get_new_event(task, timestamp);
event->type = SCHED_EVENT_SLEEP;
- nr_sleep_events++;
+ sched->nr_sleep_events++;
}
-static struct task_desc *register_pid(unsigned long pid, const char *comm)
+static struct task_desc *register_pid(struct perf_sched *sched,
+ unsigned long pid, const char *comm)
{
struct task_desc *task;
BUG_ON(pid >= MAX_PID);
- task = pid_to_task[pid];
+ task = sched->pid_to_task[pid];
if (task)
return task;
task = zalloc(sizeof(*task));
task->pid = pid;
- task->nr = nr_tasks;
+ task->nr = sched->nr_tasks;
strcpy(task->comm, comm);
/*
* every task starts in sleeping state - this gets ignored
* if there's no wakeup pointing to this sleep state:
*/
- add_sched_event_sleep(task, 0, 0);
+ add_sched_event_sleep(sched, task, 0, 0);
- pid_to_task[pid] = task;
- nr_tasks++;
- tasks = realloc(tasks, nr_tasks*sizeof(struct task_task *));
- BUG_ON(!tasks);
- tasks[task->nr] = task;
+ sched->pid_to_task[pid] = task;
+ sched->nr_tasks++;
+ sched->tasks = realloc(sched->tasks, sched->nr_tasks * sizeof(struct task_task *));
+ BUG_ON(!sched->tasks);
+ sched->tasks[task->nr] = task;
if (verbose)
- printf("registered task #%ld, PID %ld (%s)\n", nr_tasks, pid, comm);
+ printf("registered task #%ld, PID %ld (%s)\n", sched->nr_tasks, pid, comm);
return task;
}
-static void print_task_traces(void)
+static void print_task_traces(struct perf_sched *sched)
{
struct task_desc *task;
unsigned long i;
- for (i = 0; i < nr_tasks; i++) {
- task = tasks[i];
+ for (i = 0; i < sched->nr_tasks; i++) {
+ task = sched->tasks[i];
printf("task %6ld (%20s:%10ld), nr_events: %ld\n",
task->nr, task->comm, task->pid, task->nr_events);
}
}
-static void add_cross_task_wakeups(void)
+static void add_cross_task_wakeups(struct perf_sched *sched)
{
struct task_desc *task1, *task2;
unsigned long i, j;
- for (i = 0; i < nr_tasks; i++) {
- task1 = tasks[i];
+ for (i = 0; i < sched->nr_tasks; i++) {
+ task1 = sched->tasks[i];
j = i + 1;
- if (j == nr_tasks)
+ if (j == sched->nr_tasks)
j = 0;
- task2 = tasks[j];
- add_sched_event_wakeup(task1, 0, task2);
+ task2 = sched->tasks[j];
+ add_sched_event_wakeup(sched, task1, 0, task2);
}
}
-static void
-process_sched_event(struct task_desc *this_task __used, struct sched_atom *atom)
+static void perf_sched__process_event(struct perf_sched *sched,
+ struct sched_atom *atom)
{
int ret = 0;
switch (atom->type) {
case SCHED_EVENT_RUN:
- burn_nsecs(atom->duration);
+ burn_nsecs(sched, atom->duration);
break;
case SCHED_EVENT_SLEEP:
if (atom->wait_sem)
@@ -428,8 +438,8 @@
fd = sys_perf_event_open(&attr, 0, -1, -1, 0);
if (fd < 0)
- die("Error: sys_perf_event_open() syscall returned"
- "with %d (%s)\n", fd, strerror(errno));
+ pr_err("Error: sys_perf_event_open() syscall returned "
+ "with %d (%s)\n", fd, strerror(errno));
return fd;
}
@@ -444,31 +454,41 @@
return runtime;
}
+struct sched_thread_parms {
+ struct task_desc *task;
+ struct perf_sched *sched;
+};
+
static void *thread_func(void *ctx)
{
- struct task_desc *this_task = ctx;
+ struct sched_thread_parms *parms = ctx;
+ struct task_desc *this_task = parms->task;
+ struct perf_sched *sched = parms->sched;
u64 cpu_usage_0, cpu_usage_1;
unsigned long i, ret;
char comm2[22];
int fd;
+ free(parms);
+
sprintf(comm2, ":%s", this_task->comm);
prctl(PR_SET_NAME, comm2);
fd = self_open_counters();
-
+ if (fd < 0)
+ return NULL;
again:
ret = sem_post(&this_task->ready_for_work);
BUG_ON(ret);
- ret = pthread_mutex_lock(&start_work_mutex);
+ ret = pthread_mutex_lock(&sched->start_work_mutex);
BUG_ON(ret);
- ret = pthread_mutex_unlock(&start_work_mutex);
+ ret = pthread_mutex_unlock(&sched->start_work_mutex);
BUG_ON(ret);
cpu_usage_0 = get_cpu_usage_nsec_self(fd);
for (i = 0; i < this_task->nr_events; i++) {
this_task->curr_event = i;
- process_sched_event(this_task, this_task->atoms[i]);
+ perf_sched__process_event(sched, this_task->atoms[i]);
}
cpu_usage_1 = get_cpu_usage_nsec_self(fd);
@@ -476,15 +496,15 @@
ret = sem_post(&this_task->work_done_sem);
BUG_ON(ret);
- ret = pthread_mutex_lock(&work_done_wait_mutex);
+ ret = pthread_mutex_lock(&sched->work_done_wait_mutex);
BUG_ON(ret);
- ret = pthread_mutex_unlock(&work_done_wait_mutex);
+ ret = pthread_mutex_unlock(&sched->work_done_wait_mutex);
BUG_ON(ret);
goto again;
}
-static void create_tasks(void)
+static void create_tasks(struct perf_sched *sched)
{
struct task_desc *task;
pthread_attr_t attr;
@@ -496,128 +516,129 @@
err = pthread_attr_setstacksize(&attr,
(size_t) max(16 * 1024, PTHREAD_STACK_MIN));
BUG_ON(err);
- err = pthread_mutex_lock(&start_work_mutex);
+ err = pthread_mutex_lock(&sched->start_work_mutex);
BUG_ON(err);
- err = pthread_mutex_lock(&work_done_wait_mutex);
+ err = pthread_mutex_lock(&sched->work_done_wait_mutex);
BUG_ON(err);
- for (i = 0; i < nr_tasks; i++) {
- task = tasks[i];
+ for (i = 0; i < sched->nr_tasks; i++) {
+ struct sched_thread_parms *parms = malloc(sizeof(*parms));
+ BUG_ON(parms == NULL);
+ parms->task = task = sched->tasks[i];
+ parms->sched = sched;
sem_init(&task->sleep_sem, 0, 0);
sem_init(&task->ready_for_work, 0, 0);
sem_init(&task->work_done_sem, 0, 0);
task->curr_event = 0;
- err = pthread_create(&task->thread, &attr, thread_func, task);
+ err = pthread_create(&task->thread, &attr, thread_func, parms);
BUG_ON(err);
}
}
-static void wait_for_tasks(void)
+static void wait_for_tasks(struct perf_sched *sched)
{
u64 cpu_usage_0, cpu_usage_1;
struct task_desc *task;
unsigned long i, ret;
- start_time = get_nsecs();
- cpu_usage = 0;
- pthread_mutex_unlock(&work_done_wait_mutex);
+ sched->start_time = get_nsecs();
+ sched->cpu_usage = 0;
+ pthread_mutex_unlock(&sched->work_done_wait_mutex);
- for (i = 0; i < nr_tasks; i++) {
- task = tasks[i];
+ for (i = 0; i < sched->nr_tasks; i++) {
+ task = sched->tasks[i];
ret = sem_wait(&task->ready_for_work);
BUG_ON(ret);
sem_init(&task->ready_for_work, 0, 0);
}
- ret = pthread_mutex_lock(&work_done_wait_mutex);
+ ret = pthread_mutex_lock(&sched->work_done_wait_mutex);
BUG_ON(ret);
cpu_usage_0 = get_cpu_usage_nsec_parent();
- pthread_mutex_unlock(&start_work_mutex);
+ pthread_mutex_unlock(&sched->start_work_mutex);
- for (i = 0; i < nr_tasks; i++) {
- task = tasks[i];
+ for (i = 0; i < sched->nr_tasks; i++) {
+ task = sched->tasks[i];
ret = sem_wait(&task->work_done_sem);
BUG_ON(ret);
sem_init(&task->work_done_sem, 0, 0);
- cpu_usage += task->cpu_usage;
+ sched->cpu_usage += task->cpu_usage;
task->cpu_usage = 0;
}
cpu_usage_1 = get_cpu_usage_nsec_parent();
- if (!runavg_cpu_usage)
- runavg_cpu_usage = cpu_usage;
- runavg_cpu_usage = (runavg_cpu_usage*9 + cpu_usage)/10;
+ if (!sched->runavg_cpu_usage)
+ sched->runavg_cpu_usage = sched->cpu_usage;
+ sched->runavg_cpu_usage = (sched->runavg_cpu_usage * 9 + sched->cpu_usage) / 10;
- parent_cpu_usage = cpu_usage_1 - cpu_usage_0;
- if (!runavg_parent_cpu_usage)
- runavg_parent_cpu_usage = parent_cpu_usage;
- runavg_parent_cpu_usage = (runavg_parent_cpu_usage*9 +
- parent_cpu_usage)/10;
+ sched->parent_cpu_usage = cpu_usage_1 - cpu_usage_0;
+ if (!sched->runavg_parent_cpu_usage)
+ sched->runavg_parent_cpu_usage = sched->parent_cpu_usage;
+ sched->runavg_parent_cpu_usage = (sched->runavg_parent_cpu_usage * 9 +
+ sched->parent_cpu_usage)/10;
- ret = pthread_mutex_lock(&start_work_mutex);
+ ret = pthread_mutex_lock(&sched->start_work_mutex);
BUG_ON(ret);
- for (i = 0; i < nr_tasks; i++) {
- task = tasks[i];
+ for (i = 0; i < sched->nr_tasks; i++) {
+ task = sched->tasks[i];
sem_init(&task->sleep_sem, 0, 0);
task->curr_event = 0;
}
}
-static void run_one_test(void)
+static void run_one_test(struct perf_sched *sched)
{
u64 T0, T1, delta, avg_delta, fluct;
T0 = get_nsecs();
- wait_for_tasks();
+ wait_for_tasks(sched);
T1 = get_nsecs();
delta = T1 - T0;
- sum_runtime += delta;
- nr_runs++;
+ sched->sum_runtime += delta;
+ sched->nr_runs++;
- avg_delta = sum_runtime / nr_runs;
+ avg_delta = sched->sum_runtime / sched->nr_runs;
if (delta < avg_delta)
fluct = avg_delta - delta;
else
fluct = delta - avg_delta;
- sum_fluct += fluct;
- if (!run_avg)
- run_avg = delta;
- run_avg = (run_avg*9 + delta)/10;
+ sched->sum_fluct += fluct;
+ if (!sched->run_avg)
+ sched->run_avg = delta;
+ sched->run_avg = (sched->run_avg * 9 + delta) / 10;
- printf("#%-3ld: %0.3f, ",
- nr_runs, (double)delta/1000000.0);
+ printf("#%-3ld: %0.3f, ", sched->nr_runs, (double)delta / 1000000.0);
- printf("ravg: %0.2f, ",
- (double)run_avg/1e6);
+ printf("ravg: %0.2f, ", (double)sched->run_avg / 1e6);
printf("cpu: %0.2f / %0.2f",
- (double)cpu_usage/1e6, (double)runavg_cpu_usage/1e6);
+ (double)sched->cpu_usage / 1e6, (double)sched->runavg_cpu_usage / 1e6);
#if 0
/*
* rusage statistics done by the parent, these are less
- * accurate than the sum_exec_runtime based statistics:
+ * accurate than the sched->sum_exec_runtime based statistics:
*/
printf(" [%0.2f / %0.2f]",
- (double)parent_cpu_usage/1e6,
- (double)runavg_parent_cpu_usage/1e6);
+ (double)sched->parent_cpu_usage/1e6,
+ (double)sched->runavg_parent_cpu_usage/1e6);
#endif
printf("\n");
- if (nr_sleep_corrections)
- printf(" (%ld sleep corrections)\n", nr_sleep_corrections);
- nr_sleep_corrections = 0;
+ if (sched->nr_sleep_corrections)
+ printf(" (%ld sleep corrections)\n", sched->nr_sleep_corrections);
+ sched->nr_sleep_corrections = 0;
}
-static void test_calibrations(void)
+static void test_calibrations(struct perf_sched *sched)
{
u64 T0, T1;
T0 = get_nsecs();
- burn_nsecs(1e6);
+ burn_nsecs(sched, 1e6);
T1 = get_nsecs();
printf("the run test took %" PRIu64 " nsecs\n", T1 - T0);
@@ -629,236 +650,92 @@
printf("the sleep test took %" PRIu64 " nsecs\n", T1 - T0);
}
-#define FILL_FIELD(ptr, field, event, data) \
- ptr.field = (typeof(ptr.field)) raw_field_value(event, #field, data)
-
-#define FILL_ARRAY(ptr, array, event, data) \
-do { \
- void *__array = raw_field_ptr(event, #array, data); \
- memcpy(ptr.array, __array, sizeof(ptr.array)); \
-} while(0)
-
-#define FILL_COMMON_FIELDS(ptr, event, data) \
-do { \
- FILL_FIELD(ptr, common_type, event, data); \
- FILL_FIELD(ptr, common_flags, event, data); \
- FILL_FIELD(ptr, common_preempt_count, event, data); \
- FILL_FIELD(ptr, common_pid, event, data); \
- FILL_FIELD(ptr, common_tgid, event, data); \
-} while (0)
-
-
-
-struct trace_switch_event {
- u32 size;
-
- u16 common_type;
- u8 common_flags;
- u8 common_preempt_count;
- u32 common_pid;
- u32 common_tgid;
-
- char prev_comm[16];
- u32 prev_pid;
- u32 prev_prio;
- u64 prev_state;
- char next_comm[16];
- u32 next_pid;
- u32 next_prio;
-};
-
-struct trace_runtime_event {
- u32 size;
-
- u16 common_type;
- u8 common_flags;
- u8 common_preempt_count;
- u32 common_pid;
- u32 common_tgid;
-
- char comm[16];
- u32 pid;
- u64 runtime;
- u64 vruntime;
-};
-
-struct trace_wakeup_event {
- u32 size;
-
- u16 common_type;
- u8 common_flags;
- u8 common_preempt_count;
- u32 common_pid;
- u32 common_tgid;
-
- char comm[16];
- u32 pid;
-
- u32 prio;
- u32 success;
- u32 cpu;
-};
-
-struct trace_fork_event {
- u32 size;
-
- u16 common_type;
- u8 common_flags;
- u8 common_preempt_count;
- u32 common_pid;
- u32 common_tgid;
-
- char parent_comm[16];
- u32 parent_pid;
- char child_comm[16];
- u32 child_pid;
-};
-
-struct trace_migrate_task_event {
- u32 size;
-
- u16 common_type;
- u8 common_flags;
- u8 common_preempt_count;
- u32 common_pid;
- u32 common_tgid;
-
- char comm[16];
- u32 pid;
-
- u32 prio;
- u32 cpu;
-};
-
-struct trace_sched_handler {
- void (*switch_event)(struct trace_switch_event *,
- struct machine *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
-
- void (*runtime_event)(struct trace_runtime_event *,
- struct machine *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
-
- void (*wakeup_event)(struct trace_wakeup_event *,
- struct machine *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
-
- void (*fork_event)(struct trace_fork_event *,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
-
- void (*migrate_task_event)(struct trace_migrate_task_event *,
- struct machine *machine,
- struct event_format *,
- int cpu,
- u64 timestamp,
- struct thread *thread);
-};
-
-
-static void
-replay_wakeup_event(struct trace_wakeup_event *wakeup_event,
- struct machine *machine __used,
- struct event_format *event,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int
+replay_wakeup_event(struct perf_sched *sched,
+ struct perf_evsel *evsel, struct perf_sample *sample,
+ struct machine *machine __maybe_unused)
{
+ const char *comm = perf_evsel__strval(evsel, sample, "comm");
+ const u32 pid = perf_evsel__intval(evsel, sample, "pid");
struct task_desc *waker, *wakee;
if (verbose) {
- printf("sched_wakeup event %p\n", event);
+ printf("sched_wakeup event %p\n", evsel);
- printf(" ... pid %d woke up %s/%d\n",
- wakeup_event->common_pid,
- wakeup_event->comm,
- wakeup_event->pid);
+ printf(" ... pid %d woke up %s/%d\n", sample->tid, comm, pid);
}
- waker = register_pid(wakeup_event->common_pid, "<unknown>");
- wakee = register_pid(wakeup_event->pid, wakeup_event->comm);
+ waker = register_pid(sched, sample->tid, "<unknown>");
+ wakee = register_pid(sched, pid, comm);
- add_sched_event_wakeup(waker, timestamp, wakee);
+ add_sched_event_wakeup(sched, waker, sample->time, wakee);
+ return 0;
}
-static u64 cpu_last_switched[MAX_CPUS];
-
-static void
-replay_switch_event(struct trace_switch_event *switch_event,
- struct machine *machine __used,
- struct event_format *event,
- int cpu,
- u64 timestamp,
- struct thread *thread __used)
+static int replay_switch_event(struct perf_sched *sched,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine __maybe_unused)
{
- struct task_desc *prev, __used *next;
- u64 timestamp0;
+ const char *prev_comm = perf_evsel__strval(evsel, sample, "prev_comm"),
+ *next_comm = perf_evsel__strval(evsel, sample, "next_comm");
+ const u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
+ const u64 prev_state = perf_evsel__intval(evsel, sample, "prev_state");
+ struct task_desc *prev, __maybe_unused *next;
+ u64 timestamp0, timestamp = sample->time;
+ int cpu = sample->cpu;
s64 delta;
if (verbose)
- printf("sched_switch event %p\n", event);
+ printf("sched_switch event %p\n", evsel);
if (cpu >= MAX_CPUS || cpu < 0)
- return;
+ return 0;
- timestamp0 = cpu_last_switched[cpu];
+ timestamp0 = sched->cpu_last_switched[cpu];
if (timestamp0)
delta = timestamp - timestamp0;
else
delta = 0;
- if (delta < 0)
- die("hm, delta: %" PRIu64 " < 0 ?\n", delta);
-
- if (verbose) {
- printf(" ... switch from %s/%d to %s/%d [ran %" PRIu64 " nsecs]\n",
- switch_event->prev_comm, switch_event->prev_pid,
- switch_event->next_comm, switch_event->next_pid,
- delta);
+ if (delta < 0) {
+ pr_err("hm, delta: %" PRIu64 " < 0 ?\n", delta);
+ return -1;
}
- prev = register_pid(switch_event->prev_pid, switch_event->prev_comm);
- next = register_pid(switch_event->next_pid, switch_event->next_comm);
+ pr_debug(" ... switch from %s/%d to %s/%d [ran %" PRIu64 " nsecs]\n",
+ prev_comm, prev_pid, next_comm, next_pid, delta);
- cpu_last_switched[cpu] = timestamp;
+ prev = register_pid(sched, prev_pid, prev_comm);
+ next = register_pid(sched, next_pid, next_comm);
- add_sched_event_run(prev, timestamp, delta);
- add_sched_event_sleep(prev, timestamp, switch_event->prev_state);
+ sched->cpu_last_switched[cpu] = timestamp;
+
+ add_sched_event_run(sched, prev, timestamp, delta);
+ add_sched_event_sleep(sched, prev, timestamp, prev_state);
+
+ return 0;
}
-
-static void
-replay_fork_event(struct trace_fork_event *fork_event,
- struct event_format *event,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int replay_fork_event(struct perf_sched *sched, struct perf_evsel *evsel,
+ struct perf_sample *sample)
{
- if (verbose) {
- printf("sched_fork event %p\n", event);
- printf("... parent: %s/%d\n", fork_event->parent_comm, fork_event->parent_pid);
- printf("... child: %s/%d\n", fork_event->child_comm, fork_event->child_pid);
- }
- register_pid(fork_event->parent_pid, fork_event->parent_comm);
- register_pid(fork_event->child_pid, fork_event->child_comm);
-}
+ const char *parent_comm = perf_evsel__strval(evsel, sample, "parent_comm"),
+ *child_comm = perf_evsel__strval(evsel, sample, "child_comm");
+ const u32 parent_pid = perf_evsel__intval(evsel, sample, "parent_pid"),
+ child_pid = perf_evsel__intval(evsel, sample, "child_pid");
-static struct trace_sched_handler replay_ops = {
- .wakeup_event = replay_wakeup_event,
- .switch_event = replay_switch_event,
- .fork_event = replay_fork_event,
-};
+ if (verbose) {
+ printf("sched_fork event %p\n", evsel);
+ printf("... parent: %s/%d\n", parent_comm, parent_pid);
+ printf("... child: %s/%d\n", child_comm, child_pid);
+ }
+
+ register_pid(sched, parent_pid, parent_comm);
+ register_pid(sched, child_pid, child_comm);
+ return 0;
+}
struct sort_dimension {
const char *name;
@@ -866,8 +743,6 @@
struct list_head list;
};
-static LIST_HEAD(cmp_pid);
-
static int
thread_lat_cmp(struct list_head *list, struct work_atoms *l, struct work_atoms *r)
{
@@ -936,43 +811,45 @@
rb_insert_color(&data->node, root);
}
-static void thread_atoms_insert(struct thread *thread)
+static int thread_atoms_insert(struct perf_sched *sched, struct thread *thread)
{
struct work_atoms *atoms = zalloc(sizeof(*atoms));
- if (!atoms)
- die("No memory");
+ if (!atoms) {
+ pr_err("No memory at %s\n", __func__);
+ return -1;
+ }
atoms->thread = thread;
INIT_LIST_HEAD(&atoms->work_list);
- __thread_latency_insert(&atom_root, atoms, &cmp_pid);
+ __thread_latency_insert(&sched->atom_root, atoms, &sched->cmp_pid);
+ return 0;
}
-static void
-latency_fork_event(struct trace_fork_event *fork_event __used,
- struct event_format *event __used,
- int cpu __used,
- u64 timestamp __used,
- struct thread *thread __used)
+static int latency_fork_event(struct perf_sched *sched __maybe_unused,
+ struct perf_evsel *evsel __maybe_unused,
+ struct perf_sample *sample __maybe_unused)
{
/* should insert the newcomer */
+ return 0;
}
-__used
-static char sched_out_state(struct trace_switch_event *switch_event)
+static char sched_out_state(u64 prev_state)
{
const char *str = TASK_STATE_TO_CHAR_STR;
- return str[switch_event->prev_state];
+ return str[prev_state];
}
-static void
+static int
add_sched_out_event(struct work_atoms *atoms,
char run_state,
u64 timestamp)
{
struct work_atom *atom = zalloc(sizeof(*atom));
- if (!atom)
- die("Non memory");
+ if (!atom) {
+ pr_err("Non memory at %s", __func__);
+ return -1;
+ }
atom->sched_out_time = timestamp;
@@ -982,10 +859,12 @@
}
list_add_tail(&atom->list, &atoms->work_list);
+ return 0;
}
static void
-add_runtime_event(struct work_atoms *atoms, u64 delta, u64 timestamp __used)
+add_runtime_event(struct work_atoms *atoms, u64 delta,
+ u64 timestamp __maybe_unused)
{
struct work_atom *atom;
@@ -1028,106 +907,128 @@
atoms->nb_atoms++;
}
-static void
-latency_switch_event(struct trace_switch_event *switch_event,
- struct machine *machine,
- struct event_format *event __used,
- int cpu,
- u64 timestamp,
- struct thread *thread __used)
+static int latency_switch_event(struct perf_sched *sched,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
+ const u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
+ const u64 prev_state = perf_evsel__intval(evsel, sample, "prev_state");
struct work_atoms *out_events, *in_events;
struct thread *sched_out, *sched_in;
- u64 timestamp0;
+ u64 timestamp0, timestamp = sample->time;
+ int cpu = sample->cpu;
s64 delta;
BUG_ON(cpu >= MAX_CPUS || cpu < 0);
- timestamp0 = cpu_last_switched[cpu];
- cpu_last_switched[cpu] = timestamp;
+ timestamp0 = sched->cpu_last_switched[cpu];
+ sched->cpu_last_switched[cpu] = timestamp;
if (timestamp0)
delta = timestamp - timestamp0;
else
delta = 0;
- if (delta < 0)
- die("hm, delta: %" PRIu64 " < 0 ?\n", delta);
-
-
- sched_out = machine__findnew_thread(machine, switch_event->prev_pid);
- sched_in = machine__findnew_thread(machine, switch_event->next_pid);
-
- out_events = thread_atoms_search(&atom_root, sched_out, &cmp_pid);
- if (!out_events) {
- thread_atoms_insert(sched_out);
- out_events = thread_atoms_search(&atom_root, sched_out, &cmp_pid);
- if (!out_events)
- die("out-event: Internal tree error");
+ if (delta < 0) {
+ pr_err("hm, delta: %" PRIu64 " < 0 ?\n", delta);
+ return -1;
}
- add_sched_out_event(out_events, sched_out_state(switch_event), timestamp);
- in_events = thread_atoms_search(&atom_root, sched_in, &cmp_pid);
+ sched_out = machine__findnew_thread(machine, prev_pid);
+ sched_in = machine__findnew_thread(machine, next_pid);
+
+ out_events = thread_atoms_search(&sched->atom_root, sched_out, &sched->cmp_pid);
+ if (!out_events) {
+ if (thread_atoms_insert(sched, sched_out))
+ return -1;
+ out_events = thread_atoms_search(&sched->atom_root, sched_out, &sched->cmp_pid);
+ if (!out_events) {
+ pr_err("out-event: Internal tree error");
+ return -1;
+ }
+ }
+ if (add_sched_out_event(out_events, sched_out_state(prev_state), timestamp))
+ return -1;
+
+ in_events = thread_atoms_search(&sched->atom_root, sched_in, &sched->cmp_pid);
if (!in_events) {
- thread_atoms_insert(sched_in);
- in_events = thread_atoms_search(&atom_root, sched_in, &cmp_pid);
- if (!in_events)
- die("in-event: Internal tree error");
+ if (thread_atoms_insert(sched, sched_in))
+ return -1;
+ in_events = thread_atoms_search(&sched->atom_root, sched_in, &sched->cmp_pid);
+ if (!in_events) {
+ pr_err("in-event: Internal tree error");
+ return -1;
+ }
/*
* Take came in we have not heard about yet,
* add in an initial atom in runnable state:
*/
- add_sched_out_event(in_events, 'R', timestamp);
+ if (add_sched_out_event(in_events, 'R', timestamp))
+ return -1;
}
add_sched_in_event(in_events, timestamp);
+
+ return 0;
}
-static void
-latency_runtime_event(struct trace_runtime_event *runtime_event,
- struct machine *machine,
- struct event_format *event __used,
- int cpu,
- u64 timestamp,
- struct thread *this_thread __used)
+static int latency_runtime_event(struct perf_sched *sched,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
- struct thread *thread = machine__findnew_thread(machine, runtime_event->pid);
- struct work_atoms *atoms = thread_atoms_search(&atom_root, thread, &cmp_pid);
+ const u32 pid = perf_evsel__intval(evsel, sample, "pid");
+ const u64 runtime = perf_evsel__intval(evsel, sample, "runtime");
+ struct thread *thread = machine__findnew_thread(machine, pid);
+ struct work_atoms *atoms = thread_atoms_search(&sched->atom_root, thread, &sched->cmp_pid);
+ u64 timestamp = sample->time;
+ int cpu = sample->cpu;
BUG_ON(cpu >= MAX_CPUS || cpu < 0);
if (!atoms) {
- thread_atoms_insert(thread);
- atoms = thread_atoms_search(&atom_root, thread, &cmp_pid);
- if (!atoms)
- die("in-event: Internal tree error");
- add_sched_out_event(atoms, 'R', timestamp);
+ if (thread_atoms_insert(sched, thread))
+ return -1;
+ atoms = thread_atoms_search(&sched->atom_root, thread, &sched->cmp_pid);
+ if (!atoms) {
+ pr_err("in-event: Internal tree error");
+ return -1;
+ }
+ if (add_sched_out_event(atoms, 'R', timestamp))
+ return -1;
}
- add_runtime_event(atoms, runtime_event->runtime, timestamp);
+ add_runtime_event(atoms, runtime, timestamp);
+ return 0;
}
-static void
-latency_wakeup_event(struct trace_wakeup_event *wakeup_event,
- struct machine *machine,
- struct event_format *__event __used,
- int cpu __used,
- u64 timestamp,
- struct thread *thread __used)
+static int latency_wakeup_event(struct perf_sched *sched,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
+ const u32 pid = perf_evsel__intval(evsel, sample, "pid"),
+ success = perf_evsel__intval(evsel, sample, "success");
struct work_atoms *atoms;
struct work_atom *atom;
struct thread *wakee;
+ u64 timestamp = sample->time;
/* Note for later, it may be interesting to observe the failing cases */
- if (!wakeup_event->success)
- return;
+ if (!success)
+ return 0;
- wakee = machine__findnew_thread(machine, wakeup_event->pid);
- atoms = thread_atoms_search(&atom_root, wakee, &cmp_pid);
+ wakee = machine__findnew_thread(machine, pid);
+ atoms = thread_atoms_search(&sched->atom_root, wakee, &sched->cmp_pid);
if (!atoms) {
- thread_atoms_insert(wakee);
- atoms = thread_atoms_search(&atom_root, wakee, &cmp_pid);
- if (!atoms)
- die("wakeup-event: Internal tree error");
- add_sched_out_event(atoms, 'S', timestamp);
+ if (thread_atoms_insert(sched, wakee))
+ return -1;
+ atoms = thread_atoms_search(&sched->atom_root, wakee, &sched->cmp_pid);
+ if (!atoms) {
+ pr_err("wakeup-event: Internal tree error");
+ return -1;
+ }
+ if (add_sched_out_event(atoms, 'S', timestamp))
+ return -1;
}
BUG_ON(list_empty(&atoms->work_list));
@@ -1139,27 +1040,27 @@
* one CPU, or are only looking at only one, so don't
* make useless noise.
*/
- if (profile_cpu == -1 && atom->state != THREAD_SLEEPING)
- nr_state_machine_bugs++;
+ if (sched->profile_cpu == -1 && atom->state != THREAD_SLEEPING)
+ sched->nr_state_machine_bugs++;
- nr_timestamps++;
+ sched->nr_timestamps++;
if (atom->sched_out_time > timestamp) {
- nr_unordered_timestamps++;
- return;
+ sched->nr_unordered_timestamps++;
+ return 0;
}
atom->state = THREAD_WAIT_CPU;
atom->wake_up_time = timestamp;
+ return 0;
}
-static void
-latency_migrate_task_event(struct trace_migrate_task_event *migrate_task_event,
- struct machine *machine,
- struct event_format *__event __used,
- int cpu __used,
- u64 timestamp,
- struct thread *thread __used)
+static int latency_migrate_task_event(struct perf_sched *sched,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
+ const u32 pid = perf_evsel__intval(evsel, sample, "pid");
+ u64 timestamp = sample->time;
struct work_atoms *atoms;
struct work_atom *atom;
struct thread *migrant;
@@ -1167,18 +1068,22 @@
/*
* Only need to worry about migration when profiling one CPU.
*/
- if (profile_cpu == -1)
- return;
+ if (sched->profile_cpu == -1)
+ return 0;
- migrant = machine__findnew_thread(machine, migrate_task_event->pid);
- atoms = thread_atoms_search(&atom_root, migrant, &cmp_pid);
+ migrant = machine__findnew_thread(machine, pid);
+ atoms = thread_atoms_search(&sched->atom_root, migrant, &sched->cmp_pid);
if (!atoms) {
- thread_atoms_insert(migrant);
- register_pid(migrant->pid, migrant->comm);
- atoms = thread_atoms_search(&atom_root, migrant, &cmp_pid);
- if (!atoms)
- die("migration-event: Internal tree error");
- add_sched_out_event(atoms, 'R', timestamp);
+ if (thread_atoms_insert(sched, migrant))
+ return -1;
+ register_pid(sched, migrant->pid, migrant->comm);
+ atoms = thread_atoms_search(&sched->atom_root, migrant, &sched->cmp_pid);
+ if (!atoms) {
+ pr_err("migration-event: Internal tree error");
+ return -1;
+ }
+ if (add_sched_out_event(atoms, 'R', timestamp))
+ return -1;
}
BUG_ON(list_empty(&atoms->work_list));
@@ -1186,21 +1091,15 @@
atom = list_entry(atoms->work_list.prev, struct work_atom, list);
atom->sched_in_time = atom->sched_out_time = atom->wake_up_time = timestamp;
- nr_timestamps++;
+ sched->nr_timestamps++;
if (atom->sched_out_time > timestamp)
- nr_unordered_timestamps++;
+ sched->nr_unordered_timestamps++;
+
+ return 0;
}
-static struct trace_sched_handler lat_ops = {
- .wakeup_event = latency_wakeup_event,
- .switch_event = latency_switch_event,
- .runtime_event = latency_runtime_event,
- .fork_event = latency_fork_event,
- .migrate_task_event = latency_migrate_task_event,
-};
-
-static void output_lat_thread(struct work_atoms *work_list)
+static void output_lat_thread(struct perf_sched *sched, struct work_atoms *work_list)
{
int i;
int ret;
@@ -1214,8 +1113,8 @@
if (!strcmp(work_list->thread->comm, "swapper"))
return;
- all_runtime += work_list->total_runtime;
- all_count += work_list->nb_atoms;
+ sched->all_runtime += work_list->total_runtime;
+ sched->all_count += work_list->nb_atoms;
ret = printf(" %s:%d ", work_list->thread->comm, work_list->thread->pid);
@@ -1241,11 +1140,6 @@
return 0;
}
-static struct sort_dimension pid_sort_dimension = {
- .name = "pid",
- .cmp = pid_cmp,
-};
-
static int avg_cmp(struct work_atoms *l, struct work_atoms *r)
{
u64 avgl, avgr;
@@ -1267,11 +1161,6 @@
return 0;
}
-static struct sort_dimension avg_sort_dimension = {
- .name = "avg",
- .cmp = avg_cmp,
-};
-
static int max_cmp(struct work_atoms *l, struct work_atoms *r)
{
if (l->max_lat < r->max_lat)
@@ -1282,11 +1171,6 @@
return 0;
}
-static struct sort_dimension max_sort_dimension = {
- .name = "max",
- .cmp = max_cmp,
-};
-
static int switch_cmp(struct work_atoms *l, struct work_atoms *r)
{
if (l->nb_atoms < r->nb_atoms)
@@ -1297,11 +1181,6 @@
return 0;
}
-static struct sort_dimension switch_sort_dimension = {
- .name = "switch",
- .cmp = switch_cmp,
-};
-
static int runtime_cmp(struct work_atoms *l, struct work_atoms *r)
{
if (l->total_runtime < r->total_runtime)
@@ -1312,28 +1191,38 @@
return 0;
}
-static struct sort_dimension runtime_sort_dimension = {
- .name = "runtime",
- .cmp = runtime_cmp,
-};
-
-static struct sort_dimension *available_sorts[] = {
- &pid_sort_dimension,
- &avg_sort_dimension,
- &max_sort_dimension,
- &switch_sort_dimension,
- &runtime_sort_dimension,
-};
-
-#define NB_AVAILABLE_SORTS (int)(sizeof(available_sorts) / sizeof(struct sort_dimension *))
-
-static LIST_HEAD(sort_list);
-
static int sort_dimension__add(const char *tok, struct list_head *list)
{
- int i;
+ size_t i;
+ static struct sort_dimension avg_sort_dimension = {
+ .name = "avg",
+ .cmp = avg_cmp,
+ };
+ static struct sort_dimension max_sort_dimension = {
+ .name = "max",
+ .cmp = max_cmp,
+ };
+ static struct sort_dimension pid_sort_dimension = {
+ .name = "pid",
+ .cmp = pid_cmp,
+ };
+ static struct sort_dimension runtime_sort_dimension = {
+ .name = "runtime",
+ .cmp = runtime_cmp,
+ };
+ static struct sort_dimension switch_sort_dimension = {
+ .name = "switch",
+ .cmp = switch_cmp,
+ };
+ struct sort_dimension *available_sorts[] = {
+ &pid_sort_dimension,
+ &avg_sort_dimension,
+ &max_sort_dimension,
+ &switch_sort_dimension,
+ &runtime_sort_dimension,
+ };
- for (i = 0; i < NB_AVAILABLE_SORTS; i++) {
+ for (i = 0; i < ARRAY_SIZE(available_sorts); i++) {
if (!strcmp(available_sorts[i]->name, tok)) {
list_add_tail(&available_sorts[i]->list, list);
@@ -1344,126 +1233,97 @@
return -1;
}
-static void setup_sorting(void);
-
-static void sort_lat(void)
+static void perf_sched__sort_lat(struct perf_sched *sched)
{
struct rb_node *node;
for (;;) {
struct work_atoms *data;
- node = rb_first(&atom_root);
+ node = rb_first(&sched->atom_root);
if (!node)
break;
- rb_erase(node, &atom_root);
+ rb_erase(node, &sched->atom_root);
data = rb_entry(node, struct work_atoms, node);
- __thread_latency_insert(&sorted_atom_root, data, &sort_list);
+ __thread_latency_insert(&sched->sorted_atom_root, data, &sched->sort_list);
}
}
-static struct trace_sched_handler *trace_handler;
-
-static void
-process_sched_wakeup_event(struct perf_tool *tool __used,
- struct event_format *event,
- struct perf_sample *sample,
- struct machine *machine,
- struct thread *thread)
+static int process_sched_wakeup_event(struct perf_tool *tool,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
- void *data = sample->raw_data;
- struct trace_wakeup_event wakeup_event;
+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
- FILL_COMMON_FIELDS(wakeup_event, event, data);
+ if (sched->tp_handler->wakeup_event)
+ return sched->tp_handler->wakeup_event(sched, evsel, sample, machine);
- FILL_ARRAY(wakeup_event, comm, event, data);
- FILL_FIELD(wakeup_event, pid, event, data);
- FILL_FIELD(wakeup_event, prio, event, data);
- FILL_FIELD(wakeup_event, success, event, data);
- FILL_FIELD(wakeup_event, cpu, event, data);
-
- if (trace_handler->wakeup_event)
- trace_handler->wakeup_event(&wakeup_event, machine, event,
- sample->cpu, sample->time, thread);
+ return 0;
}
-/*
- * Track the current task - that way we can know whether there's any
- * weird events, such as a task being switched away that is not current.
- */
-static int max_cpu;
-
-static u32 curr_pid[MAX_CPUS] = { [0 ... MAX_CPUS-1] = -1 };
-
-static struct thread *curr_thread[MAX_CPUS];
-
-static char next_shortname1 = 'A';
-static char next_shortname2 = '0';
-
-static void
-map_switch_event(struct trace_switch_event *switch_event,
- struct machine *machine,
- struct event_format *event __used,
- int this_cpu,
- u64 timestamp,
- struct thread *thread __used)
+static int map_switch_event(struct perf_sched *sched, struct perf_evsel *evsel,
+ struct perf_sample *sample, struct machine *machine)
{
- struct thread *sched_out __used, *sched_in;
+ const u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
+ struct thread *sched_out __maybe_unused, *sched_in;
int new_shortname;
- u64 timestamp0;
+ u64 timestamp0, timestamp = sample->time;
s64 delta;
- int cpu;
+ int cpu, this_cpu = sample->cpu;
BUG_ON(this_cpu >= MAX_CPUS || this_cpu < 0);
- if (this_cpu > max_cpu)
- max_cpu = this_cpu;
+ if (this_cpu > sched->max_cpu)
+ sched->max_cpu = this_cpu;
- timestamp0 = cpu_last_switched[this_cpu];
- cpu_last_switched[this_cpu] = timestamp;
+ timestamp0 = sched->cpu_last_switched[this_cpu];
+ sched->cpu_last_switched[this_cpu] = timestamp;
if (timestamp0)
delta = timestamp - timestamp0;
else
delta = 0;
- if (delta < 0)
- die("hm, delta: %" PRIu64 " < 0 ?\n", delta);
+ if (delta < 0) {
+ pr_err("hm, delta: %" PRIu64 " < 0 ?\n", delta);
+ return -1;
+ }
+ sched_out = machine__findnew_thread(machine, prev_pid);
+ sched_in = machine__findnew_thread(machine, next_pid);
- sched_out = machine__findnew_thread(machine, switch_event->prev_pid);
- sched_in = machine__findnew_thread(machine, switch_event->next_pid);
-
- curr_thread[this_cpu] = sched_in;
+ sched->curr_thread[this_cpu] = sched_in;
printf(" ");
new_shortname = 0;
if (!sched_in->shortname[0]) {
- sched_in->shortname[0] = next_shortname1;
- sched_in->shortname[1] = next_shortname2;
+ sched_in->shortname[0] = sched->next_shortname1;
+ sched_in->shortname[1] = sched->next_shortname2;
- if (next_shortname1 < 'Z') {
- next_shortname1++;
+ if (sched->next_shortname1 < 'Z') {
+ sched->next_shortname1++;
} else {
- next_shortname1='A';
- if (next_shortname2 < '9') {
- next_shortname2++;
+ sched->next_shortname1='A';
+ if (sched->next_shortname2 < '9') {
+ sched->next_shortname2++;
} else {
- next_shortname2='0';
+ sched->next_shortname2='0';
}
}
new_shortname = 1;
}
- for (cpu = 0; cpu <= max_cpu; cpu++) {
+ for (cpu = 0; cpu <= sched->max_cpu; cpu++) {
if (cpu != this_cpu)
printf(" ");
else
printf("*");
- if (curr_thread[cpu]) {
- if (curr_thread[cpu]->pid)
- printf("%2s ", curr_thread[cpu]->shortname);
+ if (sched->curr_thread[cpu]) {
+ if (sched->curr_thread[cpu]->pid)
+ printf("%2s ", sched->curr_thread[cpu]->shortname);
else
printf(". ");
} else
@@ -1477,134 +1337,97 @@
} else {
printf("\n");
}
+
+ return 0;
}
-static void
-process_sched_switch_event(struct perf_tool *tool __used,
- struct event_format *event,
- struct perf_sample *sample,
- struct machine *machine,
- struct thread *thread)
+static int process_sched_switch_event(struct perf_tool *tool,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
- int this_cpu = sample->cpu;
- void *data = sample->raw_data;
- struct trace_switch_event switch_event;
+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
+ int this_cpu = sample->cpu, err = 0;
+ u32 prev_pid = perf_evsel__intval(evsel, sample, "prev_pid"),
+ next_pid = perf_evsel__intval(evsel, sample, "next_pid");
- FILL_COMMON_FIELDS(switch_event, event, data);
-
- FILL_ARRAY(switch_event, prev_comm, event, data);
- FILL_FIELD(switch_event, prev_pid, event, data);
- FILL_FIELD(switch_event, prev_prio, event, data);
- FILL_FIELD(switch_event, prev_state, event, data);
- FILL_ARRAY(switch_event, next_comm, event, data);
- FILL_FIELD(switch_event, next_pid, event, data);
- FILL_FIELD(switch_event, next_prio, event, data);
-
- if (curr_pid[this_cpu] != (u32)-1) {
+ if (sched->curr_pid[this_cpu] != (u32)-1) {
/*
* Are we trying to switch away a PID that is
* not current?
*/
- if (curr_pid[this_cpu] != switch_event.prev_pid)
- nr_context_switch_bugs++;
+ if (sched->curr_pid[this_cpu] != prev_pid)
+ sched->nr_context_switch_bugs++;
}
- if (trace_handler->switch_event)
- trace_handler->switch_event(&switch_event, machine, event,
- this_cpu, sample->time, thread);
- curr_pid[this_cpu] = switch_event.next_pid;
+ if (sched->tp_handler->switch_event)
+ err = sched->tp_handler->switch_event(sched, evsel, sample, machine);
+
+ sched->curr_pid[this_cpu] = next_pid;
+ return err;
}
-static void
-process_sched_runtime_event(struct perf_tool *tool __used,
- struct event_format *event,
- struct perf_sample *sample,
- struct machine *machine,
- struct thread *thread)
+static int process_sched_runtime_event(struct perf_tool *tool,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
- void *data = sample->raw_data;
- struct trace_runtime_event runtime_event;
+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
- FILL_ARRAY(runtime_event, comm, event, data);
- FILL_FIELD(runtime_event, pid, event, data);
- FILL_FIELD(runtime_event, runtime, event, data);
- FILL_FIELD(runtime_event, vruntime, event, data);
+ if (sched->tp_handler->runtime_event)
+ return sched->tp_handler->runtime_event(sched, evsel, sample, machine);
- if (trace_handler->runtime_event)
- trace_handler->runtime_event(&runtime_event, machine, event,
- sample->cpu, sample->time, thread);
+ return 0;
}
-static void
-process_sched_fork_event(struct perf_tool *tool __used,
- struct event_format *event,
- struct perf_sample *sample,
- struct machine *machine __used,
- struct thread *thread)
+static int process_sched_fork_event(struct perf_tool *tool,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine __maybe_unused)
{
- void *data = sample->raw_data;
- struct trace_fork_event fork_event;
+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
- FILL_COMMON_FIELDS(fork_event, event, data);
+ if (sched->tp_handler->fork_event)
+ return sched->tp_handler->fork_event(sched, evsel, sample);
- FILL_ARRAY(fork_event, parent_comm, event, data);
- FILL_FIELD(fork_event, parent_pid, event, data);
- FILL_ARRAY(fork_event, child_comm, event, data);
- FILL_FIELD(fork_event, child_pid, event, data);
-
- if (trace_handler->fork_event)
- trace_handler->fork_event(&fork_event, event,
- sample->cpu, sample->time, thread);
+ return 0;
}
-static void
-process_sched_exit_event(struct perf_tool *tool __used,
- struct event_format *event,
- struct perf_sample *sample __used,
- struct machine *machine __used,
- struct thread *thread __used)
+static int process_sched_exit_event(struct perf_tool *tool __maybe_unused,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
- if (verbose)
- printf("sched_exit event %p\n", event);
+ pr_debug("sched_exit event %p\n", evsel);
+ return 0;
}
-static void
-process_sched_migrate_task_event(struct perf_tool *tool __used,
- struct event_format *event,
- struct perf_sample *sample,
- struct machine *machine,
- struct thread *thread)
+static int process_sched_migrate_task_event(struct perf_tool *tool,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine)
{
- void *data = sample->raw_data;
- struct trace_migrate_task_event migrate_task_event;
+ struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
- FILL_COMMON_FIELDS(migrate_task_event, event, data);
+ if (sched->tp_handler->migrate_task_event)
+ return sched->tp_handler->migrate_task_event(sched, evsel, sample, machine);
- FILL_ARRAY(migrate_task_event, comm, event, data);
- FILL_FIELD(migrate_task_event, pid, event, data);
- FILL_FIELD(migrate_task_event, prio, event, data);
- FILL_FIELD(migrate_task_event, cpu, event, data);
-
- if (trace_handler->migrate_task_event)
- trace_handler->migrate_task_event(&migrate_task_event, machine,
- event, sample->cpu,
- sample->time, thread);
+ return 0;
}
-typedef void (*tracepoint_handler)(struct perf_tool *tool, struct event_format *event,
- struct perf_sample *sample,
- struct machine *machine,
- struct thread *thread);
+typedef int (*tracepoint_handler)(struct perf_tool *tool,
+ struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ struct machine *machine);
-static int perf_sched__process_tracepoint_sample(struct perf_tool *tool,
- union perf_event *event __used,
+static int perf_sched__process_tracepoint_sample(struct perf_tool *tool __maybe_unused,
+ union perf_event *event __maybe_unused,
struct perf_sample *sample,
struct perf_evsel *evsel,
struct machine *machine)
{
- struct perf_sched *sched = container_of(tool, struct perf_sched, tool);
- struct pevent *pevent = sched->session->pevent;
struct thread *thread = machine__findnew_thread(machine, sample->pid);
+ int err = 0;
if (thread == NULL) {
pr_debug("problem processing %s event, skipping it.\n",
@@ -1617,30 +1440,15 @@
if (evsel->handler.func != NULL) {
tracepoint_handler f = evsel->handler.func;
-
- if (evsel->handler.data == NULL)
- evsel->handler.data = pevent_find_event(pevent,
- evsel->attr.config);
-
- f(tool, evsel->handler.data, sample, machine, thread);
+ err = f(tool, evsel, sample, machine);
}
- return 0;
+ return err;
}
-static struct perf_sched perf_sched = {
- .tool = {
- .sample = perf_sched__process_tracepoint_sample,
- .comm = perf_event__process_comm,
- .lost = perf_event__process_lost,
- .fork = perf_event__process_task,
- .ordered_samples = true,
- },
-};
-
-static void read_events(bool destroy, struct perf_session **psession)
+static int perf_sched__read_events(struct perf_sched *sched, bool destroy,
+ struct perf_session **psession)
{
- int err = -EINVAL;
const struct perf_evsel_str_handler handlers[] = {
{ "sched:sched_switch", process_sched_switch_event, },
{ "sched:sched_stat_runtime", process_sched_runtime_event, },
@@ -1652,24 +1460,25 @@
};
struct perf_session *session;
- session = perf_session__new(input_name, O_RDONLY, 0, false,
- &perf_sched.tool);
- if (session == NULL)
- die("No Memory");
+ session = perf_session__new(sched->input_name, O_RDONLY, 0, false, &sched->tool);
+ if (session == NULL) {
+ pr_debug("No Memory for session\n");
+ return -1;
+ }
- perf_sched.session = session;
-
- err = perf_session__set_tracepoints_handlers(session, handlers);
- assert(err == 0);
+ if (perf_session__set_tracepoints_handlers(session, handlers))
+ goto out_delete;
if (perf_session__has_traces(session, "record -R")) {
- err = perf_session__process_events(session, &perf_sched.tool);
- if (err)
- die("Failed to process events, error %d", err);
+ int err = perf_session__process_events(session, &sched->tool);
+ if (err) {
+ pr_err("Failed to process events, error %d", err);
+ goto out_delete;
+ }
- nr_events = session->hists.stats.nr_events[0];
- nr_lost_events = session->hists.stats.total_lost;
- nr_lost_chunks = session->hists.stats.nr_events[PERF_RECORD_LOST];
+ sched->nr_events = session->hists.stats.nr_events[0];
+ sched->nr_lost_events = session->hists.stats.total_lost;
+ sched->nr_lost_chunks = session->hists.stats.nr_events[PERF_RECORD_LOST];
}
if (destroy)
@@ -1677,208 +1486,166 @@
if (psession)
*psession = session;
+
+ return 0;
+
+out_delete:
+ perf_session__delete(session);
+ return -1;
}
-static void print_bad_events(void)
+static void print_bad_events(struct perf_sched *sched)
{
- if (nr_unordered_timestamps && nr_timestamps) {
+ if (sched->nr_unordered_timestamps && sched->nr_timestamps) {
printf(" INFO: %.3f%% unordered timestamps (%ld out of %ld)\n",
- (double)nr_unordered_timestamps/(double)nr_timestamps*100.0,
- nr_unordered_timestamps, nr_timestamps);
+ (double)sched->nr_unordered_timestamps/(double)sched->nr_timestamps*100.0,
+ sched->nr_unordered_timestamps, sched->nr_timestamps);
}
- if (nr_lost_events && nr_events) {
+ if (sched->nr_lost_events && sched->nr_events) {
printf(" INFO: %.3f%% lost events (%ld out of %ld, in %ld chunks)\n",
- (double)nr_lost_events/(double)nr_events*100.0,
- nr_lost_events, nr_events, nr_lost_chunks);
+ (double)sched->nr_lost_events/(double)sched->nr_events * 100.0,
+ sched->nr_lost_events, sched->nr_events, sched->nr_lost_chunks);
}
- if (nr_state_machine_bugs && nr_timestamps) {
+ if (sched->nr_state_machine_bugs && sched->nr_timestamps) {
printf(" INFO: %.3f%% state machine bugs (%ld out of %ld)",
- (double)nr_state_machine_bugs/(double)nr_timestamps*100.0,
- nr_state_machine_bugs, nr_timestamps);
- if (nr_lost_events)
+ (double)sched->nr_state_machine_bugs/(double)sched->nr_timestamps*100.0,
+ sched->nr_state_machine_bugs, sched->nr_timestamps);
+ if (sched->nr_lost_events)
printf(" (due to lost events?)");
printf("\n");
}
- if (nr_context_switch_bugs && nr_timestamps) {
+ if (sched->nr_context_switch_bugs && sched->nr_timestamps) {
printf(" INFO: %.3f%% context switch bugs (%ld out of %ld)",
- (double)nr_context_switch_bugs/(double)nr_timestamps*100.0,
- nr_context_switch_bugs, nr_timestamps);
- if (nr_lost_events)
+ (double)sched->nr_context_switch_bugs/(double)sched->nr_timestamps*100.0,
+ sched->nr_context_switch_bugs, sched->nr_timestamps);
+ if (sched->nr_lost_events)
printf(" (due to lost events?)");
printf("\n");
}
}
-static void __cmd_lat(void)
+static int perf_sched__lat(struct perf_sched *sched)
{
struct rb_node *next;
struct perf_session *session;
setup_pager();
- read_events(false, &session);
- sort_lat();
+ if (perf_sched__read_events(sched, false, &session))
+ return -1;
+ perf_sched__sort_lat(sched);
printf("\n ---------------------------------------------------------------------------------------------------------------\n");
printf(" Task | Runtime ms | Switches | Average delay ms | Maximum delay ms | Maximum delay at |\n");
printf(" ---------------------------------------------------------------------------------------------------------------\n");
- next = rb_first(&sorted_atom_root);
+ next = rb_first(&sched->sorted_atom_root);
while (next) {
struct work_atoms *work_list;
work_list = rb_entry(next, struct work_atoms, node);
- output_lat_thread(work_list);
+ output_lat_thread(sched, work_list);
next = rb_next(next);
}
printf(" -----------------------------------------------------------------------------------------\n");
printf(" TOTAL: |%11.3f ms |%9" PRIu64 " |\n",
- (double)all_runtime/1e6, all_count);
+ (double)sched->all_runtime / 1e6, sched->all_count);
printf(" ---------------------------------------------------\n");
- print_bad_events();
+ print_bad_events(sched);
printf("\n");
perf_session__delete(session);
+ return 0;
}
-static struct trace_sched_handler map_ops = {
- .wakeup_event = NULL,
- .switch_event = map_switch_event,
- .runtime_event = NULL,
- .fork_event = NULL,
-};
-
-static void __cmd_map(void)
+static int perf_sched__map(struct perf_sched *sched)
{
- max_cpu = sysconf(_SC_NPROCESSORS_CONF);
+ sched->max_cpu = sysconf(_SC_NPROCESSORS_CONF);
setup_pager();
- read_events(true, NULL);
- print_bad_events();
+ if (perf_sched__read_events(sched, true, NULL))
+ return -1;
+ print_bad_events(sched);
+ return 0;
}
-static void __cmd_replay(void)
+static int perf_sched__replay(struct perf_sched *sched)
{
unsigned long i;
- calibrate_run_measurement_overhead();
- calibrate_sleep_measurement_overhead();
+ calibrate_run_measurement_overhead(sched);
+ calibrate_sleep_measurement_overhead(sched);
- test_calibrations();
+ test_calibrations(sched);
- read_events(true, NULL);
+ if (perf_sched__read_events(sched, true, NULL))
+ return -1;
- printf("nr_run_events: %ld\n", nr_run_events);
- printf("nr_sleep_events: %ld\n", nr_sleep_events);
- printf("nr_wakeup_events: %ld\n", nr_wakeup_events);
+ printf("nr_run_events: %ld\n", sched->nr_run_events);
+ printf("nr_sleep_events: %ld\n", sched->nr_sleep_events);
+ printf("nr_wakeup_events: %ld\n", sched->nr_wakeup_events);
- if (targetless_wakeups)
- printf("target-less wakeups: %ld\n", targetless_wakeups);
- if (multitarget_wakeups)
- printf("multi-target wakeups: %ld\n", multitarget_wakeups);
- if (nr_run_events_optimized)
+ if (sched->targetless_wakeups)
+ printf("target-less wakeups: %ld\n", sched->targetless_wakeups);
+ if (sched->multitarget_wakeups)
+ printf("multi-target wakeups: %ld\n", sched->multitarget_wakeups);
+ if (sched->nr_run_events_optimized)
printf("run atoms optimized: %ld\n",
- nr_run_events_optimized);
+ sched->nr_run_events_optimized);
- print_task_traces();
- add_cross_task_wakeups();
+ print_task_traces(sched);
+ add_cross_task_wakeups(sched);
- create_tasks();
+ create_tasks(sched);
printf("------------------------------------------------------------\n");
- for (i = 0; i < replay_repeat; i++)
- run_one_test();
+ for (i = 0; i < sched->replay_repeat; i++)
+ run_one_test(sched);
+
+ return 0;
}
-
-static const char * const sched_usage[] = {
- "perf sched [<options>] {record|latency|map|replay|script}",
- NULL
-};
-
-static const struct option sched_options[] = {
- OPT_STRING('i', "input", &input_name, "file",
- "input file name"),
- OPT_INCR('v', "verbose", &verbose,
- "be more verbose (show symbol address, etc)"),
- OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
- "dump raw trace in ASCII"),
- OPT_END()
-};
-
-static const char * const latency_usage[] = {
- "perf sched latency [<options>]",
- NULL
-};
-
-static const struct option latency_options[] = {
- OPT_STRING('s', "sort", &sort_order, "key[,key2...]",
- "sort by key(s): runtime, switch, avg, max"),
- OPT_INCR('v', "verbose", &verbose,
- "be more verbose (show symbol address, etc)"),
- OPT_INTEGER('C', "CPU", &profile_cpu,
- "CPU to profile on"),
- OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
- "dump raw trace in ASCII"),
- OPT_END()
-};
-
-static const char * const replay_usage[] = {
- "perf sched replay [<options>]",
- NULL
-};
-
-static const struct option replay_options[] = {
- OPT_UINTEGER('r', "repeat", &replay_repeat,
- "repeat the workload replay N times (-1: infinite)"),
- OPT_INCR('v', "verbose", &verbose,
- "be more verbose (show symbol address, etc)"),
- OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
- "dump raw trace in ASCII"),
- OPT_END()
-};
-
-static void setup_sorting(void)
+static void setup_sorting(struct perf_sched *sched, const struct option *options,
+ const char * const usage_msg[])
{
- char *tmp, *tok, *str = strdup(sort_order);
+ char *tmp, *tok, *str = strdup(sched->sort_order);
for (tok = strtok_r(str, ", ", &tmp);
tok; tok = strtok_r(NULL, ", ", &tmp)) {
- if (sort_dimension__add(tok, &sort_list) < 0) {
+ if (sort_dimension__add(tok, &sched->sort_list) < 0) {
error("Unknown --sort key: `%s'", tok);
- usage_with_options(latency_usage, latency_options);
+ usage_with_options(usage_msg, options);
}
}
free(str);
- sort_dimension__add("pid", &cmp_pid);
+ sort_dimension__add("pid", &sched->cmp_pid);
}
-static const char *record_args[] = {
- "record",
- "-a",
- "-R",
- "-f",
- "-m", "1024",
- "-c", "1",
- "-e", "sched:sched_switch",
- "-e", "sched:sched_stat_wait",
- "-e", "sched:sched_stat_sleep",
- "-e", "sched:sched_stat_iowait",
- "-e", "sched:sched_stat_runtime",
- "-e", "sched:sched_process_exit",
- "-e", "sched:sched_process_fork",
- "-e", "sched:sched_wakeup",
- "-e", "sched:sched_migrate_task",
-};
-
static int __cmd_record(int argc, const char **argv)
{
unsigned int rec_argc, i, j;
const char **rec_argv;
+ const char * const record_args[] = {
+ "record",
+ "-a",
+ "-R",
+ "-f",
+ "-m", "1024",
+ "-c", "1",
+ "-e", "sched:sched_switch",
+ "-e", "sched:sched_stat_wait",
+ "-e", "sched:sched_stat_sleep",
+ "-e", "sched:sched_stat_iowait",
+ "-e", "sched:sched_stat_runtime",
+ "-e", "sched:sched_process_exit",
+ "-e", "sched:sched_process_fork",
+ "-e", "sched:sched_wakeup",
+ "-e", "sched:sched_migrate_task",
+ };
rec_argc = ARRAY_SIZE(record_args) + argc - 1;
rec_argv = calloc(rec_argc + 1, sizeof(char *));
@@ -1897,8 +1664,85 @@
return cmd_record(i, rec_argv, NULL);
}
-int cmd_sched(int argc, const char **argv, const char *prefix __used)
+int cmd_sched(int argc, const char **argv, const char *prefix __maybe_unused)
{
+ const char default_sort_order[] = "avg, max, switch, runtime";
+ struct perf_sched sched = {
+ .tool = {
+ .sample = perf_sched__process_tracepoint_sample,
+ .comm = perf_event__process_comm,
+ .lost = perf_event__process_lost,
+ .fork = perf_event__process_task,
+ .ordered_samples = true,
+ },
+ .cmp_pid = LIST_HEAD_INIT(sched.cmp_pid),
+ .sort_list = LIST_HEAD_INIT(sched.sort_list),
+ .start_work_mutex = PTHREAD_MUTEX_INITIALIZER,
+ .work_done_wait_mutex = PTHREAD_MUTEX_INITIALIZER,
+ .curr_pid = { [0 ... MAX_CPUS - 1] = -1 },
+ .sort_order = default_sort_order,
+ .replay_repeat = 10,
+ .profile_cpu = -1,
+ .next_shortname1 = 'A',
+ .next_shortname2 = '0',
+ };
+ const struct option latency_options[] = {
+ OPT_STRING('s', "sort", &sched.sort_order, "key[,key2...]",
+ "sort by key(s): runtime, switch, avg, max"),
+ OPT_INCR('v', "verbose", &verbose,
+ "be more verbose (show symbol address, etc)"),
+ OPT_INTEGER('C', "CPU", &sched.profile_cpu,
+ "CPU to profile on"),
+ OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
+ "dump raw trace in ASCII"),
+ OPT_END()
+ };
+ const struct option replay_options[] = {
+ OPT_UINTEGER('r', "repeat", &sched.replay_repeat,
+ "repeat the workload replay N times (-1: infinite)"),
+ OPT_INCR('v', "verbose", &verbose,
+ "be more verbose (show symbol address, etc)"),
+ OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
+ "dump raw trace in ASCII"),
+ OPT_END()
+ };
+ const struct option sched_options[] = {
+ OPT_STRING('i', "input", &sched.input_name, "file",
+ "input file name"),
+ OPT_INCR('v', "verbose", &verbose,
+ "be more verbose (show symbol address, etc)"),
+ OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
+ "dump raw trace in ASCII"),
+ OPT_END()
+ };
+ const char * const latency_usage[] = {
+ "perf sched latency [<options>]",
+ NULL
+ };
+ const char * const replay_usage[] = {
+ "perf sched replay [<options>]",
+ NULL
+ };
+ const char * const sched_usage[] = {
+ "perf sched [<options>] {record|latency|map|replay|script}",
+ NULL
+ };
+ struct trace_sched_handler lat_ops = {
+ .wakeup_event = latency_wakeup_event,
+ .switch_event = latency_switch_event,
+ .runtime_event = latency_runtime_event,
+ .fork_event = latency_fork_event,
+ .migrate_task_event = latency_migrate_task_event,
+ };
+ struct trace_sched_handler map_ops = {
+ .switch_event = map_switch_event,
+ };
+ struct trace_sched_handler replay_ops = {
+ .wakeup_event = replay_wakeup_event,
+ .switch_event = replay_switch_event,
+ .fork_event = replay_fork_event,
+ };
+
argc = parse_options(argc, argv, sched_options, sched_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
if (!argc)
@@ -1914,26 +1758,26 @@
if (!strncmp(argv[0], "rec", 3)) {
return __cmd_record(argc, argv);
} else if (!strncmp(argv[0], "lat", 3)) {
- trace_handler = &lat_ops;
+ sched.tp_handler = &lat_ops;
if (argc > 1) {
argc = parse_options(argc, argv, latency_options, latency_usage, 0);
if (argc)
usage_with_options(latency_usage, latency_options);
}
- setup_sorting();
- __cmd_lat();
+ setup_sorting(&sched, latency_options, latency_usage);
+ return perf_sched__lat(&sched);
} else if (!strcmp(argv[0], "map")) {
- trace_handler = &map_ops;
- setup_sorting();
- __cmd_map();
+ sched.tp_handler = &map_ops;
+ setup_sorting(&sched, latency_options, latency_usage);
+ return perf_sched__map(&sched);
} else if (!strncmp(argv[0], "rep", 3)) {
- trace_handler = &replay_ops;
+ sched.tp_handler = &replay_ops;
if (argc) {
argc = parse_options(argc, argv, replay_options, replay_usage, 0);
if (argc)
usage_with_options(replay_usage, replay_options);
}
- __cmd_replay();
+ return perf_sched__replay(&sched);
} else {
usage_with_options(sched_usage, sched_options);
}
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 1e60ab7..1be843a 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -14,6 +14,7 @@
#include "util/util.h"
#include "util/evlist.h"
#include "util/evsel.h"
+#include "util/sort.h"
#include <linux/bitmap.h>
static char const *script_name;
@@ -28,11 +29,6 @@
static const char *cpu_list;
static DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
-struct perf_script {
- struct perf_tool tool;
- struct perf_session *session;
-};
-
enum perf_output_field {
PERF_OUTPUT_COMM = 1U << 0,
PERF_OUTPUT_TID = 1U << 1,
@@ -262,14 +258,11 @@
return 0;
}
-static void print_sample_start(struct pevent *pevent,
- struct perf_sample *sample,
+static void print_sample_start(struct perf_sample *sample,
struct thread *thread,
struct perf_evsel *evsel)
{
- int type;
struct perf_event_attr *attr = &evsel->attr;
- struct event_format *event;
const char *evname = NULL;
unsigned long secs;
unsigned long usecs;
@@ -307,20 +300,7 @@
}
if (PRINT_FIELD(EVNAME)) {
- if (attr->type == PERF_TYPE_TRACEPOINT) {
- /*
- * XXX Do we really need this here?
- * perf_evlist__set_tracepoint_names should have done
- * this already
- */
- type = trace_parse_common_type(pevent,
- sample->raw_data);
- event = pevent_find_event(pevent, type);
- if (event)
- evname = event->name;
- } else
- evname = perf_evsel__name(evsel);
-
+ evname = perf_evsel__name(evsel);
printf("%s: ", evname ? evname : "[unknown]");
}
}
@@ -401,7 +381,7 @@
printf(" ");
else
printf("\n");
- perf_event__print_ip(event, sample, machine,
+ perf_evsel__print_ip(evsel, event, sample, machine,
PRINT_FIELD(SYM), PRINT_FIELD(DSO),
PRINT_FIELD(SYMOFFSET));
}
@@ -415,19 +395,17 @@
printf("\n");
}
-static void process_event(union perf_event *event __unused,
- struct pevent *pevent,
- struct perf_sample *sample,
- struct perf_evsel *evsel,
- struct machine *machine,
- struct thread *thread)
+static void process_event(union perf_event *event, struct perf_sample *sample,
+ struct perf_evsel *evsel, struct machine *machine,
+ struct addr_location *al)
{
struct perf_event_attr *attr = &evsel->attr;
+ struct thread *thread = al->thread;
if (output[attr->type].fields == 0)
return;
- print_sample_start(pevent, sample, thread, evsel);
+ print_sample_start(sample, thread, evsel);
if (is_bts_event(attr)) {
print_sample_bts(event, sample, evsel, machine, thread);
@@ -435,9 +413,8 @@
}
if (PRINT_FIELD(TRACE))
- print_trace_event(pevent, sample->cpu, sample->raw_data,
- sample->raw_size);
-
+ event_format__print(evsel->tp_format, sample->cpu,
+ sample->raw_data, sample->raw_size);
if (PRINT_FIELD(ADDR))
print_sample_addr(event, sample, machine, thread, attr);
@@ -446,7 +423,7 @@
printf(" ");
else
printf("\n");
- perf_event__print_ip(event, sample, machine,
+ perf_evsel__print_ip(evsel, event, sample, machine,
PRINT_FIELD(SYM), PRINT_FIELD(DSO),
PRINT_FIELD(SYMOFFSET));
}
@@ -454,9 +431,9 @@
printf("\n");
}
-static int default_start_script(const char *script __unused,
- int argc __unused,
- const char **argv __unused)
+static int default_start_script(const char *script __maybe_unused,
+ int argc __maybe_unused,
+ const char **argv __maybe_unused)
{
return 0;
}
@@ -466,8 +443,8 @@
return 0;
}
-static int default_generate_script(struct pevent *pevent __unused,
- const char *outfile __unused)
+static int default_generate_script(struct pevent *pevent __maybe_unused,
+ const char *outfile __maybe_unused)
{
return 0;
}
@@ -498,14 +475,13 @@
static const char *input_name;
-static int process_sample_event(struct perf_tool *tool __used,
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_sample *sample,
struct perf_evsel *evsel,
struct machine *machine)
{
struct addr_location al;
- struct perf_script *scr = container_of(tool, struct perf_script, tool);
struct thread *thread = machine__findnew_thread(machine, event->ip.tid);
if (thread == NULL) {
@@ -537,32 +513,29 @@
if (cpu_list && !test_bit(sample->cpu, cpu_bitmap))
return 0;
- scripting_ops->process_event(event, scr->session->pevent,
- sample, evsel, machine, thread);
+ scripting_ops->process_event(event, sample, evsel, machine, &al);
evsel->hists.stats.total_period += sample->period;
return 0;
}
-static struct perf_script perf_script = {
- .tool = {
- .sample = process_sample_event,
- .mmap = perf_event__process_mmap,
- .comm = perf_event__process_comm,
- .exit = perf_event__process_task,
- .fork = perf_event__process_task,
- .attr = perf_event__process_attr,
- .event_type = perf_event__process_event_type,
- .tracing_data = perf_event__process_tracing_data,
- .build_id = perf_event__process_build_id,
- .ordered_samples = true,
- .ordering_requires_timestamps = true,
- },
+static struct perf_tool perf_script = {
+ .sample = process_sample_event,
+ .mmap = perf_event__process_mmap,
+ .comm = perf_event__process_comm,
+ .exit = perf_event__process_task,
+ .fork = perf_event__process_task,
+ .attr = perf_event__process_attr,
+ .event_type = perf_event__process_event_type,
+ .tracing_data = perf_event__process_tracing_data,
+ .build_id = perf_event__process_build_id,
+ .ordered_samples = true,
+ .ordering_requires_timestamps = true,
};
extern volatile int session_done;
-static void sig_handler(int sig __unused)
+static void sig_handler(int sig __maybe_unused)
{
session_done = 1;
}
@@ -573,7 +546,7 @@
signal(SIGINT, sig_handler);
- ret = perf_session__process_events(session, &perf_script.tool);
+ ret = perf_session__process_events(session, &perf_script);
if (debug_mode)
pr_err("Misordered timestamps: %" PRIu64 "\n", nr_unordered);
@@ -672,8 +645,8 @@
fprintf(stderr, "\n");
}
-static int parse_scriptname(const struct option *opt __used,
- const char *str, int unset __used)
+static int parse_scriptname(const struct option *opt __maybe_unused,
+ const char *str, int unset __maybe_unused)
{
char spec[PATH_MAX];
const char *script, *ext;
@@ -718,8 +691,8 @@
return 0;
}
-static int parse_output_fields(const struct option *opt __used,
- const char *arg, int unset __used)
+static int parse_output_fields(const struct option *opt __maybe_unused,
+ const char *arg, int unset __maybe_unused)
{
char *tok;
int i, imax = sizeof(all_output_options) / sizeof(struct output_option);
@@ -1010,8 +983,9 @@
return script_root;
}
-static int list_available_scripts(const struct option *opt __used,
- const char *s __used, int unset __used)
+static int list_available_scripts(const struct option *opt __maybe_unused,
+ const char *s __maybe_unused,
+ int unset __maybe_unused)
{
struct dirent *script_next, *lang_next, script_dirent, lang_dirent;
char scripts_path[MAXPATHLEN];
@@ -1058,6 +1032,61 @@
exit(0);
}
+/*
+ * Return -1 if none is found, otherwise the actual scripts number.
+ *
+ * Currently the only user of this function is the script browser, which
+ * will list all statically runnable scripts, select one, execute it and
+ * show the output in a perf browser.
+ */
+int find_scripts(char **scripts_array, char **scripts_path_array)
+{
+ struct dirent *script_next, *lang_next, script_dirent, lang_dirent;
+ char scripts_path[MAXPATHLEN];
+ DIR *scripts_dir, *lang_dir;
+ char lang_path[MAXPATHLEN];
+ char *temp;
+ int i = 0;
+
+ snprintf(scripts_path, MAXPATHLEN, "%s/scripts", perf_exec_path());
+
+ scripts_dir = opendir(scripts_path);
+ if (!scripts_dir)
+ return -1;
+
+ for_each_lang(scripts_path, scripts_dir, lang_dirent, lang_next) {
+ snprintf(lang_path, MAXPATHLEN, "%s/%s", scripts_path,
+ lang_dirent.d_name);
+#ifdef NO_LIBPERL
+ if (strstr(lang_path, "perl"))
+ continue;
+#endif
+#ifdef NO_LIBPYTHON
+ if (strstr(lang_path, "python"))
+ continue;
+#endif
+
+ lang_dir = opendir(lang_path);
+ if (!lang_dir)
+ continue;
+
+ for_each_script(lang_path, lang_dir, script_dirent, script_next) {
+ /* Skip those real time scripts: xxxtop.p[yl] */
+ if (strstr(script_dirent.d_name, "top."))
+ continue;
+ sprintf(scripts_path_array[i], "%s/%s", lang_path,
+ script_dirent.d_name);
+ temp = strchr(script_dirent.d_name, '.');
+ snprintf(scripts_array[i],
+ (temp - script_dirent.d_name) + 1,
+ "%s", script_dirent.d_name);
+ i++;
+ }
+ }
+
+ return i;
+}
+
static char *get_script_path(const char *script_root, const char *suffix)
{
struct dirent *script_next, *lang_next, script_dirent, lang_dirent;
@@ -1170,6 +1199,8 @@
parse_output_fields),
OPT_BOOLEAN('a', "all-cpus", &system_wide,
"system-wide collection from all CPUs"),
+ OPT_STRING('S', "symbols", &symbol_conf.sym_list_str, "symbol[,symbol...]",
+ "only consider these symbols"),
OPT_STRING('C', "cpu", &cpu_list, "cpu", "list of cpus to profile"),
OPT_STRING('c', "comms", &symbol_conf.comm_list_str, "comm[,comm...]",
"only display events for these comms"),
@@ -1181,21 +1212,26 @@
OPT_END()
};
-static bool have_cmd(int argc, const char **argv)
+static int have_cmd(int argc, const char **argv)
{
char **__argv = malloc(sizeof(const char *) * argc);
- if (!__argv)
- die("malloc");
+ if (!__argv) {
+ pr_err("malloc failed\n");
+ return -1;
+ }
+
memcpy(__argv, argv, sizeof(const char *) * argc);
argc = parse_options(argc, (const char **)__argv, record_options,
NULL, PARSE_OPT_STOP_AT_NON_OPTION);
free(__argv);
- return argc != 0;
+ system_wide = (argc == 0);
+
+ return 0;
}
-int cmd_script(int argc, const char **argv, const char *prefix __used)
+int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
{
char *rec_script_path = NULL;
char *rep_script_path = NULL;
@@ -1259,13 +1295,13 @@
if (pipe(live_pipe) < 0) {
perror("failed to create pipe");
- exit(-1);
+ return -1;
}
pid = fork();
if (pid < 0) {
perror("failed to fork");
- exit(-1);
+ return -1;
}
if (!pid) {
@@ -1277,13 +1313,18 @@
if (is_top_script(argv[0])) {
system_wide = true;
} else if (!system_wide) {
- system_wide = !have_cmd(argc - rep_args,
- &argv[rep_args]);
+ if (have_cmd(argc - rep_args, &argv[rep_args]) != 0) {
+ err = -1;
+ goto out;
+ }
}
__argv = malloc((argc + 6) * sizeof(const char *));
- if (!__argv)
- die("malloc");
+ if (!__argv) {
+ pr_err("malloc failed\n");
+ err = -ENOMEM;
+ goto out;
+ }
__argv[j++] = "/bin/sh";
__argv[j++] = rec_script_path;
@@ -1305,8 +1346,12 @@
close(live_pipe[1]);
__argv = malloc((argc + 4) * sizeof(const char *));
- if (!__argv)
- die("malloc");
+ if (!__argv) {
+ pr_err("malloc failed\n");
+ err = -ENOMEM;
+ goto out;
+ }
+
j = 0;
__argv[j++] = "/bin/sh";
__argv[j++] = rep_script_path;
@@ -1331,12 +1376,20 @@
if (!rec_script_path)
system_wide = false;
- else if (!system_wide)
- system_wide = !have_cmd(argc - 1, &argv[1]);
+ else if (!system_wide) {
+ if (have_cmd(argc - 1, &argv[1]) != 0) {
+ err = -1;
+ goto out;
+ }
+ }
__argv = malloc((argc + 2) * sizeof(const char *));
- if (!__argv)
- die("malloc");
+ if (!__argv) {
+ pr_err("malloc failed\n");
+ err = -ENOMEM;
+ goto out;
+ }
+
__argv[j++] = "/bin/sh";
__argv[j++] = script_path;
if (system_wide)
@@ -1356,12 +1409,10 @@
setup_pager();
session = perf_session__new(input_name, O_RDONLY, 0, false,
- &perf_script.tool);
+ &perf_script);
if (session == NULL)
return -ENOMEM;
- perf_script.session = session;
-
if (cpu_list) {
if (perf_session__cpu_bitmap(session, cpu_list, cpu_bitmap))
return -1;
@@ -1387,18 +1438,18 @@
input = open(session->filename, O_RDONLY); /* input_name */
if (input < 0) {
perror("failed to open file");
- exit(-1);
+ return -1;
}
err = fstat(input, &perf_stat);
if (err < 0) {
perror("failed to stat file");
- exit(-1);
+ return -1;
}
if (!perf_stat.st_size) {
fprintf(stderr, "zero-sized file, nothing to do!\n");
- exit(0);
+ return 0;
}
scripting_ops = script_spec__lookup(generate_script_lang);
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 861f0ae..e8cd4d8 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -51,13 +51,13 @@
#include "util/evsel.h"
#include "util/debug.h"
#include "util/color.h"
+#include "util/stat.h"
#include "util/header.h"
#include "util/cpumap.h"
#include "util/thread.h"
#include "util/thread_map.h"
#include <sys/prctl.h>
-#include <math.h>
#include <locale.h>
#define DEFAULT_SEPARATOR " "
@@ -199,11 +199,6 @@
static volatile int done = 0;
-struct stats
-{
- double n, mean, M2;
-};
-
struct perf_stat {
struct stats res_stats[3];
};
@@ -220,48 +215,14 @@
evsel->priv = NULL;
}
-static void update_stats(struct stats *stats, u64 val)
+static inline struct cpu_map *perf_evsel__cpus(struct perf_evsel *evsel)
{
- double delta;
-
- stats->n++;
- delta = val - stats->mean;
- stats->mean += delta / stats->n;
- stats->M2 += delta*(val - stats->mean);
+ return (evsel->cpus && !target.cpu_list) ? evsel->cpus : evsel_list->cpus;
}
-static double avg_stats(struct stats *stats)
+static inline int perf_evsel__nr_cpus(struct perf_evsel *evsel)
{
- return stats->mean;
-}
-
-/*
- * http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
- *
- * (\Sum n_i^2) - ((\Sum n_i)^2)/n
- * s^2 = -------------------------------
- * n - 1
- *
- * http://en.wikipedia.org/wiki/Stddev
- *
- * The std dev of the mean is related to the std dev by:
- *
- * s
- * s_mean = -------
- * sqrt(n)
- *
- */
-static double stddev_stats(struct stats *stats)
-{
- double variance, variance_mean;
-
- if (!stats->n)
- return 0.0;
-
- variance = stats->M2 / (stats->n - 1);
- variance_mean = variance / stats->n;
-
- return sqrt(variance_mean);
+ return perf_evsel__cpus(evsel)->nr;
}
static struct stats runtime_nsecs_stats[MAX_NR_CPUS];
@@ -281,13 +242,9 @@
struct perf_evsel *first)
{
struct perf_event_attr *attr = &evsel->attr;
- struct xyarray *group_fd = NULL;
bool exclude_guest_missing = false;
int ret;
- if (group && evsel != first)
- group_fd = first->fd;
-
if (scale)
attr->read_format = PERF_FORMAT_TOTAL_TIME_ENABLED |
PERF_FORMAT_TOTAL_TIME_RUNNING;
@@ -299,8 +256,7 @@
evsel->attr.exclude_guest = evsel->attr.exclude_host = 0;
if (perf_target__has_cpu(&target)) {
- ret = perf_evsel__open_per_cpu(evsel, evsel_list->cpus,
- group, group_fd);
+ ret = perf_evsel__open_per_cpu(evsel, perf_evsel__cpus(evsel));
if (ret)
goto check_ret;
return 0;
@@ -311,8 +267,7 @@
attr->enable_on_exec = 1;
}
- ret = perf_evsel__open_per_thread(evsel, evsel_list->threads,
- group, group_fd);
+ ret = perf_evsel__open_per_thread(evsel, evsel_list->threads);
if (!ret)
return 0;
/* fall through */
@@ -382,7 +337,7 @@
u64 *count = counter->counts->aggr.values;
int i;
- if (__perf_evsel__read(counter, evsel_list->cpus->nr,
+ if (__perf_evsel__read(counter, perf_evsel__nr_cpus(counter),
evsel_list->threads->nr, scale) < 0)
return -1;
@@ -411,7 +366,7 @@
u64 *count;
int cpu;
- for (cpu = 0; cpu < evsel_list->cpus->nr; cpu++) {
+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
if (__perf_evsel__read_on_cpu(counter, cpu, 0, scale) < 0)
return -1;
@@ -423,7 +378,7 @@
return 0;
}
-static int run_perf_stat(int argc __used, const char **argv)
+static int run_perf_stat(int argc __maybe_unused, const char **argv)
{
unsigned long long t0, t1;
struct perf_evsel *counter, *first;
@@ -434,7 +389,7 @@
if (forks && (pipe(child_ready_pipe) < 0 || pipe(go_pipe) < 0)) {
perror("failed to create pipes");
- exit(1);
+ return -1;
}
if (forks) {
@@ -483,7 +438,10 @@
close(child_ready_pipe[0]);
}
- first = list_entry(evsel_list->entries.next, struct perf_evsel, node);
+ if (group)
+ perf_evlist__set_leader(evsel_list);
+
+ first = perf_evlist__first(evsel_list);
list_for_each_entry(counter, &evsel_list->entries, node) {
if (create_perf_stat_counter(counter, first) < 0) {
@@ -513,13 +471,14 @@
}
if (child_pid != -1)
kill(child_pid, SIGTERM);
- die("Not all events could be opened.\n");
+
+ pr_err("Not all events could be opened.\n");
return -1;
}
counter->supported = true;
}
- if (perf_evlist__set_filters(evsel_list)) {
+ if (perf_evlist__apply_filters(evsel_list)) {
error("failed to set filter with %d (%s)\n", errno,
strerror(errno));
return -1;
@@ -546,12 +505,12 @@
if (no_aggr) {
list_for_each_entry(counter, &evsel_list->entries, node) {
read_counter(counter);
- perf_evsel__close_fd(counter, evsel_list->cpus->nr, 1);
+ perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter), 1);
}
} else {
list_for_each_entry(counter, &evsel_list->entries, node) {
read_counter_aggr(counter);
- perf_evsel__close_fd(counter, evsel_list->cpus->nr,
+ perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter),
evsel_list->threads->nr);
}
}
@@ -561,10 +520,7 @@
static void print_noise_pct(double total, double avg)
{
- double pct = 0.0;
-
- if (avg)
- pct = 100.0*total/avg;
+ double pct = rel_stddev_stats(total, avg);
if (csv_output)
fprintf(output, "%s%.2f%%", csv_sep, pct);
@@ -592,7 +548,7 @@
if (no_aggr)
sprintf(cpustr, "CPU%*d%s",
csv_output ? 0 : -4,
- evsel_list->cpus->map[cpu], csv_sep);
+ perf_evsel__cpus(evsel)->map[cpu], csv_sep);
fprintf(output, fmt, cpustr, msecs, csv_sep, perf_evsel__name(evsel));
@@ -636,7 +592,9 @@
return color;
}
-static void print_stalled_cycles_frontend(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_stalled_cycles_frontend(int cpu,
+ struct perf_evsel *evsel
+ __maybe_unused, double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -653,7 +611,9 @@
fprintf(output, " frontend cycles idle ");
}
-static void print_stalled_cycles_backend(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_stalled_cycles_backend(int cpu,
+ struct perf_evsel *evsel
+ __maybe_unused, double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -670,7 +630,9 @@
fprintf(output, " backend cycles idle ");
}
-static void print_branch_misses(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_branch_misses(int cpu,
+ struct perf_evsel *evsel __maybe_unused,
+ double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -687,7 +649,9 @@
fprintf(output, " of all branches ");
}
-static void print_l1_dcache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_l1_dcache_misses(int cpu,
+ struct perf_evsel *evsel __maybe_unused,
+ double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -704,7 +668,9 @@
fprintf(output, " of all L1-dcache hits ");
}
-static void print_l1_icache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_l1_icache_misses(int cpu,
+ struct perf_evsel *evsel __maybe_unused,
+ double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -721,7 +687,9 @@
fprintf(output, " of all L1-icache hits ");
}
-static void print_dtlb_cache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_dtlb_cache_misses(int cpu,
+ struct perf_evsel *evsel __maybe_unused,
+ double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -738,7 +706,9 @@
fprintf(output, " of all dTLB cache hits ");
}
-static void print_itlb_cache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_itlb_cache_misses(int cpu,
+ struct perf_evsel *evsel __maybe_unused,
+ double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -755,7 +725,9 @@
fprintf(output, " of all iTLB cache hits ");
}
-static void print_ll_cache_misses(int cpu, struct perf_evsel *evsel __used, double avg)
+static void print_ll_cache_misses(int cpu,
+ struct perf_evsel *evsel __maybe_unused,
+ double avg)
{
double total, ratio = 0.0;
const char *color;
@@ -788,7 +760,7 @@
if (no_aggr)
sprintf(cpustr, "CPU%*d%s",
csv_output ? 0 : -4,
- evsel_list->cpus->map[cpu], csv_sep);
+ perf_evsel__cpus(evsel)->map[cpu], csv_sep);
else
cpu = 0;
@@ -949,14 +921,14 @@
u64 ena, run, val;
int cpu;
- for (cpu = 0; cpu < evsel_list->cpus->nr; cpu++) {
+ for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
val = counter->counts->cpu[cpu].val;
ena = counter->counts->cpu[cpu].ena;
run = counter->counts->cpu[cpu].run;
if (run == 0 || ena == 0) {
fprintf(output, "CPU%*d%s%*s%s%*s",
csv_output ? 0 : -4,
- evsel_list->cpus->map[cpu], csv_sep,
+ perf_evsel__cpus(counter)->map[cpu], csv_sep,
csv_output ? 0 : 18,
counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
csv_sep,
@@ -1061,8 +1033,8 @@
NULL
};
-static int stat__set_big_num(const struct option *opt __used,
- const char *s __used, int unset)
+static int stat__set_big_num(const struct option *opt __maybe_unused,
+ const char *s __maybe_unused, int unset)
{
big_num_opt = unset ? 0 : 1;
return 0;
@@ -1156,7 +1128,7 @@
return perf_evlist__add_default_attrs(evsel_list, very_very_detailed_attrs);
}
-int cmd_stat(int argc, const char **argv, const char *prefix __used)
+int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
{
struct perf_evsel *pos;
int status = -ENOMEM;
@@ -1192,7 +1164,7 @@
output = fopen(output_name, mode);
if (!output) {
perror("failed to create output file");
- exit(-1);
+ return -1;
}
clock_gettime(CLOCK_REALTIME, &tm);
fprintf(output, "# started on %s\n", ctime(&tm.tv_sec));
@@ -1255,7 +1227,7 @@
list_for_each_entry(pos, &evsel_list->entries, node) {
if (perf_evsel__alloc_stat_priv(pos) < 0 ||
- perf_evsel__alloc_counts(pos, evsel_list->cpus->nr) < 0)
+ perf_evsel__alloc_counts(pos, perf_evsel__nr_cpus(pos)) < 0)
goto out_free_fd;
}
diff --git a/tools/perf/builtin-test.c b/tools/perf/builtin-test.c
index 1d592f5..484f26c 100644
--- a/tools/perf/builtin-test.c
+++ b/tools/perf/builtin-test.c
@@ -14,11 +14,13 @@
#include "util/symbol.h"
#include "util/thread_map.h"
#include "util/pmu.h"
+#include "event-parse.h"
#include "../../include/linux/hw_breakpoint.h"
#include <sys/mman.h>
-static int vmlinux_matches_kallsyms_filter(struct map *map __used, struct symbol *sym)
+static int vmlinux_matches_kallsyms_filter(struct map *map __maybe_unused,
+ struct symbol *sym)
{
bool *visited = symbol__priv(sym);
*visited = true;
@@ -294,7 +296,7 @@
goto out_thread_map_delete;
}
- if (perf_evsel__open_per_thread(evsel, threads, false, NULL) < 0) {
+ if (perf_evsel__open_per_thread(evsel, threads) < 0) {
pr_debug("failed to open counter: %s, "
"tweak /proc/sys/kernel/perf_event_paranoid?\n",
strerror(errno));
@@ -369,7 +371,7 @@
goto out_thread_map_delete;
}
- if (perf_evsel__open(evsel, cpus, threads, false, NULL) < 0) {
+ if (perf_evsel__open(evsel, cpus, threads) < 0) {
pr_debug("failed to open counter: %s, "
"tweak /proc/sys/kernel/perf_event_paranoid?\n",
strerror(errno));
@@ -533,7 +535,7 @@
perf_evlist__add(evlist, evsels[i]);
- if (perf_evsel__open(evsels[i], cpus, threads, false, NULL) < 0) {
+ if (perf_evsel__open(evsels[i], cpus, threads) < 0) {
pr_debug("failed to open counter: %s, "
"tweak /proc/sys/kernel/perf_event_paranoid?\n",
strerror(errno));
@@ -562,7 +564,7 @@
goto out_munmap;
}
- err = perf_evlist__parse_sample(evlist, event, &sample, false);
+ err = perf_evlist__parse_sample(evlist, event, &sample);
if (err) {
pr_err("Can't parse sample, err = %d\n", err);
goto out_munmap;
@@ -710,7 +712,7 @@
/*
* Config the evsels, setting attr->comm on the first one, etc.
*/
- evsel = list_entry(evlist->entries.next, struct perf_evsel, node);
+ evsel = perf_evlist__first(evlist);
evsel->attr.sample_type |= PERF_SAMPLE_CPU;
evsel->attr.sample_type |= PERF_SAMPLE_TID;
evsel->attr.sample_type |= PERF_SAMPLE_TIME;
@@ -737,7 +739,7 @@
* Call sys_perf_event_open on all the fds on all the evsels,
* grouping them if asked to.
*/
- err = perf_evlist__open(evlist, opts.group);
+ err = perf_evlist__open(evlist);
if (err < 0) {
pr_debug("perf_evlist__open: %s\n", strerror(errno));
goto out_delete_evlist;
@@ -779,7 +781,7 @@
if (type < PERF_RECORD_MAX)
nr_events[type]++;
- err = perf_evlist__parse_sample(evlist, event, &sample, false);
+ err = perf_evlist__parse_sample(evlist, event, &sample);
if (err < 0) {
if (verbose)
perf_event__fprintf(event, stderr);
@@ -996,7 +998,9 @@
/*
* If the RDPMC instruction faults then signal this back to the test parent task:
*/
-static void segfault_handler(int sig __used, siginfo_t *info __used, void *uc __used)
+static void segfault_handler(int sig __maybe_unused,
+ siginfo_t *info __maybe_unused,
+ void *uc __maybe_unused)
{
exit(-1);
}
@@ -1023,14 +1027,16 @@
fd = sys_perf_event_open(&attr, 0, -1, -1, 0);
if (fd < 0) {
- die("Error: sys_perf_event_open() syscall returned "
- "with %d (%s)\n", fd, strerror(errno));
+ pr_err("Error: sys_perf_event_open() syscall returned "
+ "with %d (%s)\n", fd, strerror(errno));
+ return -1;
}
addr = mmap(NULL, page_size, PROT_READ, MAP_SHARED, fd, 0);
if (addr == (void *)(-1)) {
- die("Error: mmap() syscall returned "
- "with (%s)\n", strerror(errno));
+ pr_err("Error: mmap() syscall returned with (%s)\n",
+ strerror(errno));
+ goto out_close;
}
for (n = 0; n < 6; n++) {
@@ -1051,9 +1057,9 @@
}
munmap(addr, page_size);
- close(fd);
-
pr_debug(" ");
+out_close:
+ close(fd);
if (!delta_sum)
return -1;
@@ -1092,6 +1098,309 @@
return perf_pmu__test();
}
+static int perf_evsel__roundtrip_cache_name_test(void)
+{
+ char name[128];
+ int type, op, err = 0, ret = 0, i, idx;
+ struct perf_evsel *evsel;
+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+
+ if (evlist == NULL)
+ return -ENOMEM;
+
+ for (type = 0; type < PERF_COUNT_HW_CACHE_MAX; type++) {
+ for (op = 0; op < PERF_COUNT_HW_CACHE_OP_MAX; op++) {
+ /* skip invalid cache type */
+ if (!perf_evsel__is_cache_op_valid(type, op))
+ continue;
+
+ for (i = 0; i < PERF_COUNT_HW_CACHE_RESULT_MAX; i++) {
+ __perf_evsel__hw_cache_type_op_res_name(type, op, i,
+ name, sizeof(name));
+ err = parse_events(evlist, name, 0);
+ if (err)
+ ret = err;
+ }
+ }
+ }
+
+ idx = 0;
+ evsel = perf_evlist__first(evlist);
+
+ for (type = 0; type < PERF_COUNT_HW_CACHE_MAX; type++) {
+ for (op = 0; op < PERF_COUNT_HW_CACHE_OP_MAX; op++) {
+ /* skip invalid cache type */
+ if (!perf_evsel__is_cache_op_valid(type, op))
+ continue;
+
+ for (i = 0; i < PERF_COUNT_HW_CACHE_RESULT_MAX; i++) {
+ __perf_evsel__hw_cache_type_op_res_name(type, op, i,
+ name, sizeof(name));
+ if (evsel->idx != idx)
+ continue;
+
+ ++idx;
+
+ if (strcmp(perf_evsel__name(evsel), name)) {
+ pr_debug("%s != %s\n", perf_evsel__name(evsel), name);
+ ret = -1;
+ }
+
+ evsel = perf_evsel__next(evsel);
+ }
+ }
+ }
+
+ perf_evlist__delete(evlist);
+ return ret;
+}
+
+static int __perf_evsel__name_array_test(const char *names[], int nr_names)
+{
+ int i, err;
+ struct perf_evsel *evsel;
+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+
+ if (evlist == NULL)
+ return -ENOMEM;
+
+ for (i = 0; i < nr_names; ++i) {
+ err = parse_events(evlist, names[i], 0);
+ if (err) {
+ pr_debug("failed to parse event '%s', err %d\n",
+ names[i], err);
+ goto out_delete_evlist;
+ }
+ }
+
+ err = 0;
+ list_for_each_entry(evsel, &evlist->entries, node) {
+ if (strcmp(perf_evsel__name(evsel), names[evsel->idx])) {
+ --err;
+ pr_debug("%s != %s\n", perf_evsel__name(evsel), names[evsel->idx]);
+ }
+ }
+
+out_delete_evlist:
+ perf_evlist__delete(evlist);
+ return err;
+}
+
+#define perf_evsel__name_array_test(names) \
+ __perf_evsel__name_array_test(names, ARRAY_SIZE(names))
+
+static int perf_evsel__roundtrip_name_test(void)
+{
+ int err = 0, ret = 0;
+
+ err = perf_evsel__name_array_test(perf_evsel__hw_names);
+ if (err)
+ ret = err;
+
+ err = perf_evsel__name_array_test(perf_evsel__sw_names);
+ if (err)
+ ret = err;
+
+ err = perf_evsel__roundtrip_cache_name_test();
+ if (err)
+ ret = err;
+
+ return ret;
+}
+
+static int perf_evsel__test_field(struct perf_evsel *evsel, const char *name,
+ int size, bool should_be_signed)
+{
+ struct format_field *field = perf_evsel__field(evsel, name);
+ int is_signed;
+ int ret = 0;
+
+ if (field == NULL) {
+ pr_debug("%s: \"%s\" field not found!\n", evsel->name, name);
+ return -1;
+ }
+
+ is_signed = !!(field->flags | FIELD_IS_SIGNED);
+ if (should_be_signed && !is_signed) {
+ pr_debug("%s: \"%s\" signedness(%d) is wrong, should be %d\n",
+ evsel->name, name, is_signed, should_be_signed);
+ ret = -1;
+ }
+
+ if (field->size != size) {
+ pr_debug("%s: \"%s\" size (%d) should be %d!\n",
+ evsel->name, name, field->size, size);
+ ret = -1;
+ }
+
+ return ret;
+}
+
+static int perf_evsel__tp_sched_test(void)
+{
+ struct perf_evsel *evsel = perf_evsel__newtp("sched", "sched_switch", 0);
+ int ret = 0;
+
+ if (evsel == NULL) {
+ pr_debug("perf_evsel__new\n");
+ return -1;
+ }
+
+ if (perf_evsel__test_field(evsel, "prev_comm", 16, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "prev_pid", 4, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "prev_prio", 4, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "prev_state", 8, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "next_comm", 16, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "next_pid", 4, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "next_prio", 4, true))
+ ret = -1;
+
+ perf_evsel__delete(evsel);
+
+ evsel = perf_evsel__newtp("sched", "sched_wakeup", 0);
+
+ if (perf_evsel__test_field(evsel, "comm", 16, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "pid", 4, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "prio", 4, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "success", 4, true))
+ ret = -1;
+
+ if (perf_evsel__test_field(evsel, "target_cpu", 4, true))
+ ret = -1;
+
+ return ret;
+}
+
+static int test__syscall_open_tp_fields(void)
+{
+ struct perf_record_opts opts = {
+ .target = {
+ .uid = UINT_MAX,
+ .uses_mmap = true,
+ },
+ .no_delay = true,
+ .freq = 1,
+ .mmap_pages = 256,
+ .raw_samples = true,
+ };
+ const char *filename = "/etc/passwd";
+ int flags = O_RDONLY | O_DIRECTORY;
+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+ struct perf_evsel *evsel;
+ int err = -1, i, nr_events = 0, nr_polls = 0;
+
+ if (evlist == NULL) {
+ pr_debug("%s: perf_evlist__new\n", __func__);
+ goto out;
+ }
+
+ evsel = perf_evsel__newtp("syscalls", "sys_enter_open", 0);
+ if (evsel == NULL) {
+ pr_debug("%s: perf_evsel__newtp\n", __func__);
+ goto out_delete_evlist;
+ }
+
+ perf_evlist__add(evlist, evsel);
+
+ err = perf_evlist__create_maps(evlist, &opts.target);
+ if (err < 0) {
+ pr_debug("%s: perf_evlist__create_maps\n", __func__);
+ goto out_delete_evlist;
+ }
+
+ perf_evsel__config(evsel, &opts, evsel);
+
+ evlist->threads->map[0] = getpid();
+
+ err = perf_evlist__open(evlist);
+ if (err < 0) {
+ pr_debug("perf_evlist__open: %s\n", strerror(errno));
+ goto out_delete_evlist;
+ }
+
+ err = perf_evlist__mmap(evlist, UINT_MAX, false);
+ if (err < 0) {
+ pr_debug("perf_evlist__mmap: %s\n", strerror(errno));
+ goto out_delete_evlist;
+ }
+
+ perf_evlist__enable(evlist);
+
+ /*
+ * Generate the event:
+ */
+ open(filename, flags);
+
+ while (1) {
+ int before = nr_events;
+
+ for (i = 0; i < evlist->nr_mmaps; i++) {
+ union perf_event *event;
+
+ while ((event = perf_evlist__mmap_read(evlist, i)) != NULL) {
+ const u32 type = event->header.type;
+ int tp_flags;
+ struct perf_sample sample;
+
+ ++nr_events;
+
+ if (type != PERF_RECORD_SAMPLE)
+ continue;
+
+ err = perf_evsel__parse_sample(evsel, event, &sample);
+ if (err) {
+ pr_err("Can't parse sample, err = %d\n", err);
+ goto out_munmap;
+ }
+
+ tp_flags = perf_evsel__intval(evsel, &sample, "flags");
+
+ if (flags != tp_flags) {
+ pr_debug("%s: Expected flags=%#x, got %#x\n",
+ __func__, flags, tp_flags);
+ goto out_munmap;
+ }
+
+ goto out_ok;
+ }
+ }
+
+ if (nr_events == before)
+ poll(evlist->pollfd, evlist->nr_fds, 10);
+
+ if (++nr_polls > 5) {
+ pr_debug("%s: no events!\n", __func__);
+ goto out_munmap;
+ }
+ }
+out_ok:
+ err = 0;
+out_munmap:
+ perf_evlist__munmap(evlist);
+out_delete_evlist:
+ perf_evlist__delete(evlist);
+out:
+ return err;
+}
+
static struct test {
const char *desc;
int (*func)(void);
@@ -1135,6 +1444,18 @@
.func = dso__test_data,
},
{
+ .desc = "roundtrip evsel->name check",
+ .func = perf_evsel__roundtrip_name_test,
+ },
+ {
+ .desc = "Check parsing of sched tracepoints fields",
+ .func = perf_evsel__tp_sched_test,
+ },
+ {
+ .desc = "Generate and check syscalls:sys_enter_open event fields",
+ .func = test__syscall_open_tp_fields,
+ },
+ {
.func = NULL,
},
};
@@ -1199,7 +1520,7 @@
return 0;
}
-int cmd_test(int argc, const char **argv, const char *prefix __used)
+int cmd_test(int argc, const char **argv, const char *prefix __maybe_unused)
{
const char * const test_usage[] = {
"perf test [<options>] [{list <test-name-fragment>|[<test-name-fragments>|<test-numbers>]}]",
diff --git a/tools/perf/builtin-timechart.c b/tools/perf/builtin-timechart.c
index 3b75b2e..b1a8a3b 100644
--- a/tools/perf/builtin-timechart.c
+++ b/tools/perf/builtin-timechart.c
@@ -168,9 +168,8 @@
return cursor;
cursor = cursor->next;
}
- cursor = malloc(sizeof(struct per_pid));
+ cursor = zalloc(sizeof(*cursor));
assert(cursor != NULL);
- memset(cursor, 0, sizeof(struct per_pid));
cursor->pid = pid;
cursor->next = all_data;
all_data = cursor;
@@ -195,9 +194,8 @@
}
c = c->next;
}
- c = malloc(sizeof(struct per_pidcomm));
+ c = zalloc(sizeof(*c));
assert(c != NULL);
- memset(c, 0, sizeof(struct per_pidcomm));
c->comm = strdup(comm);
p->current = c;
c->next = p->all;
@@ -239,17 +237,15 @@
p = find_create_pid(pid);
c = p->current;
if (!c) {
- c = malloc(sizeof(struct per_pidcomm));
+ c = zalloc(sizeof(*c));
assert(c != NULL);
- memset(c, 0, sizeof(struct per_pidcomm));
p->current = c;
c->next = p->all;
p->all = c;
}
- sample = malloc(sizeof(struct cpu_sample));
+ sample = zalloc(sizeof(*sample));
assert(sample != NULL);
- memset(sample, 0, sizeof(struct cpu_sample));
sample->start_time = start;
sample->end_time = end;
sample->type = type;
@@ -275,28 +271,28 @@
static u64 cpus_pstate_start_times[MAX_CPUS];
static u64 cpus_pstate_state[MAX_CPUS];
-static int process_comm_event(struct perf_tool *tool __used,
+static int process_comm_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
- struct machine *machine __used)
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
pid_set_comm(event->comm.tid, event->comm.comm);
return 0;
}
-static int process_fork_event(struct perf_tool *tool __used,
+static int process_fork_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
- struct machine *machine __used)
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
pid_fork(event->fork.pid, event->fork.ppid, event->fork.time);
return 0;
}
-static int process_exit_event(struct perf_tool *tool __used,
+static int process_exit_event(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
- struct machine *machine __used)
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
pid_exit(event->fork.pid, event->fork.time);
return 0;
@@ -373,11 +369,10 @@
static void c_state_end(int cpu, u64 timestamp)
{
- struct power_event *pwr;
- pwr = malloc(sizeof(struct power_event));
+ struct power_event *pwr = zalloc(sizeof(*pwr));
+
if (!pwr)
return;
- memset(pwr, 0, sizeof(struct power_event));
pwr->state = cpus_cstate_state[cpu];
pwr->start_time = cpus_cstate_start_times[cpu];
@@ -392,14 +387,13 @@
static void p_state_change(int cpu, u64 timestamp, u64 new_freq)
{
struct power_event *pwr;
- pwr = malloc(sizeof(struct power_event));
if (new_freq > 8000000) /* detect invalid data */
return;
+ pwr = zalloc(sizeof(*pwr));
if (!pwr)
return;
- memset(pwr, 0, sizeof(struct power_event));
pwr->state = cpus_pstate_state[cpu];
pwr->start_time = cpus_pstate_start_times[cpu];
@@ -429,15 +423,13 @@
static void
sched_wakeup(int cpu, u64 timestamp, int pid, struct trace_entry *te)
{
- struct wake_event *we;
struct per_pid *p;
struct wakeup_entry *wake = (void *)te;
+ struct wake_event *we = zalloc(sizeof(*we));
- we = malloc(sizeof(struct wake_event));
if (!we)
return;
- memset(we, 0, sizeof(struct wake_event));
we->time = timestamp;
we->waker = pid;
@@ -491,11 +483,11 @@
}
-static int process_sample_event(struct perf_tool *tool __used,
- union perf_event *event __used,
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
+ union perf_event *event __maybe_unused,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine __used)
+ struct machine *machine __maybe_unused)
{
struct trace_entry *te;
@@ -579,13 +571,12 @@
struct power_event *pwr;
for (cpu = 0; cpu <= numcpus; cpu++) {
- pwr = malloc(sizeof(struct power_event));
- if (!pwr)
- return;
- memset(pwr, 0, sizeof(struct power_event));
-
/* C state */
#if 0
+ pwr = zalloc(sizeof(*pwr));
+ if (!pwr)
+ return;
+
pwr->state = cpus_cstate_state[cpu];
pwr->start_time = cpus_cstate_start_times[cpu];
pwr->end_time = last_time;
@@ -597,10 +588,9 @@
#endif
/* P state */
- pwr = malloc(sizeof(struct power_event));
+ pwr = zalloc(sizeof(*pwr));
if (!pwr)
return;
- memset(pwr, 0, sizeof(struct power_event));
pwr->state = cpus_pstate_state[cpu];
pwr->start_time = cpus_pstate_start_times[cpu];
@@ -830,11 +820,9 @@
static void add_process_filter(const char *string)
{
- struct process_filter *filt;
- int pid;
+ int pid = strtoull(string, NULL, 10);
+ struct process_filter *filt = malloc(sizeof(*filt));
- pid = strtoull(string, NULL, 10);
- filt = malloc(sizeof(struct process_filter));
if (!filt)
return;
@@ -1081,7 +1069,8 @@
}
static int
-parse_process(const struct option *opt __used, const char *arg, int __used unset)
+parse_process(const struct option *opt __maybe_unused, const char *arg,
+ int __maybe_unused unset)
{
if (arg)
add_process_filter(arg);
@@ -1106,7 +1095,8 @@
};
-int cmd_timechart(int argc, const char **argv, const char *prefix __used)
+int cmd_timechart(int argc, const char **argv,
+ const char *prefix __maybe_unused)
{
argc = parse_options(argc, argv, options, timechart_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 68cd61e..e434a16 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -95,7 +95,8 @@
top->print_entries -= 9;
}
-static void perf_top__sig_winch(int sig __used, siginfo_t *info __used, void *arg)
+static void perf_top__sig_winch(int sig __maybe_unused,
+ siginfo_t *info __maybe_unused, void *arg)
{
struct perf_top *top = arg;
@@ -509,7 +510,7 @@
prompt_integer(&counter, "Enter details event counter");
if (counter >= top->evlist->nr_entries) {
- top->sym_evsel = list_entry(top->evlist->entries.next, struct perf_evsel, node);
+ top->sym_evsel = perf_evlist__first(top->evlist);
fprintf(stderr, "Sorry, no such event, using %s.\n", perf_evsel__name(top->sym_evsel));
sleep(1);
break;
@@ -518,7 +519,7 @@
if (top->sym_evsel->idx == counter)
break;
} else
- top->sym_evsel = list_entry(top->evlist->entries.next, struct perf_evsel, node);
+ top->sym_evsel = perf_evlist__first(top->evlist);
break;
case 'f':
prompt_integer(&top->count_filter, "Enter display event count filter");
@@ -663,7 +664,7 @@
NULL
};
-static int symbol_filter(struct map *map __used, struct symbol *sym)
+static int symbol_filter(struct map *map __maybe_unused, struct symbol *sym)
{
const char *name = sym->name;
int i;
@@ -783,8 +784,10 @@
if ((sort__has_parent || symbol_conf.use_callchain) &&
sample->callchain) {
- err = machine__resolve_callchain(machine, al.thread,
- sample->callchain, &parent);
+ err = machine__resolve_callchain(machine, evsel,
+ al.thread, sample,
+ &parent);
+
if (err)
return;
}
@@ -820,7 +823,7 @@
int ret;
while ((event = perf_evlist__mmap_read(top->evlist, idx)) != NULL) {
- ret = perf_evlist__parse_sample(top->evlist, event, &sample, false);
+ ret = perf_evlist__parse_sample(top->evlist, event, &sample);
if (ret) {
pr_err("Can't parse sample, err = %d\n", ret);
continue;
@@ -884,17 +887,14 @@
static void perf_top__start_counters(struct perf_top *top)
{
- struct perf_evsel *counter, *first;
+ struct perf_evsel *counter;
struct perf_evlist *evlist = top->evlist;
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ if (top->group)
+ perf_evlist__set_leader(evlist);
list_for_each_entry(counter, &evlist->entries, node) {
struct perf_event_attr *attr = &counter->attr;
- struct xyarray *group_fd = NULL;
-
- if (top->group && counter != first)
- group_fd = first->fd;
attr->sample_type = PERF_SAMPLE_IP | PERF_SAMPLE_TID;
@@ -925,8 +925,7 @@
attr->sample_id_all = top->sample_id_all_missing ? 0 : 1;
try_again:
if (perf_evsel__open(counter, top->evlist->cpus,
- top->evlist->threads, top->group,
- group_fd) < 0) {
+ top->evlist->threads) < 0) {
int err = errno;
if (err == EPERM || err == EACCES) {
@@ -1165,7 +1164,7 @@
NULL
};
-int cmd_top(int argc, const char **argv, const char *prefix __used)
+int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
{
struct perf_evsel *pos;
int status;
@@ -1328,7 +1327,7 @@
pos->attr.sample_period = top.default_interval;
}
- top.sym_evsel = list_entry(top.evlist->entries.next, struct perf_evsel, node);
+ top.sym_evsel = perf_evlist__first(top.evlist);
symbol_conf.priv_size = sizeof(struct annotation);
diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
new file mode 100644
index 0000000..8f113da
--- /dev/null
+++ b/tools/perf/builtin-trace.c
@@ -0,0 +1,310 @@
+#include "builtin.h"
+#include "util/evlist.h"
+#include "util/parse-options.h"
+#include "util/thread_map.h"
+#include "event-parse.h"
+
+#include <libaudit.h>
+#include <stdlib.h>
+
+static struct syscall_fmt {
+ const char *name;
+ const char *alias;
+ bool errmsg;
+ bool timeout;
+} syscall_fmts[] = {
+ { .name = "arch_prctl", .errmsg = true, .alias = "prctl", },
+ { .name = "fstat", .errmsg = true, .alias = "newfstat", },
+ { .name = "fstatat", .errmsg = true, .alias = "newfstatat", },
+ { .name = "futex", .errmsg = true, },
+ { .name = "poll", .errmsg = true, .timeout = true, },
+ { .name = "ppoll", .errmsg = true, .timeout = true, },
+ { .name = "read", .errmsg = true, },
+ { .name = "recvfrom", .errmsg = true, },
+ { .name = "select", .errmsg = true, .timeout = true, },
+ { .name = "stat", .errmsg = true, .alias = "newstat", },
+};
+
+static int syscall_fmt__cmp(const void *name, const void *fmtp)
+{
+ const struct syscall_fmt *fmt = fmtp;
+ return strcmp(name, fmt->name);
+}
+
+static struct syscall_fmt *syscall_fmt__find(const char *name)
+{
+ const int nmemb = ARRAY_SIZE(syscall_fmts);
+ return bsearch(name, syscall_fmts, nmemb, sizeof(struct syscall_fmt), syscall_fmt__cmp);
+}
+
+struct syscall {
+ struct event_format *tp_format;
+ const char *name;
+ struct syscall_fmt *fmt;
+};
+
+struct trace {
+ int audit_machine;
+ struct {
+ int max;
+ struct syscall *table;
+ } syscalls;
+ struct perf_record_opts opts;
+};
+
+static int trace__read_syscall_info(struct trace *trace, int id)
+{
+ char tp_name[128];
+ struct syscall *sc;
+
+ if (id > trace->syscalls.max) {
+ struct syscall *nsyscalls = realloc(trace->syscalls.table, (id + 1) * sizeof(*sc));
+
+ if (nsyscalls == NULL)
+ return -1;
+
+ if (trace->syscalls.max != -1) {
+ memset(nsyscalls + trace->syscalls.max + 1, 0,
+ (id - trace->syscalls.max) * sizeof(*sc));
+ } else {
+ memset(nsyscalls, 0, (id + 1) * sizeof(*sc));
+ }
+
+ trace->syscalls.table = nsyscalls;
+ trace->syscalls.max = id;
+ }
+
+ sc = trace->syscalls.table + id;
+ sc->name = audit_syscall_to_name(id, trace->audit_machine);
+ if (sc->name == NULL)
+ return -1;
+
+ sc->fmt = syscall_fmt__find(sc->name);
+
+ snprintf(tp_name, sizeof(tp_name), "sys_enter_%s", sc->name);
+ sc->tp_format = event_format__new("syscalls", tp_name);
+
+ if (sc->tp_format == NULL && sc->fmt && sc->fmt->alias) {
+ snprintf(tp_name, sizeof(tp_name), "sys_enter_%s", sc->fmt->alias);
+ sc->tp_format = event_format__new("syscalls", tp_name);
+ }
+
+ return sc->tp_format != NULL ? 0 : -1;
+}
+
+static size_t syscall__fprintf_args(struct syscall *sc, unsigned long *args, FILE *fp)
+{
+ int i = 0;
+ size_t printed = 0;
+
+ if (sc->tp_format != NULL) {
+ struct format_field *field;
+
+ for (field = sc->tp_format->format.fields->next; field; field = field->next) {
+ printed += fprintf(fp, "%s%s: %ld", printed ? ", " : "",
+ field->name, args[i++]);
+ }
+ } else {
+ while (i < 6) {
+ printed += fprintf(fp, "%sarg%d: %ld", printed ? ", " : "", i, args[i]);
+ ++i;
+ }
+ }
+
+ return printed;
+}
+
+static int trace__run(struct trace *trace)
+{
+ struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+ struct perf_evsel *evsel, *evsel_enter, *evsel_exit;
+ int err = -1, i, nr_events = 0, before;
+
+ if (evlist == NULL) {
+ printf("Not enough memory to run!\n");
+ goto out;
+ }
+
+ evsel_enter = perf_evsel__newtp("raw_syscalls", "sys_enter", 0);
+ if (evsel_enter == NULL) {
+ printf("Couldn't read the raw_syscalls:sys_enter tracepoint information!\n");
+ goto out_delete_evlist;
+ }
+
+ perf_evlist__add(evlist, evsel_enter);
+
+ evsel_exit = perf_evsel__newtp("raw_syscalls", "sys_exit", 1);
+ if (evsel_exit == NULL) {
+ printf("Couldn't read the raw_syscalls:sys_exit tracepoint information!\n");
+ goto out_delete_evlist;
+ }
+
+ perf_evlist__add(evlist, evsel_exit);
+
+ err = perf_evlist__create_maps(evlist, &trace->opts.target);
+ if (err < 0) {
+ printf("Problems parsing the target to trace, check your options!\n");
+ goto out_delete_evlist;
+ }
+
+ perf_evlist__config_attrs(evlist, &trace->opts);
+
+ err = perf_evlist__open(evlist);
+ if (err < 0) {
+ printf("Couldn't create the events: %s\n", strerror(errno));
+ goto out_delete_evlist;
+ }
+
+ err = perf_evlist__mmap(evlist, UINT_MAX, false);
+ if (err < 0) {
+ printf("Couldn't mmap the events: %s\n", strerror(errno));
+ goto out_delete_evlist;
+ }
+
+ perf_evlist__enable(evlist);
+again:
+ before = nr_events;
+
+ for (i = 0; i < evlist->nr_mmaps; i++) {
+ union perf_event *event;
+
+ while ((event = perf_evlist__mmap_read(evlist, i)) != NULL) {
+ const u32 type = event->header.type;
+ struct syscall *sc;
+ struct perf_sample sample;
+ int id;
+
+ ++nr_events;
+
+ switch (type) {
+ case PERF_RECORD_SAMPLE:
+ break;
+ case PERF_RECORD_LOST:
+ printf("LOST %" PRIu64 " events!\n", event->lost.lost);
+ continue;
+ default:
+ printf("Unexpected %s event, skipping...\n",
+ perf_event__name(type));
+ continue;
+ }
+
+ err = perf_evlist__parse_sample(evlist, event, &sample);
+ if (err) {
+ printf("Can't parse sample, err = %d, skipping...\n", err);
+ continue;
+ }
+
+ evsel = perf_evlist__id2evsel(evlist, sample.id);
+ if (evsel == NULL) {
+ printf("Unknown tp ID %" PRIu64 ", skipping...\n", sample.id);
+ continue;
+ }
+
+ id = perf_evsel__intval(evsel, &sample, "id");
+ if (id < 0) {
+ printf("Invalid syscall %d id, skipping...\n", id);
+ continue;
+ }
+
+ if ((id > trace->syscalls.max || trace->syscalls.table[id].name == NULL) &&
+ trace__read_syscall_info(trace, id))
+ continue;
+
+ if ((id > trace->syscalls.max || trace->syscalls.table[id].name == NULL))
+ continue;
+
+ sc = &trace->syscalls.table[id];
+
+ if (evlist->threads->map[0] == -1 || evlist->threads->nr > 1)
+ printf("%d ", sample.tid);
+
+ if (evsel == evsel_enter) {
+ void *args = perf_evsel__rawptr(evsel, &sample, "args");
+
+ printf("%s(", sc->name);
+ syscall__fprintf_args(sc, args, stdout);
+ } else if (evsel == evsel_exit) {
+ int ret = perf_evsel__intval(evsel, &sample, "ret");
+
+ if (ret < 0 && sc->fmt && sc->fmt->errmsg) {
+ char bf[256];
+ const char *emsg = strerror_r(-ret, bf, sizeof(bf)),
+ *e = audit_errno_to_name(-ret);
+
+ printf(") = -1 %s %s", e, emsg);
+ } else if (ret == 0 && sc->fmt && sc->fmt->timeout)
+ printf(") = 0 Timeout");
+ else
+ printf(") = %d", ret);
+
+ putchar('\n');
+ }
+ }
+ }
+
+ if (nr_events == before)
+ poll(evlist->pollfd, evlist->nr_fds, -1);
+
+ goto again;
+
+out_delete_evlist:
+ perf_evlist__delete(evlist);
+out:
+ return err;
+}
+
+int cmd_trace(int argc, const char **argv, const char *prefix __maybe_unused)
+{
+ const char * const trace_usage[] = {
+ "perf trace [<options>]",
+ NULL
+ };
+ struct trace trace = {
+ .audit_machine = audit_detect_machine(),
+ .syscalls = {
+ . max = -1,
+ },
+ .opts = {
+ .target = {
+ .uid = UINT_MAX,
+ .uses_mmap = true,
+ },
+ .user_freq = UINT_MAX,
+ .user_interval = ULLONG_MAX,
+ .no_delay = true,
+ .mmap_pages = 1024,
+ },
+ };
+ const struct option trace_options[] = {
+ OPT_STRING('p', "pid", &trace.opts.target.pid, "pid",
+ "trace events on existing process id"),
+ OPT_STRING(0, "tid", &trace.opts.target.tid, "tid",
+ "trace events on existing thread id"),
+ OPT_BOOLEAN(0, "all-cpus", &trace.opts.target.system_wide,
+ "system-wide collection from all CPUs"),
+ OPT_STRING(0, "cpu", &trace.opts.target.cpu_list, "cpu",
+ "list of cpus to monitor"),
+ OPT_BOOLEAN(0, "no-inherit", &trace.opts.no_inherit,
+ "child tasks do not inherit counters"),
+ OPT_UINTEGER(0, "mmap-pages", &trace.opts.mmap_pages,
+ "number of mmap data pages"),
+ OPT_STRING(0, "uid", &trace.opts.target.uid_str, "user",
+ "user to profile"),
+ OPT_END()
+ };
+ int err;
+
+ argc = parse_options(argc, argv, trace_options, trace_usage, 0);
+ if (argc)
+ usage_with_options(trace_usage, trace_options);
+
+ err = perf_target__parse_uid(&trace.opts.target);
+ if (err) {
+ char bf[BUFSIZ];
+ perf_target__strerror(&trace.opts.target, err, bf, sizeof(bf));
+ printf("%s", bf);
+ return err;
+ }
+
+ return trace__run(&trace);
+}
diff --git a/tools/perf/builtin.h b/tools/perf/builtin.h
index b382bd5..08143bd 100644
--- a/tools/perf/builtin.h
+++ b/tools/perf/builtin.h
@@ -34,6 +34,8 @@
extern int cmd_lock(int argc, const char **argv, const char *prefix);
extern int cmd_kvm(int argc, const char **argv, const char *prefix);
extern int cmd_test(int argc, const char **argv, const char *prefix);
+extern int cmd_trace(int argc, const char **argv, const char *prefix);
extern int cmd_inject(int argc, const char **argv, const char *prefix);
+extern int find_scripts(char **scripts_array, char **scripts_path_array);
#endif
diff --git a/tools/perf/command-list.txt b/tools/perf/command-list.txt
index d695fe4..3e86bbd 100644
--- a/tools/perf/command-list.txt
+++ b/tools/perf/command-list.txt
@@ -17,8 +17,9 @@
perf-stat mainporcelain common
perf-timechart mainporcelain common
perf-top mainporcelain common
+perf-trace mainporcelain common
perf-script mainporcelain common
-perf-probe mainporcelain common
+perf-probe mainporcelain full
perf-kmem mainporcelain common
perf-lock mainporcelain common
perf-kvm mainporcelain common
diff --git a/tools/perf/config/feature-tests.mak b/tools/perf/config/feature-tests.mak
index 6c18785..4add41b 100644
--- a/tools/perf/config/feature-tests.mak
+++ b/tools/perf/config/feature-tests.mak
@@ -154,3 +154,53 @@
return 0;
}
endef
+
+ifndef NO_LIBUNWIND
+define SOURCE_LIBUNWIND
+#include <libunwind.h>
+#include <stdlib.h>
+
+extern int UNW_OBJ(dwarf_search_unwind_table) (unw_addr_space_t as,
+ unw_word_t ip,
+ unw_dyn_info_t *di,
+ unw_proc_info_t *pi,
+ int need_unwind_info, void *arg);
+
+
+#define dwarf_search_unwind_table UNW_OBJ(dwarf_search_unwind_table)
+
+int main(void)
+{
+ unw_addr_space_t addr_space;
+ addr_space = unw_create_addr_space(NULL, 0);
+ unw_init_remote(NULL, addr_space, NULL);
+ dwarf_search_unwind_table(addr_space, 0, NULL, NULL, 0, NULL);
+ return 0;
+}
+endef
+endif
+
+ifndef NO_BACKTRACE
+define SOURCE_BACKTRACE
+#include <execinfo.h>
+#include <stdio.h>
+
+int main(void)
+{
+ backtrace(NULL, 0);
+ backtrace_symbols(NULL, 0);
+ return 0;
+}
+endef
+endif
+
+ifndef NO_LIBAUDIT
+define SOURCE_LIBAUDIT
+#include <libaudit.h>
+
+int main(void)
+{
+ return audit_open();
+}
+endef
+endif
\ No newline at end of file
diff --git a/tools/perf/perf-archive.sh b/tools/perf/perf-archive.sh
index 95b6f8b..e919306 100644
--- a/tools/perf/perf-archive.sh
+++ b/tools/perf/perf-archive.sh
@@ -24,7 +24,7 @@
perf buildid-list -i $PERF_DATA --with-hits | grep -v "^$NOBUILDID " > $BUILDIDS
if [ ! -s $BUILDIDS ] ; then
echo "perf archive: no build-ids found"
- rm -f $BUILDIDS
+ rm $BUILDIDS || true
exit 1
fi
@@ -39,8 +39,8 @@
echo ${filename#$PERF_BUILDID_LINKDIR} >> $MANIFEST
done
-tar cfj $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
-rm -f $MANIFEST $BUILDIDS
+tar cjf $PERF_DATA.tar.bz2 -C $PERF_BUILDID_DIR -T $MANIFEST
+rm $MANIFEST $BUILDIDS || true
echo -e "Now please run:\n"
echo -e "$ tar xvf $PERF_DATA.tar.bz2 -C ~/.debug\n"
echo "wherever you need to run 'perf report' on."
diff --git a/tools/perf/perf.c b/tools/perf/perf.c
index 2b2e225..fc2f770 100644
--- a/tools/perf/perf.c
+++ b/tools/perf/perf.c
@@ -14,6 +14,7 @@
#include "util/run-command.h"
#include "util/parse-events.h"
#include "util/debugfs.h"
+#include <pthread.h>
const char perf_usage_string[] =
"perf [--version] [--help] COMMAND [ARGS]";
@@ -24,6 +25,42 @@
int use_browser = -1;
static int use_pager = -1;
+struct cmd_struct {
+ const char *cmd;
+ int (*fn)(int, const char **, const char *);
+ int option;
+};
+
+static struct cmd_struct commands[] = {
+ { "buildid-cache", cmd_buildid_cache, 0 },
+ { "buildid-list", cmd_buildid_list, 0 },
+ { "diff", cmd_diff, 0 },
+ { "evlist", cmd_evlist, 0 },
+ { "help", cmd_help, 0 },
+ { "list", cmd_list, 0 },
+ { "record", cmd_record, 0 },
+ { "report", cmd_report, 0 },
+ { "bench", cmd_bench, 0 },
+ { "stat", cmd_stat, 0 },
+ { "timechart", cmd_timechart, 0 },
+ { "top", cmd_top, 0 },
+ { "annotate", cmd_annotate, 0 },
+ { "version", cmd_version, 0 },
+ { "script", cmd_script, 0 },
+ { "sched", cmd_sched, 0 },
+#ifndef NO_LIBELF_SUPPORT
+ { "probe", cmd_probe, 0 },
+#endif
+ { "kmem", cmd_kmem, 0 },
+ { "lock", cmd_lock, 0 },
+ { "kvm", cmd_kvm, 0 },
+ { "test", cmd_test, 0 },
+#ifndef NO_LIBAUDIT_SUPPORT
+ { "trace", cmd_trace, 0 },
+#endif
+ { "inject", cmd_inject, 0 },
+};
+
struct pager_config {
const char *cmd;
int val;
@@ -160,6 +197,14 @@
fprintf(stderr, "dir: %s\n", debugfs_mountpoint);
if (envchanged)
*envchanged = 1;
+ } else if (!strcmp(cmd, "--list-cmds")) {
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(commands); i++) {
+ struct cmd_struct *p = commands+i;
+ printf("%s ", p->cmd);
+ }
+ exit(0);
} else {
fprintf(stderr, "Unknown option: %s\n", cmd);
usage(perf_usage_string);
@@ -245,12 +290,6 @@
*/
#define NEED_WORK_TREE (1<<2)
-struct cmd_struct {
- const char *cmd;
- int (*fn)(int, const char **, const char *);
- int option;
-};
-
static int run_builtin(struct cmd_struct *p, int argc, const char **argv)
{
int status;
@@ -296,30 +335,6 @@
static void handle_internal_command(int argc, const char **argv)
{
const char *cmd = argv[0];
- static struct cmd_struct commands[] = {
- { "buildid-cache", cmd_buildid_cache, 0 },
- { "buildid-list", cmd_buildid_list, 0 },
- { "diff", cmd_diff, 0 },
- { "evlist", cmd_evlist, 0 },
- { "help", cmd_help, 0 },
- { "list", cmd_list, 0 },
- { "record", cmd_record, 0 },
- { "report", cmd_report, 0 },
- { "bench", cmd_bench, 0 },
- { "stat", cmd_stat, 0 },
- { "timechart", cmd_timechart, 0 },
- { "top", cmd_top, 0 },
- { "annotate", cmd_annotate, 0 },
- { "version", cmd_version, 0 },
- { "script", cmd_script, 0 },
- { "sched", cmd_sched, 0 },
- { "probe", cmd_probe, 0 },
- { "kmem", cmd_kmem, 0 },
- { "lock", cmd_lock, 0 },
- { "kvm", cmd_kvm, 0 },
- { "test", cmd_test, 0 },
- { "inject", cmd_inject, 0 },
- };
unsigned int i;
static const char ext[] = STRIP_EXTENSION;
diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index f960ccb..a89cbbb 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -88,6 +88,12 @@
#define CPUINFO_PROC "Processor"
#endif
+#ifdef __aarch64__
+#include "../../arch/arm64/include/asm/unistd.h"
+#define rmb() asm volatile("dmb ld" ::: "memory")
+#define cpu_relax() asm volatile("yield" ::: "memory")
+#endif
+
#ifdef __mips__
#include "../../arch/mips/include/asm/unistd.h"
#define rmb() asm volatile( \
@@ -209,9 +215,15 @@
#include "util/target.h"
+enum perf_call_graph_mode {
+ CALLCHAIN_NONE,
+ CALLCHAIN_FP,
+ CALLCHAIN_DWARF
+};
+
struct perf_record_opts {
struct perf_target target;
- bool call_graph;
+ int call_graph;
bool group;
bool inherit_stat;
bool no_delay;
@@ -230,6 +242,7 @@
u64 branch_stack;
u64 default_interval;
u64 user_interval;
+ u16 stack_dump_size;
};
#endif
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py b/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py
new file mode 100755
index 0000000..9e098579
--- /dev/null
+++ b/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py
@@ -0,0 +1,94 @@
+# EventClass.py
+#
+# This is a library defining some events types classes, which could
+# be used by other scripts to analyzing the perf samples.
+#
+# Currently there are just a few classes defined for examples,
+# PerfEvent is the base class for all perf event sample, PebsEvent
+# is a HW base Intel x86 PEBS event, and user could add more SW/HW
+# event classes based on requirements.
+
+import struct
+
+# Event types, user could add more here
+EVTYPE_GENERIC = 0
+EVTYPE_PEBS = 1 # Basic PEBS event
+EVTYPE_PEBS_LL = 2 # PEBS event with load latency info
+EVTYPE_IBS = 3
+
+#
+# Currently we don't have good way to tell the event type, but by
+# the size of raw buffer, raw PEBS event with load latency data's
+# size is 176 bytes, while the pure PEBS event's size is 144 bytes.
+#
+def create_event(name, comm, dso, symbol, raw_buf):
+ if (len(raw_buf) == 144):
+ event = PebsEvent(name, comm, dso, symbol, raw_buf)
+ elif (len(raw_buf) == 176):
+ event = PebsNHM(name, comm, dso, symbol, raw_buf)
+ else:
+ event = PerfEvent(name, comm, dso, symbol, raw_buf)
+
+ return event
+
+class PerfEvent(object):
+ event_num = 0
+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_GENERIC):
+ self.name = name
+ self.comm = comm
+ self.dso = dso
+ self.symbol = symbol
+ self.raw_buf = raw_buf
+ self.ev_type = ev_type
+ PerfEvent.event_num += 1
+
+ def show(self):
+ print "PMU event: name=%12s, symbol=%24s, comm=%8s, dso=%12s" % (self.name, self.symbol, self.comm, self.dso)
+
+#
+# Basic Intel PEBS (Precise Event-based Sampling) event, whose raw buffer
+# contains the context info when that event happened: the EFLAGS and
+# linear IP info, as well as all the registers.
+#
+class PebsEvent(PerfEvent):
+ pebs_num = 0
+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS):
+ tmp_buf=raw_buf[0:80]
+ flags, ip, ax, bx, cx, dx, si, di, bp, sp = struct.unpack('QQQQQQQQQQ', tmp_buf)
+ self.flags = flags
+ self.ip = ip
+ self.ax = ax
+ self.bx = bx
+ self.cx = cx
+ self.dx = dx
+ self.si = si
+ self.di = di
+ self.bp = bp
+ self.sp = sp
+
+ PerfEvent.__init__(self, name, comm, dso, symbol, raw_buf, ev_type)
+ PebsEvent.pebs_num += 1
+ del tmp_buf
+
+#
+# Intel Nehalem and Westmere support PEBS plus Load Latency info which lie
+# in the four 64 bit words write after the PEBS data:
+# Status: records the IA32_PERF_GLOBAL_STATUS register value
+# DLA: Data Linear Address (EIP)
+# DSE: Data Source Encoding, where the latency happens, hit or miss
+# in L1/L2/L3 or IO operations
+# LAT: the actual latency in cycles
+#
+class PebsNHM(PebsEvent):
+ pebs_nhm_num = 0
+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS_LL):
+ tmp_buf=raw_buf[144:176]
+ status, dla, dse, lat = struct.unpack('QQQQ', tmp_buf)
+ self.status = status
+ self.dla = dla
+ self.dse = dse
+ self.lat = lat
+
+ PebsEvent.__init__(self, name, comm, dso, symbol, raw_buf, ev_type)
+ PebsNHM.pebs_nhm_num += 1
+ del tmp_buf
diff --git a/tools/perf/scripts/python/bin/event_analyzing_sample-record b/tools/perf/scripts/python/bin/event_analyzing_sample-record
new file mode 100644
index 0000000..5ce652d
--- /dev/null
+++ b/tools/perf/scripts/python/bin/event_analyzing_sample-record
@@ -0,0 +1,8 @@
+#!/bin/bash
+
+#
+# event_analyzing_sample.py can cover all type of perf samples including
+# the tracepoints, so no special record requirements, just record what
+# you want to analyze.
+#
+perf record $@
diff --git a/tools/perf/scripts/python/bin/event_analyzing_sample-report b/tools/perf/scripts/python/bin/event_analyzing_sample-report
new file mode 100644
index 0000000..0941fc9
--- /dev/null
+++ b/tools/perf/scripts/python/bin/event_analyzing_sample-report
@@ -0,0 +1,3 @@
+#!/bin/bash
+# description: analyze all perf samples
+perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/event_analyzing_sample.py
diff --git a/tools/perf/scripts/python/event_analyzing_sample.py b/tools/perf/scripts/python/event_analyzing_sample.py
new file mode 100644
index 0000000..163c39f
--- /dev/null
+++ b/tools/perf/scripts/python/event_analyzing_sample.py
@@ -0,0 +1,189 @@
+# event_analyzing_sample.py: general event handler in python
+#
+# Current perf report is already very powerful with the annotation integrated,
+# and this script is not trying to be as powerful as perf report, but
+# providing end user/developer a flexible way to analyze the events other
+# than trace points.
+#
+# The 2 database related functions in this script just show how to gather
+# the basic information, and users can modify and write their own functions
+# according to their specific requirement.
+#
+# The first function "show_general_events" just does a basic grouping for all
+# generic events with the help of sqlite, and the 2nd one "show_pebs_ll" is
+# for a x86 HW PMU event: PEBS with load latency data.
+#
+
+import os
+import sys
+import math
+import struct
+import sqlite3
+
+sys.path.append(os.environ['PERF_EXEC_PATH'] + \
+ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
+
+from perf_trace_context import *
+from EventClass import *
+
+#
+# If the perf.data has a big number of samples, then the insert operation
+# will be very time consuming (about 10+ minutes for 10000 samples) if the
+# .db database is on disk. Move the .db file to RAM based FS to speedup
+# the handling, which will cut the time down to several seconds.
+#
+con = sqlite3.connect("/dev/shm/perf.db")
+con.isolation_level = None
+
+def trace_begin():
+ print "In trace_begin:\n"
+
+ #
+ # Will create several tables at the start, pebs_ll is for PEBS data with
+ # load latency info, while gen_events is for general event.
+ #
+ con.execute("""
+ create table if not exists gen_events (
+ name text,
+ symbol text,
+ comm text,
+ dso text
+ );""")
+ con.execute("""
+ create table if not exists pebs_ll (
+ name text,
+ symbol text,
+ comm text,
+ dso text,
+ flags integer,
+ ip integer,
+ status integer,
+ dse integer,
+ dla integer,
+ lat integer
+ );""")
+
+#
+# Create and insert event object to a database so that user could
+# do more analysis with simple database commands.
+#
+def process_event(param_dict):
+ event_attr = param_dict["attr"]
+ sample = param_dict["sample"]
+ raw_buf = param_dict["raw_buf"]
+ comm = param_dict["comm"]
+ name = param_dict["ev_name"]
+
+ # Symbol and dso info are not always resolved
+ if (param_dict.has_key("dso")):
+ dso = param_dict["dso"]
+ else:
+ dso = "Unknown_dso"
+
+ if (param_dict.has_key("symbol")):
+ symbol = param_dict["symbol"]
+ else:
+ symbol = "Unknown_symbol"
+
+ # Create the event object and insert it to the right table in database
+ event = create_event(name, comm, dso, symbol, raw_buf)
+ insert_db(event)
+
+def insert_db(event):
+ if event.ev_type == EVTYPE_GENERIC:
+ con.execute("insert into gen_events values(?, ?, ?, ?)",
+ (event.name, event.symbol, event.comm, event.dso))
+ elif event.ev_type == EVTYPE_PEBS_LL:
+ event.ip &= 0x7fffffffffffffff
+ event.dla &= 0x7fffffffffffffff
+ con.execute("insert into pebs_ll values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
+ (event.name, event.symbol, event.comm, event.dso, event.flags,
+ event.ip, event.status, event.dse, event.dla, event.lat))
+
+def trace_end():
+ print "In trace_end:\n"
+ # We show the basic info for the 2 type of event classes
+ show_general_events()
+ show_pebs_ll()
+ con.close()
+
+#
+# As the event number may be very big, so we can't use linear way
+# to show the histogram in real number, but use a log2 algorithm.
+#
+
+def num2sym(num):
+ # Each number will have at least one '#'
+ snum = '#' * (int)(math.log(num, 2) + 1)
+ return snum
+
+def show_general_events():
+
+ # Check the total record number in the table
+ count = con.execute("select count(*) from gen_events")
+ for t in count:
+ print "There is %d records in gen_events table" % t[0]
+ if t[0] == 0:
+ return
+
+ print "Statistics about the general events grouped by thread/symbol/dso: \n"
+
+ # Group by thread
+ commq = con.execute("select comm, count(comm) from gen_events group by comm order by -count(comm)")
+ print "\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42)
+ for row in commq:
+ print "%16s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+ # Group by symbol
+ print "\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58)
+ symbolq = con.execute("select symbol, count(symbol) from gen_events group by symbol order by -count(symbol)")
+ for row in symbolq:
+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+ # Group by dso
+ print "\n%40s %8s %16s\n%s" % ("dso", "number", "histogram", "="*74)
+ dsoq = con.execute("select dso, count(dso) from gen_events group by dso order by -count(dso)")
+ for row in dsoq:
+ print "%40s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+#
+# This function just shows the basic info, and we could do more with the
+# data in the tables, like checking the function parameters when some
+# big latency events happen.
+#
+def show_pebs_ll():
+
+ count = con.execute("select count(*) from pebs_ll")
+ for t in count:
+ print "There is %d records in pebs_ll table" % t[0]
+ if t[0] == 0:
+ return
+
+ print "Statistics about the PEBS Load Latency events grouped by thread/symbol/dse/latency: \n"
+
+ # Group by thread
+ commq = con.execute("select comm, count(comm) from pebs_ll group by comm order by -count(comm)")
+ print "\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42)
+ for row in commq:
+ print "%16s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+ # Group by symbol
+ print "\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58)
+ symbolq = con.execute("select symbol, count(symbol) from pebs_ll group by symbol order by -count(symbol)")
+ for row in symbolq:
+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+ # Group by dse
+ dseq = con.execute("select dse, count(dse) from pebs_ll group by dse order by -count(dse)")
+ print "\n%32s %8s %16s\n%s" % ("dse", "number", "histogram", "="*58)
+ for row in dseq:
+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+ # Group by latency
+ latq = con.execute("select lat, count(lat) from pebs_ll group by lat order by lat")
+ print "\n%32s %8s %16s\n%s" % ("latency", "number", "histogram", "="*58)
+ for row in latq:
+ print "%32s %8d %s" % (row[0], row[1], num2sym(row[1]))
+
+def trace_unhandled(event_name, context, event_fields_dict):
+ print ' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())])
diff --git a/tools/perf/ui/browser.c b/tools/perf/ui/browser.c
index 1818a53..4aeb7d5 100644
--- a/tools/perf/ui/browser.c
+++ b/tools/perf/ui/browser.c
@@ -269,7 +269,7 @@
return err ? 0 : -1;
}
-void ui_browser__hide(struct ui_browser *browser __used)
+void ui_browser__hide(struct ui_browser *browser __maybe_unused)
{
pthread_mutex_lock(&ui__lock);
ui_helpline__pop();
@@ -518,7 +518,7 @@
static int ui_browser__color_config(const char *var, const char *value,
- void *data __used)
+ void *data __maybe_unused)
{
char *fg = NULL, *bg;
int i;
@@ -602,7 +602,8 @@
SLsmg_set_char_set(0);
}
-void ui_browser__write_graph(struct ui_browser *browser __used, int graph)
+void ui_browser__write_graph(struct ui_browser *browser __maybe_unused,
+ int graph)
{
SLsmg_set_char_set(1);
SLsmg_write_char(graph);
diff --git a/tools/perf/ui/browsers/annotate.c b/tools/perf/ui/browsers/annotate.c
index 67a2703..8f8cd2d 100644
--- a/tools/perf/ui/browsers/annotate.c
+++ b/tools/perf/ui/browsers/annotate.c
@@ -54,7 +54,8 @@
return (struct browser_disasm_line *)(dl + 1);
}
-static bool disasm_line__filter(struct ui_browser *browser __used, void *entry)
+static bool disasm_line__filter(struct ui_browser *browser __maybe_unused,
+ void *entry)
{
if (annotate_browser__opts.hide_src_code) {
struct disasm_line *dl = list_entry(entry, struct disasm_line, node);
@@ -928,7 +929,8 @@
return strcmp(name, cfg->name);
}
-static int annotate__config(const char *var, const char *value, void *data __used)
+static int annotate__config(const char *var, const char *value,
+ void *data __maybe_unused)
{
struct annotate__config *cfg;
const char *name;
diff --git a/tools/perf/ui/browsers/hists.c b/tools/perf/ui/browsers/hists.c
index 413bd62..a21f40b 100644
--- a/tools/perf/ui/browsers/hists.c
+++ b/tools/perf/ui/browsers/hists.c
@@ -24,9 +24,12 @@
struct hist_entry *he_selection;
struct map_symbol *selection;
int print_seq;
+ bool show_dso;
bool has_symbols;
};
+extern void hist_browser__init_hpp(void);
+
static int hists__browser_title(struct hists *hists, char *bf, size_t size,
const char *ev_name);
@@ -376,12 +379,19 @@
}
static char *callchain_list__sym_name(struct callchain_list *cl,
- char *bf, size_t bfsize)
+ char *bf, size_t bfsize, bool show_dso)
{
- if (cl->ms.sym)
- return cl->ms.sym->name;
+ int printed;
- snprintf(bf, bfsize, "%#" PRIx64, cl->ip);
+ if (cl->ms.sym)
+ printed = scnprintf(bf, bfsize, "%s", cl->ms.sym->name);
+ else
+ printed = scnprintf(bf, bfsize, "%#" PRIx64, cl->ip);
+
+ if (show_dso)
+ scnprintf(bf + printed, bfsize - printed, " %s",
+ cl->ms.map ? cl->ms.map->dso->short_name : "unknown");
+
return bf;
}
@@ -417,7 +427,7 @@
remaining -= cumul;
list_for_each_entry(chain, &child->val, list) {
- char ipstr[BITS_PER_LONG / 4 + 1], *alloc_str;
+ char bf[1024], *alloc_str;
const char *str;
int color;
bool was_first = first;
@@ -434,7 +444,8 @@
}
alloc_str = NULL;
- str = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
+ str = callchain_list__sym_name(chain, bf, sizeof(bf),
+ browser->show_dso);
if (was_first) {
double percent = cumul * 100.0 / new_total;
@@ -493,7 +504,7 @@
char folded_sign = ' ';
list_for_each_entry(chain, &node->val, list) {
- char ipstr[BITS_PER_LONG / 4 + 1], *s;
+ char bf[1024], *s;
int color;
folded_sign = callchain_list__folded(chain);
@@ -510,7 +521,8 @@
*is_current_entry = true;
}
- s = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
+ s = callchain_list__sym_name(chain, bf, sizeof(bf),
+ browser->show_dso);
ui_browser__gotorc(&browser->b, row, 0);
ui_browser__set_color(&browser->b, color);
slsmg_write_nstring(" ", offset);
@@ -553,14 +565,47 @@
return row - first_row;
}
+#define HPP__COLOR_FN(_name, _field) \
+static int hist_browser__hpp_color_ ## _name(struct perf_hpp *hpp, \
+ struct hist_entry *he) \
+{ \
+ double percent = 100.0 * he->_field / hpp->total_period; \
+ *(double *)hpp->ptr = percent; \
+ return scnprintf(hpp->buf, hpp->size, "%6.2f%%", percent); \
+}
+
+HPP__COLOR_FN(overhead, period)
+HPP__COLOR_FN(overhead_sys, period_sys)
+HPP__COLOR_FN(overhead_us, period_us)
+HPP__COLOR_FN(overhead_guest_sys, period_guest_sys)
+HPP__COLOR_FN(overhead_guest_us, period_guest_us)
+
+#undef HPP__COLOR_FN
+
+void hist_browser__init_hpp(void)
+{
+ perf_hpp__init(false, false);
+
+ perf_hpp__format[PERF_HPP__OVERHEAD].color =
+ hist_browser__hpp_color_overhead;
+ perf_hpp__format[PERF_HPP__OVERHEAD_SYS].color =
+ hist_browser__hpp_color_overhead_sys;
+ perf_hpp__format[PERF_HPP__OVERHEAD_US].color =
+ hist_browser__hpp_color_overhead_us;
+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_SYS].color =
+ hist_browser__hpp_color_overhead_guest_sys;
+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_US].color =
+ hist_browser__hpp_color_overhead_guest_us;
+}
+
static int hist_browser__show_entry(struct hist_browser *browser,
struct hist_entry *entry,
unsigned short row)
{
char s[256];
double percent;
- int printed = 0;
- int width = browser->b.width - 6; /* The percentage */
+ int i, printed = 0;
+ int width = browser->b.width;
char folded_sign = ' ';
bool current_entry = ui_browser__is_current_entry(&browser->b, row);
off_t row_offset = entry->row_offset;
@@ -576,35 +621,50 @@
}
if (row_offset == 0) {
- hist_entry__snprintf(entry, s, sizeof(s), browser->hists);
- percent = (entry->period * 100.0) / browser->hists->stats.total_period;
+ struct perf_hpp hpp = {
+ .buf = s,
+ .size = sizeof(s),
+ .total_period = browser->hists->stats.total_period,
+ };
- ui_browser__set_percent_color(&browser->b, percent, current_entry);
ui_browser__gotorc(&browser->b, row, 0);
- if (symbol_conf.use_callchain) {
- slsmg_printf("%c ", folded_sign);
- width -= 2;
- }
- slsmg_printf(" %5.2f%%", percent);
+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
+ if (!perf_hpp__format[i].cond)
+ continue;
+
+ if (i) {
+ slsmg_printf(" ");
+ width -= 2;
+ }
+
+ if (perf_hpp__format[i].color) {
+ hpp.ptr = &percent;
+ /* It will set percent for us. See HPP__COLOR_FN above. */
+ width -= perf_hpp__format[i].color(&hpp, entry);
+
+ ui_browser__set_percent_color(&browser->b, percent, current_entry);
+
+ if (i == 0 && symbol_conf.use_callchain) {
+ slsmg_printf("%c ", folded_sign);
+ width -= 2;
+ }
+
+ slsmg_printf("%s", s);
+
+ if (!current_entry || !browser->b.navkeypressed)
+ ui_browser__set_color(&browser->b, HE_COLORSET_NORMAL);
+ } else {
+ width -= perf_hpp__format[i].entry(&hpp, entry);
+ slsmg_printf("%s", s);
+ }
+ }
/* The scroll bar isn't being used */
if (!browser->b.navkeypressed)
width += 1;
- if (!current_entry || !browser->b.navkeypressed)
- ui_browser__set_color(&browser->b, HE_COLORSET_NORMAL);
-
- if (symbol_conf.show_nr_samples) {
- slsmg_printf(" %11u", entry->nr_events);
- width -= 12;
- }
-
- if (symbol_conf.show_total_period) {
- slsmg_printf(" %12" PRIu64, entry->period);
- width -= 13;
- }
-
+ hist_entry__sort_snprintf(entry, s, sizeof(s), browser->hists);
slsmg_write_nstring(s, width);
++row;
++printed;
@@ -830,7 +890,7 @@
remaining -= cumul;
list_for_each_entry(chain, &child->val, list) {
- char ipstr[BITS_PER_LONG / 4 + 1], *alloc_str;
+ char bf[1024], *alloc_str;
const char *str;
bool was_first = first;
@@ -842,7 +902,8 @@
folded_sign = callchain_list__folded(chain);
alloc_str = NULL;
- str = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
+ str = callchain_list__sym_name(chain, bf, sizeof(bf),
+ browser->show_dso);
if (was_first) {
double percent = cumul * 100.0 / new_total;
@@ -880,10 +941,10 @@
int printed = 0;
list_for_each_entry(chain, &node->val, list) {
- char ipstr[BITS_PER_LONG / 4 + 1], *s;
+ char bf[1024], *s;
folded_sign = callchain_list__folded(chain);
- s = callchain_list__sym_name(chain, ipstr, sizeof(ipstr));
+ s = callchain_list__sym_name(chain, bf, sizeof(bf), browser->show_dso);
printed += fprintf(fp, "%*s%c %s\n", offset, " ", folded_sign, s);
}
@@ -920,7 +981,7 @@
if (symbol_conf.use_callchain)
folded_sign = hist_entry__folded(he);
- hist_entry__snprintf(he, s, sizeof(s), browser->hists);
+ hist_entry__sort_snprintf(he, s, sizeof(s), browser->hists);
percent = (he->period * 100.0) / browser->hists->stats.total_period;
if (symbol_conf.use_callchain)
@@ -1133,6 +1194,9 @@
continue;
case 'd':
goto zoom_dso;
+ case 'V':
+ browser->show_dso = !browser->show_dso;
+ continue;
case 't':
goto zoom_thread;
case '/':
@@ -1164,6 +1228,7 @@
"d Zoom into current DSO\n"
"t Zoom into current Thread\n"
"P Print histograms to perf.hist.N\n"
+ "V Verbose (DSO names in callchains, etc)\n"
"/ Filter symbol by name");
continue;
case K_ENTER:
diff --git a/tools/perf/ui/gtk/browser.c b/tools/perf/ui/gtk/browser.c
index ec12e0b..7ff99ec 100644
--- a/tools/perf/ui/gtk/browser.c
+++ b/tools/perf/ui/gtk/browser.c
@@ -3,6 +3,7 @@
#include "../evsel.h"
#include "../sort.h"
#include "../hist.h"
+#include "../helpline.h"
#include "gtk.h"
#include <signal.h>
@@ -35,6 +36,57 @@
gtk_window_resize(GTK_WINDOW(window), width, height);
}
+static const char *perf_gtk__get_percent_color(double percent)
+{
+ if (percent >= MIN_RED)
+ return "<span fgcolor='red'>";
+ if (percent >= MIN_GREEN)
+ return "<span fgcolor='dark green'>";
+ return NULL;
+}
+
+#define HPP__COLOR_FN(_name, _field) \
+static int perf_gtk__hpp_color_ ## _name(struct perf_hpp *hpp, \
+ struct hist_entry *he) \
+{ \
+ double percent = 100.0 * he->_field / hpp->total_period; \
+ const char *markup; \
+ int ret = 0; \
+ \
+ markup = perf_gtk__get_percent_color(percent); \
+ if (markup) \
+ ret += scnprintf(hpp->buf, hpp->size, "%s", markup); \
+ ret += scnprintf(hpp->buf + ret, hpp->size - ret, "%6.2f%%", percent); \
+ if (markup) \
+ ret += scnprintf(hpp->buf + ret, hpp->size - ret, "</span>"); \
+ \
+ return ret; \
+}
+
+HPP__COLOR_FN(overhead, period)
+HPP__COLOR_FN(overhead_sys, period_sys)
+HPP__COLOR_FN(overhead_us, period_us)
+HPP__COLOR_FN(overhead_guest_sys, period_guest_sys)
+HPP__COLOR_FN(overhead_guest_us, period_guest_us)
+
+#undef HPP__COLOR_FN
+
+void perf_gtk__init_hpp(void)
+{
+ perf_hpp__init(false, false);
+
+ perf_hpp__format[PERF_HPP__OVERHEAD].color =
+ perf_gtk__hpp_color_overhead;
+ perf_hpp__format[PERF_HPP__OVERHEAD_SYS].color =
+ perf_gtk__hpp_color_overhead_sys;
+ perf_hpp__format[PERF_HPP__OVERHEAD_US].color =
+ perf_gtk__hpp_color_overhead_us;
+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_SYS].color =
+ perf_gtk__hpp_color_overhead_guest_sys;
+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_US].color =
+ perf_gtk__hpp_color_overhead_guest_us;
+}
+
static void perf_gtk__show_hists(GtkWidget *window, struct hists *hists)
{
GType col_types[MAX_COLUMNS];
@@ -42,15 +94,25 @@
struct sort_entry *se;
GtkListStore *store;
struct rb_node *nd;
- u64 total_period;
GtkWidget *view;
- int col_idx;
+ int i, col_idx;
int nr_cols;
+ char s[512];
+
+ struct perf_hpp hpp = {
+ .buf = s,
+ .size = sizeof(s),
+ .total_period = hists->stats.total_period,
+ };
nr_cols = 0;
- /* The percentage column */
- col_types[nr_cols++] = G_TYPE_STRING;
+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
+ if (!perf_hpp__format[i].cond)
+ continue;
+
+ col_types[nr_cols++] = G_TYPE_STRING;
+ }
list_for_each_entry(se, &hist_entry__sort_list, list) {
if (se->elide)
@@ -67,11 +129,17 @@
col_idx = 0;
- /* The percentage column */
- gtk_tree_view_insert_column_with_attributes(GTK_TREE_VIEW(view),
- -1, "Overhead (%)",
- renderer, "text",
- col_idx++, NULL);
+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
+ if (!perf_hpp__format[i].cond)
+ continue;
+
+ perf_hpp__format[i].header(&hpp);
+
+ gtk_tree_view_insert_column_with_attributes(GTK_TREE_VIEW(view),
+ -1, s,
+ renderer, "markup",
+ col_idx++, NULL);
+ }
list_for_each_entry(se, &hist_entry__sort_list, list) {
if (se->elide)
@@ -87,13 +155,9 @@
g_object_unref(GTK_TREE_MODEL(store));
- total_period = hists->stats.total_period;
-
for (nd = rb_first(&hists->entries); nd; nd = rb_next(nd)) {
struct hist_entry *h = rb_entry(nd, struct hist_entry, rb_node);
GtkTreeIter iter;
- double percent;
- char s[512];
if (h->filtered)
continue;
@@ -102,11 +166,17 @@
col_idx = 0;
- percent = (h->period * 100.0) / total_period;
+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
+ if (!perf_hpp__format[i].cond)
+ continue;
- snprintf(s, ARRAY_SIZE(s), "%.2f", percent);
+ if (perf_hpp__format[i].color)
+ perf_hpp__format[i].color(&hpp, h);
+ else
+ perf_hpp__format[i].entry(&hpp, h);
- gtk_list_store_set(store, &iter, col_idx++, s, -1);
+ gtk_list_store_set(store, &iter, col_idx++, s, -1);
+ }
list_for_each_entry(se, &hist_entry__sort_list, list) {
if (se->elide)
@@ -166,9 +236,10 @@
}
int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist,
- const char *help __used,
- void (*timer) (void *arg)__used,
- void *arg __used, int delay_secs __used)
+ const char *help,
+ void (*timer) (void *arg)__maybe_unused,
+ void *arg __maybe_unused,
+ int delay_secs __maybe_unused)
{
struct perf_evsel *pos;
GtkWidget *vbox;
@@ -233,6 +304,8 @@
gtk_window_set_position(GTK_WINDOW(window), GTK_WIN_POS_CENTER);
+ ui_helpline__push(help);
+
gtk_main();
perf_gtk__deactivate_context(&pgctx);
diff --git a/tools/perf/ui/gtk/gtk.h b/tools/perf/ui/gtk/gtk.h
index a4d0f2b..687af0b 100644
--- a/tools/perf/ui/gtk/gtk.h
+++ b/tools/perf/ui/gtk/gtk.h
@@ -29,6 +29,9 @@
struct perf_gtk_context *perf_gtk__activate_context(GtkWidget *window);
int perf_gtk__deactivate_context(struct perf_gtk_context **ctx);
+void perf_gtk__init_helpline(void);
+void perf_gtk__init_hpp(void);
+
#ifndef HAVE_GTK_INFO_BAR
static inline GtkWidget *perf_gtk__setup_info_bar(void)
{
diff --git a/tools/perf/ui/gtk/helpline.c b/tools/perf/ui/gtk/helpline.c
new file mode 100644
index 0000000..5db4432
--- /dev/null
+++ b/tools/perf/ui/gtk/helpline.c
@@ -0,0 +1,56 @@
+#include <stdio.h>
+#include <string.h>
+
+#include "gtk.h"
+#include "../ui.h"
+#include "../helpline.h"
+#include "../../util/debug.h"
+
+static void gtk_helpline_pop(void)
+{
+ if (!perf_gtk__is_active_context(pgctx))
+ return;
+
+ gtk_statusbar_pop(GTK_STATUSBAR(pgctx->statbar),
+ pgctx->statbar_ctx_id);
+}
+
+static void gtk_helpline_push(const char *msg)
+{
+ if (!perf_gtk__is_active_context(pgctx))
+ return;
+
+ gtk_statusbar_push(GTK_STATUSBAR(pgctx->statbar),
+ pgctx->statbar_ctx_id, msg);
+}
+
+static struct ui_helpline gtk_helpline_fns = {
+ .pop = gtk_helpline_pop,
+ .push = gtk_helpline_push,
+};
+
+void perf_gtk__init_helpline(void)
+{
+ helpline_fns = >k_helpline_fns;
+}
+
+int perf_gtk__show_helpline(const char *fmt, va_list ap)
+{
+ int ret;
+ char *ptr;
+ static int backlog;
+
+ ret = vscnprintf(ui_helpline__current + backlog,
+ sizeof(ui_helpline__current) - backlog, fmt, ap);
+ backlog += ret;
+
+ /* only first line can be displayed */
+ ptr = strchr(ui_helpline__current, '\n');
+ if (ptr && (ptr - ui_helpline__current) <= backlog) {
+ *ptr = '\0';
+ ui_helpline__puts(ui_helpline__current);
+ backlog = 0;
+ }
+
+ return ret;
+}
diff --git a/tools/perf/ui/gtk/setup.c b/tools/perf/ui/gtk/setup.c
index 92879ce..3c4c6ef 100644
--- a/tools/perf/ui/gtk/setup.c
+++ b/tools/perf/ui/gtk/setup.c
@@ -7,11 +7,15 @@
int perf_gtk__init(void)
{
perf_error__register(&perf_gtk_eops);
+ perf_gtk__init_helpline();
+ perf_gtk__init_hpp();
return gtk_init_check(NULL, NULL) ? 0 : -1;
}
-void perf_gtk__exit(bool wait_for_ok __used)
+void perf_gtk__exit(bool wait_for_ok __maybe_unused)
{
+ if (!perf_gtk__is_active_context(pgctx))
+ return;
perf_error__unregister(&perf_gtk_eops);
gtk_main_quit();
}
diff --git a/tools/perf/ui/gtk/util.c b/tools/perf/ui/gtk/util.c
index 0ead373..8aada5b 100644
--- a/tools/perf/ui/gtk/util.c
+++ b/tools/perf/ui/gtk/util.c
@@ -117,13 +117,8 @@
* For now, just add stubs for NO_NEWT=1 build.
*/
#ifdef NO_NEWT_SUPPORT
-int ui_helpline__show_help(const char *format __used, va_list ap __used)
-{
- return 0;
-}
-
-void ui_progress__update(u64 curr __used, u64 total __used,
- const char *title __used)
+void ui_progress__update(u64 curr __maybe_unused, u64 total __maybe_unused,
+ const char *title __maybe_unused)
{
}
#endif
diff --git a/tools/perf/ui/helpline.c b/tools/perf/ui/helpline.c
index 2f950c2..a49bcf3 100644
--- a/tools/perf/ui/helpline.c
+++ b/tools/perf/ui/helpline.c
@@ -5,23 +5,32 @@
#include "../debug.h"
#include "helpline.h"
#include "ui.h"
-#include "libslang.h"
-
-void ui_helpline__pop(void)
-{
-}
char ui_helpline__current[512];
+static void nop_helpline__pop(void)
+{
+}
+
+static void nop_helpline__push(const char *msg __maybe_unused)
+{
+}
+
+static struct ui_helpline default_helpline_fns = {
+ .pop = nop_helpline__pop,
+ .push = nop_helpline__push,
+};
+
+struct ui_helpline *helpline_fns = &default_helpline_fns;
+
+void ui_helpline__pop(void)
+{
+ helpline_fns->pop();
+}
+
void ui_helpline__push(const char *msg)
{
- const size_t sz = sizeof(ui_helpline__current);
-
- SLsmg_gotorc(SLtt_Screen_Rows - 1, 0);
- SLsmg_set_color(0);
- SLsmg_write_nstring((char *)msg, SLtt_Screen_Cols);
- SLsmg_refresh();
- strncpy(ui_helpline__current, msg, sz)[sz - 1] = '\0';
+ helpline_fns->push(msg);
}
void ui_helpline__vpush(const char *fmt, va_list ap)
@@ -50,30 +59,3 @@
ui_helpline__pop();
ui_helpline__push(msg);
}
-
-void ui_helpline__init(void)
-{
- ui_helpline__puts(" ");
-}
-
-char ui_helpline__last_msg[1024];
-
-int ui_helpline__show_help(const char *format, va_list ap)
-{
- int ret;
- static int backlog;
-
- pthread_mutex_lock(&ui__lock);
- ret = vscnprintf(ui_helpline__last_msg + backlog,
- sizeof(ui_helpline__last_msg) - backlog, format, ap);
- backlog += ret;
-
- if (ui_helpline__last_msg[backlog - 1] == '\n') {
- ui_helpline__puts(ui_helpline__last_msg);
- SLsmg_refresh();
- backlog = 0;
- }
- pthread_mutex_unlock(&ui__lock);
-
- return ret;
-}
diff --git a/tools/perf/ui/helpline.h b/tools/perf/ui/helpline.h
index 7bab6b3..2b667ee 100644
--- a/tools/perf/ui/helpline.h
+++ b/tools/perf/ui/helpline.h
@@ -4,13 +4,44 @@
#include <stdio.h>
#include <stdarg.h>
+#include "../util/cache.h"
+
+struct ui_helpline {
+ void (*pop)(void);
+ void (*push)(const char *msg);
+};
+
+extern struct ui_helpline *helpline_fns;
+
void ui_helpline__init(void);
+
void ui_helpline__pop(void);
void ui_helpline__push(const char *msg);
void ui_helpline__vpush(const char *fmt, va_list ap);
void ui_helpline__fpush(const char *fmt, ...);
void ui_helpline__puts(const char *msg);
-extern char ui_helpline__current[];
+extern char ui_helpline__current[512];
+
+#ifdef NO_NEWT_SUPPORT
+static inline int ui_helpline__show_help(const char *format __maybe_unused,
+ va_list ap __maybe_unused)
+{
+ return 0;
+}
+#else
+extern char ui_helpline__last_msg[];
+int ui_helpline__show_help(const char *format, va_list ap);
+#endif /* NO_NEWT_SUPPORT */
+
+#ifdef NO_GTK2_SUPPORT
+static inline int perf_gtk__show_helpline(const char *format __maybe_unused,
+ va_list ap __maybe_unused)
+{
+ return 0;
+}
+#else
+int perf_gtk__show_helpline(const char *format, va_list ap);
+#endif /* NO_GTK2_SUPPORT */
#endif /* _PERF_UI_HELPLINE_H_ */
diff --git a/tools/perf/ui/hist.c b/tools/perf/ui/hist.c
new file mode 100644
index 0000000..e3f8cd4
--- /dev/null
+++ b/tools/perf/ui/hist.c
@@ -0,0 +1,390 @@
+#include <math.h>
+
+#include "../util/hist.h"
+#include "../util/util.h"
+#include "../util/sort.h"
+
+
+/* hist period print (hpp) functions */
+static int hpp__header_overhead(struct perf_hpp *hpp)
+{
+ const char *fmt = hpp->ptr ? "Baseline" : "Overhead";
+
+ return scnprintf(hpp->buf, hpp->size, fmt);
+}
+
+static int hpp__width_overhead(struct perf_hpp *hpp __maybe_unused)
+{
+ return 8;
+}
+
+static int hpp__color_overhead(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ double percent = 100.0 * he->period / hpp->total_period;
+
+ if (hpp->ptr) {
+ struct hists *old_hists = hpp->ptr;
+ u64 total_period = old_hists->stats.total_period;
+ u64 base_period = he->pair ? he->pair->period : 0;
+
+ if (total_period)
+ percent = 100.0 * base_period / total_period;
+ else
+ percent = 0.0;
+ }
+
+ return percent_color_snprintf(hpp->buf, hpp->size, " %6.2f%%", percent);
+}
+
+static int hpp__entry_overhead(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ double percent = 100.0 * he->period / hpp->total_period;
+ const char *fmt = symbol_conf.field_sep ? "%.2f" : " %6.2f%%";
+
+ if (hpp->ptr) {
+ struct hists *old_hists = hpp->ptr;
+ u64 total_period = old_hists->stats.total_period;
+ u64 base_period = he->pair ? he->pair->period : 0;
+
+ if (total_period)
+ percent = 100.0 * base_period / total_period;
+ else
+ percent = 0.0;
+ }
+
+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
+}
+
+static int hpp__header_overhead_sys(struct perf_hpp *hpp)
+{
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7s";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, "sys");
+}
+
+static int hpp__width_overhead_sys(struct perf_hpp *hpp __maybe_unused)
+{
+ return 7;
+}
+
+static int hpp__color_overhead_sys(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_sys / hpp->total_period;
+ return percent_color_snprintf(hpp->buf, hpp->size, "%6.2f%%", percent);
+}
+
+static int hpp__entry_overhead_sys(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_sys / hpp->total_period;
+ const char *fmt = symbol_conf.field_sep ? "%.2f" : "%6.2f%%";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
+}
+
+static int hpp__header_overhead_us(struct perf_hpp *hpp)
+{
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7s";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, "user");
+}
+
+static int hpp__width_overhead_us(struct perf_hpp *hpp __maybe_unused)
+{
+ return 7;
+}
+
+static int hpp__color_overhead_us(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_us / hpp->total_period;
+ return percent_color_snprintf(hpp->buf, hpp->size, "%6.2f%%", percent);
+}
+
+static int hpp__entry_overhead_us(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_us / hpp->total_period;
+ const char *fmt = symbol_conf.field_sep ? "%.2f" : "%6.2f%%";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
+}
+
+static int hpp__header_overhead_guest_sys(struct perf_hpp *hpp)
+{
+ return scnprintf(hpp->buf, hpp->size, "guest sys");
+}
+
+static int hpp__width_overhead_guest_sys(struct perf_hpp *hpp __maybe_unused)
+{
+ return 9;
+}
+
+static int hpp__color_overhead_guest_sys(struct perf_hpp *hpp,
+ struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_guest_sys / hpp->total_period;
+ return percent_color_snprintf(hpp->buf, hpp->size, " %6.2f%% ", percent);
+}
+
+static int hpp__entry_overhead_guest_sys(struct perf_hpp *hpp,
+ struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_guest_sys / hpp->total_period;
+ const char *fmt = symbol_conf.field_sep ? "%.2f" : " %6.2f%% ";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
+}
+
+static int hpp__header_overhead_guest_us(struct perf_hpp *hpp)
+{
+ return scnprintf(hpp->buf, hpp->size, "guest usr");
+}
+
+static int hpp__width_overhead_guest_us(struct perf_hpp *hpp __maybe_unused)
+{
+ return 9;
+}
+
+static int hpp__color_overhead_guest_us(struct perf_hpp *hpp,
+ struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_guest_us / hpp->total_period;
+ return percent_color_snprintf(hpp->buf, hpp->size, " %6.2f%% ", percent);
+}
+
+static int hpp__entry_overhead_guest_us(struct perf_hpp *hpp,
+ struct hist_entry *he)
+{
+ double percent = 100.0 * he->period_guest_us / hpp->total_period;
+ const char *fmt = symbol_conf.field_sep ? "%.2f" : " %6.2f%% ";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, percent);
+}
+
+static int hpp__header_samples(struct perf_hpp *hpp)
+{
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%11s";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, "Samples");
+}
+
+static int hpp__width_samples(struct perf_hpp *hpp __maybe_unused)
+{
+ return 11;
+}
+
+static int hpp__entry_samples(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ const char *fmt = symbol_conf.field_sep ? "%" PRIu64 : "%11" PRIu64;
+
+ return scnprintf(hpp->buf, hpp->size, fmt, he->nr_events);
+}
+
+static int hpp__header_period(struct perf_hpp *hpp)
+{
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%12s";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, "Period");
+}
+
+static int hpp__width_period(struct perf_hpp *hpp __maybe_unused)
+{
+ return 12;
+}
+
+static int hpp__entry_period(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ const char *fmt = symbol_conf.field_sep ? "%" PRIu64 : "%12" PRIu64;
+
+ return scnprintf(hpp->buf, hpp->size, fmt, he->period);
+}
+
+static int hpp__header_delta(struct perf_hpp *hpp)
+{
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7s";
+
+ return scnprintf(hpp->buf, hpp->size, fmt, "Delta");
+}
+
+static int hpp__width_delta(struct perf_hpp *hpp __maybe_unused)
+{
+ return 7;
+}
+
+static int hpp__entry_delta(struct perf_hpp *hpp, struct hist_entry *he)
+{
+ struct hists *pair_hists = hpp->ptr;
+ u64 old_total, new_total;
+ double old_percent = 0, new_percent = 0;
+ double diff;
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%7.7s";
+ char buf[32] = " ";
+
+ old_total = pair_hists->stats.total_period;
+ if (old_total > 0 && he->pair)
+ old_percent = 100.0 * he->pair->period / old_total;
+
+ new_total = hpp->total_period;
+ if (new_total > 0)
+ new_percent = 100.0 * he->period / new_total;
+
+ diff = new_percent - old_percent;
+ if (fabs(diff) >= 0.01)
+ scnprintf(buf, sizeof(buf), "%+4.2F%%", diff);
+
+ return scnprintf(hpp->buf, hpp->size, fmt, buf);
+}
+
+static int hpp__header_displ(struct perf_hpp *hpp)
+{
+ return scnprintf(hpp->buf, hpp->size, "Displ.");
+}
+
+static int hpp__width_displ(struct perf_hpp *hpp __maybe_unused)
+{
+ return 6;
+}
+
+static int hpp__entry_displ(struct perf_hpp *hpp,
+ struct hist_entry *he __maybe_unused)
+{
+ const char *fmt = symbol_conf.field_sep ? "%s" : "%6.6s";
+ char buf[32] = " ";
+
+ if (hpp->displacement)
+ scnprintf(buf, sizeof(buf), "%+4ld", hpp->displacement);
+
+ return scnprintf(hpp->buf, hpp->size, fmt, buf);
+}
+
+#define HPP__COLOR_PRINT_FNS(_name) \
+ .header = hpp__header_ ## _name, \
+ .width = hpp__width_ ## _name, \
+ .color = hpp__color_ ## _name, \
+ .entry = hpp__entry_ ## _name
+
+#define HPP__PRINT_FNS(_name) \
+ .header = hpp__header_ ## _name, \
+ .width = hpp__width_ ## _name, \
+ .entry = hpp__entry_ ## _name
+
+struct perf_hpp_fmt perf_hpp__format[] = {
+ { .cond = true, HPP__COLOR_PRINT_FNS(overhead) },
+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_sys) },
+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_us) },
+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_guest_sys) },
+ { .cond = false, HPP__COLOR_PRINT_FNS(overhead_guest_us) },
+ { .cond = false, HPP__PRINT_FNS(samples) },
+ { .cond = false, HPP__PRINT_FNS(period) },
+ { .cond = false, HPP__PRINT_FNS(delta) },
+ { .cond = false, HPP__PRINT_FNS(displ) }
+};
+
+#undef HPP__COLOR_PRINT_FNS
+#undef HPP__PRINT_FNS
+
+void perf_hpp__init(bool need_pair, bool show_displacement)
+{
+ if (symbol_conf.show_cpu_utilization) {
+ perf_hpp__format[PERF_HPP__OVERHEAD_SYS].cond = true;
+ perf_hpp__format[PERF_HPP__OVERHEAD_US].cond = true;
+
+ if (perf_guest) {
+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_SYS].cond = true;
+ perf_hpp__format[PERF_HPP__OVERHEAD_GUEST_US].cond = true;
+ }
+ }
+
+ if (symbol_conf.show_nr_samples)
+ perf_hpp__format[PERF_HPP__SAMPLES].cond = true;
+
+ if (symbol_conf.show_total_period)
+ perf_hpp__format[PERF_HPP__PERIOD].cond = true;
+
+ if (need_pair) {
+ perf_hpp__format[PERF_HPP__DELTA].cond = true;
+
+ if (show_displacement)
+ perf_hpp__format[PERF_HPP__DISPL].cond = true;
+ }
+}
+
+static inline void advance_hpp(struct perf_hpp *hpp, int inc)
+{
+ hpp->buf += inc;
+ hpp->size -= inc;
+}
+
+int hist_entry__period_snprintf(struct perf_hpp *hpp, struct hist_entry *he,
+ bool color)
+{
+ const char *sep = symbol_conf.field_sep;
+ char *start = hpp->buf;
+ int i, ret;
+
+ if (symbol_conf.exclude_other && !he->parent)
+ return 0;
+
+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
+ if (!perf_hpp__format[i].cond)
+ continue;
+
+ if (!sep || i > 0) {
+ ret = scnprintf(hpp->buf, hpp->size, "%s", sep ?: " ");
+ advance_hpp(hpp, ret);
+ }
+
+ if (color && perf_hpp__format[i].color)
+ ret = perf_hpp__format[i].color(hpp, he);
+ else
+ ret = perf_hpp__format[i].entry(hpp, he);
+
+ advance_hpp(hpp, ret);
+ }
+
+ return hpp->buf - start;
+}
+
+int hist_entry__sort_snprintf(struct hist_entry *he, char *s, size_t size,
+ struct hists *hists)
+{
+ const char *sep = symbol_conf.field_sep;
+ struct sort_entry *se;
+ int ret = 0;
+
+ list_for_each_entry(se, &hist_entry__sort_list, list) {
+ if (se->elide)
+ continue;
+
+ ret += scnprintf(s + ret, size - ret, "%s", sep ?: " ");
+ ret += se->se_snprintf(he, s + ret, size - ret,
+ hists__col_len(hists, se->se_width_idx));
+ }
+
+ return ret;
+}
+
+/*
+ * See hists__fprintf to match the column widths
+ */
+unsigned int hists__sort_list_width(struct hists *hists)
+{
+ struct sort_entry *se;
+ int i, ret = 0;
+
+ for (i = 0; i < PERF_HPP__MAX_INDEX; i++) {
+ if (!perf_hpp__format[i].cond)
+ continue;
+ if (i)
+ ret += 2;
+
+ ret += perf_hpp__format[i].width(NULL);
+ }
+
+ list_for_each_entry(se, &hist_entry__sort_list, list)
+ if (!se->elide)
+ ret += 2 + hists__col_len(hists, se->se_width_idx);
+
+ if (verbose) /* Addr + origin */
+ ret += 3 + BITS_PER_LONG / 4;
+
+ return ret;
+}
diff --git a/tools/perf/ui/setup.c b/tools/perf/ui/setup.c
index 791fb15..bd7d460 100644
--- a/tools/perf/ui/setup.c
+++ b/tools/perf/ui/setup.c
@@ -1,6 +1,10 @@
-#include "../cache.h"
-#include "../debug.h"
+#include <pthread.h>
+#include "../util/cache.h"
+#include "../util/debug.h"
+#include "../util/hist.h"
+
+pthread_mutex_t ui__lock = PTHREAD_MUTEX_INITIALIZER;
void setup_browser(bool fallback_to_pager)
{
@@ -25,6 +29,8 @@
use_browser = 0;
if (fallback_to_pager)
setup_pager();
+
+ perf_hpp__init(false, false);
break;
}
}
diff --git a/tools/perf/ui/stdio/hist.c b/tools/perf/ui/stdio/hist.c
new file mode 100644
index 0000000..882461a
--- /dev/null
+++ b/tools/perf/ui/stdio/hist.c
@@ -0,0 +1,498 @@
+#include <stdio.h>
+
+#include "../../util/util.h"
+#include "../../util/hist.h"
+#include "../../util/sort.h"
+
+
+static size_t callchain__fprintf_left_margin(FILE *fp, int left_margin)
+{
+ int i;
+ int ret = fprintf(fp, " ");
+
+ for (i = 0; i < left_margin; i++)
+ ret += fprintf(fp, " ");
+
+ return ret;
+}
+
+static size_t ipchain__fprintf_graph_line(FILE *fp, int depth, int depth_mask,
+ int left_margin)
+{
+ int i;
+ size_t ret = callchain__fprintf_left_margin(fp, left_margin);
+
+ for (i = 0; i < depth; i++)
+ if (depth_mask & (1 << i))
+ ret += fprintf(fp, "| ");
+ else
+ ret += fprintf(fp, " ");
+
+ ret += fprintf(fp, "\n");
+
+ return ret;
+}
+
+static size_t ipchain__fprintf_graph(FILE *fp, struct callchain_list *chain,
+ int depth, int depth_mask, int period,
+ u64 total_samples, u64 hits,
+ int left_margin)
+{
+ int i;
+ size_t ret = 0;
+
+ ret += callchain__fprintf_left_margin(fp, left_margin);
+ for (i = 0; i < depth; i++) {
+ if (depth_mask & (1 << i))
+ ret += fprintf(fp, "|");
+ else
+ ret += fprintf(fp, " ");
+ if (!period && i == depth - 1) {
+ double percent;
+
+ percent = hits * 100.0 / total_samples;
+ ret += percent_color_fprintf(fp, "--%2.2f%%-- ", percent);
+ } else
+ ret += fprintf(fp, "%s", " ");
+ }
+ if (chain->ms.sym)
+ ret += fprintf(fp, "%s\n", chain->ms.sym->name);
+ else
+ ret += fprintf(fp, "0x%0" PRIx64 "\n", chain->ip);
+
+ return ret;
+}
+
+static struct symbol *rem_sq_bracket;
+static struct callchain_list rem_hits;
+
+static void init_rem_hits(void)
+{
+ rem_sq_bracket = malloc(sizeof(*rem_sq_bracket) + 6);
+ if (!rem_sq_bracket) {
+ fprintf(stderr, "Not enough memory to display remaining hits\n");
+ return;
+ }
+
+ strcpy(rem_sq_bracket->name, "[...]");
+ rem_hits.ms.sym = rem_sq_bracket;
+}
+
+static size_t __callchain__fprintf_graph(FILE *fp, struct rb_root *root,
+ u64 total_samples, int depth,
+ int depth_mask, int left_margin)
+{
+ struct rb_node *node, *next;
+ struct callchain_node *child;
+ struct callchain_list *chain;
+ int new_depth_mask = depth_mask;
+ u64 remaining;
+ size_t ret = 0;
+ int i;
+ uint entries_printed = 0;
+
+ remaining = total_samples;
+
+ node = rb_first(root);
+ while (node) {
+ u64 new_total;
+ u64 cumul;
+
+ child = rb_entry(node, struct callchain_node, rb_node);
+ cumul = callchain_cumul_hits(child);
+ remaining -= cumul;
+
+ /*
+ * The depth mask manages the output of pipes that show
+ * the depth. We don't want to keep the pipes of the current
+ * level for the last child of this depth.
+ * Except if we have remaining filtered hits. They will
+ * supersede the last child
+ */
+ next = rb_next(node);
+ if (!next && (callchain_param.mode != CHAIN_GRAPH_REL || !remaining))
+ new_depth_mask &= ~(1 << (depth - 1));
+
+ /*
+ * But we keep the older depth mask for the line separator
+ * to keep the level link until we reach the last child
+ */
+ ret += ipchain__fprintf_graph_line(fp, depth, depth_mask,
+ left_margin);
+ i = 0;
+ list_for_each_entry(chain, &child->val, list) {
+ ret += ipchain__fprintf_graph(fp, chain, depth,
+ new_depth_mask, i++,
+ total_samples,
+ cumul,
+ left_margin);
+ }
+
+ if (callchain_param.mode == CHAIN_GRAPH_REL)
+ new_total = child->children_hit;
+ else
+ new_total = total_samples;
+
+ ret += __callchain__fprintf_graph(fp, &child->rb_root, new_total,
+ depth + 1,
+ new_depth_mask | (1 << depth),
+ left_margin);
+ node = next;
+ if (++entries_printed == callchain_param.print_limit)
+ break;
+ }
+
+ if (callchain_param.mode == CHAIN_GRAPH_REL &&
+ remaining && remaining != total_samples) {
+
+ if (!rem_sq_bracket)
+ return ret;
+
+ new_depth_mask &= ~(1 << (depth - 1));
+ ret += ipchain__fprintf_graph(fp, &rem_hits, depth,
+ new_depth_mask, 0, total_samples,
+ remaining, left_margin);
+ }
+
+ return ret;
+}
+
+static size_t callchain__fprintf_graph(FILE *fp, struct rb_root *root,
+ u64 total_samples, int left_margin)
+{
+ struct callchain_node *cnode;
+ struct callchain_list *chain;
+ u32 entries_printed = 0;
+ bool printed = false;
+ struct rb_node *node;
+ int i = 0;
+ int ret = 0;
+
+ /*
+ * If have one single callchain root, don't bother printing
+ * its percentage (100 % in fractal mode and the same percentage
+ * than the hist in graph mode). This also avoid one level of column.
+ */
+ node = rb_first(root);
+ if (node && !rb_next(node)) {
+ cnode = rb_entry(node, struct callchain_node, rb_node);
+ list_for_each_entry(chain, &cnode->val, list) {
+ /*
+ * If we sort by symbol, the first entry is the same than
+ * the symbol. No need to print it otherwise it appears as
+ * displayed twice.
+ */
+ if (!i++ && sort__first_dimension == SORT_SYM)
+ continue;
+ if (!printed) {
+ ret += callchain__fprintf_left_margin(fp, left_margin);
+ ret += fprintf(fp, "|\n");
+ ret += callchain__fprintf_left_margin(fp, left_margin);
+ ret += fprintf(fp, "---");
+ left_margin += 3;
+ printed = true;
+ } else
+ ret += callchain__fprintf_left_margin(fp, left_margin);
+
+ if (chain->ms.sym)
+ ret += fprintf(fp, " %s\n", chain->ms.sym->name);
+ else
+ ret += fprintf(fp, " %p\n", (void *)(long)chain->ip);
+
+ if (++entries_printed == callchain_param.print_limit)
+ break;
+ }
+ root = &cnode->rb_root;
+ }
+
+ ret += __callchain__fprintf_graph(fp, root, total_samples,
+ 1, 1, left_margin);
+ ret += fprintf(fp, "\n");
+
+ return ret;
+}
+
+static size_t __callchain__fprintf_flat(FILE *fp,
+ struct callchain_node *self,
+ u64 total_samples)
+{
+ struct callchain_list *chain;
+ size_t ret = 0;
+
+ if (!self)
+ return 0;
+
+ ret += __callchain__fprintf_flat(fp, self->parent, total_samples);
+
+
+ list_for_each_entry(chain, &self->val, list) {
+ if (chain->ip >= PERF_CONTEXT_MAX)
+ continue;
+ if (chain->ms.sym)
+ ret += fprintf(fp, " %s\n", chain->ms.sym->name);
+ else
+ ret += fprintf(fp, " %p\n",
+ (void *)(long)chain->ip);
+ }
+
+ return ret;
+}
+
+static size_t callchain__fprintf_flat(FILE *fp, struct rb_root *self,
+ u64 total_samples)
+{
+ size_t ret = 0;
+ u32 entries_printed = 0;
+ struct rb_node *rb_node;
+ struct callchain_node *chain;
+
+ rb_node = rb_first(self);
+ while (rb_node) {
+ double percent;
+
+ chain = rb_entry(rb_node, struct callchain_node, rb_node);
+ percent = chain->hit * 100.0 / total_samples;
+
+ ret = percent_color_fprintf(fp, " %6.2f%%\n", percent);
+ ret += __callchain__fprintf_flat(fp, chain, total_samples);
+ ret += fprintf(fp, "\n");
+ if (++entries_printed == callchain_param.print_limit)
+ break;
+
+ rb_node = rb_next(rb_node);
+ }
+
+ return ret;
+}
+
+static size_t hist_entry_callchain__fprintf(struct hist_entry *he,
+ u64 total_samples, int left_margin,
+ FILE *fp)
+{
+ switch (callchain_param.mode) {
+ case CHAIN_GRAPH_REL:
+ return callchain__fprintf_graph(fp, &he->sorted_chain, he->period,
+ left_margin);
+ break;
+ case CHAIN_GRAPH_ABS:
+ return callchain__fprintf_graph(fp, &he->sorted_chain, total_samples,
+ left_margin);
+ break;
+ case CHAIN_FLAT:
+ return callchain__fprintf_flat(fp, &he->sorted_chain, total_samples);
+ break;
+ case CHAIN_NONE:
+ break;
+ default:
+ pr_err("Bad callchain mode\n");
+ }
+
+ return 0;
+}
+
+static size_t hist_entry__callchain_fprintf(struct hist_entry *he,
+ struct hists *hists,
+ u64 total_period, FILE *fp)
+{
+ int left_margin = 0;
+
+ if (sort__first_dimension == SORT_COMM) {
+ struct sort_entry *se = list_first_entry(&hist_entry__sort_list,
+ typeof(*se), list);
+ left_margin = hists__col_len(hists, se->se_width_idx);
+ left_margin -= thread__comm_len(he->thread);
+ }
+
+ return hist_entry_callchain__fprintf(he, total_period, left_margin, fp);
+}
+
+static int hist_entry__fprintf(struct hist_entry *he, size_t size,
+ struct hists *hists, struct hists *pair_hists,
+ long displacement, u64 total_period, FILE *fp)
+{
+ char bf[512];
+ int ret;
+ struct perf_hpp hpp = {
+ .buf = bf,
+ .size = size,
+ .total_period = total_period,
+ .displacement = displacement,
+ .ptr = pair_hists,
+ };
+ bool color = !symbol_conf.field_sep;
+
+ if (size == 0 || size > sizeof(bf))
+ size = hpp.size = sizeof(bf);
+
+ ret = hist_entry__period_snprintf(&hpp, he, color);
+ hist_entry__sort_snprintf(he, bf + ret, size - ret, hists);
+
+ ret = fprintf(fp, "%s\n", bf);
+
+ if (symbol_conf.use_callchain)
+ ret += hist_entry__callchain_fprintf(he, hists,
+ total_period, fp);
+
+ return ret;
+}
+
+size_t hists__fprintf(struct hists *hists, struct hists *pair,
+ bool show_displacement, bool show_header, int max_rows,
+ int max_cols, FILE *fp)
+{
+ struct sort_entry *se;
+ struct rb_node *nd;
+ size_t ret = 0;
+ u64 total_period;
+ unsigned long position = 1;
+ long displacement = 0;
+ unsigned int width;
+ const char *sep = symbol_conf.field_sep;
+ const char *col_width = symbol_conf.col_width_list_str;
+ int idx, nr_rows = 0;
+ char bf[64];
+ struct perf_hpp dummy_hpp = {
+ .buf = bf,
+ .size = sizeof(bf),
+ .ptr = pair,
+ };
+
+ init_rem_hits();
+
+ if (!show_header)
+ goto print_entries;
+
+ fprintf(fp, "# ");
+ for (idx = 0; idx < PERF_HPP__MAX_INDEX; idx++) {
+ if (!perf_hpp__format[idx].cond)
+ continue;
+
+ if (idx)
+ fprintf(fp, "%s", sep ?: " ");
+
+ perf_hpp__format[idx].header(&dummy_hpp);
+ fprintf(fp, "%s", bf);
+ }
+
+ list_for_each_entry(se, &hist_entry__sort_list, list) {
+ if (se->elide)
+ continue;
+ if (sep) {
+ fprintf(fp, "%c%s", *sep, se->se_header);
+ continue;
+ }
+ width = strlen(se->se_header);
+ if (symbol_conf.col_width_list_str) {
+ if (col_width) {
+ hists__set_col_len(hists, se->se_width_idx,
+ atoi(col_width));
+ col_width = strchr(col_width, ',');
+ if (col_width)
+ ++col_width;
+ }
+ }
+ if (!hists__new_col_len(hists, se->se_width_idx, width))
+ width = hists__col_len(hists, se->se_width_idx);
+ fprintf(fp, " %*s", width, se->se_header);
+ }
+
+ fprintf(fp, "\n");
+ if (max_rows && ++nr_rows >= max_rows)
+ goto out;
+
+ if (sep)
+ goto print_entries;
+
+ fprintf(fp, "# ");
+ for (idx = 0; idx < PERF_HPP__MAX_INDEX; idx++) {
+ unsigned int i;
+
+ if (!perf_hpp__format[idx].cond)
+ continue;
+
+ if (idx)
+ fprintf(fp, "%s", sep ?: " ");
+
+ width = perf_hpp__format[idx].width(&dummy_hpp);
+ for (i = 0; i < width; i++)
+ fprintf(fp, ".");
+ }
+
+ list_for_each_entry(se, &hist_entry__sort_list, list) {
+ unsigned int i;
+
+ if (se->elide)
+ continue;
+
+ fprintf(fp, " ");
+ width = hists__col_len(hists, se->se_width_idx);
+ if (width == 0)
+ width = strlen(se->se_header);
+ for (i = 0; i < width; i++)
+ fprintf(fp, ".");
+ }
+
+ fprintf(fp, "\n");
+ if (max_rows && ++nr_rows >= max_rows)
+ goto out;
+
+ fprintf(fp, "#\n");
+ if (max_rows && ++nr_rows >= max_rows)
+ goto out;
+
+print_entries:
+ total_period = hists->stats.total_period;
+
+ for (nd = rb_first(&hists->entries); nd; nd = rb_next(nd)) {
+ struct hist_entry *h = rb_entry(nd, struct hist_entry, rb_node);
+
+ if (h->filtered)
+ continue;
+
+ if (show_displacement) {
+ if (h->pair != NULL)
+ displacement = ((long)h->pair->position -
+ (long)position);
+ else
+ displacement = 0;
+ ++position;
+ }
+ ret += hist_entry__fprintf(h, max_cols, hists, pair, displacement,
+ total_period, fp);
+
+ if (max_rows && ++nr_rows >= max_rows)
+ goto out;
+
+ if (h->ms.map == NULL && verbose > 1) {
+ __map_groups__fprintf_maps(&h->thread->mg,
+ MAP__FUNCTION, verbose, fp);
+ fprintf(fp, "%.10s end\n", graph_dotted_line);
+ }
+ }
+out:
+ free(rem_sq_bracket);
+
+ return ret;
+}
+
+size_t hists__fprintf_nr_events(struct hists *hists, FILE *fp)
+{
+ int i;
+ size_t ret = 0;
+
+ for (i = 0; i < PERF_RECORD_HEADER_MAX; ++i) {
+ const char *name;
+
+ if (hists->stats.nr_events[i] == 0)
+ continue;
+
+ name = perf_event__name(i);
+ if (!strcmp(name, "UNKNOWN"))
+ continue;
+
+ ret += fprintf(fp, "%16s events: %10d\n", name,
+ hists->stats.nr_events[i]);
+ }
+
+ return ret;
+}
diff --git a/tools/perf/ui/tui/helpline.c b/tools/perf/ui/tui/helpline.c
new file mode 100644
index 0000000..2884d2f
--- /dev/null
+++ b/tools/perf/ui/tui/helpline.c
@@ -0,0 +1,57 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <pthread.h>
+
+#include "../../util/debug.h"
+#include "../helpline.h"
+#include "../ui.h"
+#include "../libslang.h"
+
+static void tui_helpline__pop(void)
+{
+}
+
+static void tui_helpline__push(const char *msg)
+{
+ const size_t sz = sizeof(ui_helpline__current);
+
+ SLsmg_gotorc(SLtt_Screen_Rows - 1, 0);
+ SLsmg_set_color(0);
+ SLsmg_write_nstring((char *)msg, SLtt_Screen_Cols);
+ SLsmg_refresh();
+ strncpy(ui_helpline__current, msg, sz)[sz - 1] = '\0';
+}
+
+struct ui_helpline tui_helpline_fns = {
+ .pop = tui_helpline__pop,
+ .push = tui_helpline__push,
+};
+
+void ui_helpline__init(void)
+{
+ helpline_fns = &tui_helpline_fns;
+ ui_helpline__puts(" ");
+}
+
+char ui_helpline__last_msg[1024];
+
+int ui_helpline__show_help(const char *format, va_list ap)
+{
+ int ret;
+ static int backlog;
+
+ pthread_mutex_lock(&ui__lock);
+ ret = vscnprintf(ui_helpline__last_msg + backlog,
+ sizeof(ui_helpline__last_msg) - backlog, format, ap);
+ backlog += ret;
+
+ if (ui_helpline__last_msg[backlog - 1] == '\n') {
+ ui_helpline__puts(ui_helpline__last_msg);
+ SLsmg_refresh();
+ backlog = 0;
+ }
+ pthread_mutex_unlock(&ui__lock);
+
+ return ret;
+}
diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
index e813c1d..60debb8 100644
--- a/tools/perf/ui/tui/setup.c
+++ b/tools/perf/ui/tui/setup.c
@@ -11,12 +11,12 @@
#include "../libslang.h"
#include "../keysyms.h"
-pthread_mutex_t ui__lock = PTHREAD_MUTEX_INITIALIZER;
-
static volatile int ui__need_resize;
extern struct perf_error_ops perf_tui_eops;
+extern void hist_browser__init_hpp(void);
+
void ui__refresh_dimensions(bool force)
{
if (force || ui__need_resize) {
@@ -28,7 +28,7 @@
}
}
-static void ui__sigwinch(int sig __used)
+static void ui__sigwinch(int sig __maybe_unused)
{
ui__need_resize = 1;
}
@@ -88,7 +88,7 @@
return SLkp_getkey();
}
-static void newt_suspend(void *d __used)
+static void newt_suspend(void *d __maybe_unused)
{
newtSuspend();
raise(SIGTSTP);
@@ -126,6 +126,8 @@
signal(SIGTERM, ui__signal);
perf_error__register(&perf_tui_eops);
+
+ hist_browser__init_hpp();
out:
return err;
}
diff --git a/tools/perf/util/alias.c b/tools/perf/util/alias.c
index b8144e8..e6d1347 100644
--- a/tools/perf/util/alias.c
+++ b/tools/perf/util/alias.c
@@ -3,7 +3,8 @@
static const char *alias_key;
static char *alias_val;
-static int alias_lookup_cb(const char *k, const char *v, void *cb __used)
+static int alias_lookup_cb(const char *k, const char *v,
+ void *cb __maybe_unused)
{
if (!prefixcmp(k, "alias.") && !strcmp(k+6, alias_key)) {
if (!v)
diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
index 3a282c0..f0a9103 100644
--- a/tools/perf/util/annotate.c
+++ b/tools/perf/util/annotate.c
@@ -17,6 +17,7 @@
#include <pthread.h>
const char *disassembler_style;
+const char *objdump_path;
static struct ins *ins__find(const char *name);
static int disasm_line__parse(char *line, char **namep, char **rawp);
@@ -312,8 +313,8 @@
.scnprintf = dec__scnprintf,
};
-static int nop__scnprintf(struct ins *ins __used, char *bf, size_t size,
- struct ins_operands *ops __used)
+static int nop__scnprintf(struct ins *ins __maybe_unused, char *bf, size_t size,
+ struct ins_operands *ops __maybe_unused)
{
return scnprintf(bf, size, "%-6.6s", "nop");
}
@@ -415,7 +416,7 @@
return bsearch(name, instructions, nmemb, sizeof(struct ins), ins__cmp);
}
-int symbol__annotate_init(struct map *map __used, struct symbol *sym)
+int symbol__annotate_init(struct map *map __maybe_unused, struct symbol *sym)
{
struct annotation *notes = symbol__annotation(sym);
pthread_mutex_init(¬es->lock, NULL);
@@ -820,9 +821,10 @@
dso, dso->long_name, sym, sym->name);
snprintf(command, sizeof(command),
- "objdump %s%s --start-address=0x%016" PRIx64
+ "%s %s%s --start-address=0x%016" PRIx64
" --stop-address=0x%016" PRIx64
" -d %s %s -C %s|grep -v %s|expand",
+ objdump_path ? objdump_path : "objdump",
disassembler_style ? "-M " : "",
disassembler_style ? disassembler_style : "",
map__rip_2objdump(map, sym->start),
@@ -982,7 +984,8 @@
int context)
{
struct dso *dso = map->dso;
- const char *filename = dso->long_name, *d_filename;
+ char *filename;
+ const char *d_filename;
struct annotation *notes = symbol__annotation(sym);
struct disasm_line *pos, *queue = NULL;
u64 start = map__rip_2objdump(map, sym->start);
@@ -990,6 +993,10 @@
int more = 0;
u64 len;
+ filename = strdup(dso->long_name);
+ if (!filename)
+ return -ENOMEM;
+
if (full_paths)
d_filename = filename;
else
@@ -1040,6 +1047,8 @@
}
}
+ free(filename);
+
return more;
}
diff --git a/tools/perf/util/annotate.h b/tools/perf/util/annotate.h
index 78a5692..9b5b21e 100644
--- a/tools/perf/util/annotate.h
+++ b/tools/perf/util/annotate.h
@@ -7,6 +7,7 @@
#include "symbol.h"
#include <linux/list.h>
#include <linux/rbtree.h>
+#include <pthread.h>
struct ins;
@@ -125,7 +126,7 @@
void symbol__annotate_zero_histograms(struct symbol *sym);
int symbol__annotate(struct symbol *sym, struct map *map, size_t privsize);
-int symbol__annotate_init(struct map *map __used, struct symbol *sym);
+int symbol__annotate_init(struct map *map __maybe_unused, struct symbol *sym);
int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
bool full_paths, int min_pcnt, int max_lines,
int context);
@@ -138,11 +139,12 @@
int max_lines);
#ifdef NO_NEWT_SUPPORT
-static inline int symbol__tui_annotate(struct symbol *sym __used,
- struct map *map __used,
- int evidx __used,
- void(*timer)(void *arg) __used,
- void *arg __used, int delay_secs __used)
+static inline int symbol__tui_annotate(struct symbol *sym __maybe_unused,
+ struct map *map __maybe_unused,
+ int evidx __maybe_unused,
+ void(*timer)(void *arg) __maybe_unused,
+ void *arg __maybe_unused,
+ int delay_secs __maybe_unused)
{
return 0;
}
@@ -152,5 +154,6 @@
#endif
extern const char *disassembler_style;
+extern const char *objdump_path;
#endif /* __PERF_ANNOTATE_H */
diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
index fd9a594..8e3a740 100644
--- a/tools/perf/util/build-id.c
+++ b/tools/perf/util/build-id.c
@@ -16,10 +16,10 @@
#include "session.h"
#include "tool.h"
-static int build_id__mark_dso_hit(struct perf_tool *tool __used,
+static int build_id__mark_dso_hit(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
- struct perf_evsel *evsel __used,
+ struct perf_sample *sample __maybe_unused,
+ struct perf_evsel *evsel __maybe_unused,
struct machine *machine)
{
struct addr_location al;
@@ -41,9 +41,10 @@
return 0;
}
-static int perf_event__exit_del_thread(struct perf_tool *tool __used,
+static int perf_event__exit_del_thread(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
+ struct perf_sample *sample
+ __maybe_unused,
struct machine *machine)
{
struct thread *thread = machine__findnew_thread(machine, event->fork.tid);
diff --git a/tools/perf/util/cache.h b/tools/perf/util/cache.h
index cff18c6..ab17694 100644
--- a/tools/perf/util/cache.h
+++ b/tools/perf/util/cache.h
@@ -39,7 +39,7 @@
if (fallback_to_pager)
setup_pager();
}
-static inline void exit_browser(bool wait_for_ok __used) {}
+static inline void exit_browser(bool wait_for_ok __maybe_unused) {}
#else
void setup_browser(bool fallback_to_pager);
void exit_browser(bool wait_for_ok);
@@ -49,7 +49,7 @@
{
return -1;
}
-static inline void ui__exit(bool wait_for_ok __used) {}
+static inline void ui__exit(bool wait_for_ok __maybe_unused) {}
#else
int ui__init(void);
void ui__exit(bool wait_for_ok);
@@ -60,7 +60,7 @@
{
return -1;
}
-static inline void perf_gtk__exit(bool wait_for_ok __used) {}
+static inline void perf_gtk__exit(bool wait_for_ok __maybe_unused) {}
#else
int perf_gtk__init(void);
void perf_gtk__exit(bool wait_for_ok);
diff --git a/tools/perf/util/callchain.c b/tools/perf/util/callchain.c
index 3a6bff4..d3b3f5d 100644
--- a/tools/perf/util/callchain.c
+++ b/tools/perf/util/callchain.c
@@ -93,7 +93,7 @@
*/
static void
sort_chain_flat(struct rb_root *rb_root, struct callchain_root *root,
- u64 min_hit, struct callchain_param *param __used)
+ u64 min_hit, struct callchain_param *param __maybe_unused)
{
__sort_chain_flat(rb_root, &root->node, min_hit);
}
@@ -115,7 +115,7 @@
static void
sort_chain_graph_abs(struct rb_root *rb_root, struct callchain_root *chain_root,
- u64 min_hit, struct callchain_param *param __used)
+ u64 min_hit, struct callchain_param *param __maybe_unused)
{
__sort_chain_graph_abs(&chain_root->node, min_hit);
rb_root->rb_node = chain_root->node.rb_root.rb_node;
@@ -140,7 +140,7 @@
static void
sort_chain_graph_rel(struct rb_root *rb_root, struct callchain_root *chain_root,
- u64 min_hit __used, struct callchain_param *param)
+ u64 min_hit __maybe_unused, struct callchain_param *param)
{
__sort_chain_graph_rel(&chain_root->node, param->min_percent / 100.0);
rb_root->rb_node = chain_root->node.rb_root.rb_node;
diff --git a/tools/perf/util/callchain.h b/tools/perf/util/callchain.h
index 3bdb407..eb34057 100644
--- a/tools/perf/util/callchain.h
+++ b/tools/perf/util/callchain.h
@@ -58,7 +58,7 @@
/*
* A callchain cursor is a single linked list that
* let one feed a callchain progressively.
- * It keeps persitent allocated entries to minimize
+ * It keeps persistent allocated entries to minimize
* allocations.
*/
struct callchain_cursor_node {
diff --git a/tools/perf/util/cgroup.c b/tools/perf/util/cgroup.c
index dbe2f16..96bbda1 100644
--- a/tools/perf/util/cgroup.c
+++ b/tools/perf/util/cgroup.c
@@ -138,8 +138,8 @@
}
}
-int parse_cgroups(const struct option *opt __used, const char *str,
- int unset __used)
+int parse_cgroups(const struct option *opt __maybe_unused, const char *str,
+ int unset __maybe_unused)
{
struct perf_evlist *evlist = *(struct perf_evlist **)opt->value;
const char *p, *e, *eos = str + strlen(str);
diff --git a/tools/perf/util/config.c b/tools/perf/util/config.c
index 6faa3a1..3e0fdd3 100644
--- a/tools/perf/util/config.c
+++ b/tools/perf/util/config.c
@@ -342,13 +342,15 @@
return value;
}
-static int perf_default_core_config(const char *var __used, const char *value __used)
+static int perf_default_core_config(const char *var __maybe_unused,
+ const char *value __maybe_unused)
{
/* Add other config variables here. */
return 0;
}
-int perf_default_config(const char *var, const char *value, void *dummy __used)
+int perf_default_config(const char *var, const char *value,
+ void *dummy __maybe_unused)
{
if (!prefixcmp(var, "core."))
return perf_default_core_config(var, value);
diff --git a/tools/perf/util/cpumap.c b/tools/perf/util/cpumap.c
index adc72f0..2b32ffa 100644
--- a/tools/perf/util/cpumap.c
+++ b/tools/perf/util/cpumap.c
@@ -38,24 +38,19 @@
return cpus;
}
-static struct cpu_map *cpu_map__read_all_cpu_map(void)
+struct cpu_map *cpu_map__read(FILE *file)
{
struct cpu_map *cpus = NULL;
- FILE *onlnf;
int nr_cpus = 0;
int *tmp_cpus = NULL, *tmp;
int max_entries = 0;
int n, cpu, prev;
char sep;
- onlnf = fopen("/sys/devices/system/cpu/online", "r");
- if (!onlnf)
- return cpu_map__default_new();
-
sep = 0;
prev = -1;
for (;;) {
- n = fscanf(onlnf, "%u%c", &cpu, &sep);
+ n = fscanf(file, "%u%c", &cpu, &sep);
if (n <= 0)
break;
if (prev >= 0) {
@@ -95,6 +90,19 @@
cpus = cpu_map__default_new();
out_free_tmp:
free(tmp_cpus);
+ return cpus;
+}
+
+static struct cpu_map *cpu_map__read_all_cpu_map(void)
+{
+ struct cpu_map *cpus = NULL;
+ FILE *onlnf;
+
+ onlnf = fopen("/sys/devices/system/cpu/online", "r");
+ if (!onlnf)
+ return cpu_map__default_new();
+
+ cpus = cpu_map__read(onlnf);
fclose(onlnf);
return cpus;
}
diff --git a/tools/perf/util/cpumap.h b/tools/perf/util/cpumap.h
index c415185..2f68a3b 100644
--- a/tools/perf/util/cpumap.h
+++ b/tools/perf/util/cpumap.h
@@ -2,6 +2,7 @@
#define __PERF_CPUMAP_H
#include <stdio.h>
+#include <stdbool.h>
struct cpu_map {
int nr;
@@ -11,7 +12,17 @@
struct cpu_map *cpu_map__new(const char *cpu_list);
struct cpu_map *cpu_map__dummy_new(void);
void cpu_map__delete(struct cpu_map *map);
-
+struct cpu_map *cpu_map__read(FILE *file);
size_t cpu_map__fprintf(struct cpu_map *map, FILE *fp);
+static inline int cpu_map__nr(const struct cpu_map *map)
+{
+ return map ? map->nr : 1;
+}
+
+static inline bool cpu_map__all(const struct cpu_map *map)
+{
+ return map ? map->map[0] == -1 : true;
+}
+
#endif /* __PERF_CPUMAP_H */
diff --git a/tools/perf/util/debug.c b/tools/perf/util/debug.c
index 4dfe0bb..66eb382 100644
--- a/tools/perf/util/debug.c
+++ b/tools/perf/util/debug.c
@@ -23,8 +23,10 @@
if (verbose >= level) {
va_start(args, fmt);
- if (use_browser > 0)
+ if (use_browser == 1)
ret = ui_helpline__show_help(fmt, args);
+ else if (use_browser == 2)
+ ret = perf_gtk__show_helpline(fmt, args);
else
ret = vfprintf(stderr, fmt, args);
va_end(args);
diff --git a/tools/perf/util/debug.h b/tools/perf/util/debug.h
index 015c91d..bb2e7d1 100644
--- a/tools/perf/util/debug.h
+++ b/tools/perf/util/debug.h
@@ -4,6 +4,7 @@
#include <stdbool.h>
#include "event.h"
+#include "../ui/helpline.h"
extern int verbose;
extern bool quiet, dump_trace;
@@ -15,32 +16,26 @@
struct perf_error_ops;
#if defined(NO_NEWT_SUPPORT) && defined(NO_GTK2_SUPPORT)
-static inline int ui_helpline__show_help(const char *format __used, va_list ap __used)
-{
- return 0;
-}
-
-static inline void ui_progress__update(u64 curr __used, u64 total __used,
- const char *title __used) {}
+static inline void ui_progress__update(u64 curr __maybe_unused,
+ u64 total __maybe_unused,
+ const char *title __maybe_unused) {}
#define ui__error(format, arg...) ui__warning(format, ##arg)
static inline int
-perf_error__register(struct perf_error_ops *eops __used)
+perf_error__register(struct perf_error_ops *eops __maybe_unused)
{
return 0;
}
static inline int
-perf_error__unregister(struct perf_error_ops *eops __used)
+perf_error__unregister(struct perf_error_ops *eops __maybe_unused)
{
return 0;
}
#else /* NO_NEWT_SUPPORT && NO_GTK2_SUPPORT */
-extern char ui_helpline__last_msg[];
-int ui_helpline__show_help(const char *format, va_list ap);
#include "../ui/progress.h"
int ui__error(const char *format, ...) __attribute__((format(printf, 1, 2)));
#include "../ui/util.h"
diff --git a/tools/perf/util/dso-test-data.c b/tools/perf/util/dso-test-data.c
index 541cdc7..c6caede 100644
--- a/tools/perf/util/dso-test-data.c
+++ b/tools/perf/util/dso-test-data.c
@@ -23,7 +23,7 @@
int fd, i;
unsigned char *buf;
- fd = mkostemp(templ, O_CREAT|O_WRONLY|O_TRUNC);
+ fd = mkstemp(templ);
buf = malloc(size);
if (!buf) {
diff --git a/tools/perf/util/dwarf-aux.c b/tools/perf/util/dwarf-aux.c
index ee51e9b..3e5f543 100644
--- a/tools/perf/util/dwarf-aux.c
+++ b/tools/perf/util/dwarf-aux.c
@@ -804,6 +804,8 @@
tmp = "union ";
else if (tag == DW_TAG_structure_type)
tmp = "struct ";
+ else if (tag == DW_TAG_enumeration_type)
+ tmp = "enum ";
/* Write a base name */
ret = snprintf(buf, len, "%s%s", tmp, dwarf_diename(&type));
return (ret >= len) ? -E2BIG : ret;
diff --git a/tools/perf/util/event.c b/tools/perf/util/event.c
index 2a6f33c..6715b19 100644
--- a/tools/perf/util/event.c
+++ b/tools/perf/util/event.c
@@ -112,7 +112,7 @@
event->comm.header.type = PERF_RECORD_COMM;
size = strlen(event->comm.comm) + 1;
- size = ALIGN(size, sizeof(u64));
+ size = PERF_ALIGN(size, sizeof(u64));
memset(event->comm.comm + size, 0, machine->id_hdr_size);
event->comm.header.size = (sizeof(event->comm) -
(sizeof(event->comm.comm) - size) +
@@ -120,7 +120,9 @@
if (!full) {
event->comm.tid = pid;
- process(tool, event, &synth_sample, machine);
+ if (process(tool, event, &synth_sample, machine) != 0)
+ return -1;
+
goto out;
}
@@ -143,7 +145,7 @@
sizeof(event->comm.comm));
size = strlen(event->comm.comm) + 1;
- size = ALIGN(size, sizeof(u64));
+ size = PERF_ALIGN(size, sizeof(u64));
memset(event->comm.comm + size, 0, machine->id_hdr_size);
event->comm.header.size = (sizeof(event->comm) -
(sizeof(event->comm.comm) - size) +
@@ -151,7 +153,10 @@
event->comm.tid = pid;
- process(tool, event, &synth_sample, machine);
+ if (process(tool, event, &synth_sample, machine) != 0) {
+ tgid = -1;
+ break;
+ }
}
closedir(tasks);
@@ -167,6 +172,7 @@
{
char filename[PATH_MAX];
FILE *fp;
+ int rc = 0;
snprintf(filename, sizeof(filename), "/proc/%d/maps", pid);
@@ -222,7 +228,7 @@
size = strlen(execname);
execname[size - 1] = '\0'; /* Remove \n */
memcpy(event->mmap.filename, execname, size);
- size = ALIGN(size, sizeof(u64));
+ size = PERF_ALIGN(size, sizeof(u64));
event->mmap.len -= event->mmap.start;
event->mmap.header.size = (sizeof(event->mmap) -
(sizeof(event->mmap.filename) - size));
@@ -231,18 +237,22 @@
event->mmap.pid = tgid;
event->mmap.tid = pid;
- process(tool, event, &synth_sample, machine);
+ if (process(tool, event, &synth_sample, machine) != 0) {
+ rc = -1;
+ break;
+ }
}
}
fclose(fp);
- return 0;
+ return rc;
}
int perf_event__synthesize_modules(struct perf_tool *tool,
perf_event__handler_t process,
struct machine *machine)
{
+ int rc = 0;
struct rb_node *nd;
struct map_groups *kmaps = &machine->kmaps;
union perf_event *event = zalloc((sizeof(event->mmap) +
@@ -272,7 +282,7 @@
if (pos->dso->kernel)
continue;
- size = ALIGN(pos->dso->long_name_len + 1, sizeof(u64));
+ size = PERF_ALIGN(pos->dso->long_name_len + 1, sizeof(u64));
event->mmap.header.type = PERF_RECORD_MMAP;
event->mmap.header.size = (sizeof(event->mmap) -
(sizeof(event->mmap.filename) - size));
@@ -284,11 +294,14 @@
memcpy(event->mmap.filename, pos->dso->long_name,
pos->dso->long_name_len + 1);
- process(tool, event, &synth_sample, machine);
+ if (process(tool, event, &synth_sample, machine) != 0) {
+ rc = -1;
+ break;
+ }
}
free(event);
- return 0;
+ return rc;
}
static int __event__synthesize_thread(union perf_event *comm_event,
@@ -392,12 +405,16 @@
if (*end) /* only interested in proper numerical dirents */
continue;
- __event__synthesize_thread(comm_event, mmap_event, pid, 1,
- process, tool, machine);
+ if (__event__synthesize_thread(comm_event, mmap_event, pid, 1,
+ process, tool, machine) != 0) {
+ err = -1;
+ goto out_closedir;
+ }
}
- closedir(proc);
err = 0;
+out_closedir:
+ closedir(proc);
out_free_mmap:
free(mmap_event);
out_free_comm:
@@ -412,7 +429,7 @@
};
static int find_symbol_cb(void *arg, const char *name, char type,
- u64 start, u64 end __used)
+ u64 start)
{
struct process_symbol_args *args = arg;
@@ -477,7 +494,7 @@
map = machine->vmlinux_maps[MAP__FUNCTION];
size = snprintf(event->mmap.filename, sizeof(event->mmap.filename),
"%s%s", mmap_name, symbol_name) + 1;
- size = ALIGN(size, sizeof(u64));
+ size = PERF_ALIGN(size, sizeof(u64));
event->mmap.header.type = PERF_RECORD_MMAP;
event->mmap.header.size = (sizeof(event->mmap) -
(sizeof(event->mmap.filename) - size) + machine->id_hdr_size);
@@ -497,9 +514,9 @@
return fprintf(fp, ": %s:%d\n", event->comm.comm, event->comm.tid);
}
-int perf_event__process_comm(struct perf_tool *tool __used,
+int perf_event__process_comm(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
+ struct perf_sample *sample __maybe_unused,
struct machine *machine)
{
struct thread *thread = machine__findnew_thread(machine, event->comm.tid);
@@ -515,10 +532,10 @@
return 0;
}
-int perf_event__process_lost(struct perf_tool *tool __used,
+int perf_event__process_lost(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
- struct machine *machine __used)
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
dump_printf(": id:%" PRIu64 ": lost:%" PRIu64 "\n",
event->lost.id, event->lost.lost);
@@ -538,7 +555,8 @@
maps[MAP__FUNCTION]->end = ~0ULL;
}
-static int perf_event__process_kernel_mmap(struct perf_tool *tool __used,
+static int perf_event__process_kernel_mmap(struct perf_tool *tool
+ __maybe_unused,
union perf_event *event,
struct machine *machine)
{
@@ -640,7 +658,7 @@
int perf_event__process_mmap(struct perf_tool *tool,
union perf_event *event,
- struct perf_sample *sample __used,
+ struct perf_sample *sample __maybe_unused,
struct machine *machine)
{
struct thread *thread;
@@ -684,9 +702,9 @@
event->fork.ppid, event->fork.ptid);
}
-int perf_event__process_task(struct perf_tool *tool __used,
+int perf_event__process_task(struct perf_tool *tool __maybe_unused,
union perf_event *event,
- struct perf_sample *sample __used,
+ struct perf_sample *sample __maybe_unused,
struct machine *machine)
{
struct thread *thread = machine__findnew_thread(machine, event->fork.tid);
@@ -886,8 +904,9 @@
al->sym = map__find_symbol(al->map, al->addr, filter);
}
- if (symbol_conf.sym_list && al->sym &&
- !strlist__has_entry(symbol_conf.sym_list, al->sym->name))
+ if (symbol_conf.sym_list &&
+ (!al->sym || !strlist__has_entry(symbol_conf.sym_list,
+ al->sym->name)))
goto out_filtered;
return 0;
diff --git a/tools/perf/util/event.h b/tools/perf/util/event.h
index d84870b..21b99e7 100644
--- a/tools/perf/util/event.h
+++ b/tools/perf/util/event.h
@@ -69,6 +69,16 @@
u64 array[];
};
+struct regs_dump {
+ u64 *regs;
+};
+
+struct stack_dump {
+ u16 offset;
+ u64 size;
+ char *data;
+};
+
struct perf_sample {
u64 ip;
u32 pid, tid;
@@ -82,6 +92,8 @@
void *raw_data;
struct ip_callchain *callchain;
struct branch_stack *branch_stack;
+ struct regs_dump user_regs;
+ struct stack_dump user_stack;
};
#define BUILD_ID_SIZE 20
@@ -89,7 +101,7 @@
struct build_id_event {
struct perf_event_header header;
pid_t pid;
- u8 build_id[ALIGN(BUILD_ID_SIZE, sizeof(u64))];
+ u8 build_id[PERF_ALIGN(BUILD_ID_SIZE, sizeof(u64))];
char filename[];
};
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index 9b38681..ae89686 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -57,7 +57,7 @@
if (evlist->cpus->map[0] < 0)
opts->no_inherit = true;
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ first = perf_evlist__first(evlist);
list_for_each_entry(evsel, &evlist->entries, node) {
perf_evsel__config(evsel, opts, first);
@@ -108,6 +108,25 @@
evlist->nr_entries += nr_entries;
}
+void __perf_evlist__set_leader(struct list_head *list)
+{
+ struct perf_evsel *evsel, *leader;
+
+ leader = list_entry(list->next, struct perf_evsel, node);
+ leader->leader = NULL;
+
+ list_for_each_entry(evsel, list, node) {
+ if (evsel != leader)
+ evsel->leader = leader;
+ }
+}
+
+void perf_evlist__set_leader(struct perf_evlist *evlist)
+{
+ if (evlist->nr_entries)
+ __perf_evlist__set_leader(&evlist->entries);
+}
+
int perf_evlist__add_default(struct perf_evlist *evlist)
{
struct perf_event_attr attr = {
@@ -285,7 +304,7 @@
int cpu, thread;
struct perf_evsel *pos;
- for (cpu = 0; cpu < evlist->cpus->nr; cpu++) {
+ for (cpu = 0; cpu < cpu_map__nr(evlist->cpus); cpu++) {
list_for_each_entry(pos, &evlist->entries, node) {
for (thread = 0; thread < evlist->threads->nr; thread++)
ioctl(FD(pos, cpu, thread),
@@ -296,7 +315,7 @@
static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist)
{
- int nfds = evlist->cpus->nr * evlist->threads->nr * evlist->nr_entries;
+ int nfds = cpu_map__nr(evlist->cpus) * evlist->threads->nr * evlist->nr_entries;
evlist->pollfd = malloc(sizeof(struct pollfd) * nfds);
return evlist->pollfd != NULL ? 0 : -ENOMEM;
}
@@ -357,7 +376,7 @@
int hash;
if (evlist->nr_entries == 1)
- return list_entry(evlist->entries.next, struct perf_evsel, node);
+ return perf_evlist__first(evlist);
hash = hash_64(id, PERF_EVLIST__HLIST_BITS);
head = &evlist->heads[hash];
@@ -367,7 +386,7 @@
return sid->evsel;
if (!perf_evlist__sample_id_all(evlist))
- return list_entry(evlist->entries.next, struct perf_evsel, node);
+ return perf_evlist__first(evlist);
return NULL;
}
@@ -456,8 +475,8 @@
static int perf_evlist__alloc_mmap(struct perf_evlist *evlist)
{
- evlist->nr_mmaps = evlist->cpus->nr;
- if (evlist->cpus->map[0] == -1)
+ evlist->nr_mmaps = cpu_map__nr(evlist->cpus);
+ if (cpu_map__all(evlist->cpus))
evlist->nr_mmaps = evlist->threads->nr;
evlist->mmap = zalloc(evlist->nr_mmaps * sizeof(struct perf_mmap));
return evlist->mmap != NULL ? 0 : -ENOMEM;
@@ -603,11 +622,11 @@
list_for_each_entry(evsel, &evlist->entries, node) {
if ((evsel->attr.read_format & PERF_FORMAT_ID) &&
evsel->sample_id == NULL &&
- perf_evsel__alloc_id(evsel, cpus->nr, threads->nr) < 0)
+ perf_evsel__alloc_id(evsel, cpu_map__nr(cpus), threads->nr) < 0)
return -ENOMEM;
}
- if (evlist->cpus->map[0] == -1)
+ if (cpu_map__all(cpus))
return perf_evlist__mmap_per_thread(evlist, prot, mask);
return perf_evlist__mmap_per_cpu(evlist, prot, mask);
@@ -647,39 +666,44 @@
evlist->threads = NULL;
}
-int perf_evlist__set_filters(struct perf_evlist *evlist)
+int perf_evlist__apply_filters(struct perf_evlist *evlist)
{
- const struct thread_map *threads = evlist->threads;
- const struct cpu_map *cpus = evlist->cpus;
struct perf_evsel *evsel;
- char *filter;
- int thread;
- int cpu;
- int err;
- int fd;
+ int err = 0;
+ const int ncpus = cpu_map__nr(evlist->cpus),
+ nthreads = evlist->threads->nr;
list_for_each_entry(evsel, &evlist->entries, node) {
- filter = evsel->filter;
- if (!filter)
+ if (evsel->filter == NULL)
continue;
- for (cpu = 0; cpu < cpus->nr; cpu++) {
- for (thread = 0; thread < threads->nr; thread++) {
- fd = FD(evsel, cpu, thread);
- err = ioctl(fd, PERF_EVENT_IOC_SET_FILTER, filter);
- if (err)
- return err;
- }
- }
+
+ err = perf_evsel__set_filter(evsel, ncpus, nthreads, evsel->filter);
+ if (err)
+ break;
}
- return 0;
+ return err;
}
-bool perf_evlist__valid_sample_type(const struct perf_evlist *evlist)
+int perf_evlist__set_filter(struct perf_evlist *evlist, const char *filter)
{
- struct perf_evsel *pos, *first;
+ struct perf_evsel *evsel;
+ int err = 0;
+ const int ncpus = cpu_map__nr(evlist->cpus),
+ nthreads = evlist->threads->nr;
- pos = first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ list_for_each_entry(evsel, &evlist->entries, node) {
+ err = perf_evsel__set_filter(evsel, ncpus, nthreads, filter);
+ if (err)
+ break;
+ }
+
+ return err;
+}
+
+bool perf_evlist__valid_sample_type(struct perf_evlist *evlist)
+{
+ struct perf_evsel *first = perf_evlist__first(evlist), *pos = first;
list_for_each_entry_continue(pos, &evlist->entries, node) {
if (first->attr.sample_type != pos->attr.sample_type)
@@ -689,23 +713,19 @@
return true;
}
-u64 perf_evlist__sample_type(const struct perf_evlist *evlist)
+u64 perf_evlist__sample_type(struct perf_evlist *evlist)
{
- struct perf_evsel *first;
-
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ struct perf_evsel *first = perf_evlist__first(evlist);
return first->attr.sample_type;
}
-u16 perf_evlist__id_hdr_size(const struct perf_evlist *evlist)
+u16 perf_evlist__id_hdr_size(struct perf_evlist *evlist)
{
- struct perf_evsel *first;
+ struct perf_evsel *first = perf_evlist__first(evlist);
struct perf_sample *data;
u64 sample_type;
u16 size = 0;
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
-
if (!first->attr.sample_id_all)
goto out;
@@ -729,11 +749,9 @@
return size;
}
-bool perf_evlist__valid_sample_id_all(const struct perf_evlist *evlist)
+bool perf_evlist__valid_sample_id_all(struct perf_evlist *evlist)
{
- struct perf_evsel *pos, *first;
-
- pos = first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ struct perf_evsel *first = perf_evlist__first(evlist), *pos = first;
list_for_each_entry_continue(pos, &evlist->entries, node) {
if (first->attr.sample_id_all != pos->attr.sample_id_all)
@@ -743,11 +761,9 @@
return true;
}
-bool perf_evlist__sample_id_all(const struct perf_evlist *evlist)
+bool perf_evlist__sample_id_all(struct perf_evlist *evlist)
{
- struct perf_evsel *first;
-
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
+ struct perf_evsel *first = perf_evlist__first(evlist);
return first->attr.sample_id_all;
}
@@ -757,21 +773,13 @@
evlist->selected = evsel;
}
-int perf_evlist__open(struct perf_evlist *evlist, bool group)
+int perf_evlist__open(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel, *first;
+ struct perf_evsel *evsel;
int err, ncpus, nthreads;
- first = list_entry(evlist->entries.next, struct perf_evsel, node);
-
list_for_each_entry(evsel, &evlist->entries, node) {
- struct xyarray *group_fd = NULL;
-
- if (group && evsel != first)
- group_fd = first->fd;
-
- err = perf_evsel__open(evsel, evlist->cpus, evlist->threads,
- group, group_fd);
+ err = perf_evsel__open(evsel, evlist->cpus, evlist->threads);
if (err < 0)
goto out_err;
}
@@ -883,8 +891,21 @@
}
int perf_evlist__parse_sample(struct perf_evlist *evlist, union perf_event *event,
- struct perf_sample *sample, bool swapped)
+ struct perf_sample *sample)
{
- struct perf_evsel *e = list_entry(evlist->entries.next, struct perf_evsel, node);
- return perf_evsel__parse_sample(e, event, sample, swapped);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
+ return perf_evsel__parse_sample(evsel, event, sample);
+}
+
+size_t perf_evlist__fprintf(struct perf_evlist *evlist, FILE *fp)
+{
+ struct perf_evsel *evsel;
+ size_t printed = 0;
+
+ list_for_each_entry(evsel, &evlist->entries, node) {
+ printed += fprintf(fp, "%s%s", evsel->idx ? ", " : "",
+ perf_evsel__name(evsel));
+ }
+
+ return printed + fprintf(fp, "\n");;
}
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index 528c1ac..3f1fb66 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -5,6 +5,7 @@
#include <stdio.h>
#include "../perf.h"
#include "event.h"
+#include "evsel.h"
#include "util.h"
#include <unistd.h>
@@ -41,8 +42,6 @@
void *handler;
};
-struct perf_evsel;
-
struct perf_evlist *perf_evlist__new(struct cpu_map *cpus,
struct thread_map *threads);
void perf_evlist__init(struct perf_evlist *evlist, struct cpu_map *cpus,
@@ -73,6 +72,8 @@
#define perf_evlist__set_tracepoints_handlers_array(evlist, array) \
perf_evlist__set_tracepoints_handlers(evlist, array, ARRAY_SIZE(array))
+int perf_evlist__set_filter(struct perf_evlist *evlist, const char *filter);
+
struct perf_evsel *
perf_evlist__find_tracepoint_by_id(struct perf_evlist *evlist, int id);
@@ -85,7 +86,7 @@
union perf_event *perf_evlist__mmap_read(struct perf_evlist *self, int idx);
-int perf_evlist__open(struct perf_evlist *evlist, bool group);
+int perf_evlist__open(struct perf_evlist *evlist);
void perf_evlist__config_attrs(struct perf_evlist *evlist,
struct perf_record_opts *opts);
@@ -116,20 +117,34 @@
int perf_evlist__create_maps(struct perf_evlist *evlist,
struct perf_target *target);
void perf_evlist__delete_maps(struct perf_evlist *evlist);
-int perf_evlist__set_filters(struct perf_evlist *evlist);
+int perf_evlist__apply_filters(struct perf_evlist *evlist);
-u64 perf_evlist__sample_type(const struct perf_evlist *evlist);
-bool perf_evlist__sample_id_all(const const struct perf_evlist *evlist);
-u16 perf_evlist__id_hdr_size(const struct perf_evlist *evlist);
+void __perf_evlist__set_leader(struct list_head *list);
+void perf_evlist__set_leader(struct perf_evlist *evlist);
+
+u64 perf_evlist__sample_type(struct perf_evlist *evlist);
+bool perf_evlist__sample_id_all(struct perf_evlist *evlist);
+u16 perf_evlist__id_hdr_size(struct perf_evlist *evlist);
int perf_evlist__parse_sample(struct perf_evlist *evlist, union perf_event *event,
- struct perf_sample *sample, bool swapped);
+ struct perf_sample *sample);
-bool perf_evlist__valid_sample_type(const struct perf_evlist *evlist);
-bool perf_evlist__valid_sample_id_all(const struct perf_evlist *evlist);
+bool perf_evlist__valid_sample_type(struct perf_evlist *evlist);
+bool perf_evlist__valid_sample_id_all(struct perf_evlist *evlist);
void perf_evlist__splice_list_tail(struct perf_evlist *evlist,
struct list_head *list,
int nr_entries);
+static inline struct perf_evsel *perf_evlist__first(struct perf_evlist *evlist)
+{
+ return list_entry(evlist->entries.next, struct perf_evsel, node);
+}
+
+static inline struct perf_evsel *perf_evlist__last(struct perf_evlist *evlist)
+{
+ return list_entry(evlist->entries.prev, struct perf_evsel, node);
+}
+
+size_t perf_evlist__fprintf(struct perf_evlist *evlist, FILE *fp);
#endif /* __PERF_EVLIST_H */
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 2eaae14..ffdd94e 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -8,7 +8,10 @@
*/
#include <byteswap.h>
+#include <linux/bitops.h>
#include "asm/bug.h"
+#include "debugfs.h"
+#include "event-parse.h"
#include "evsel.h"
#include "evlist.h"
#include "util.h"
@@ -16,9 +19,10 @@
#include "thread_map.h"
#include "target.h"
#include "../../../include/linux/hw_breakpoint.h"
+#include "../../include/linux/perf_event.h"
+#include "perf_regs.h"
#define FD(e, x, y) (*(int *)xyarray__entry(e->fd, x, y))
-#define GROUP_FD(group_fd, cpu) (*(int *)xyarray__entry(group_fd, cpu, 0))
static int __perf_evsel__sample_size(u64 sample_type)
{
@@ -66,7 +70,80 @@
return evsel;
}
-static const char *perf_evsel__hw_names[PERF_COUNT_HW_MAX] = {
+struct event_format *event_format__new(const char *sys, const char *name)
+{
+ int fd, n;
+ char *filename;
+ void *bf = NULL, *nbf;
+ size_t size = 0, alloc_size = 0;
+ struct event_format *format = NULL;
+
+ if (asprintf(&filename, "%s/%s/%s/format", tracing_events_path, sys, name) < 0)
+ goto out;
+
+ fd = open(filename, O_RDONLY);
+ if (fd < 0)
+ goto out_free_filename;
+
+ do {
+ if (size == alloc_size) {
+ alloc_size += BUFSIZ;
+ nbf = realloc(bf, alloc_size);
+ if (nbf == NULL)
+ goto out_free_bf;
+ bf = nbf;
+ }
+
+ n = read(fd, bf + size, BUFSIZ);
+ if (n < 0)
+ goto out_free_bf;
+ size += n;
+ } while (n > 0);
+
+ pevent_parse_format(&format, bf, size, sys);
+
+out_free_bf:
+ free(bf);
+ close(fd);
+out_free_filename:
+ free(filename);
+out:
+ return format;
+}
+
+struct perf_evsel *perf_evsel__newtp(const char *sys, const char *name, int idx)
+{
+ struct perf_evsel *evsel = zalloc(sizeof(*evsel));
+
+ if (evsel != NULL) {
+ struct perf_event_attr attr = {
+ .type = PERF_TYPE_TRACEPOINT,
+ .sample_type = (PERF_SAMPLE_RAW | PERF_SAMPLE_TIME |
+ PERF_SAMPLE_CPU | PERF_SAMPLE_PERIOD),
+ };
+
+ if (asprintf(&evsel->name, "%s:%s", sys, name) < 0)
+ goto out_free;
+
+ evsel->tp_format = event_format__new(sys, name);
+ if (evsel->tp_format == NULL)
+ goto out_free;
+
+ event_attr_init(&attr);
+ attr.config = evsel->tp_format->id;
+ attr.sample_period = 1;
+ perf_evsel__init(evsel, &attr, idx);
+ }
+
+ return evsel;
+
+out_free:
+ free(evsel->name);
+ free(evsel);
+ return NULL;
+}
+
+const char *perf_evsel__hw_names[PERF_COUNT_HW_MAX] = {
"cycles",
"instructions",
"cache-references",
@@ -129,12 +206,12 @@
return r + perf_evsel__add_modifiers(evsel, bf + r, size - r);
}
-static const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX] = {
+const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX] = {
"cpu-clock",
"task-clock",
"page-faults",
"context-switches",
- "CPU-migrations",
+ "cpu-migrations",
"minor-faults",
"major-faults",
"alignment-faults",
@@ -317,7 +394,8 @@
break;
default:
- scnprintf(bf, sizeof(bf), "%s", "unknown attr type");
+ scnprintf(bf, sizeof(bf), "unknown attr type: %d",
+ evsel->attr.type);
break;
}
@@ -367,9 +445,18 @@
attr->mmap_data = track;
}
- if (opts->call_graph)
+ if (opts->call_graph) {
attr->sample_type |= PERF_SAMPLE_CALLCHAIN;
+ if (opts->call_graph == CALLCHAIN_DWARF) {
+ attr->sample_type |= PERF_SAMPLE_REGS_USER |
+ PERF_SAMPLE_STACK_USER;
+ attr->sample_regs_user = PERF_REGS_MASK;
+ attr->sample_stack_user = opts->stack_dump_size;
+ attr->exclude_callchain_user = 1;
+ }
+ }
+
if (perf_target__has_cpu(&opts->target))
attr->sample_type |= PERF_SAMPLE_CPU;
@@ -421,6 +508,24 @@
return evsel->fd != NULL ? 0 : -ENOMEM;
}
+int perf_evsel__set_filter(struct perf_evsel *evsel, int ncpus, int nthreads,
+ const char *filter)
+{
+ int cpu, thread;
+
+ for (cpu = 0; cpu < ncpus; cpu++) {
+ for (thread = 0; thread < nthreads; thread++) {
+ int fd = FD(evsel, cpu, thread),
+ err = ioctl(fd, PERF_EVENT_IOC_SET_FILTER, filter);
+
+ if (err)
+ return err;
+ }
+ }
+
+ return 0;
+}
+
int perf_evsel__alloc_id(struct perf_evsel *evsel, int ncpus, int nthreads)
{
evsel->sample_id = xyarray__new(ncpus, nthreads, sizeof(struct perf_sample_id));
@@ -481,6 +586,9 @@
{
perf_evsel__exit(evsel);
close_cgroup(evsel->cgrp);
+ free(evsel->group_name);
+ if (evsel->tp_format)
+ pevent_free_format(evsel->tp_format);
free(evsel->name);
free(evsel);
}
@@ -556,9 +664,28 @@
return 0;
}
+static int get_group_fd(struct perf_evsel *evsel, int cpu, int thread)
+{
+ struct perf_evsel *leader = evsel->leader;
+ int fd;
+
+ if (!leader)
+ return -1;
+
+ /*
+ * Leader must be already processed/open,
+ * if not it's a bug.
+ */
+ BUG_ON(!leader->fd);
+
+ fd = FD(leader, cpu, thread);
+ BUG_ON(fd == -1);
+
+ return fd;
+}
+
static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
- struct thread_map *threads, bool group,
- struct xyarray *group_fds)
+ struct thread_map *threads)
{
int cpu, thread;
unsigned long flags = 0;
@@ -574,13 +701,15 @@
}
for (cpu = 0; cpu < cpus->nr; cpu++) {
- int group_fd = group_fds ? GROUP_FD(group_fds, cpu) : -1;
for (thread = 0; thread < threads->nr; thread++) {
+ int group_fd;
if (!evsel->cgrp)
pid = threads->map[thread];
+ group_fd = get_group_fd(evsel, cpu, thread);
+
FD(evsel, cpu, thread) = sys_perf_event_open(&evsel->attr,
pid,
cpus->map[cpu],
@@ -589,9 +718,6 @@
err = -errno;
goto out_close;
}
-
- if (group && group_fd == -1)
- group_fd = FD(evsel, cpu, thread);
}
}
@@ -635,8 +761,7 @@
};
int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
- struct thread_map *threads, bool group,
- struct xyarray *group_fd)
+ struct thread_map *threads)
{
if (cpus == NULL) {
/* Work around old compiler warnings about strict aliasing */
@@ -646,30 +771,28 @@
if (threads == NULL)
threads = &empty_thread_map.map;
- return __perf_evsel__open(evsel, cpus, threads, group, group_fd);
+ return __perf_evsel__open(evsel, cpus, threads);
}
int perf_evsel__open_per_cpu(struct perf_evsel *evsel,
- struct cpu_map *cpus, bool group,
- struct xyarray *group_fd)
+ struct cpu_map *cpus)
{
- return __perf_evsel__open(evsel, cpus, &empty_thread_map.map, group,
- group_fd);
+ return __perf_evsel__open(evsel, cpus, &empty_thread_map.map);
}
int perf_evsel__open_per_thread(struct perf_evsel *evsel,
- struct thread_map *threads, bool group,
- struct xyarray *group_fd)
+ struct thread_map *threads)
{
- return __perf_evsel__open(evsel, &empty_cpu_map.map, threads, group,
- group_fd);
+ return __perf_evsel__open(evsel, &empty_cpu_map.map, threads);
}
-static int perf_event__parse_id_sample(const union perf_event *event, u64 type,
- struct perf_sample *sample,
- bool swapped)
+static int perf_evsel__parse_id_sample(const struct perf_evsel *evsel,
+ const union perf_event *event,
+ struct perf_sample *sample)
{
+ u64 type = evsel->attr.sample_type;
const u64 *array = event->sample.array;
+ bool swapped = evsel->needs_swap;
union u64_swap u;
array += ((event->header.size -
@@ -730,9 +853,11 @@
}
int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
- struct perf_sample *data, bool swapped)
+ struct perf_sample *data)
{
u64 type = evsel->attr.sample_type;
+ u64 regs_user = evsel->attr.sample_regs_user;
+ bool swapped = evsel->needs_swap;
const u64 *array;
/*
@@ -749,7 +874,7 @@
if (event->header.type != PERF_RECORD_SAMPLE) {
if (!evsel->attr.sample_id_all)
return 0;
- return perf_event__parse_id_sample(event, type, data, swapped);
+ return perf_evsel__parse_id_sample(evsel, event, data);
}
array = event->sample.array;
@@ -869,6 +994,32 @@
sz /= sizeof(u64);
array += sz;
}
+
+ if (type & PERF_SAMPLE_REGS_USER) {
+ /* First u64 tells us if we have any regs in sample. */
+ u64 avail = *array++;
+
+ if (avail) {
+ data->user_regs.regs = (u64 *)array;
+ array += hweight_long(regs_user);
+ }
+ }
+
+ if (type & PERF_SAMPLE_STACK_USER) {
+ u64 size = *array++;
+
+ data->user_stack.offset = ((char *)(array - 1)
+ - (char *) event);
+
+ if (!size) {
+ data->user_stack.size = 0;
+ } else {
+ data->user_stack.data = (char *)array;
+ array += size / sizeof(*array);
+ data->user_stack.size = *array;
+ }
+ }
+
return 0;
}
@@ -947,3 +1098,72 @@
return 0;
}
+
+struct format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name)
+{
+ return pevent_find_field(evsel->tp_format, name);
+}
+
+void *perf_evsel__rawptr(struct perf_evsel *evsel, struct perf_sample *sample,
+ const char *name)
+{
+ struct format_field *field = perf_evsel__field(evsel, name);
+ int offset;
+
+ if (!field)
+ return NULL;
+
+ offset = field->offset;
+
+ if (field->flags & FIELD_IS_DYNAMIC) {
+ offset = *(int *)(sample->raw_data + field->offset);
+ offset &= 0xffff;
+ }
+
+ return sample->raw_data + offset;
+}
+
+u64 perf_evsel__intval(struct perf_evsel *evsel, struct perf_sample *sample,
+ const char *name)
+{
+ struct format_field *field = perf_evsel__field(evsel, name);
+ void *ptr;
+ u64 value;
+
+ if (!field)
+ return 0;
+
+ ptr = sample->raw_data + field->offset;
+
+ switch (field->size) {
+ case 1:
+ return *(u8 *)ptr;
+ case 2:
+ value = *(u16 *)ptr;
+ break;
+ case 4:
+ value = *(u32 *)ptr;
+ break;
+ case 8:
+ value = *(u64 *)ptr;
+ break;
+ default:
+ return 0;
+ }
+
+ if (!evsel->needs_swap)
+ return value;
+
+ switch (field->size) {
+ case 2:
+ return bswap_16(value);
+ case 4:
+ return bswap_32(value);
+ case 8:
+ return bswap_64(value);
+ default:
+ return 0;
+ }
+
+ return 0;
+}
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index b559929..3ead0d5 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -53,9 +53,10 @@
u64 *id;
struct perf_counts *counts;
int idx;
- int ids;
+ u32 ids;
struct hists hists;
char *name;
+ struct event_format *tp_format;
union {
void *priv;
off_t id_offset;
@@ -65,8 +66,14 @@
void *func;
void *data;
} handler;
+ struct cpu_map *cpus;
unsigned int sample_size;
bool supported;
+ bool needs_swap;
+ /* parse modifier helper */
+ int exclude_GH;
+ struct perf_evsel *leader;
+ char *group_name;
};
struct cpu_map;
@@ -75,6 +82,10 @@
struct perf_record_opts;
struct perf_evsel *perf_evsel__new(struct perf_event_attr *attr, int idx);
+struct perf_evsel *perf_evsel__newtp(const char *sys, const char *name, int idx);
+
+struct event_format *event_format__new(const char *sys, const char *name);
+
void perf_evsel__init(struct perf_evsel *evsel,
struct perf_event_attr *attr, int idx);
void perf_evsel__exit(struct perf_evsel *evsel);
@@ -92,8 +103,10 @@
[PERF_EVSEL__MAX_ALIASES];
extern const char *perf_evsel__hw_cache_op[PERF_COUNT_HW_CACHE_OP_MAX]
[PERF_EVSEL__MAX_ALIASES];
-const char *perf_evsel__hw_cache_result[PERF_COUNT_HW_CACHE_RESULT_MAX]
- [PERF_EVSEL__MAX_ALIASES];
+extern const char *perf_evsel__hw_cache_result[PERF_COUNT_HW_CACHE_RESULT_MAX]
+ [PERF_EVSEL__MAX_ALIASES];
+extern const char *perf_evsel__hw_names[PERF_COUNT_HW_MAX];
+extern const char *perf_evsel__sw_names[PERF_COUNT_SW_MAX];
int __perf_evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result,
char *bf, size_t size);
const char *perf_evsel__name(struct perf_evsel *evsel);
@@ -105,21 +118,46 @@
void perf_evsel__free_id(struct perf_evsel *evsel);
void perf_evsel__close_fd(struct perf_evsel *evsel, int ncpus, int nthreads);
+int perf_evsel__set_filter(struct perf_evsel *evsel, int ncpus, int nthreads,
+ const char *filter);
+
int perf_evsel__open_per_cpu(struct perf_evsel *evsel,
- struct cpu_map *cpus, bool group,
- struct xyarray *group_fds);
+ struct cpu_map *cpus);
int perf_evsel__open_per_thread(struct perf_evsel *evsel,
- struct thread_map *threads, bool group,
- struct xyarray *group_fds);
+ struct thread_map *threads);
int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
- struct thread_map *threads, bool group,
- struct xyarray *group_fds);
+ struct thread_map *threads);
void perf_evsel__close(struct perf_evsel *evsel, int ncpus, int nthreads);
+struct perf_sample;
+
+void *perf_evsel__rawptr(struct perf_evsel *evsel, struct perf_sample *sample,
+ const char *name);
+u64 perf_evsel__intval(struct perf_evsel *evsel, struct perf_sample *sample,
+ const char *name);
+
+static inline char *perf_evsel__strval(struct perf_evsel *evsel,
+ struct perf_sample *sample,
+ const char *name)
+{
+ return perf_evsel__rawptr(evsel, sample, name);
+}
+
+struct format_field;
+
+struct format_field *perf_evsel__field(struct perf_evsel *evsel, const char *name);
+
#define perf_evsel__match(evsel, t, c) \
(evsel->attr.type == PERF_TYPE_##t && \
evsel->attr.config == PERF_COUNT_##c)
+static inline bool perf_evsel__match2(struct perf_evsel *e1,
+ struct perf_evsel *e2)
+{
+ return (e1->attr.type == e2->attr.type) &&
+ (e1->attr.config == e2->attr.config);
+}
+
int __perf_evsel__read_on_cpu(struct perf_evsel *evsel,
int cpu, int thread, bool scale);
@@ -181,5 +219,10 @@
void hists__init(struct hists *hists);
int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
- struct perf_sample *sample, bool swapped);
+ struct perf_sample *sample);
+
+static inline struct perf_evsel *perf_evsel__next(struct perf_evsel *evsel)
+{
+ return list_entry(evsel->node.next, struct perf_evsel, node);
+}
#endif /* __PERF_EVSEL_H */
diff --git a/tools/perf/util/generate-cmdlist.sh b/tools/perf/util/generate-cmdlist.sh
index f06f6fd..389590c 100755
--- a/tools/perf/util/generate-cmdlist.sh
+++ b/tools/perf/util/generate-cmdlist.sh
@@ -21,4 +21,19 @@
p
}' "Documentation/perf-$cmd.txt"
done
+
+echo "#ifndef NO_LIBELF_SUPPORT"
+sed -n -e 's/^perf-\([^ ]*\)[ ].* full.*/\1/p' command-list.txt |
+sort |
+while read cmd
+do
+ sed -n '
+ /^NAME/,/perf-'"$cmd"'/H
+ ${
+ x
+ s/.*perf-'"$cmd"' - \(.*\)/ {"'"$cmd"'", "\1"},/
+ p
+ }' "Documentation/perf-$cmd.txt"
+done
+echo "#endif /* NO_LIBELF_SUPPORT */"
echo "};"
diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index 74ea3c2..7daad23 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -20,11 +20,14 @@
#include "symbol.h"
#include "debug.h"
#include "cpumap.h"
+#include "pmu.h"
+#include "vdso.h"
+#include "strbuf.h"
static bool no_buildid_cache = false;
-static int event_count;
-static struct perf_trace_event_type *events;
+static int trace_event_count;
+static struct perf_trace_event_type *trace_events;
static u32 header_argc;
static const char **header_argv;
@@ -36,24 +39,24 @@
if (strlen(name) > MAX_EVENT_NAME)
pr_warning("Event %s will be truncated\n", name);
- nevents = realloc(events, (event_count + 1) * sizeof(*events));
+ nevents = realloc(trace_events, (trace_event_count + 1) * sizeof(*trace_events));
if (nevents == NULL)
return -ENOMEM;
- events = nevents;
+ trace_events = nevents;
- memset(&events[event_count], 0, sizeof(struct perf_trace_event_type));
- events[event_count].event_id = id;
- strncpy(events[event_count].name, name, MAX_EVENT_NAME - 1);
- event_count++;
+ memset(&trace_events[trace_event_count], 0, sizeof(struct perf_trace_event_type));
+ trace_events[trace_event_count].event_id = id;
+ strncpy(trace_events[trace_event_count].name, name, MAX_EVENT_NAME - 1);
+ trace_event_count++;
return 0;
}
char *perf_header__find_event(u64 id)
{
int i;
- for (i = 0 ; i < event_count; i++) {
- if (events[i].event_id == id)
- return events[i].name;
+ for (i = 0 ; i < trace_event_count; i++) {
+ if (trace_events[i].event_id == id)
+ return trace_events[i].name;
}
return NULL;
}
@@ -128,7 +131,7 @@
int ret;
olen = strlen(str) + 1;
- len = ALIGN(olen, NAME_ALIGN);
+ len = PERF_ALIGN(olen, NAME_ALIGN);
/* write len, incl. \0 */
ret = do_write(fd, &len, sizeof(len));
@@ -206,6 +209,29 @@
continue; \
else
+static int write_buildid(char *name, size_t name_len, u8 *build_id,
+ pid_t pid, u16 misc, int fd)
+{
+ int err;
+ struct build_id_event b;
+ size_t len;
+
+ len = name_len + 1;
+ len = PERF_ALIGN(len, NAME_ALIGN);
+
+ memset(&b, 0, sizeof(b));
+ memcpy(&b.build_id, build_id, BUILD_ID_SIZE);
+ b.pid = pid;
+ b.header.misc = misc;
+ b.header.size = sizeof(b) + len;
+
+ err = do_write(fd, &b, sizeof(b));
+ if (err < 0)
+ return err;
+
+ return write_padded(fd, name, name_len + 1, len);
+}
+
static int __dsos__write_buildid_table(struct list_head *head, pid_t pid,
u16 misc, int fd)
{
@@ -213,24 +239,23 @@
dsos__for_each_with_build_id(pos, head) {
int err;
- struct build_id_event b;
- size_t len;
+ char *name;
+ size_t name_len;
if (!pos->hit)
continue;
- len = pos->long_name_len + 1;
- len = ALIGN(len, NAME_ALIGN);
- memset(&b, 0, sizeof(b));
- memcpy(&b.build_id, pos->build_id, sizeof(pos->build_id));
- b.pid = pid;
- b.header.misc = misc;
- b.header.size = sizeof(b) + len;
- err = do_write(fd, &b, sizeof(b));
- if (err < 0)
- return err;
- err = write_padded(fd, pos->long_name,
- pos->long_name_len + 1, len);
- if (err < 0)
+
+ if (is_vdso_map(pos->short_name)) {
+ name = (char *) VDSO__MAP_NAME;
+ name_len = sizeof(VDSO__MAP_NAME) + 1;
+ } else {
+ name = pos->long_name;
+ name_len = pos->long_name_len + 1;
+ }
+
+ err = write_buildid(name, name_len, pos->build_id,
+ pid, misc, fd);
+ if (err)
return err;
}
@@ -276,19 +301,20 @@
}
int build_id_cache__add_s(const char *sbuild_id, const char *debugdir,
- const char *name, bool is_kallsyms)
+ const char *name, bool is_kallsyms, bool is_vdso)
{
const size_t size = PATH_MAX;
char *realname, *filename = zalloc(size),
*linkname = zalloc(size), *targetname;
int len, err = -1;
+ bool slash = is_kallsyms || is_vdso;
if (is_kallsyms) {
if (symbol_conf.kptr_restrict) {
pr_debug("Not caching a kptr_restrict'ed /proc/kallsyms\n");
return 0;
}
- realname = (char *)name;
+ realname = (char *) name;
} else
realname = realpath(name, NULL);
@@ -296,7 +322,8 @@
goto out_free;
len = scnprintf(filename, size, "%s%s%s",
- debugdir, is_kallsyms ? "/" : "", realname);
+ debugdir, slash ? "/" : "",
+ is_vdso ? VDSO__MAP_NAME : realname);
if (mkdir_p(filename, 0755))
goto out_free;
@@ -332,13 +359,14 @@
static int build_id_cache__add_b(const u8 *build_id, size_t build_id_size,
const char *name, const char *debugdir,
- bool is_kallsyms)
+ bool is_kallsyms, bool is_vdso)
{
char sbuild_id[BUILD_ID_SIZE * 2 + 1];
build_id__sprintf(build_id, build_id_size, sbuild_id);
- return build_id_cache__add_s(sbuild_id, debugdir, name, is_kallsyms);
+ return build_id_cache__add_s(sbuild_id, debugdir, name,
+ is_kallsyms, is_vdso);
}
int build_id_cache__remove_s(const char *sbuild_id, const char *debugdir)
@@ -382,9 +410,11 @@
static int dso__cache_build_id(struct dso *dso, const char *debugdir)
{
bool is_kallsyms = dso->kernel && dso->long_name[0] != '/';
+ bool is_vdso = is_vdso_map(dso->short_name);
return build_id_cache__add_b(dso->build_id, sizeof(dso->build_id),
- dso->long_name, debugdir, is_kallsyms);
+ dso->long_name, debugdir,
+ is_kallsyms, is_vdso);
}
static int __dsos__cache_build_ids(struct list_head *head, const char *debugdir)
@@ -446,7 +476,7 @@
return ret;
}
-static int write_tracing_data(int fd, struct perf_header *h __used,
+static int write_tracing_data(int fd, struct perf_header *h __maybe_unused,
struct perf_evlist *evlist)
{
return read_tracing_data(fd, &evlist->entries);
@@ -454,7 +484,7 @@
static int write_build_id(int fd, struct perf_header *h,
- struct perf_evlist *evlist __used)
+ struct perf_evlist *evlist __maybe_unused)
{
struct perf_session *session;
int err;
@@ -475,8 +505,8 @@
return 0;
}
-static int write_hostname(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_hostname(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
struct utsname uts;
int ret;
@@ -488,8 +518,8 @@
return do_write_string(fd, uts.nodename);
}
-static int write_osrelease(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_osrelease(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
struct utsname uts;
int ret;
@@ -501,8 +531,8 @@
return do_write_string(fd, uts.release);
}
-static int write_arch(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_arch(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
struct utsname uts;
int ret;
@@ -514,14 +544,14 @@
return do_write_string(fd, uts.machine);
}
-static int write_version(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_version(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
return do_write_string(fd, perf_version_string);
}
-static int write_cpudesc(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_cpudesc(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
#ifndef CPUINFO_PROC
#define CPUINFO_PROC NULL
@@ -579,8 +609,8 @@
return ret;
}
-static int write_nrcpus(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_nrcpus(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
long nr;
u32 nrc, nra;
@@ -605,15 +635,14 @@
return do_write(fd, &nra, sizeof(nra));
}
-static int write_event_desc(int fd, struct perf_header *h __used,
+static int write_event_desc(int fd, struct perf_header *h __maybe_unused,
struct perf_evlist *evlist)
{
- struct perf_evsel *attr;
- u32 nre = 0, nri, sz;
+ struct perf_evsel *evsel;
+ u32 nre, nri, sz;
int ret;
- list_for_each_entry(attr, &evlist->entries, node)
- nre++;
+ nre = evlist->nr_entries;
/*
* write number of events
@@ -625,14 +654,14 @@
/*
* size of perf_event_attr struct
*/
- sz = (u32)sizeof(attr->attr);
+ sz = (u32)sizeof(evsel->attr);
ret = do_write(fd, &sz, sizeof(sz));
if (ret < 0)
return ret;
- list_for_each_entry(attr, &evlist->entries, node) {
+ list_for_each_entry(evsel, &evlist->entries, node) {
- ret = do_write(fd, &attr->attr, sz);
+ ret = do_write(fd, &evsel->attr, sz);
if (ret < 0)
return ret;
/*
@@ -642,7 +671,7 @@
* copy into an nri to be independent of the
* type of ids,
*/
- nri = attr->ids;
+ nri = evsel->ids;
ret = do_write(fd, &nri, sizeof(nri));
if (ret < 0)
return ret;
@@ -650,21 +679,21 @@
/*
* write event string as passed on cmdline
*/
- ret = do_write_string(fd, perf_evsel__name(attr));
+ ret = do_write_string(fd, perf_evsel__name(evsel));
if (ret < 0)
return ret;
/*
* write unique ids for this event
*/
- ret = do_write(fd, attr->id, attr->ids * sizeof(u64));
+ ret = do_write(fd, evsel->id, evsel->ids * sizeof(u64));
if (ret < 0)
return ret;
}
return 0;
}
-static int write_cmdline(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_cmdline(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
char buf[MAXPATHLEN];
char proc[32];
@@ -832,8 +861,8 @@
return tp;
}
-static int write_cpu_topology(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_cpu_topology(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
struct cpu_topo *tp;
u32 i;
@@ -868,8 +897,8 @@
-static int write_total_mem(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_total_mem(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
char *buf = NULL;
FILE *fp;
@@ -954,8 +983,8 @@
return ret;
}
-static int write_numa_topology(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_numa_topology(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
char *buf = NULL;
size_t len = 0;
@@ -1004,16 +1033,56 @@
}
/*
+ * File format:
+ *
+ * struct pmu_mappings {
+ * u32 pmu_num;
+ * struct pmu_map {
+ * u32 type;
+ * char name[];
+ * }[pmu_num];
+ * };
+ */
+
+static int write_pmu_mappings(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
+{
+ struct perf_pmu *pmu = NULL;
+ off_t offset = lseek(fd, 0, SEEK_CUR);
+ __u32 pmu_num = 0;
+
+ /* write real pmu_num later */
+ do_write(fd, &pmu_num, sizeof(pmu_num));
+
+ while ((pmu = perf_pmu__scan(pmu))) {
+ if (!pmu->name)
+ continue;
+ pmu_num++;
+ do_write(fd, &pmu->type, sizeof(pmu->type));
+ do_write_string(fd, pmu->name);
+ }
+
+ if (pwrite(fd, &pmu_num, sizeof(pmu_num), offset) != sizeof(pmu_num)) {
+ /* discard all */
+ lseek(fd, offset, SEEK_SET);
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
* default get_cpuid(): nothing gets recorded
* actual implementation must be in arch/$(ARCH)/util/header.c
*/
-int __attribute__((weak)) get_cpuid(char *buffer __used, size_t sz __used)
+int __attribute__ ((weak)) get_cpuid(char *buffer __maybe_unused,
+ size_t sz __maybe_unused)
{
return -1;
}
-static int write_cpuid(int fd, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_cpuid(int fd, struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
char buffer[64];
int ret;
@@ -1027,133 +1096,113 @@
return do_write_string(fd, buffer);
}
-static int write_branch_stack(int fd __used, struct perf_header *h __used,
- struct perf_evlist *evlist __used)
+static int write_branch_stack(int fd __maybe_unused,
+ struct perf_header *h __maybe_unused,
+ struct perf_evlist *evlist __maybe_unused)
{
return 0;
}
-static void print_hostname(struct perf_header *ph, int fd, FILE *fp)
+static void print_hostname(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- char *str = do_read_string(fd, ph);
- fprintf(fp, "# hostname : %s\n", str);
- free(str);
+ fprintf(fp, "# hostname : %s\n", ph->env.hostname);
}
-static void print_osrelease(struct perf_header *ph, int fd, FILE *fp)
+static void print_osrelease(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- char *str = do_read_string(fd, ph);
- fprintf(fp, "# os release : %s\n", str);
- free(str);
+ fprintf(fp, "# os release : %s\n", ph->env.os_release);
}
-static void print_arch(struct perf_header *ph, int fd, FILE *fp)
+static void print_arch(struct perf_header *ph, int fd __maybe_unused, FILE *fp)
{
- char *str = do_read_string(fd, ph);
- fprintf(fp, "# arch : %s\n", str);
- free(str);
+ fprintf(fp, "# arch : %s\n", ph->env.arch);
}
-static void print_cpudesc(struct perf_header *ph, int fd, FILE *fp)
+static void print_cpudesc(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- char *str = do_read_string(fd, ph);
- fprintf(fp, "# cpudesc : %s\n", str);
- free(str);
+ fprintf(fp, "# cpudesc : %s\n", ph->env.cpu_desc);
}
-static void print_nrcpus(struct perf_header *ph, int fd, FILE *fp)
+static void print_nrcpus(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- ssize_t ret;
- u32 nr;
-
- ret = read(fd, &nr, sizeof(nr));
- if (ret != (ssize_t)sizeof(nr))
- nr = -1; /* interpreted as error */
-
- if (ph->needs_swap)
- nr = bswap_32(nr);
-
- fprintf(fp, "# nrcpus online : %u\n", nr);
-
- ret = read(fd, &nr, sizeof(nr));
- if (ret != (ssize_t)sizeof(nr))
- nr = -1; /* interpreted as error */
-
- if (ph->needs_swap)
- nr = bswap_32(nr);
-
- fprintf(fp, "# nrcpus avail : %u\n", nr);
+ fprintf(fp, "# nrcpus online : %u\n", ph->env.nr_cpus_online);
+ fprintf(fp, "# nrcpus avail : %u\n", ph->env.nr_cpus_avail);
}
-static void print_version(struct perf_header *ph, int fd, FILE *fp)
+static void print_version(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- char *str = do_read_string(fd, ph);
- fprintf(fp, "# perf version : %s\n", str);
- free(str);
+ fprintf(fp, "# perf version : %s\n", ph->env.version);
}
-static void print_cmdline(struct perf_header *ph, int fd, FILE *fp)
+static void print_cmdline(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- ssize_t ret;
+ int nr, i;
char *str;
- u32 nr, i;
- ret = read(fd, &nr, sizeof(nr));
- if (ret != (ssize_t)sizeof(nr))
- return;
-
- if (ph->needs_swap)
- nr = bswap_32(nr);
+ nr = ph->env.nr_cmdline;
+ str = ph->env.cmdline;
fprintf(fp, "# cmdline : ");
for (i = 0; i < nr; i++) {
- str = do_read_string(fd, ph);
fprintf(fp, "%s ", str);
- free(str);
+ str += strlen(str) + 1;
}
fputc('\n', fp);
}
-static void print_cpu_topology(struct perf_header *ph, int fd, FILE *fp)
+static void print_cpu_topology(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- ssize_t ret;
- u32 nr, i;
+ int nr, i;
char *str;
- ret = read(fd, &nr, sizeof(nr));
- if (ret != (ssize_t)sizeof(nr))
- return;
-
- if (ph->needs_swap)
- nr = bswap_32(nr);
+ nr = ph->env.nr_sibling_cores;
+ str = ph->env.sibling_cores;
for (i = 0; i < nr; i++) {
- str = do_read_string(fd, ph);
fprintf(fp, "# sibling cores : %s\n", str);
- free(str);
+ str += strlen(str) + 1;
}
- ret = read(fd, &nr, sizeof(nr));
- if (ret != (ssize_t)sizeof(nr))
- return;
-
- if (ph->needs_swap)
- nr = bswap_32(nr);
+ nr = ph->env.nr_sibling_threads;
+ str = ph->env.sibling_threads;
for (i = 0; i < nr; i++) {
- str = do_read_string(fd, ph);
fprintf(fp, "# sibling threads : %s\n", str);
- free(str);
+ str += strlen(str) + 1;
}
}
-static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
+static void free_event_desc(struct perf_evsel *events)
{
- struct perf_event_attr attr;
- uint64_t id;
+ struct perf_evsel *evsel;
+
+ if (!events)
+ return;
+
+ for (evsel = events; evsel->attr.size; evsel++) {
+ if (evsel->name)
+ free(evsel->name);
+ if (evsel->id)
+ free(evsel->id);
+ }
+
+ free(events);
+}
+
+static struct perf_evsel *
+read_event_desc(struct perf_header *ph, int fd)
+{
+ struct perf_evsel *evsel, *events = NULL;
+ u64 *id;
void *buf = NULL;
- char *str;
u32 nre, sz, nr, i, j;
ssize_t ret;
size_t msz;
@@ -1173,18 +1222,22 @@
if (ph->needs_swap)
sz = bswap_32(sz);
- memset(&attr, 0, sizeof(attr));
-
/* buffer to hold on file attr struct */
buf = malloc(sz);
if (!buf)
goto error;
- msz = sizeof(attr);
+ /* the last event terminates with evsel->attr.size == 0: */
+ events = calloc(nre + 1, sizeof(*events));
+ if (!events)
+ goto error;
+
+ msz = sizeof(evsel->attr);
if (sz < msz)
msz = sz;
- for (i = 0 ; i < nre; i++) {
+ for (i = 0, evsel = events; i < nre; evsel++, i++) {
+ evsel->idx = i;
/*
* must read entire on-file attr struct to
@@ -1197,146 +1250,188 @@
if (ph->needs_swap)
perf_event__attr_swap(buf);
- memcpy(&attr, buf, msz);
+ memcpy(&evsel->attr, buf, msz);
ret = read(fd, &nr, sizeof(nr));
if (ret != (ssize_t)sizeof(nr))
goto error;
- if (ph->needs_swap)
+ if (ph->needs_swap) {
nr = bswap_32(nr);
+ evsel->needs_swap = true;
+ }
- str = do_read_string(fd, ph);
- fprintf(fp, "# event : name = %s, ", str);
- free(str);
+ evsel->name = do_read_string(fd, ph);
+
+ if (!nr)
+ continue;
+
+ id = calloc(nr, sizeof(*id));
+ if (!id)
+ goto error;
+ evsel->ids = nr;
+ evsel->id = id;
+
+ for (j = 0 ; j < nr; j++) {
+ ret = read(fd, id, sizeof(*id));
+ if (ret != (ssize_t)sizeof(*id))
+ goto error;
+ if (ph->needs_swap)
+ *id = bswap_64(*id);
+ id++;
+ }
+ }
+out:
+ if (buf)
+ free(buf);
+ return events;
+error:
+ if (events)
+ free_event_desc(events);
+ events = NULL;
+ goto out;
+}
+
+static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
+{
+ struct perf_evsel *evsel, *events = read_event_desc(ph, fd);
+ u32 j;
+ u64 *id;
+
+ if (!events) {
+ fprintf(fp, "# event desc: not available or unable to read\n");
+ return;
+ }
+
+ for (evsel = events; evsel->attr.size; evsel++) {
+ fprintf(fp, "# event : name = %s, ", evsel->name);
fprintf(fp, "type = %d, config = 0x%"PRIx64
", config1 = 0x%"PRIx64", config2 = 0x%"PRIx64,
- attr.type,
- (u64)attr.config,
- (u64)attr.config1,
- (u64)attr.config2);
+ evsel->attr.type,
+ (u64)evsel->attr.config,
+ (u64)evsel->attr.config1,
+ (u64)evsel->attr.config2);
fprintf(fp, ", excl_usr = %d, excl_kern = %d",
- attr.exclude_user,
- attr.exclude_kernel);
+ evsel->attr.exclude_user,
+ evsel->attr.exclude_kernel);
fprintf(fp, ", excl_host = %d, excl_guest = %d",
- attr.exclude_host,
- attr.exclude_guest);
+ evsel->attr.exclude_host,
+ evsel->attr.exclude_guest);
- fprintf(fp, ", precise_ip = %d", attr.precise_ip);
+ fprintf(fp, ", precise_ip = %d", evsel->attr.precise_ip);
- if (nr)
+ if (evsel->ids) {
fprintf(fp, ", id = {");
-
- for (j = 0 ; j < nr; j++) {
- ret = read(fd, &id, sizeof(id));
- if (ret != (ssize_t)sizeof(id))
- goto error;
-
- if (ph->needs_swap)
- id = bswap_64(id);
-
- if (j)
- fputc(',', fp);
-
- fprintf(fp, " %"PRIu64, id);
- }
- if (nr && j == nr)
+ for (j = 0, id = evsel->id; j < evsel->ids; j++, id++) {
+ if (j)
+ fputc(',', fp);
+ fprintf(fp, " %"PRIu64, *id);
+ }
fprintf(fp, " }");
+ }
+
fputc('\n', fp);
}
- free(buf);
- return;
-error:
- fprintf(fp, "# event desc: not available or unable to read\n");
+
+ free_event_desc(events);
}
-static void print_total_mem(struct perf_header *h __used, int fd, FILE *fp)
+static void print_total_mem(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- uint64_t mem;
- ssize_t ret;
-
- ret = read(fd, &mem, sizeof(mem));
- if (ret != sizeof(mem))
- goto error;
-
- if (h->needs_swap)
- mem = bswap_64(mem);
-
- fprintf(fp, "# total memory : %"PRIu64" kB\n", mem);
- return;
-error:
- fprintf(fp, "# total memory : unknown\n");
+ fprintf(fp, "# total memory : %Lu kB\n", ph->env.total_mem);
}
-static void print_numa_topology(struct perf_header *h __used, int fd, FILE *fp)
+static void print_numa_topology(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
{
- ssize_t ret;
u32 nr, c, i;
- char *str;
+ char *str, *tmp;
uint64_t mem_total, mem_free;
/* nr nodes */
- ret = read(fd, &nr, sizeof(nr));
- if (ret != (ssize_t)sizeof(nr))
- goto error;
-
- if (h->needs_swap)
- nr = bswap_32(nr);
+ nr = ph->env.nr_numa_nodes;
+ str = ph->env.numa_nodes;
for (i = 0; i < nr; i++) {
-
/* node number */
- ret = read(fd, &c, sizeof(c));
- if (ret != (ssize_t)sizeof(c))
+ c = strtoul(str, &tmp, 0);
+ if (*tmp != ':')
goto error;
- if (h->needs_swap)
- c = bswap_32(c);
-
- ret = read(fd, &mem_total, sizeof(u64));
- if (ret != sizeof(u64))
+ str = tmp + 1;
+ mem_total = strtoull(str, &tmp, 0);
+ if (*tmp != ':')
goto error;
- ret = read(fd, &mem_free, sizeof(u64));
- if (ret != sizeof(u64))
+ str = tmp + 1;
+ mem_free = strtoull(str, &tmp, 0);
+ if (*tmp != ':')
goto error;
- if (h->needs_swap) {
- mem_total = bswap_64(mem_total);
- mem_free = bswap_64(mem_free);
- }
-
fprintf(fp, "# node%u meminfo : total = %"PRIu64" kB,"
" free = %"PRIu64" kB\n",
- c,
- mem_total,
- mem_free);
+ c, mem_total, mem_free);
- str = do_read_string(fd, h);
+ str = tmp + 1;
fprintf(fp, "# node%u cpu list : %s\n", c, str);
- free(str);
}
return;
error:
fprintf(fp, "# numa topology : not available\n");
}
-static void print_cpuid(struct perf_header *ph, int fd, FILE *fp)
+static void print_cpuid(struct perf_header *ph, int fd __maybe_unused, FILE *fp)
{
- char *str = do_read_string(fd, ph);
- fprintf(fp, "# cpuid : %s\n", str);
- free(str);
+ fprintf(fp, "# cpuid : %s\n", ph->env.cpuid);
}
-static void print_branch_stack(struct perf_header *ph __used, int fd __used,
- FILE *fp)
+static void print_branch_stack(struct perf_header *ph __maybe_unused,
+ int fd __maybe_unused, FILE *fp)
{
fprintf(fp, "# contains samples with branch stack\n");
}
+static void print_pmu_mappings(struct perf_header *ph, int fd __maybe_unused,
+ FILE *fp)
+{
+ const char *delimiter = "# pmu mappings: ";
+ char *str, *tmp;
+ u32 pmu_num;
+ u32 type;
+
+ pmu_num = ph->env.nr_pmu_mappings;
+ if (!pmu_num) {
+ fprintf(fp, "# pmu mappings: not available\n");
+ return;
+ }
+
+ str = ph->env.pmu_mappings;
+
+ while (pmu_num) {
+ type = strtoul(str, &tmp, 0);
+ if (*tmp != ':')
+ goto error;
+
+ str = tmp + 1;
+ fprintf(fp, "%s%s = %" PRIu32, delimiter, str, type);
+
+ delimiter = ", ";
+ str += strlen(str) + 1;
+ pmu_num--;
+ }
+
+ fprintf(fp, "\n");
+
+ if (!pmu_num)
+ return;
+error:
+ fprintf(fp, "# pmu mappings: unable to read\n");
+}
+
static int __event_process_build_id(struct build_id_event *bev,
char *filename,
struct perf_session *session)
@@ -1398,7 +1493,7 @@
struct perf_session *session = container_of(header, struct perf_session, header);
struct {
struct perf_event_header header;
- u8 build_id[ALIGN(BUILD_ID_SIZE, sizeof(u64))];
+ u8 build_id[PERF_ALIGN(BUILD_ID_SIZE, sizeof(u64))];
char filename[0];
} old_bev;
struct build_id_event bev;
@@ -1487,28 +1582,375 @@
return err;
}
-static int process_tracing_data(struct perf_file_section *section __unused,
- struct perf_header *ph __unused,
- int feat __unused, int fd, void *data)
+static int process_tracing_data(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph __maybe_unused,
+ int fd, void *data)
{
trace_report(fd, data, false);
return 0;
}
static int process_build_id(struct perf_file_section *section,
- struct perf_header *ph,
- int feat __unused, int fd, void *data __used)
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
{
if (perf_header__read_build_ids(ph, fd, section->offset, section->size))
pr_debug("Failed to read buildids, continuing...\n");
return 0;
}
+static int process_hostname(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ ph->env.hostname = do_read_string(fd, ph);
+ return ph->env.hostname ? 0 : -ENOMEM;
+}
+
+static int process_osrelease(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ ph->env.os_release = do_read_string(fd, ph);
+ return ph->env.os_release ? 0 : -ENOMEM;
+}
+
+static int process_version(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ ph->env.version = do_read_string(fd, ph);
+ return ph->env.version ? 0 : -ENOMEM;
+}
+
+static int process_arch(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ ph->env.arch = do_read_string(fd, ph);
+ return ph->env.arch ? 0 : -ENOMEM;
+}
+
+static int process_nrcpus(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ size_t ret;
+ u32 nr;
+
+ ret = read(fd, &nr, sizeof(nr));
+ if (ret != sizeof(nr))
+ return -1;
+
+ if (ph->needs_swap)
+ nr = bswap_32(nr);
+
+ ph->env.nr_cpus_online = nr;
+
+ ret = read(fd, &nr, sizeof(nr));
+ if (ret != sizeof(nr))
+ return -1;
+
+ if (ph->needs_swap)
+ nr = bswap_32(nr);
+
+ ph->env.nr_cpus_avail = nr;
+ return 0;
+}
+
+static int process_cpudesc(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ ph->env.cpu_desc = do_read_string(fd, ph);
+ return ph->env.cpu_desc ? 0 : -ENOMEM;
+}
+
+static int process_cpuid(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ ph->env.cpuid = do_read_string(fd, ph);
+ return ph->env.cpuid ? 0 : -ENOMEM;
+}
+
+static int process_total_mem(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ uint64_t mem;
+ size_t ret;
+
+ ret = read(fd, &mem, sizeof(mem));
+ if (ret != sizeof(mem))
+ return -1;
+
+ if (ph->needs_swap)
+ mem = bswap_64(mem);
+
+ ph->env.total_mem = mem;
+ return 0;
+}
+
+static struct perf_evsel *
+perf_evlist__find_by_index(struct perf_evlist *evlist, int idx)
+{
+ struct perf_evsel *evsel;
+
+ list_for_each_entry(evsel, &evlist->entries, node) {
+ if (evsel->idx == idx)
+ return evsel;
+ }
+
+ return NULL;
+}
+
+static void
+perf_evlist__set_event_name(struct perf_evlist *evlist,
+ struct perf_evsel *event)
+{
+ struct perf_evsel *evsel;
+
+ if (!event->name)
+ return;
+
+ evsel = perf_evlist__find_by_index(evlist, event->idx);
+ if (!evsel)
+ return;
+
+ if (evsel->name)
+ return;
+
+ evsel->name = strdup(event->name);
+}
+
+static int
+process_event_desc(struct perf_file_section *section __maybe_unused,
+ struct perf_header *header, int fd,
+ void *data __maybe_unused)
+{
+ struct perf_session *session;
+ struct perf_evsel *evsel, *events = read_event_desc(header, fd);
+
+ if (!events)
+ return 0;
+
+ session = container_of(header, struct perf_session, header);
+ for (evsel = events; evsel->attr.size; evsel++)
+ perf_evlist__set_event_name(session->evlist, evsel);
+
+ free_event_desc(events);
+
+ return 0;
+}
+
+static int process_cmdline(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ size_t ret;
+ char *str;
+ u32 nr, i;
+ struct strbuf sb;
+
+ ret = read(fd, &nr, sizeof(nr));
+ if (ret != sizeof(nr))
+ return -1;
+
+ if (ph->needs_swap)
+ nr = bswap_32(nr);
+
+ ph->env.nr_cmdline = nr;
+ strbuf_init(&sb, 128);
+
+ for (i = 0; i < nr; i++) {
+ str = do_read_string(fd, ph);
+ if (!str)
+ goto error;
+
+ /* include a NULL character at the end */
+ strbuf_add(&sb, str, strlen(str) + 1);
+ free(str);
+ }
+ ph->env.cmdline = strbuf_detach(&sb, NULL);
+ return 0;
+
+error:
+ strbuf_release(&sb);
+ return -1;
+}
+
+static int process_cpu_topology(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ size_t ret;
+ u32 nr, i;
+ char *str;
+ struct strbuf sb;
+
+ ret = read(fd, &nr, sizeof(nr));
+ if (ret != sizeof(nr))
+ return -1;
+
+ if (ph->needs_swap)
+ nr = bswap_32(nr);
+
+ ph->env.nr_sibling_cores = nr;
+ strbuf_init(&sb, 128);
+
+ for (i = 0; i < nr; i++) {
+ str = do_read_string(fd, ph);
+ if (!str)
+ goto error;
+
+ /* include a NULL character at the end */
+ strbuf_add(&sb, str, strlen(str) + 1);
+ free(str);
+ }
+ ph->env.sibling_cores = strbuf_detach(&sb, NULL);
+
+ ret = read(fd, &nr, sizeof(nr));
+ if (ret != sizeof(nr))
+ return -1;
+
+ if (ph->needs_swap)
+ nr = bswap_32(nr);
+
+ ph->env.nr_sibling_threads = nr;
+
+ for (i = 0; i < nr; i++) {
+ str = do_read_string(fd, ph);
+ if (!str)
+ goto error;
+
+ /* include a NULL character at the end */
+ strbuf_add(&sb, str, strlen(str) + 1);
+ free(str);
+ }
+ ph->env.sibling_threads = strbuf_detach(&sb, NULL);
+ return 0;
+
+error:
+ strbuf_release(&sb);
+ return -1;
+}
+
+static int process_numa_topology(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ size_t ret;
+ u32 nr, node, i;
+ char *str;
+ uint64_t mem_total, mem_free;
+ struct strbuf sb;
+
+ /* nr nodes */
+ ret = read(fd, &nr, sizeof(nr));
+ if (ret != sizeof(nr))
+ goto error;
+
+ if (ph->needs_swap)
+ nr = bswap_32(nr);
+
+ ph->env.nr_numa_nodes = nr;
+ strbuf_init(&sb, 256);
+
+ for (i = 0; i < nr; i++) {
+ /* node number */
+ ret = read(fd, &node, sizeof(node));
+ if (ret != sizeof(node))
+ goto error;
+
+ ret = read(fd, &mem_total, sizeof(u64));
+ if (ret != sizeof(u64))
+ goto error;
+
+ ret = read(fd, &mem_free, sizeof(u64));
+ if (ret != sizeof(u64))
+ goto error;
+
+ if (ph->needs_swap) {
+ node = bswap_32(node);
+ mem_total = bswap_64(mem_total);
+ mem_free = bswap_64(mem_free);
+ }
+
+ strbuf_addf(&sb, "%u:%"PRIu64":%"PRIu64":",
+ node, mem_total, mem_free);
+
+ str = do_read_string(fd, ph);
+ if (!str)
+ goto error;
+
+ /* include a NULL character at the end */
+ strbuf_add(&sb, str, strlen(str) + 1);
+ free(str);
+ }
+ ph->env.numa_nodes = strbuf_detach(&sb, NULL);
+ return 0;
+
+error:
+ strbuf_release(&sb);
+ return -1;
+}
+
+static int process_pmu_mappings(struct perf_file_section *section __maybe_unused,
+ struct perf_header *ph, int fd,
+ void *data __maybe_unused)
+{
+ size_t ret;
+ char *name;
+ u32 pmu_num;
+ u32 type;
+ struct strbuf sb;
+
+ ret = read(fd, &pmu_num, sizeof(pmu_num));
+ if (ret != sizeof(pmu_num))
+ return -1;
+
+ if (ph->needs_swap)
+ pmu_num = bswap_32(pmu_num);
+
+ if (!pmu_num) {
+ pr_debug("pmu mappings not available\n");
+ return 0;
+ }
+
+ ph->env.nr_pmu_mappings = pmu_num;
+ strbuf_init(&sb, 128);
+
+ while (pmu_num) {
+ if (read(fd, &type, sizeof(type)) != sizeof(type))
+ goto error;
+ if (ph->needs_swap)
+ type = bswap_32(type);
+
+ name = do_read_string(fd, ph);
+ if (!name)
+ goto error;
+
+ strbuf_addf(&sb, "%u:%s", type, name);
+ /* include a NULL character at the end */
+ strbuf_add(&sb, "", 1);
+
+ free(name);
+ pmu_num--;
+ }
+ ph->env.pmu_mappings = strbuf_detach(&sb, NULL);
+ return 0;
+
+error:
+ strbuf_release(&sb);
+ return -1;
+}
+
struct feature_ops {
int (*write)(int fd, struct perf_header *h, struct perf_evlist *evlist);
void (*print)(struct perf_header *h, int fd, FILE *fp);
int (*process)(struct perf_file_section *section,
- struct perf_header *h, int feat, int fd, void *data);
+ struct perf_header *h, int fd, void *data);
const char *name;
bool full_only;
};
@@ -1520,7 +1962,7 @@
.process = process_##func }
#define FEAT_OPF(n, func) \
[n] = { .name = #n, .write = write_##func, .print = print_##func, \
- .full_only = true }
+ .process = process_##func, .full_only = true }
/* feature_ops not implemented: */
#define print_tracing_data NULL
@@ -1529,19 +1971,20 @@
static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {
FEAT_OPP(HEADER_TRACING_DATA, tracing_data),
FEAT_OPP(HEADER_BUILD_ID, build_id),
- FEAT_OPA(HEADER_HOSTNAME, hostname),
- FEAT_OPA(HEADER_OSRELEASE, osrelease),
- FEAT_OPA(HEADER_VERSION, version),
- FEAT_OPA(HEADER_ARCH, arch),
- FEAT_OPA(HEADER_NRCPUS, nrcpus),
- FEAT_OPA(HEADER_CPUDESC, cpudesc),
- FEAT_OPA(HEADER_CPUID, cpuid),
- FEAT_OPA(HEADER_TOTAL_MEM, total_mem),
- FEAT_OPA(HEADER_EVENT_DESC, event_desc),
- FEAT_OPA(HEADER_CMDLINE, cmdline),
+ FEAT_OPP(HEADER_HOSTNAME, hostname),
+ FEAT_OPP(HEADER_OSRELEASE, osrelease),
+ FEAT_OPP(HEADER_VERSION, version),
+ FEAT_OPP(HEADER_ARCH, arch),
+ FEAT_OPP(HEADER_NRCPUS, nrcpus),
+ FEAT_OPP(HEADER_CPUDESC, cpudesc),
+ FEAT_OPP(HEADER_CPUID, cpuid),
+ FEAT_OPP(HEADER_TOTAL_MEM, total_mem),
+ FEAT_OPP(HEADER_EVENT_DESC, event_desc),
+ FEAT_OPP(HEADER_CMDLINE, cmdline),
FEAT_OPF(HEADER_CPU_TOPOLOGY, cpu_topology),
FEAT_OPF(HEADER_NUMA_TOPOLOGY, numa_topology),
FEAT_OPA(HEADER_BRANCH_STACK, branch_stack),
+ FEAT_OPP(HEADER_PMU_MAPPINGS, pmu_mappings),
};
struct header_print_data {
@@ -1683,17 +2126,17 @@
struct perf_file_header f_header;
struct perf_file_attr f_attr;
struct perf_header *header = &session->header;
- struct perf_evsel *attr, *pair = NULL;
+ struct perf_evsel *evsel, *pair = NULL;
int err;
lseek(fd, sizeof(f_header), SEEK_SET);
if (session->evlist != evlist)
- pair = list_entry(session->evlist->entries.next, struct perf_evsel, node);
+ pair = perf_evlist__first(session->evlist);
- list_for_each_entry(attr, &evlist->entries, node) {
- attr->id_offset = lseek(fd, 0, SEEK_CUR);
- err = do_write(fd, attr->id, attr->ids * sizeof(u64));
+ list_for_each_entry(evsel, &evlist->entries, node) {
+ evsel->id_offset = lseek(fd, 0, SEEK_CUR);
+ err = do_write(fd, evsel->id, evsel->ids * sizeof(u64));
if (err < 0) {
out_err_write:
pr_debug("failed to write perf header\n");
@@ -1703,19 +2146,19 @@
err = do_write(fd, pair->id, pair->ids * sizeof(u64));
if (err < 0)
goto out_err_write;
- attr->ids += pair->ids;
- pair = list_entry(pair->node.next, struct perf_evsel, node);
+ evsel->ids += pair->ids;
+ pair = perf_evsel__next(pair);
}
}
header->attr_offset = lseek(fd, 0, SEEK_CUR);
- list_for_each_entry(attr, &evlist->entries, node) {
+ list_for_each_entry(evsel, &evlist->entries, node) {
f_attr = (struct perf_file_attr){
- .attr = attr->attr,
+ .attr = evsel->attr,
.ids = {
- .offset = attr->id_offset,
- .size = attr->ids * sizeof(u64),
+ .offset = evsel->id_offset,
+ .size = evsel->ids * sizeof(u64),
}
};
err = do_write(fd, &f_attr, sizeof(f_attr));
@@ -1726,9 +2169,9 @@
}
header->event_offset = lseek(fd, 0, SEEK_CUR);
- header->event_size = event_count * sizeof(struct perf_trace_event_type);
- if (events) {
- err = do_write(fd, events, header->event_size);
+ header->event_size = trace_event_count * sizeof(struct perf_trace_event_type);
+ if (trace_events) {
+ err = do_write(fd, trace_events, header->event_size);
if (err < 0) {
pr_debug("failed to write perf header events\n");
return err;
@@ -1829,6 +2272,8 @@
static const int attr_file_abi_sizes[] = {
[0] = PERF_ATTR_SIZE_VER0,
[1] = PERF_ATTR_SIZE_VER1,
+ [2] = PERF_ATTR_SIZE_VER2,
+ [3] = PERF_ATTR_SIZE_VER3,
0,
};
@@ -2019,7 +2464,7 @@
if (!feat_ops[feat].process)
return 0;
- return feat_ops[feat].process(section, ph, feat, fd, data);
+ return feat_ops[feat].process(section, ph, fd, data);
}
static int perf_file_header__read_pipe(struct perf_pipe_file_header *header,
@@ -2108,32 +2553,39 @@
return ret <= 0 ? -1 : 0;
}
-static int perf_evsel__set_tracepoint_name(struct perf_evsel *evsel,
- struct pevent *pevent)
+static int perf_evsel__prepare_tracepoint_event(struct perf_evsel *evsel,
+ struct pevent *pevent)
{
- struct event_format *event = pevent_find_event(pevent,
- evsel->attr.config);
+ struct event_format *event;
char bf[128];
+ /* already prepared */
+ if (evsel->tp_format)
+ return 0;
+
+ event = pevent_find_event(pevent, evsel->attr.config);
if (event == NULL)
return -1;
- snprintf(bf, sizeof(bf), "%s:%s", event->system, event->name);
- evsel->name = strdup(bf);
- if (event->name == NULL)
- return -1;
+ if (!evsel->name) {
+ snprintf(bf, sizeof(bf), "%s:%s", event->system, event->name);
+ evsel->name = strdup(bf);
+ if (evsel->name == NULL)
+ return -1;
+ }
+ evsel->tp_format = event;
return 0;
}
-static int perf_evlist__set_tracepoint_names(struct perf_evlist *evlist,
- struct pevent *pevent)
+static int perf_evlist__prepare_tracepoint_events(struct perf_evlist *evlist,
+ struct pevent *pevent)
{
struct perf_evsel *pos;
list_for_each_entry(pos, &evlist->entries, node) {
if (pos->attr.type == PERF_TYPE_TRACEPOINT &&
- perf_evsel__set_tracepoint_name(pos, pevent))
+ perf_evsel__prepare_tracepoint_event(pos, pevent))
return -1;
}
@@ -2176,6 +2628,8 @@
if (evsel == NULL)
goto out_delete_evlist;
+
+ evsel->needs_swap = header->needs_swap;
/*
* Do it before so that if perf_evsel__alloc_id fails, this
* entry gets purged too at perf_evlist__delete().
@@ -2207,13 +2661,13 @@
if (f_header.event_types.size) {
lseek(fd, f_header.event_types.offset, SEEK_SET);
- events = malloc(f_header.event_types.size);
- if (events == NULL)
+ trace_events = malloc(f_header.event_types.size);
+ if (trace_events == NULL)
return -ENOMEM;
- if (perf_header__getbuffer64(header, fd, events,
+ if (perf_header__getbuffer64(header, fd, trace_events,
f_header.event_types.size))
goto out_errno;
- event_count = f_header.event_types.size / sizeof(struct perf_trace_event_type);
+ trace_event_count = f_header.event_types.size / sizeof(struct perf_trace_event_type);
}
perf_header__process_sections(header, fd, &session->pevent,
@@ -2221,7 +2675,8 @@
lseek(fd, header->data_offset, SEEK_SET);
- if (perf_evlist__set_tracepoint_names(session->evlist, session->pevent))
+ if (perf_evlist__prepare_tracepoint_events(session->evlist,
+ session->pevent))
goto out_delete_evlist;
header->frozen = 1;
@@ -2236,7 +2691,7 @@
}
int perf_event__synthesize_attr(struct perf_tool *tool,
- struct perf_event_attr *attr, u16 ids, u64 *id,
+ struct perf_event_attr *attr, u32 ids, u64 *id,
perf_event__handler_t process)
{
union perf_event *ev;
@@ -2244,7 +2699,7 @@
int err;
size = sizeof(struct perf_event_attr);
- size = ALIGN(size, sizeof(u64));
+ size = PERF_ALIGN(size, sizeof(u64));
size += sizeof(struct perf_event_header);
size += ids * sizeof(u64);
@@ -2257,9 +2712,12 @@
memcpy(ev->attr.id, id, ids * sizeof(u64));
ev->attr.header.type = PERF_RECORD_HEADER_ATTR;
- ev->attr.header.size = size;
+ ev->attr.header.size = (u16)size;
- err = process(tool, ev, NULL, NULL);
+ if (ev->attr.header.size == size)
+ err = process(tool, ev, NULL, NULL);
+ else
+ err = -E2BIG;
free(ev);
@@ -2270,12 +2728,12 @@
struct perf_session *session,
perf_event__handler_t process)
{
- struct perf_evsel *attr;
+ struct perf_evsel *evsel;
int err = 0;
- list_for_each_entry(attr, &session->evlist->entries, node) {
- err = perf_event__synthesize_attr(tool, &attr->attr, attr->ids,
- attr->id, process);
+ list_for_each_entry(evsel, &session->evlist->entries, node) {
+ err = perf_event__synthesize_attr(tool, &evsel->attr, evsel->ids,
+ evsel->id, process);
if (err) {
pr_debug("failed to create perf header attribute\n");
return err;
@@ -2288,7 +2746,7 @@
int perf_event__process_attr(union perf_event *event,
struct perf_evlist **pevlist)
{
- unsigned int i, ids, n_ids;
+ u32 i, ids, n_ids;
struct perf_evsel *evsel;
struct perf_evlist *evlist = *pevlist;
@@ -2339,7 +2797,7 @@
ev.event_type.header.type = PERF_RECORD_HEADER_EVENT_TYPE;
size = strlen(ev.event_type.event_type.name);
- size = ALIGN(size, sizeof(u64));
+ size = PERF_ALIGN(size, sizeof(u64));
ev.event_type.header.size = sizeof(ev.event_type) -
(sizeof(ev.event_type.event_type.name) - size);
@@ -2355,8 +2813,8 @@
struct perf_trace_event_type *type;
int i, err = 0;
- for (i = 0; i < event_count; i++) {
- type = &events[i];
+ for (i = 0; i < trace_event_count; i++) {
+ type = &trace_events[i];
err = perf_event__synthesize_event_type(tool, type->event_id,
type->name, process,
@@ -2370,7 +2828,7 @@
return err;
}
-int perf_event__process_event_type(struct perf_tool *tool __unused,
+int perf_event__process_event_type(struct perf_tool *tool __maybe_unused,
union perf_event *event)
{
if (perf_header__push_event(event->event_type.event_type.event_id,
@@ -2387,7 +2845,7 @@
union perf_event ev;
struct tracing_data *tdata;
ssize_t size = 0, aligned_size = 0, padding;
- int err __used = 0;
+ int err __maybe_unused = 0;
/*
* We are going to store the size of the data followed
@@ -2408,7 +2866,7 @@
ev.tracing_data.header.type = PERF_RECORD_HEADER_TRACING_DATA;
size = tdata->size;
- aligned_size = ALIGN(size, sizeof(u64));
+ aligned_size = PERF_ALIGN(size, sizeof(u64));
padding = aligned_size - size;
ev.tracing_data.header.size = sizeof(ev.tracing_data);
ev.tracing_data.size = aligned_size;
@@ -2439,7 +2897,7 @@
size_read = trace_report(session->fd, &session->pevent,
session->repipe);
- padding = ALIGN(size_read, sizeof(u64)) - size_read;
+ padding = PERF_ALIGN(size_read, sizeof(u64)) - size_read;
if (read(session->fd, buf, padding) < 0)
die("reading input file");
@@ -2452,6 +2910,9 @@
if (size_read + padding != size)
die("tracing data size mismatch");
+ perf_evlist__prepare_tracepoint_events(session->evlist,
+ session->pevent);
+
return size_read + padding;
}
@@ -2470,7 +2931,7 @@
memset(&ev, 0, sizeof(ev));
len = pos->long_name_len + 1;
- len = ALIGN(len, NAME_ALIGN);
+ len = PERF_ALIGN(len, NAME_ALIGN);
memcpy(&ev.build_id.build_id, pos->build_id, sizeof(pos->build_id));
ev.build_id.header.type = PERF_RECORD_HEADER_BUILD_ID;
ev.build_id.header.misc = misc;
@@ -2483,7 +2944,7 @@
return err;
}
-int perf_event__process_build_id(struct perf_tool *tool __used,
+int perf_event__process_build_id(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_session *session)
{
diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h
index 2d42b3e..99bdd3a 100644
--- a/tools/perf/util/header.h
+++ b/tools/perf/util/header.h
@@ -28,6 +28,7 @@
HEADER_CPU_TOPOLOGY,
HEADER_NUMA_TOPOLOGY,
HEADER_BRANCH_STACK,
+ HEADER_PMU_MAPPINGS,
HEADER_LAST_FEATURE,
HEADER_FEAT_BITS = 256,
};
@@ -57,6 +58,29 @@
int perf_file_header__read(struct perf_file_header *header,
struct perf_header *ph, int fd);
+struct perf_session_env {
+ char *hostname;
+ char *os_release;
+ char *version;
+ char *arch;
+ int nr_cpus_online;
+ int nr_cpus_avail;
+ char *cpu_desc;
+ char *cpuid;
+ unsigned long long total_mem;
+
+ int nr_cmdline;
+ char *cmdline;
+ int nr_sibling_cores;
+ char *sibling_cores;
+ int nr_sibling_threads;
+ char *sibling_threads;
+ int nr_numa_nodes;
+ char *numa_nodes;
+ int nr_pmu_mappings;
+ char *pmu_mappings;
+};
+
struct perf_header {
int frozen;
bool needs_swap;
@@ -66,6 +90,7 @@
u64 event_offset;
u64 event_size;
DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
+ struct perf_session_env env;
};
struct perf_evlist;
@@ -95,11 +120,11 @@
int perf_header__fprintf_info(struct perf_session *s, FILE *fp, bool full);
int build_id_cache__add_s(const char *sbuild_id, const char *debugdir,
- const char *name, bool is_kallsyms);
+ const char *name, bool is_kallsyms, bool is_vdso);
int build_id_cache__remove_s(const char *sbuild_id, const char *debugdir);
int perf_event__synthesize_attr(struct perf_tool *tool,
- struct perf_event_attr *attr, u16 ids, u64 *id,
+ struct perf_event_attr *attr, u32 ids, u64 *id,
perf_event__handler_t process);
int perf_event__synthesize_attrs(struct perf_tool *tool,
struct perf_session *session,
diff --git a/tools/perf/util/help.c b/tools/perf/util/help.c
index 6f2975a..8b1f6e8 100644
--- a/tools/perf/util/help.c
+++ b/tools/perf/util/help.c
@@ -3,6 +3,7 @@
#include "exec_cmd.h"
#include "levenshtein.h"
#include "help.h"
+#include <termios.h>
void add_cmdname(struct cmdnames *cmds, const char *name, size_t len)
{
@@ -331,7 +332,8 @@
exit(1);
}
-int cmd_version(int argc __used, const char **argv __used, const char *prefix __used)
+int cmd_version(int argc __maybe_unused, const char **argv __maybe_unused,
+ const char *prefix __maybe_unused)
{
printf("perf version %s\n", perf_version_string);
return 0;
diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c
index f247ef2..236bc9d 100644
--- a/tools/perf/util/hist.c
+++ b/tools/perf/util/hist.c
@@ -45,7 +45,7 @@
return false;
}
-static void hists__reset_col_len(struct hists *hists)
+void hists__reset_col_len(struct hists *hists)
{
enum hist_column col;
@@ -63,7 +63,7 @@
hists__set_col_len(hists, dso, unresolved_col_width);
}
-static void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
+void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
{
const unsigned int unresolved_col_width = BITS_PER_LONG / 4;
u16 len;
@@ -114,6 +114,22 @@
}
}
+void hists__output_recalc_col_len(struct hists *hists, int max_rows)
+{
+ struct rb_node *next = rb_first(&hists->entries);
+ struct hist_entry *n;
+ int row = 0;
+
+ hists__reset_col_len(hists);
+
+ while (next && row++ < max_rows) {
+ n = rb_entry(next, struct hist_entry, rb_node);
+ if (!n->filtered)
+ hists__calc_col_len(hists, n);
+ next = rb_next(&n->rb_node);
+ }
+}
+
static void hist_entry__add_cpumode_period(struct hist_entry *he,
unsigned int cpumode, u64 period)
{
@@ -378,7 +394,7 @@
* collapse the histogram
*/
-static bool hists__collapse_insert_entry(struct hists *hists __used,
+static bool hists__collapse_insert_entry(struct hists *hists __maybe_unused,
struct rb_root *root,
struct hist_entry *he)
{
@@ -394,8 +410,13 @@
cmp = hist_entry__collapse(iter, he);
if (!cmp) {
- iter->period += he->period;
- iter->nr_events += he->nr_events;
+ iter->period += he->period;
+ iter->period_sys += he->period_sys;
+ iter->period_us += he->period_us;
+ iter->period_guest_sys += he->period_guest_sys;
+ iter->period_guest_us += he->period_guest_us;
+ iter->nr_events += he->nr_events;
+
if (symbol_conf.use_callchain) {
callchain_cursor_reset(&callchain_cursor);
callchain_merge(&callchain_cursor,
@@ -547,674 +568,6 @@
return __hists__output_resort(hists, true);
}
-static size_t callchain__fprintf_left_margin(FILE *fp, int left_margin)
-{
- int i;
- int ret = fprintf(fp, " ");
-
- for (i = 0; i < left_margin; i++)
- ret += fprintf(fp, " ");
-
- return ret;
-}
-
-static size_t ipchain__fprintf_graph_line(FILE *fp, int depth, int depth_mask,
- int left_margin)
-{
- int i;
- size_t ret = callchain__fprintf_left_margin(fp, left_margin);
-
- for (i = 0; i < depth; i++)
- if (depth_mask & (1 << i))
- ret += fprintf(fp, "| ");
- else
- ret += fprintf(fp, " ");
-
- ret += fprintf(fp, "\n");
-
- return ret;
-}
-
-static size_t ipchain__fprintf_graph(FILE *fp, struct callchain_list *chain,
- int depth, int depth_mask, int period,
- u64 total_samples, u64 hits,
- int left_margin)
-{
- int i;
- size_t ret = 0;
-
- ret += callchain__fprintf_left_margin(fp, left_margin);
- for (i = 0; i < depth; i++) {
- if (depth_mask & (1 << i))
- ret += fprintf(fp, "|");
- else
- ret += fprintf(fp, " ");
- if (!period && i == depth - 1) {
- double percent;
-
- percent = hits * 100.0 / total_samples;
- ret += percent_color_fprintf(fp, "--%2.2f%%-- ", percent);
- } else
- ret += fprintf(fp, "%s", " ");
- }
- if (chain->ms.sym)
- ret += fprintf(fp, "%s\n", chain->ms.sym->name);
- else
- ret += fprintf(fp, "0x%0" PRIx64 "\n", chain->ip);
-
- return ret;
-}
-
-static struct symbol *rem_sq_bracket;
-static struct callchain_list rem_hits;
-
-static void init_rem_hits(void)
-{
- rem_sq_bracket = malloc(sizeof(*rem_sq_bracket) + 6);
- if (!rem_sq_bracket) {
- fprintf(stderr, "Not enough memory to display remaining hits\n");
- return;
- }
-
- strcpy(rem_sq_bracket->name, "[...]");
- rem_hits.ms.sym = rem_sq_bracket;
-}
-
-static size_t __callchain__fprintf_graph(FILE *fp, struct rb_root *root,
- u64 total_samples, int depth,
- int depth_mask, int left_margin)
-{
- struct rb_node *node, *next;
- struct callchain_node *child;
- struct callchain_list *chain;
- int new_depth_mask = depth_mask;
- u64 remaining;
- size_t ret = 0;
- int i;
- uint entries_printed = 0;
-
- remaining = total_samples;
-
- node = rb_first(root);
- while (node) {
- u64 new_total;
- u64 cumul;
-
- child = rb_entry(node, struct callchain_node, rb_node);
- cumul = callchain_cumul_hits(child);
- remaining -= cumul;
-
- /*
- * The depth mask manages the output of pipes that show
- * the depth. We don't want to keep the pipes of the current
- * level for the last child of this depth.
- * Except if we have remaining filtered hits. They will
- * supersede the last child
- */
- next = rb_next(node);
- if (!next && (callchain_param.mode != CHAIN_GRAPH_REL || !remaining))
- new_depth_mask &= ~(1 << (depth - 1));
-
- /*
- * But we keep the older depth mask for the line separator
- * to keep the level link until we reach the last child
- */
- ret += ipchain__fprintf_graph_line(fp, depth, depth_mask,
- left_margin);
- i = 0;
- list_for_each_entry(chain, &child->val, list) {
- ret += ipchain__fprintf_graph(fp, chain, depth,
- new_depth_mask, i++,
- total_samples,
- cumul,
- left_margin);
- }
-
- if (callchain_param.mode == CHAIN_GRAPH_REL)
- new_total = child->children_hit;
- else
- new_total = total_samples;
-
- ret += __callchain__fprintf_graph(fp, &child->rb_root, new_total,
- depth + 1,
- new_depth_mask | (1 << depth),
- left_margin);
- node = next;
- if (++entries_printed == callchain_param.print_limit)
- break;
- }
-
- if (callchain_param.mode == CHAIN_GRAPH_REL &&
- remaining && remaining != total_samples) {
-
- if (!rem_sq_bracket)
- return ret;
-
- new_depth_mask &= ~(1 << (depth - 1));
- ret += ipchain__fprintf_graph(fp, &rem_hits, depth,
- new_depth_mask, 0, total_samples,
- remaining, left_margin);
- }
-
- return ret;
-}
-
-static size_t callchain__fprintf_graph(FILE *fp, struct rb_root *root,
- u64 total_samples, int left_margin)
-{
- struct callchain_node *cnode;
- struct callchain_list *chain;
- u32 entries_printed = 0;
- bool printed = false;
- struct rb_node *node;
- int i = 0;
- int ret = 0;
-
- /*
- * If have one single callchain root, don't bother printing
- * its percentage (100 % in fractal mode and the same percentage
- * than the hist in graph mode). This also avoid one level of column.
- */
- node = rb_first(root);
- if (node && !rb_next(node)) {
- cnode = rb_entry(node, struct callchain_node, rb_node);
- list_for_each_entry(chain, &cnode->val, list) {
- /*
- * If we sort by symbol, the first entry is the same than
- * the symbol. No need to print it otherwise it appears as
- * displayed twice.
- */
- if (!i++ && sort__first_dimension == SORT_SYM)
- continue;
- if (!printed) {
- ret += callchain__fprintf_left_margin(fp, left_margin);
- ret += fprintf(fp, "|\n");
- ret += callchain__fprintf_left_margin(fp, left_margin);
- ret += fprintf(fp, "---");
- left_margin += 3;
- printed = true;
- } else
- ret += callchain__fprintf_left_margin(fp, left_margin);
-
- if (chain->ms.sym)
- ret += fprintf(fp, " %s\n", chain->ms.sym->name);
- else
- ret += fprintf(fp, " %p\n", (void *)(long)chain->ip);
-
- if (++entries_printed == callchain_param.print_limit)
- break;
- }
- root = &cnode->rb_root;
- }
-
- ret += __callchain__fprintf_graph(fp, root, total_samples,
- 1, 1, left_margin);
- ret += fprintf(fp, "\n");
-
- return ret;
-}
-
-static size_t __callchain__fprintf_flat(FILE *fp,
- struct callchain_node *self,
- u64 total_samples)
-{
- struct callchain_list *chain;
- size_t ret = 0;
-
- if (!self)
- return 0;
-
- ret += __callchain__fprintf_flat(fp, self->parent, total_samples);
-
-
- list_for_each_entry(chain, &self->val, list) {
- if (chain->ip >= PERF_CONTEXT_MAX)
- continue;
- if (chain->ms.sym)
- ret += fprintf(fp, " %s\n", chain->ms.sym->name);
- else
- ret += fprintf(fp, " %p\n",
- (void *)(long)chain->ip);
- }
-
- return ret;
-}
-
-static size_t callchain__fprintf_flat(FILE *fp, struct rb_root *self,
- u64 total_samples)
-{
- size_t ret = 0;
- u32 entries_printed = 0;
- struct rb_node *rb_node;
- struct callchain_node *chain;
-
- rb_node = rb_first(self);
- while (rb_node) {
- double percent;
-
- chain = rb_entry(rb_node, struct callchain_node, rb_node);
- percent = chain->hit * 100.0 / total_samples;
-
- ret = percent_color_fprintf(fp, " %6.2f%%\n", percent);
- ret += __callchain__fprintf_flat(fp, chain, total_samples);
- ret += fprintf(fp, "\n");
- if (++entries_printed == callchain_param.print_limit)
- break;
-
- rb_node = rb_next(rb_node);
- }
-
- return ret;
-}
-
-static size_t hist_entry_callchain__fprintf(struct hist_entry *he,
- u64 total_samples, int left_margin,
- FILE *fp)
-{
- switch (callchain_param.mode) {
- case CHAIN_GRAPH_REL:
- return callchain__fprintf_graph(fp, &he->sorted_chain, he->period,
- left_margin);
- break;
- case CHAIN_GRAPH_ABS:
- return callchain__fprintf_graph(fp, &he->sorted_chain, total_samples,
- left_margin);
- break;
- case CHAIN_FLAT:
- return callchain__fprintf_flat(fp, &he->sorted_chain, total_samples);
- break;
- case CHAIN_NONE:
- break;
- default:
- pr_err("Bad callchain mode\n");
- }
-
- return 0;
-}
-
-void hists__output_recalc_col_len(struct hists *hists, int max_rows)
-{
- struct rb_node *next = rb_first(&hists->entries);
- struct hist_entry *n;
- int row = 0;
-
- hists__reset_col_len(hists);
-
- while (next && row++ < max_rows) {
- n = rb_entry(next, struct hist_entry, rb_node);
- if (!n->filtered)
- hists__calc_col_len(hists, n);
- next = rb_next(&n->rb_node);
- }
-}
-
-static int hist_entry__pcnt_snprintf(struct hist_entry *he, char *s,
- size_t size, struct hists *pair_hists,
- bool show_displacement, long displacement,
- bool color, u64 total_period)
-{
- u64 period, total, period_sys, period_us, period_guest_sys, period_guest_us;
- u64 nr_events;
- const char *sep = symbol_conf.field_sep;
- int ret;
-
- if (symbol_conf.exclude_other && !he->parent)
- return 0;
-
- if (pair_hists) {
- period = he->pair ? he->pair->period : 0;
- nr_events = he->pair ? he->pair->nr_events : 0;
- total = pair_hists->stats.total_period;
- period_sys = he->pair ? he->pair->period_sys : 0;
- period_us = he->pair ? he->pair->period_us : 0;
- period_guest_sys = he->pair ? he->pair->period_guest_sys : 0;
- period_guest_us = he->pair ? he->pair->period_guest_us : 0;
- } else {
- period = he->period;
- nr_events = he->nr_events;
- total = total_period;
- period_sys = he->period_sys;
- period_us = he->period_us;
- period_guest_sys = he->period_guest_sys;
- period_guest_us = he->period_guest_us;
- }
-
- if (total) {
- if (color)
- ret = percent_color_snprintf(s, size,
- sep ? "%.2f" : " %6.2f%%",
- (period * 100.0) / total);
- else
- ret = scnprintf(s, size, sep ? "%.2f" : " %6.2f%%",
- (period * 100.0) / total);
- if (symbol_conf.show_cpu_utilization) {
- ret += percent_color_snprintf(s + ret, size - ret,
- sep ? "%.2f" : " %6.2f%%",
- (period_sys * 100.0) / total);
- ret += percent_color_snprintf(s + ret, size - ret,
- sep ? "%.2f" : " %6.2f%%",
- (period_us * 100.0) / total);
- if (perf_guest) {
- ret += percent_color_snprintf(s + ret,
- size - ret,
- sep ? "%.2f" : " %6.2f%%",
- (period_guest_sys * 100.0) /
- total);
- ret += percent_color_snprintf(s + ret,
- size - ret,
- sep ? "%.2f" : " %6.2f%%",
- (period_guest_us * 100.0) /
- total);
- }
- }
- } else
- ret = scnprintf(s, size, sep ? "%" PRIu64 : "%12" PRIu64 " ", period);
-
- if (symbol_conf.show_nr_samples) {
- if (sep)
- ret += scnprintf(s + ret, size - ret, "%c%" PRIu64, *sep, nr_events);
- else
- ret += scnprintf(s + ret, size - ret, "%11" PRIu64, nr_events);
- }
-
- if (symbol_conf.show_total_period) {
- if (sep)
- ret += scnprintf(s + ret, size - ret, "%c%" PRIu64, *sep, period);
- else
- ret += scnprintf(s + ret, size - ret, " %12" PRIu64, period);
- }
-
- if (pair_hists) {
- char bf[32];
- double old_percent = 0, new_percent = 0, diff;
-
- if (total > 0)
- old_percent = (period * 100.0) / total;
- if (total_period > 0)
- new_percent = (he->period * 100.0) / total_period;
-
- diff = new_percent - old_percent;
-
- if (fabs(diff) >= 0.01)
- scnprintf(bf, sizeof(bf), "%+4.2F%%", diff);
- else
- scnprintf(bf, sizeof(bf), " ");
-
- if (sep)
- ret += scnprintf(s + ret, size - ret, "%c%s", *sep, bf);
- else
- ret += scnprintf(s + ret, size - ret, "%11.11s", bf);
-
- if (show_displacement) {
- if (displacement)
- scnprintf(bf, sizeof(bf), "%+4ld", displacement);
- else
- scnprintf(bf, sizeof(bf), " ");
-
- if (sep)
- ret += scnprintf(s + ret, size - ret, "%c%s", *sep, bf);
- else
- ret += scnprintf(s + ret, size - ret, "%6.6s", bf);
- }
- }
-
- return ret;
-}
-
-int hist_entry__snprintf(struct hist_entry *he, char *s, size_t size,
- struct hists *hists)
-{
- const char *sep = symbol_conf.field_sep;
- struct sort_entry *se;
- int ret = 0;
-
- list_for_each_entry(se, &hist_entry__sort_list, list) {
- if (se->elide)
- continue;
-
- ret += scnprintf(s + ret, size - ret, "%s", sep ?: " ");
- ret += se->se_snprintf(he, s + ret, size - ret,
- hists__col_len(hists, se->se_width_idx));
- }
-
- return ret;
-}
-
-static int hist_entry__fprintf(struct hist_entry *he, size_t size,
- struct hists *hists, struct hists *pair_hists,
- bool show_displacement, long displacement,
- u64 total_period, FILE *fp)
-{
- char bf[512];
- int ret;
-
- if (size == 0 || size > sizeof(bf))
- size = sizeof(bf);
-
- ret = hist_entry__pcnt_snprintf(he, bf, size, pair_hists,
- show_displacement, displacement,
- true, total_period);
- hist_entry__snprintf(he, bf + ret, size - ret, hists);
- return fprintf(fp, "%s\n", bf);
-}
-
-static size_t hist_entry__fprintf_callchain(struct hist_entry *he,
- struct hists *hists,
- u64 total_period, FILE *fp)
-{
- int left_margin = 0;
-
- if (sort__first_dimension == SORT_COMM) {
- struct sort_entry *se = list_first_entry(&hist_entry__sort_list,
- typeof(*se), list);
- left_margin = hists__col_len(hists, se->se_width_idx);
- left_margin -= thread__comm_len(he->thread);
- }
-
- return hist_entry_callchain__fprintf(he, total_period, left_margin, fp);
-}
-
-size_t hists__fprintf(struct hists *hists, struct hists *pair,
- bool show_displacement, bool show_header, int max_rows,
- int max_cols, FILE *fp)
-{
- struct sort_entry *se;
- struct rb_node *nd;
- size_t ret = 0;
- u64 total_period;
- unsigned long position = 1;
- long displacement = 0;
- unsigned int width;
- const char *sep = symbol_conf.field_sep;
- const char *col_width = symbol_conf.col_width_list_str;
- int nr_rows = 0;
-
- init_rem_hits();
-
- if (!show_header)
- goto print_entries;
-
- fprintf(fp, "# %s", pair ? "Baseline" : "Overhead");
-
- if (symbol_conf.show_cpu_utilization) {
- if (sep) {
- ret += fprintf(fp, "%csys", *sep);
- ret += fprintf(fp, "%cus", *sep);
- if (perf_guest) {
- ret += fprintf(fp, "%cguest sys", *sep);
- ret += fprintf(fp, "%cguest us", *sep);
- }
- } else {
- ret += fprintf(fp, " sys ");
- ret += fprintf(fp, " us ");
- if (perf_guest) {
- ret += fprintf(fp, " guest sys ");
- ret += fprintf(fp, " guest us ");
- }
- }
- }
-
- if (symbol_conf.show_nr_samples) {
- if (sep)
- fprintf(fp, "%cSamples", *sep);
- else
- fputs(" Samples ", fp);
- }
-
- if (symbol_conf.show_total_period) {
- if (sep)
- ret += fprintf(fp, "%cPeriod", *sep);
- else
- ret += fprintf(fp, " Period ");
- }
-
- if (pair) {
- if (sep)
- ret += fprintf(fp, "%cDelta", *sep);
- else
- ret += fprintf(fp, " Delta ");
-
- if (show_displacement) {
- if (sep)
- ret += fprintf(fp, "%cDisplacement", *sep);
- else
- ret += fprintf(fp, " Displ");
- }
- }
-
- list_for_each_entry(se, &hist_entry__sort_list, list) {
- if (se->elide)
- continue;
- if (sep) {
- fprintf(fp, "%c%s", *sep, se->se_header);
- continue;
- }
- width = strlen(se->se_header);
- if (symbol_conf.col_width_list_str) {
- if (col_width) {
- hists__set_col_len(hists, se->se_width_idx,
- atoi(col_width));
- col_width = strchr(col_width, ',');
- if (col_width)
- ++col_width;
- }
- }
- if (!hists__new_col_len(hists, se->se_width_idx, width))
- width = hists__col_len(hists, se->se_width_idx);
- fprintf(fp, " %*s", width, se->se_header);
- }
-
- fprintf(fp, "\n");
- if (max_rows && ++nr_rows >= max_rows)
- goto out;
-
- if (sep)
- goto print_entries;
-
- fprintf(fp, "# ........");
- if (symbol_conf.show_cpu_utilization)
- fprintf(fp, " ....... .......");
- if (symbol_conf.show_nr_samples)
- fprintf(fp, " ..........");
- if (symbol_conf.show_total_period)
- fprintf(fp, " ............");
- if (pair) {
- fprintf(fp, " ..........");
- if (show_displacement)
- fprintf(fp, " .....");
- }
- list_for_each_entry(se, &hist_entry__sort_list, list) {
- unsigned int i;
-
- if (se->elide)
- continue;
-
- fprintf(fp, " ");
- width = hists__col_len(hists, se->se_width_idx);
- if (width == 0)
- width = strlen(se->se_header);
- for (i = 0; i < width; i++)
- fprintf(fp, ".");
- }
-
- fprintf(fp, "\n");
- if (max_rows && ++nr_rows >= max_rows)
- goto out;
-
- fprintf(fp, "#\n");
- if (max_rows && ++nr_rows >= max_rows)
- goto out;
-
-print_entries:
- total_period = hists->stats.total_period;
-
- for (nd = rb_first(&hists->entries); nd; nd = rb_next(nd)) {
- struct hist_entry *h = rb_entry(nd, struct hist_entry, rb_node);
-
- if (h->filtered)
- continue;
-
- if (show_displacement) {
- if (h->pair != NULL)
- displacement = ((long)h->pair->position -
- (long)position);
- else
- displacement = 0;
- ++position;
- }
- ret += hist_entry__fprintf(h, max_cols, hists, pair, show_displacement,
- displacement, total_period, fp);
-
- if (symbol_conf.use_callchain)
- ret += hist_entry__fprintf_callchain(h, hists, total_period, fp);
- if (max_rows && ++nr_rows >= max_rows)
- goto out;
-
- if (h->ms.map == NULL && verbose > 1) {
- __map_groups__fprintf_maps(&h->thread->mg,
- MAP__FUNCTION, verbose, fp);
- fprintf(fp, "%.10s end\n", graph_dotted_line);
- }
- }
-out:
- free(rem_sq_bracket);
-
- return ret;
-}
-
-/*
- * See hists__fprintf to match the column widths
- */
-unsigned int hists__sort_list_width(struct hists *hists)
-{
- struct sort_entry *se;
- int ret = 9; /* total % */
-
- if (symbol_conf.show_cpu_utilization) {
- ret += 7; /* count_sys % */
- ret += 6; /* count_us % */
- if (perf_guest) {
- ret += 13; /* count_guest_sys % */
- ret += 12; /* count_guest_us % */
- }
- }
-
- if (symbol_conf.show_nr_samples)
- ret += 11;
-
- if (symbol_conf.show_total_period)
- ret += 13;
-
- list_for_each_entry(se, &hist_entry__sort_list, list)
- if (!se->elide)
- ret += 2 + hists__col_len(hists, se->se_width_idx);
-
- if (verbose) /* Addr + origin */
- ret += 3 + BITS_PER_LONG / 4;
-
- return ret;
-}
-
static void hists__remove_entry_filter(struct hists *hists, struct hist_entry *h,
enum hist_filter filter)
{
@@ -1342,25 +695,3 @@
++hists->stats.nr_events[0];
++hists->stats.nr_events[type];
}
-
-size_t hists__fprintf_nr_events(struct hists *hists, FILE *fp)
-{
- int i;
- size_t ret = 0;
-
- for (i = 0; i < PERF_RECORD_HEADER_MAX; ++i) {
- const char *name;
-
- if (hists->stats.nr_events[i] == 0)
- continue;
-
- name = perf_event__name(i);
- if (!strcmp(name, "UNKNOWN"))
- continue;
-
- ret += fprintf(fp, "%16s events: %10d\n", name,
- hists->stats.nr_events[i]);
- }
-
- return ret;
-}
diff --git a/tools/perf/util/hist.h b/tools/perf/util/hist.h
index 0b096c2..f011ad4 100644
--- a/tools/perf/util/hist.h
+++ b/tools/perf/util/hist.h
@@ -75,8 +75,8 @@
struct symbol *parent, u64 period);
int64_t hist_entry__cmp(struct hist_entry *left, struct hist_entry *right);
int64_t hist_entry__collapse(struct hist_entry *left, struct hist_entry *right);
-int hist_entry__snprintf(struct hist_entry *self, char *bf, size_t size,
- struct hists *hists);
+int hist_entry__sort_snprintf(struct hist_entry *self, char *bf, size_t size,
+ struct hists *hists);
void hist_entry__free(struct hist_entry *);
struct hist_entry *__hists__add_branch_entry(struct hists *self,
@@ -112,25 +112,66 @@
u16 hists__col_len(struct hists *self, enum hist_column col);
void hists__set_col_len(struct hists *self, enum hist_column col, u16 len);
bool hists__new_col_len(struct hists *self, enum hist_column col, u16 len);
+void hists__reset_col_len(struct hists *hists);
+void hists__calc_col_len(struct hists *hists, struct hist_entry *he);
+
+struct perf_hpp {
+ char *buf;
+ size_t size;
+ u64 total_period;
+ const char *sep;
+ long displacement;
+ void *ptr;
+};
+
+struct perf_hpp_fmt {
+ bool cond;
+ int (*header)(struct perf_hpp *hpp);
+ int (*width)(struct perf_hpp *hpp);
+ int (*color)(struct perf_hpp *hpp, struct hist_entry *he);
+ int (*entry)(struct perf_hpp *hpp, struct hist_entry *he);
+};
+
+extern struct perf_hpp_fmt perf_hpp__format[];
+
+enum {
+ PERF_HPP__OVERHEAD,
+ PERF_HPP__OVERHEAD_SYS,
+ PERF_HPP__OVERHEAD_US,
+ PERF_HPP__OVERHEAD_GUEST_SYS,
+ PERF_HPP__OVERHEAD_GUEST_US,
+ PERF_HPP__SAMPLES,
+ PERF_HPP__PERIOD,
+ PERF_HPP__DELTA,
+ PERF_HPP__DISPL,
+
+ PERF_HPP__MAX_INDEX
+};
+
+void perf_hpp__init(bool need_pair, bool show_displacement);
+int hist_entry__period_snprintf(struct perf_hpp *hpp, struct hist_entry *he,
+ bool color);
struct perf_evlist;
#ifdef NO_NEWT_SUPPORT
static inline
-int perf_evlist__tui_browse_hists(struct perf_evlist *evlist __used,
- const char *help __used,
- void(*timer)(void *arg) __used,
- void *arg __used,
- int refresh __used)
+int perf_evlist__tui_browse_hists(struct perf_evlist *evlist __maybe_unused,
+ const char *help __maybe_unused,
+ void(*timer)(void *arg) __maybe_unused,
+ void *arg __maybe_unused,
+ int refresh __maybe_unused)
{
return 0;
}
-static inline int hist_entry__tui_annotate(struct hist_entry *self __used,
- int evidx __used,
- void(*timer)(void *arg) __used,
- void *arg __used,
- int delay_secs __used)
+static inline int hist_entry__tui_annotate(struct hist_entry *self
+ __maybe_unused,
+ int evidx __maybe_unused,
+ void(*timer)(void *arg)
+ __maybe_unused,
+ void *arg __maybe_unused,
+ int delay_secs __maybe_unused)
{
return 0;
}
@@ -148,11 +189,11 @@
#ifdef NO_GTK2_SUPPORT
static inline
-int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist __used,
- const char *help __used,
- void(*timer)(void *arg) __used,
- void *arg __used,
- int refresh __used)
+int perf_evlist__gtk_browse_hists(struct perf_evlist *evlist __maybe_unused,
+ const char *help __maybe_unused,
+ void(*timer)(void *arg) __maybe_unused,
+ void *arg __maybe_unused,
+ int refresh __maybe_unused)
{
return 0;
}
diff --git a/tools/perf/util/include/linux/bitops.h b/tools/perf/util/include/linux/bitops.h
index 587a230..a55d8cf0 100644
--- a/tools/perf/util/include/linux/bitops.h
+++ b/tools/perf/util/include/linux/bitops.h
@@ -5,6 +5,10 @@
#include <linux/compiler.h>
#include <asm/hweight.h>
+#ifndef __WORDSIZE
+#define __WORDSIZE (__SIZEOF_LONG__ * 8)
+#endif
+
#define BITS_PER_LONG __WORDSIZE
#define BITS_PER_BYTE 8
#define BITS_TO_LONGS(nr) DIV_ROUND_UP(nr, BITS_PER_BYTE * sizeof(long))
diff --git a/tools/perf/util/include/linux/compiler.h b/tools/perf/util/include/linux/compiler.h
index 547628e..96b919d 100644
--- a/tools/perf/util/include/linux/compiler.h
+++ b/tools/perf/util/include/linux/compiler.h
@@ -9,6 +9,13 @@
#define __attribute_const__
#endif
-#define __used __attribute__((__unused__))
+#ifndef __maybe_unused
+#define __maybe_unused __attribute__((unused))
+#endif
+#define __packed __attribute__((__packed__))
+
+#ifndef __force
+#define __force
+#endif
#endif
diff --git a/tools/perf/util/include/linux/kernel.h b/tools/perf/util/include/linux/kernel.h
index b6842c1..d8c927c 100644
--- a/tools/perf/util/include/linux/kernel.h
+++ b/tools/perf/util/include/linux/kernel.h
@@ -8,8 +8,8 @@
#define DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d))
-#define ALIGN(x,a) __ALIGN_MASK(x,(typeof(x))(a)-1)
-#define __ALIGN_MASK(x,mask) (((x)+(mask))&~(mask))
+#define PERF_ALIGN(x, a) __PERF_ALIGN_MASK(x, (typeof(x))(a)-1)
+#define __PERF_ALIGN_MASK(x, mask) (((x)+(mask))&~(mask))
#ifndef offsetof
#define offsetof(TYPE, MEMBER) ((size_t) &((TYPE *)0)->MEMBER)
@@ -46,9 +46,22 @@
_min1 < _min2 ? _min1 : _min2; })
#endif
+#ifndef roundup
+#define roundup(x, y) ( \
+{ \
+ const typeof(y) __y = y; \
+ (((x) + (__y - 1)) / __y) * __y; \
+} \
+)
+#endif
+
#ifndef BUG_ON
+#ifdef NDEBUG
+#define BUG_ON(cond) do { if (cond) {} } while (0)
+#else
#define BUG_ON(cond) assert(!(cond))
#endif
+#endif
/*
* Both need more care to handle endianness
diff --git a/tools/perf/util/include/linux/magic.h b/tools/perf/util/include/linux/magic.h
new file mode 100644
index 0000000..58b64ed
--- /dev/null
+++ b/tools/perf/util/include/linux/magic.h
@@ -0,0 +1,12 @@
+#ifndef _PERF_LINUX_MAGIC_H_
+#define _PERF_LINUX_MAGIC_H_
+
+#ifndef DEBUGFS_MAGIC
+#define DEBUGFS_MAGIC 0x64626720
+#endif
+
+#ifndef SYSFS_MAGIC
+#define SYSFS_MAGIC 0x62656572
+#endif
+
+#endif
diff --git a/tools/perf/util/include/linux/rbtree.h b/tools/perf/util/include/linux/rbtree.h
index 7a243a1..2a030c5 100644
--- a/tools/perf/util/include/linux/rbtree.h
+++ b/tools/perf/util/include/linux/rbtree.h
@@ -1 +1,2 @@
+#include <stdbool.h>
#include "../../../../include/linux/rbtree.h"
diff --git a/tools/perf/util/include/linux/string.h b/tools/perf/util/include/linux/string.h
index 3b2f590..6f19c54 100644
--- a/tools/perf/util/include/linux/string.h
+++ b/tools/perf/util/include/linux/string.h
@@ -1 +1,3 @@
#include <string.h>
+
+void *memdup(const void *src, size_t len);
diff --git a/tools/perf/util/include/linux/types.h b/tools/perf/util/include/linux/types.h
index 12de3b81..eb46478 100644
--- a/tools/perf/util/include/linux/types.h
+++ b/tools/perf/util/include/linux/types.h
@@ -3,6 +3,14 @@
#include <asm/types.h>
+#ifndef __bitwise
+#define __bitwise
+#endif
+
+#ifndef __le32
+typedef __u32 __bitwise __le32;
+#endif
+
#define DECLARE_BITMAP(name,bits) \
unsigned long name[BITS_TO_LONGS(bits)]
diff --git a/tools/perf/util/intlist.c b/tools/perf/util/intlist.c
index fd530dc..9d07400 100644
--- a/tools/perf/util/intlist.c
+++ b/tools/perf/util/intlist.c
@@ -11,7 +11,7 @@
#include "intlist.h"
-static struct rb_node *intlist__node_new(struct rblist *rblist __used,
+static struct rb_node *intlist__node_new(struct rblist *rblist __maybe_unused,
const void *entry)
{
int i = (int)((long)entry);
@@ -31,7 +31,7 @@
free(ilist);
}
-static void intlist__node_delete(struct rblist *rblist __used,
+static void intlist__node_delete(struct rblist *rblist __maybe_unused,
struct rb_node *rb_node)
{
struct int_node *node = container_of(rb_node, struct int_node, rb_node);
@@ -52,9 +52,9 @@
return rblist__add_node(&ilist->rblist, (void *)((long)i));
}
-void intlist__remove(struct intlist *ilist __used, struct int_node *node)
+void intlist__remove(struct intlist *ilist, struct int_node *node)
{
- int_node__delete(node);
+ rblist__remove_node(&ilist->rblist, &node->rb_node);
}
struct int_node *intlist__find(struct intlist *ilist, int i)
diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index cc33486..ead5316 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -9,6 +9,7 @@
#include "map.h"
#include "thread.h"
#include "strlist.h"
+#include "vdso.h"
const char *map_type__name[MAP__NR_TYPES] = {
[MAP__FUNCTION] = "Functions",
@@ -23,7 +24,6 @@
static inline int is_no_dso_memory(const char *filename)
{
return !strcmp(filename, "[stack]") ||
- !strcmp(filename, "[vdso]") ||
!strcmp(filename, "[heap]");
}
@@ -52,9 +52,10 @@
if (self != NULL) {
char newfilename[PATH_MAX];
struct dso *dso;
- int anon, no_dso;
+ int anon, no_dso, vdso;
anon = is_anon_memory(filename);
+ vdso = is_vdso_map(filename);
no_dso = is_no_dso_memory(filename);
if (anon) {
@@ -62,7 +63,12 @@
filename = newfilename;
}
- dso = __dsos__findnew(dsos__list, filename);
+ if (vdso) {
+ pgoff = 0;
+ dso = vdso__dso_findnew(dsos__list);
+ } else
+ dso = __dsos__findnew(dsos__list, filename);
+
if (dso == NULL)
goto out_delete;
@@ -86,6 +92,25 @@
return NULL;
}
+/*
+ * Constructor variant for modules (where we know from /proc/modules where
+ * they are loaded) and for vmlinux, where only after we load all the
+ * symbols we'll know where it starts and ends.
+ */
+struct map *map__new2(u64 start, struct dso *dso, enum map_type type)
+{
+ struct map *map = calloc(1, (sizeof(*map) +
+ (dso->kernel ? sizeof(struct kmap) : 0)));
+ if (map != NULL) {
+ /*
+ * ->end will be filled after we load all the symbols
+ */
+ map__init(map, type, start, 0, 0, dso);
+ }
+
+ return map;
+}
+
void map__delete(struct map *self)
{
free(self);
@@ -137,6 +162,7 @@
pr_warning(", continuing without symbols\n");
return -1;
} else if (nr == 0) {
+#ifndef NO_LIBELF_SUPPORT
const size_t len = strlen(name);
const size_t real_len = len - sizeof(DSO__DELETED);
@@ -149,7 +175,7 @@
pr_warning("no symbols found in %s, maybe install "
"a debug package?\n", name);
}
-
+#endif
return -1;
}
/*
@@ -217,15 +243,14 @@
size_t map__fprintf_dsoname(struct map *map, FILE *fp)
{
- const char *dsoname;
+ const char *dsoname = "[unknown]";
if (map && map->dso && (map->dso->name || map->dso->long_name)) {
if (symbol_conf.show_kernel_path && map->dso->long_name)
dsoname = map->dso->long_name;
else if (map->dso->name)
dsoname = map->dso->name;
- } else
- dsoname = "[unknown]";
+ }
return fprintf(fp, "%s", dsoname);
}
@@ -242,14 +267,6 @@
return addr;
}
-u64 map__objdump_2ip(struct map *map, u64 addr)
-{
- u64 ip = map->dso->adjust_symbols ?
- addr :
- map->unmap_ip(map, addr); /* RIP -> IP */
- return ip;
-}
-
void map_groups__init(struct map_groups *mg)
{
int i;
diff --git a/tools/perf/util/map.h b/tools/perf/util/map.h
index 03a1e9b..d2250fc 100644
--- a/tools/perf/util/map.h
+++ b/tools/perf/util/map.h
@@ -96,7 +96,7 @@
return ip + map->start - map->pgoff;
}
-static inline u64 identity__map_ip(struct map *map __used, u64 ip)
+static inline u64 identity__map_ip(struct map *map __maybe_unused, u64 ip)
{
return ip;
}
@@ -104,7 +104,6 @@
/* rip/ip <-> addr suitable for passing to `objdump --start-address=` */
u64 map__rip_2objdump(struct map *map, u64 rip);
-u64 map__objdump_2ip(struct map *map, u64 addr);
struct symbol;
@@ -115,6 +114,7 @@
struct map *map__new(struct list_head *dsos__list, u64 start, u64 len,
u64 pgoff, u32 pid, char *filename,
enum map_type type);
+struct map *map__new2(u64 start, struct dso *dso, enum map_type type);
void map__delete(struct map *self);
struct map *map__clone(struct map *self);
int map__overlap(struct map *l, struct map *r);
@@ -157,9 +157,12 @@
void machine__exit(struct machine *self);
void machine__delete(struct machine *self);
+struct perf_evsel;
+struct perf_sample;
int machine__resolve_callchain(struct machine *machine,
+ struct perf_evsel *evsel,
struct thread *thread,
- struct ip_callchain *chain,
+ struct perf_sample *sample,
struct symbol **parent);
int maps__set_kallsyms_ref_reloc_sym(struct map **maps, const char *symbol_name,
u64 addr);
diff --git a/tools/perf/util/parse-events-test.c b/tools/perf/util/parse-events-test.c
index 127d648..28c18d1 100644
--- a/tools/perf/util/parse-events-test.c
+++ b/tools/perf/util/parse-events-test.c
@@ -18,8 +18,7 @@
static int test__checkevent_tracepoint(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_TRACEPOINT == evsel->attr.type);
@@ -48,8 +47,7 @@
static int test__checkevent_raw(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_RAW == evsel->attr.type);
@@ -59,8 +57,7 @@
static int test__checkevent_numeric(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", 1 == evsel->attr.type);
@@ -70,8 +67,7 @@
static int test__checkevent_symbolic_name(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
@@ -82,8 +78,7 @@
static int test__checkevent_symbolic_name_config(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
@@ -100,8 +95,7 @@
static int test__checkevent_symbolic_alias(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_SOFTWARE == evsel->attr.type);
@@ -112,8 +106,7 @@
static int test__checkevent_genhw(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_HW_CACHE == evsel->attr.type);
@@ -123,8 +116,7 @@
static int test__checkevent_breakpoint(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_BREAKPOINT == evsel->attr.type);
@@ -138,8 +130,7 @@
static int test__checkevent_breakpoint_x(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_BREAKPOINT == evsel->attr.type);
@@ -152,8 +143,7 @@
static int test__checkevent_breakpoint_r(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type",
@@ -168,8 +158,7 @@
static int test__checkevent_breakpoint_w(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type",
@@ -184,8 +173,7 @@
static int test__checkevent_breakpoint_rw(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type",
@@ -200,8 +188,7 @@
static int test__checkevent_tracepoint_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
@@ -232,8 +219,7 @@
static int test__checkevent_raw_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
@@ -245,8 +231,7 @@
static int test__checkevent_numeric_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
@@ -258,8 +243,7 @@
static int test__checkevent_symbolic_name_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
@@ -271,8 +255,7 @@
static int test__checkevent_exclude_host_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
@@ -282,8 +265,7 @@
static int test__checkevent_exclude_guest_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude guest", evsel->attr.exclude_guest);
TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
@@ -293,8 +275,7 @@
static int test__checkevent_symbolic_alias_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
@@ -306,8 +287,7 @@
static int test__checkevent_genhw_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
@@ -319,75 +299,71 @@
static int test__checkevent_breakpoint_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
+
TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
TEST_ASSERT_VAL("wrong name",
- !strcmp(perf_evsel__name(evsel), "mem:0x0:rw:u"));
+ !strcmp(perf_evsel__name(evsel), "mem:0:u"));
return test__checkevent_breakpoint(evlist);
}
static int test__checkevent_breakpoint_x_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
TEST_ASSERT_VAL("wrong name",
- !strcmp(perf_evsel__name(evsel), "mem:0x0:x:k"));
+ !strcmp(perf_evsel__name(evsel), "mem:0:x:k"));
return test__checkevent_breakpoint_x(evlist);
}
static int test__checkevent_breakpoint_r_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip);
TEST_ASSERT_VAL("wrong name",
- !strcmp(perf_evsel__name(evsel), "mem:0x0:r:hp"));
+ !strcmp(perf_evsel__name(evsel), "mem:0:r:hp"));
return test__checkevent_breakpoint_r(evlist);
}
static int test__checkevent_breakpoint_w_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip);
TEST_ASSERT_VAL("wrong name",
- !strcmp(perf_evsel__name(evsel), "mem:0x0:w:up"));
+ !strcmp(perf_evsel__name(evsel), "mem:0:w:up"));
return test__checkevent_breakpoint_w(evlist);
}
static int test__checkevent_breakpoint_rw_modifier(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip);
TEST_ASSERT_VAL("wrong name",
- !strcmp(perf_evsel__name(evsel), "mem:0x0:rw:kp"));
+ !strcmp(perf_evsel__name(evsel), "mem:0:rw:kp"));
return test__checkevent_breakpoint_rw(evlist);
}
@@ -395,8 +371,7 @@
static int test__checkevent_pmu(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel = list_entry(evlist->entries.next,
- struct perf_evsel, node);
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_RAW == evsel->attr.type);
@@ -410,12 +385,11 @@
static int test__checkevent_list(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel;
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
TEST_ASSERT_VAL("wrong number of entries", 3 == evlist->nr_entries);
/* r1 */
- evsel = list_entry(evlist->entries.next, struct perf_evsel, node);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_RAW == evsel->attr.type);
TEST_ASSERT_VAL("wrong config", 1 == evsel->attr.config);
TEST_ASSERT_VAL("wrong config1", 0 == evsel->attr.config1);
@@ -426,7 +400,7 @@
TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
/* syscalls:sys_enter_open:k */
- evsel = list_entry(evsel->node.next, struct perf_evsel, node);
+ evsel = perf_evsel__next(evsel);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_TRACEPOINT == evsel->attr.type);
TEST_ASSERT_VAL("wrong sample_type",
PERF_TP_SAMPLE_TYPE == evsel->attr.sample_type);
@@ -437,7 +411,7 @@
TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
/* 1:1:hp */
- evsel = list_entry(evsel->node.next, struct perf_evsel, node);
+ evsel = perf_evsel__next(evsel);
TEST_ASSERT_VAL("wrong type", 1 == evsel->attr.type);
TEST_ASSERT_VAL("wrong config", 1 == evsel->attr.config);
TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
@@ -450,22 +424,21 @@
static int test__checkevent_pmu_name(struct perf_evlist *evlist)
{
- struct perf_evsel *evsel;
+ struct perf_evsel *evsel = perf_evlist__first(evlist);
/* cpu/config=1,name=krava/u */
- evsel = list_entry(evlist->entries.next, struct perf_evsel, node);
TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_RAW == evsel->attr.type);
TEST_ASSERT_VAL("wrong config", 1 == evsel->attr.config);
TEST_ASSERT_VAL("wrong name", !strcmp(perf_evsel__name(evsel), "krava"));
/* cpu/config=2/u" */
- evsel = list_entry(evsel->node.next, struct perf_evsel, node);
+ evsel = perf_evsel__next(evsel);
TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->nr_entries);
TEST_ASSERT_VAL("wrong type", PERF_TYPE_RAW == evsel->attr.type);
TEST_ASSERT_VAL("wrong config", 2 == evsel->attr.config);
TEST_ASSERT_VAL("wrong name",
- !strcmp(perf_evsel__name(evsel), "raw 0x2:u"));
+ !strcmp(perf_evsel__name(evsel), "cpu/config=2/u"));
return 0;
}
@@ -513,6 +486,280 @@
return 0;
}
+static int test__group1(struct perf_evlist *evlist)
+{
+ struct perf_evsel *evsel, *leader;
+
+ TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->nr_entries);
+
+ /* instructions:k */
+ evsel = leader = perf_evlist__first(evlist);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_INSTRUCTIONS == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ /* cycles:upp */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip == 2);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+
+ return 0;
+}
+
+static int test__group2(struct perf_evlist *evlist)
+{
+ struct perf_evsel *evsel, *leader;
+
+ TEST_ASSERT_VAL("wrong number of entries", 3 == evlist->nr_entries);
+
+ /* faults + :ku modifier */
+ evsel = leader = perf_evlist__first(evlist);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_SOFTWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_SW_PAGE_FAULTS == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ /* cache-references + :u modifier */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CACHE_REFERENCES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+
+ /* cycles:k */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ return 0;
+}
+
+static int test__group3(struct perf_evlist *evlist __maybe_unused)
+{
+ struct perf_evsel *evsel, *leader;
+
+ TEST_ASSERT_VAL("wrong number of entries", 5 == evlist->nr_entries);
+
+ /* group1 syscalls:sys_enter_open:H */
+ evsel = leader = perf_evlist__first(evlist);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_TRACEPOINT == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong sample_type",
+ PERF_TP_SAMPLE_TYPE == evsel->attr.sample_type);
+ TEST_ASSERT_VAL("wrong sample_period", 1 == evsel->attr.sample_period);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+ TEST_ASSERT_VAL("wrong group name",
+ !strcmp(leader->group_name, "group1"));
+
+ /* group1 cycles:kppp */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip == 3);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+ TEST_ASSERT_VAL("wrong group name", !evsel->group_name);
+
+ /* group2 cycles + G modifier */
+ evsel = leader = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+ TEST_ASSERT_VAL("wrong group name",
+ !strcmp(leader->group_name, "group2"));
+
+ /* group2 1:3 + G modifier */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", 1 == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config", 3 == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+
+ /* instructions:u */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_INSTRUCTIONS == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ return 0;
+}
+
+static int test__group4(struct perf_evlist *evlist __maybe_unused)
+{
+ struct perf_evsel *evsel, *leader;
+
+ TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->nr_entries);
+
+ /* cycles:u + p */
+ evsel = leader = perf_evlist__first(evlist);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip == 1);
+ TEST_ASSERT_VAL("wrong group name", !evsel->group_name);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ /* instructions:kp + p */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_INSTRUCTIONS == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", evsel->attr.precise_ip == 2);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+
+ return 0;
+}
+
+static int test__group5(struct perf_evlist *evlist __maybe_unused)
+{
+ struct perf_evsel *evsel, *leader;
+
+ TEST_ASSERT_VAL("wrong number of entries", 5 == evlist->nr_entries);
+
+ /* cycles + G */
+ evsel = leader = perf_evlist__first(evlist);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong group name", !evsel->group_name);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ /* instructions + G */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_INSTRUCTIONS == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+
+ /* cycles:G */
+ evsel = leader = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong group name", !evsel->group_name);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ /* instructions:G */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_INSTRUCTIONS == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", !evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == leader);
+
+ /* cycles */
+ evsel = perf_evsel__next(evsel);
+ TEST_ASSERT_VAL("wrong type", PERF_TYPE_HARDWARE == evsel->attr.type);
+ TEST_ASSERT_VAL("wrong config",
+ PERF_COUNT_HW_CPU_CYCLES == evsel->attr.config);
+ TEST_ASSERT_VAL("wrong exclude_user", !evsel->attr.exclude_user);
+ TEST_ASSERT_VAL("wrong exclude_kernel", !evsel->attr.exclude_kernel);
+ TEST_ASSERT_VAL("wrong exclude_hv", !evsel->attr.exclude_hv);
+ TEST_ASSERT_VAL("wrong exclude guest", evsel->attr.exclude_guest);
+ TEST_ASSERT_VAL("wrong exclude host", !evsel->attr.exclude_host);
+ TEST_ASSERT_VAL("wrong precise_ip", !evsel->attr.precise_ip);
+ TEST_ASSERT_VAL("wrong leader", evsel->leader == NULL);
+
+ return 0;
+}
+
struct test__event_st {
const char *name;
__u32 type;
@@ -632,6 +879,26 @@
.name = "mem:0:rw:kp",
.check = test__checkevent_breakpoint_rw_modifier,
},
+ [28] = {
+ .name = "{instructions:k,cycles:upp}",
+ .check = test__group1,
+ },
+ [29] = {
+ .name = "{faults:k,cache-references}:u,cycles:k",
+ .check = test__group2,
+ },
+ [30] = {
+ .name = "group1{syscalls:sys_enter_open:H,cycles:kppp},group2{cycles,1:3}:G,instructions:u",
+ .check = test__group3,
+ },
+ [31] = {
+ .name = "{cycles:u,instructions:kp}:p",
+ .check = test__group4,
+ },
+ [32] = {
+ .name = "{cycles,instructions}:G,{cycles:G,instructions:G},cycles",
+ .check = test__group5,
+ },
};
static struct test__event_st test__events_pmu[] = {
@@ -658,9 +925,6 @@
},
};
-#define TEST__TERMS_CNT (sizeof(test__terms) / \
- sizeof(struct test__term))
-
static int test_event(struct test__event_st *e)
{
struct perf_evlist *evlist;
@@ -685,19 +949,19 @@
static int test_events(struct test__event_st *events, unsigned cnt)
{
- int ret = 0;
+ int ret1, ret2 = 0;
unsigned i;
for (i = 0; i < cnt; i++) {
struct test__event_st *e = &events[i];
pr_debug("running test %d '%s'\n", i, e->name);
- ret = test_event(e);
- if (ret)
- break;
+ ret1 = test_event(e);
+ if (ret1)
+ ret2 = ret1;
}
- return ret;
+ return ret2;
}
static int test_term(struct test__term *t)
@@ -752,19 +1016,19 @@
ret = stat(path, &st);
if (ret)
- pr_debug("ommiting PMU cpu tests\n");
+ pr_debug("omitting PMU cpu tests\n");
return !ret;
}
int parse_events__test(void)
{
- int ret;
+ int ret1, ret2 = 0;
#define TEST_EVENTS(tests) \
do { \
- ret = test_events(tests, ARRAY_SIZE(tests)); \
- if (ret) \
- return ret; \
+ ret1 = test_events(tests, ARRAY_SIZE(tests)); \
+ if (!ret2) \
+ ret2 = ret1; \
} while (0)
TEST_EVENTS(test__events);
@@ -772,5 +1036,9 @@
if (test_pmu())
TEST_EVENTS(test__events_pmu);
- return test_terms(test__terms, ARRAY_SIZE(test__terms));
+ ret1 = test_terms(test__terms, ARRAY_SIZE(test__terms));
+ if (!ret2)
+ ret2 = ret1;
+
+ return ret2;
}
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 74a5af4..aed38e4 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -239,8 +239,11 @@
return "unknown";
}
-static int add_event(struct list_head **_list, int *idx,
- struct perf_event_attr *attr, char *name)
+
+
+static int __add_event(struct list_head **_list, int *idx,
+ struct perf_event_attr *attr,
+ char *name, struct cpu_map *cpus)
{
struct perf_evsel *evsel;
struct list_head *list = *_list;
@@ -260,6 +263,7 @@
return -ENOMEM;
}
+ evsel->cpus = cpus;
if (name)
evsel->name = strdup(name);
list_add_tail(&evsel->node, list);
@@ -267,6 +271,12 @@
return 0;
}
+static int add_event(struct list_head **_list, int *idx,
+ struct perf_event_attr *attr, char *name)
+{
+ return __add_event(_list, idx, attr, name, NULL);
+}
+
static int parse_aliases(char *str, const char *names[][PERF_EVSEL__MAX_ALIASES], int size)
{
int i, j;
@@ -308,7 +318,7 @@
for (i = 0; (i < 2) && (op_result[i]); i++) {
char *str = op_result[i];
- snprintf(name + n, MAX_NAME_LEN - n, "-%s\n", str);
+ n += snprintf(name + n, MAX_NAME_LEN - n, "-%s", str);
if (cache_op == -1) {
cache_op = parse_aliases(str, perf_evsel__hw_cache_op,
@@ -346,42 +356,28 @@
return add_event(list, idx, &attr, name);
}
-static int add_tracepoint(struct list_head **list, int *idx,
+static int add_tracepoint(struct list_head **listp, int *idx,
char *sys_name, char *evt_name)
{
- struct perf_event_attr attr;
- char name[MAX_NAME_LEN];
- char evt_path[MAXPATHLEN];
- char id_buf[4];
- u64 id;
- int fd;
+ struct perf_evsel *evsel;
+ struct list_head *list = *listp;
- snprintf(evt_path, MAXPATHLEN, "%s/%s/%s/id", tracing_events_path,
- sys_name, evt_name);
-
- fd = open(evt_path, O_RDONLY);
- if (fd < 0)
- return -1;
-
- if (read(fd, id_buf, sizeof(id_buf)) < 0) {
- close(fd);
- return -1;
+ if (!list) {
+ list = malloc(sizeof(*list));
+ if (!list)
+ return -ENOMEM;
+ INIT_LIST_HEAD(list);
}
- close(fd);
- id = atoll(id_buf);
+ evsel = perf_evsel__newtp(sys_name, evt_name, (*idx)++);
+ if (!evsel) {
+ free(list);
+ return -ENOMEM;
+ }
- memset(&attr, 0, sizeof(attr));
- attr.config = id;
- attr.type = PERF_TYPE_TRACEPOINT;
- attr.sample_type |= PERF_SAMPLE_RAW;
- attr.sample_type |= PERF_SAMPLE_TIME;
- attr.sample_type |= PERF_SAMPLE_CPU;
- attr.sample_type |= PERF_SAMPLE_PERIOD;
- attr.sample_period = 1;
-
- snprintf(name, MAX_NAME_LEN, "%s:%s", sys_name, evt_name);
- return add_event(list, idx, &attr, name);
+ list_add_tail(&evsel->node, list);
+ *listp = list;
+ return 0;
}
static int add_tracepoint_multi(struct list_head **list, int *idx,
@@ -551,7 +547,7 @@
}
int parse_events_add_numeric(struct list_head **list, int *idx,
- unsigned long type, unsigned long config,
+ u32 type, u64 config,
struct list_head *head_config)
{
struct perf_event_attr attr;
@@ -607,8 +603,23 @@
if (perf_pmu__config(pmu, &attr, head_config))
return -EINVAL;
- return add_event(list, idx, &attr,
- pmu_event_name(head_config));
+ return __add_event(list, idx, &attr, pmu_event_name(head_config),
+ pmu->cpus);
+}
+
+int parse_events__modifier_group(struct list_head *list,
+ char *event_mod)
+{
+ return parse_events__modifier_event(list, event_mod, true);
+}
+
+void parse_events__set_leader(char *name, struct list_head *list)
+{
+ struct perf_evsel *leader;
+
+ __perf_evlist__set_leader(list);
+ leader = list_entry(list->next, struct perf_evsel, node);
+ leader->group_name = name ? strdup(name) : NULL;
}
void parse_events_update_lists(struct list_head *list_event,
@@ -616,21 +627,45 @@
{
/*
* Called for single event definition. Update the
- * 'all event' list, and reinit the 'signle event'
+ * 'all event' list, and reinit the 'single event'
* list, for next event definition.
*/
list_splice_tail(list_event, list_all);
free(list_event);
}
-int parse_events_modifier(struct list_head *list, char *str)
-{
- struct perf_evsel *evsel;
- int exclude = 0, exclude_GH = 0;
- int eu = 0, ek = 0, eh = 0, eH = 0, eG = 0, precise = 0;
+struct event_modifier {
+ int eu;
+ int ek;
+ int eh;
+ int eH;
+ int eG;
+ int precise;
+ int exclude_GH;
+};
- if (str == NULL)
- return 0;
+static int get_event_modifier(struct event_modifier *mod, char *str,
+ struct perf_evsel *evsel)
+{
+ int eu = evsel ? evsel->attr.exclude_user : 0;
+ int ek = evsel ? evsel->attr.exclude_kernel : 0;
+ int eh = evsel ? evsel->attr.exclude_hv : 0;
+ int eH = evsel ? evsel->attr.exclude_host : 0;
+ int eG = evsel ? evsel->attr.exclude_guest : 0;
+ int precise = evsel ? evsel->attr.precise_ip : 0;
+
+ int exclude = eu | ek | eh;
+ int exclude_GH = evsel ? evsel->exclude_GH : 0;
+
+ /*
+ * We are here for group and 'GH' was not set as event
+ * modifier and whatever event/group modifier override
+ * default 'GH' setup.
+ */
+ if (evsel && !exclude_GH)
+ eH = eG = 0;
+
+ memset(mod, 0, sizeof(*mod));
while (*str) {
if (*str == 'u') {
@@ -674,13 +709,51 @@
if (precise > 3)
return -EINVAL;
+ mod->eu = eu;
+ mod->ek = ek;
+ mod->eh = eh;
+ mod->eH = eH;
+ mod->eG = eG;
+ mod->precise = precise;
+ mod->exclude_GH = exclude_GH;
+ return 0;
+}
+
+int parse_events__modifier_event(struct list_head *list, char *str, bool add)
+{
+ struct perf_evsel *evsel;
+ struct event_modifier mod;
+
+ if (str == NULL)
+ return 0;
+
+ if (!add && get_event_modifier(&mod, str, NULL))
+ return -EINVAL;
+
list_for_each_entry(evsel, list, node) {
- evsel->attr.exclude_user = eu;
- evsel->attr.exclude_kernel = ek;
- evsel->attr.exclude_hv = eh;
- evsel->attr.precise_ip = precise;
- evsel->attr.exclude_host = eH;
- evsel->attr.exclude_guest = eG;
+
+ if (add && get_event_modifier(&mod, str, evsel))
+ return -EINVAL;
+
+ evsel->attr.exclude_user = mod.eu;
+ evsel->attr.exclude_kernel = mod.ek;
+ evsel->attr.exclude_hv = mod.eh;
+ evsel->attr.precise_ip = mod.precise;
+ evsel->attr.exclude_host = mod.eH;
+ evsel->attr.exclude_guest = mod.eG;
+ evsel->exclude_GH = mod.exclude_GH;
+ }
+
+ return 0;
+}
+
+int parse_events_name(struct list_head *list, char *name)
+{
+ struct perf_evsel *evsel;
+
+ list_for_each_entry(evsel, list, node) {
+ if (!evsel->name)
+ evsel->name = strdup(name);
}
return 0;
@@ -730,7 +803,8 @@
return ret;
}
-int parse_events(struct perf_evlist *evlist, const char *str, int unset __used)
+int parse_events(struct perf_evlist *evlist, const char *str,
+ int unset __maybe_unused)
{
struct parse_events_data__events data = {
.list = LIST_HEAD_INIT(data.list),
@@ -756,20 +830,20 @@
}
int parse_events_option(const struct option *opt, const char *str,
- int unset __used)
+ int unset __maybe_unused)
{
struct perf_evlist *evlist = *(struct perf_evlist **)opt->value;
return parse_events(evlist, str, unset);
}
int parse_filter(const struct option *opt, const char *str,
- int unset __used)
+ int unset __maybe_unused)
{
struct perf_evlist *evlist = *(struct perf_evlist **)opt->value;
struct perf_evsel *last = NULL;
if (evlist->nr_entries > 0)
- last = list_entry(evlist->entries.prev, struct perf_evsel, node);
+ last = perf_evlist__last(evlist);
if (last == NULL || last->attr.type != PERF_TYPE_TRACEPOINT) {
fprintf(stderr,
@@ -799,7 +873,8 @@
* Print the events from <debugfs_mount_point>/tracing/events
*/
-void print_tracepoint_events(const char *subsys_glob, const char *event_glob)
+void print_tracepoint_events(const char *subsys_glob, const char *event_glob,
+ bool name_only)
{
DIR *sys_dir, *evt_dir;
struct dirent *sys_next, *evt_next, sys_dirent, evt_dirent;
@@ -829,6 +904,11 @@
!strglobmatch(evt_dirent.d_name, event_glob))
continue;
+ if (name_only) {
+ printf("%s:%s ", sys_dirent.d_name, evt_dirent.d_name);
+ continue;
+ }
+
snprintf(evt_path, MAXPATHLEN, "%s:%s",
sys_dirent.d_name, evt_dirent.d_name);
printf(" %-50s [%s]\n", evt_path,
@@ -906,7 +986,7 @@
__print_events_type(type, event_symbols_hw, PERF_COUNT_HW_MAX);
}
-int print_hwcache_events(const char *event_glob)
+int print_hwcache_events(const char *event_glob, bool name_only)
{
unsigned int type, op, i, printed = 0;
char name[64];
@@ -923,8 +1003,11 @@
if (event_glob != NULL && !strglobmatch(name, event_glob))
continue;
- printf(" %-50s [%s]\n", name,
- event_type_descriptors[PERF_TYPE_HW_CACHE]);
+ if (name_only)
+ printf("%s ", name);
+ else
+ printf(" %-50s [%s]\n", name,
+ event_type_descriptors[PERF_TYPE_HW_CACHE]);
++printed;
}
}
@@ -934,7 +1017,8 @@
}
static void print_symbol_events(const char *event_glob, unsigned type,
- struct event_symbol *syms, unsigned max)
+ struct event_symbol *syms, unsigned max,
+ bool name_only)
{
unsigned i, printed = 0;
char name[MAX_NAME_LEN];
@@ -946,6 +1030,11 @@
(syms->alias && strglobmatch(syms->alias, event_glob))))
continue;
+ if (name_only) {
+ printf("%s ", syms->symbol);
+ continue;
+ }
+
if (strlen(syms->alias))
snprintf(name, MAX_NAME_LEN, "%s OR %s", syms->symbol, syms->alias);
else
@@ -963,39 +1052,42 @@
/*
* Print the help text for the event symbols:
*/
-void print_events(const char *event_glob)
+void print_events(const char *event_glob, bool name_only)
{
-
- printf("\n");
- printf("List of pre-defined events (to be used in -e):\n");
+ if (!name_only) {
+ printf("\n");
+ printf("List of pre-defined events (to be used in -e):\n");
+ }
print_symbol_events(event_glob, PERF_TYPE_HARDWARE,
- event_symbols_hw, PERF_COUNT_HW_MAX);
+ event_symbols_hw, PERF_COUNT_HW_MAX, name_only);
print_symbol_events(event_glob, PERF_TYPE_SOFTWARE,
- event_symbols_sw, PERF_COUNT_SW_MAX);
+ event_symbols_sw, PERF_COUNT_SW_MAX, name_only);
- print_hwcache_events(event_glob);
+ print_hwcache_events(event_glob, name_only);
if (event_glob != NULL)
return;
- printf("\n");
- printf(" %-50s [%s]\n",
- "rNNN",
- event_type_descriptors[PERF_TYPE_RAW]);
- printf(" %-50s [%s]\n",
- "cpu/t1=v1[,t2=v2,t3 ...]/modifier",
- event_type_descriptors[PERF_TYPE_RAW]);
- printf(" (see 'perf list --help' on how to encode it)\n");
- printf("\n");
+ if (!name_only) {
+ printf("\n");
+ printf(" %-50s [%s]\n",
+ "rNNN",
+ event_type_descriptors[PERF_TYPE_RAW]);
+ printf(" %-50s [%s]\n",
+ "cpu/t1=v1[,t2=v2,t3 ...]/modifier",
+ event_type_descriptors[PERF_TYPE_RAW]);
+ printf(" (see 'perf list --help' on how to encode it)\n");
+ printf("\n");
- printf(" %-50s [%s]\n",
- "mem:<addr>[:access]",
+ printf(" %-50s [%s]\n",
+ "mem:<addr>[:access]",
event_type_descriptors[PERF_TYPE_BREAKPOINT]);
- printf("\n");
+ printf("\n");
+ }
- print_tracepoint_events(NULL, NULL);
+ print_tracepoint_events(NULL, NULL, name_only);
}
int parse_events__is_hardcoded_term(struct parse_events__term *term)
@@ -1005,7 +1097,7 @@
static int new_term(struct parse_events__term **_term, int type_val,
int type_term, char *config,
- char *str, long num)
+ char *str, u64 num)
{
struct parse_events__term *term;
@@ -1034,7 +1126,7 @@
}
int parse_events__term_num(struct parse_events__term **term,
- int type_term, char *config, long num)
+ int type_term, char *config, u64 num)
{
return new_term(term, PARSE_EVENTS__TERM_TYPE_NUM, type_term,
config, NULL, num);
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index ee9c218..c356e44 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -55,7 +55,7 @@
char *config;
union {
char *str;
- long num;
+ u64 num;
} val;
int type_val;
int type_term;
@@ -73,17 +73,19 @@
int parse_events__is_hardcoded_term(struct parse_events__term *term);
int parse_events__term_num(struct parse_events__term **_term,
- int type_term, char *config, long num);
+ int type_term, char *config, u64 num);
int parse_events__term_str(struct parse_events__term **_term,
int type_term, char *config, char *str);
int parse_events__term_clone(struct parse_events__term **new,
struct parse_events__term *term);
void parse_events__free_terms(struct list_head *terms);
-int parse_events_modifier(struct list_head *list, char *str);
+int parse_events__modifier_event(struct list_head *list, char *str, bool add);
+int parse_events__modifier_group(struct list_head *list, char *event_mod);
+int parse_events_name(struct list_head *list, char *name);
int parse_events_add_tracepoint(struct list_head **list, int *idx,
char *sys, char *event);
int parse_events_add_numeric(struct list_head **list, int *idx,
- unsigned long type, unsigned long config,
+ u32 type, u64 config,
struct list_head *head_config);
int parse_events_add_cache(struct list_head **list, int *idx,
char *type, char *op_result1, char *op_result2);
@@ -91,15 +93,17 @@
void *ptr, char *type);
int parse_events_add_pmu(struct list_head **list, int *idx,
char *pmu , struct list_head *head_config);
+void parse_events__set_leader(char *name, struct list_head *list);
void parse_events_update_lists(struct list_head *list_event,
struct list_head *list_all);
void parse_events_error(void *data, void *scanner, char const *msg);
int parse_events__test(void);
-void print_events(const char *event_glob);
+void print_events(const char *event_glob, bool name_only);
void print_events_type(u8 type);
-void print_tracepoint_events(const char *subsys_glob, const char *event_glob);
-int print_hwcache_events(const char *event_glob);
+void print_tracepoint_events(const char *subsys_glob, const char *event_glob,
+ bool name_only);
+int print_hwcache_events(const char *event_glob, bool name_only);
extern int is_valid_tracepoint(const char *event_string);
extern int valid_debugfs_mount(const char *debugfs);
diff --git a/tools/perf/util/parse-events.l b/tools/perf/util/parse-events.l
index 384ca74..c87efc1 100644
--- a/tools/perf/util/parse-events.l
+++ b/tools/perf/util/parse-events.l
@@ -15,10 +15,10 @@
static int __value(YYSTYPE *yylval, char *str, int base, int token)
{
- long num;
+ u64 num;
errno = 0;
- num = strtoul(str, NULL, base);
+ num = strtoull(str, NULL, base);
if (errno)
return PE_ERROR;
@@ -70,6 +70,12 @@
%}
%x mem
+%s config
+%x event
+
+group [^,{}/]*[{][^}]*[}][^,{}/]*
+event_pmu [^,{}/]+[/][^/]*[/][^,{}/]*
+event [^,{}/]+
num_dec [0-9]+
num_hex 0x[a-fA-F0-9]+
@@ -84,7 +90,13 @@
{
int start_token;
- start_token = (int) parse_events_get_extra(yyscanner);
+ start_token = parse_events_get_extra(yyscanner);
+
+ if (start_token == PE_START_TERMS)
+ BEGIN(config);
+ else if (start_token == PE_START_EVENTS)
+ BEGIN(event);
+
if (start_token) {
parse_events_set_extra(NULL, yyscanner);
return start_token;
@@ -92,6 +104,26 @@
}
%}
+<event>{
+
+{group} {
+ BEGIN(INITIAL); yyless(0);
+ }
+
+{event_pmu} |
+{event} {
+ str(yyscanner, PE_EVENT_NAME);
+ BEGIN(INITIAL); yyless(0);
+ return PE_EVENT_NAME;
+ }
+
+. |
+<<EOF>> {
+ BEGIN(INITIAL); yyless(0);
+ }
+
+}
+
cpu-cycles|cycles { return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_CPU_CYCLES); }
stalled-cycles-frontend|idle-cycles-frontend { return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_STALLED_CYCLES_FRONTEND); }
stalled-cycles-backend|idle-cycles-backend { return sym(yyscanner, PERF_TYPE_HARDWARE, PERF_COUNT_HW_STALLED_CYCLES_BACKEND); }
@@ -127,18 +159,16 @@
refs|Reference|ops|access |
misses|miss { return str(yyscanner, PE_NAME_CACHE_OP_RESULT); }
- /*
- * These are event config hardcoded term names to be specified
- * within xxx/.../ syntax. So far we dont clash with other names,
- * so we can put them here directly. In case the we have a conflict
- * in future, this needs to go into '//' condition block.
- */
+<config>{
config { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG); }
config1 { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG1); }
config2 { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_CONFIG2); }
name { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NAME); }
period { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_SAMPLE_PERIOD); }
branch_type { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE); }
+, { return ','; }
+"/" { BEGIN(INITIAL); return '/'; }
+}
mem: { BEGIN(mem); return PE_PREFIX_MEM; }
r{num_raw_hex} { return raw(yyscanner); }
@@ -147,10 +177,12 @@
{modifier_event} { return str(yyscanner, PE_MODIFIER_EVENT); }
{name} { return str(yyscanner, PE_NAME); }
-"/" { return '/'; }
+"/" { BEGIN(config); return '/'; }
- { return '-'; }
-, { return ','; }
+, { BEGIN(event); return ','; }
: { return ':'; }
+"{" { BEGIN(event); return '{'; }
+"}" { return '}'; }
= { return '='; }
\n { }
@@ -175,7 +207,7 @@
%%
-int parse_events_wrap(void *scanner __used)
+int parse_events_wrap(void *scanner __maybe_unused)
{
return 1;
}
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 2bc5fbf..cd88209 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -27,10 +27,11 @@
%token PE_START_EVENTS PE_START_TERMS
%token PE_VALUE PE_VALUE_SYM_HW PE_VALUE_SYM_SW PE_RAW PE_TERM
+%token PE_EVENT_NAME
%token PE_NAME
%token PE_MODIFIER_EVENT PE_MODIFIER_BP
%token PE_NAME_CACHE_TYPE PE_NAME_CACHE_OP_RESULT
-%token PE_PREFIX_MEM PE_PREFIX_RAW
+%token PE_PREFIX_MEM PE_PREFIX_RAW PE_PREFIX_GROUP
%token PE_ERROR
%type <num> PE_VALUE
%type <num> PE_VALUE_SYM_HW
@@ -42,6 +43,7 @@
%type <str> PE_NAME_CACHE_OP_RESULT
%type <str> PE_MODIFIER_EVENT
%type <str> PE_MODIFIER_BP
+%type <str> PE_EVENT_NAME
%type <num> value_sym
%type <head> event_config
%type <term> event_term
@@ -53,44 +55,125 @@
%type <head> event_legacy_numeric
%type <head> event_legacy_raw
%type <head> event_def
+%type <head> event_mod
+%type <head> event_name
+%type <head> event
+%type <head> events
+%type <head> group_def
+%type <head> group
+%type <head> groups
%union
{
char *str;
- unsigned long num;
+ u64 num;
struct list_head *head;
struct parse_events__term *term;
}
%%
start:
-PE_START_EVENTS events
+PE_START_EVENTS start_events
|
-PE_START_TERMS terms
+PE_START_TERMS start_terms
-events:
-events ',' event | event
-
-event:
-event_def PE_MODIFIER_EVENT
+start_events: groups
{
struct parse_events_data__events *data = _data;
+ parse_events_update_lists($1, &data->list);
+}
+
+groups:
+groups ',' group
+{
+ struct list_head *list = $1;
+ struct list_head *group = $3;
+
+ parse_events_update_lists(group, list);
+ $$ = list;
+}
+|
+groups ',' event
+{
+ struct list_head *list = $1;
+ struct list_head *event = $3;
+
+ parse_events_update_lists(event, list);
+ $$ = list;
+}
+|
+group
+|
+event
+
+group:
+group_def ':' PE_MODIFIER_EVENT
+{
+ struct list_head *list = $1;
+
+ ABORT_ON(parse_events__modifier_group(list, $3));
+ $$ = list;
+}
+|
+group_def
+
+group_def:
+PE_NAME '{' events '}'
+{
+ struct list_head *list = $3;
+
+ parse_events__set_leader($1, list);
+ $$ = list;
+}
+|
+'{' events '}'
+{
+ struct list_head *list = $2;
+
+ parse_events__set_leader(NULL, list);
+ $$ = list;
+}
+
+events:
+events ',' event
+{
+ struct list_head *event = $3;
+ struct list_head *list = $1;
+
+ parse_events_update_lists(event, list);
+ $$ = list;
+}
+|
+event
+
+event: event_mod
+
+event_mod:
+event_name PE_MODIFIER_EVENT
+{
+ struct list_head *list = $1;
+
/*
* Apply modifier on all events added by single event definition
* (there could be more events added for multiple tracepoint
* definitions via '*?'.
*/
- ABORT_ON(parse_events_modifier($1, $2));
- parse_events_update_lists($1, &data->list);
+ ABORT_ON(parse_events__modifier_event(list, $2, false));
+ $$ = list;
+}
+|
+event_name
+
+event_name:
+PE_EVENT_NAME event_def
+{
+ ABORT_ON(parse_events_name($2, $1));
+ free($1);
+ $$ = $2;
}
|
event_def
-{
- struct parse_events_data__events *data = _data;
-
- parse_events_update_lists($1, &data->list);
-}
event_def: event_pmu |
event_legacy_symbol |
@@ -207,7 +290,7 @@
struct parse_events_data__events *data = _data;
struct list_head *list = NULL;
- ABORT_ON(parse_events_add_numeric(&list, &data->idx, $1, $3, NULL));
+ ABORT_ON(parse_events_add_numeric(&list, &data->idx, (u32)$1, $3, NULL));
$$ = list;
}
@@ -222,7 +305,7 @@
$$ = list;
}
-terms: event_config
+start_terms: event_config
{
struct parse_events_data__terms *data = _data;
data->terms = $1;
@@ -282,7 +365,7 @@
{
struct parse_events__term *term;
- ABORT_ON(parse_events__term_str(&term, $1, NULL, $3));
+ ABORT_ON(parse_events__term_str(&term, (int)$1, NULL, $3));
$$ = term;
}
|
@@ -290,7 +373,7 @@
{
struct parse_events__term *term;
- ABORT_ON(parse_events__term_num(&term, $1, NULL, $3));
+ ABORT_ON(parse_events__term_num(&term, (int)$1, NULL, $3));
$$ = term;
}
|
@@ -298,7 +381,7 @@
{
struct parse_events__term *term;
- ABORT_ON(parse_events__term_num(&term, $1, NULL, 1));
+ ABORT_ON(parse_events__term_num(&term, (int)$1, NULL, 1));
$$ = term;
}
@@ -308,7 +391,7 @@
%%
-void parse_events_error(void *data __used, void *scanner __used,
- char const *msg __used)
+void parse_events_error(void *data __maybe_unused, void *scanner __maybe_unused,
+ char const *msg __maybe_unused)
{
}
diff --git a/tools/perf/util/parse-options.c b/tools/perf/util/parse-options.c
index 594f8fa..443fc11 100644
--- a/tools/perf/util/parse-options.c
+++ b/tools/perf/util/parse-options.c
@@ -557,7 +557,8 @@
}
-int parse_opt_verbosity_cb(const struct option *opt, const char *arg __used,
+int parse_opt_verbosity_cb(const struct option *opt,
+ const char *arg __maybe_unused,
int unset)
{
int *target = opt->value;
diff --git a/tools/perf/util/perf_regs.h b/tools/perf/util/perf_regs.h
new file mode 100644
index 0000000..316dbe7
--- /dev/null
+++ b/tools/perf/util/perf_regs.h
@@ -0,0 +1,14 @@
+#ifndef __PERF_REGS_H
+#define __PERF_REGS_H
+
+#ifndef NO_PERF_REGS
+#include <perf_regs.h>
+#else
+#define PERF_REGS_MASK 0
+
+static inline const char *perf_reg_name(int id __maybe_unused)
+{
+ return NULL;
+}
+#endif /* NO_PERF_REGS */
+#endif /* __PERF_REGS_H */
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index 67715a4..8a2229d 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -9,6 +9,9 @@
#include "util.h"
#include "pmu.h"
#include "parse-events.h"
+#include "cpumap.h"
+
+#define EVENT_SOURCE_DEVICE_PATH "/bus/event_source/devices/"
int perf_pmu_parse(struct list_head *list, char *name);
extern FILE *perf_pmu_in;
@@ -69,7 +72,7 @@
return -1;
snprintf(path, PATH_MAX,
- "%s/bus/event_source/devices/%s/format", sysfs, name);
+ "%s" EVENT_SOURCE_DEVICE_PATH "%s/format", sysfs, name);
if (stat(path, &st) < 0)
return 0; /* no error if format does not exist */
@@ -206,7 +209,7 @@
return -1;
snprintf(path, PATH_MAX,
- "%s/bus/event_source/devices/%s/type", sysfs, name);
+ "%s" EVENT_SOURCE_DEVICE_PATH "%s/type", sysfs, name);
if (stat(path, &st) < 0)
return -1;
@@ -222,6 +225,62 @@
return ret;
}
+/* Add all pmus in sysfs to pmu list: */
+static void pmu_read_sysfs(void)
+{
+ char path[PATH_MAX];
+ const char *sysfs;
+ DIR *dir;
+ struct dirent *dent;
+
+ sysfs = sysfs_find_mountpoint();
+ if (!sysfs)
+ return;
+
+ snprintf(path, PATH_MAX,
+ "%s" EVENT_SOURCE_DEVICE_PATH, sysfs);
+
+ dir = opendir(path);
+ if (!dir)
+ return;
+
+ while ((dent = readdir(dir))) {
+ if (!strcmp(dent->d_name, ".") || !strcmp(dent->d_name, ".."))
+ continue;
+ /* add to static LIST_HEAD(pmus): */
+ perf_pmu__find(dent->d_name);
+ }
+
+ closedir(dir);
+}
+
+static struct cpu_map *pmu_cpumask(char *name)
+{
+ struct stat st;
+ char path[PATH_MAX];
+ const char *sysfs;
+ FILE *file;
+ struct cpu_map *cpus;
+
+ sysfs = sysfs_find_mountpoint();
+ if (!sysfs)
+ return NULL;
+
+ snprintf(path, PATH_MAX,
+ "%s/bus/event_source/devices/%s/cpumask", sysfs, name);
+
+ if (stat(path, &st) < 0)
+ return NULL;
+
+ file = fopen(path, "r");
+ if (!file)
+ return NULL;
+
+ cpus = cpu_map__read(file);
+ fclose(file);
+ return cpus;
+}
+
static struct perf_pmu *pmu_lookup(char *name)
{
struct perf_pmu *pmu;
@@ -244,6 +303,8 @@
if (!pmu)
return NULL;
+ pmu->cpus = pmu_cpumask(name);
+
pmu_aliases(name, &aliases);
INIT_LIST_HEAD(&pmu->format);
@@ -267,6 +328,21 @@
return NULL;
}
+struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu)
+{
+ /*
+ * pmu iterator: If pmu is NULL, we start at the begin,
+ * otherwise return the next pmu. Returns NULL on end.
+ */
+ if (!pmu) {
+ pmu_read_sysfs();
+ pmu = list_prepare_entry(pmu, &pmus, list);
+ }
+ list_for_each_entry_continue(pmu, &pmus, list)
+ return pmu;
+ return NULL;
+}
+
struct perf_pmu *perf_pmu__find(char *name)
{
struct perf_pmu *pmu;
diff --git a/tools/perf/util/pmu.h b/tools/perf/util/pmu.h
index 535f2c5..53c7794 100644
--- a/tools/perf/util/pmu.h
+++ b/tools/perf/util/pmu.h
@@ -28,6 +28,7 @@
struct perf_pmu {
char *name;
__u32 type;
+ struct cpu_map *cpus;
struct list_head format;
struct list_head aliases;
struct list_head list;
@@ -46,5 +47,7 @@
int config, unsigned long *bits);
void perf_pmu__set_format(unsigned long *bits, long from, long to);
+struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu);
+
int perf_pmu__test(void);
#endif /* __PMU_H */
diff --git a/tools/perf/util/pmu.y b/tools/perf/util/pmu.y
index 20ea77e..ec89804 100644
--- a/tools/perf/util/pmu.y
+++ b/tools/perf/util/pmu.y
@@ -86,8 +86,8 @@
%%
-void perf_pmu_error(struct list_head *list __used,
- char *name __used,
- char const *msg __used)
+void perf_pmu_error(struct list_head *list __maybe_unused,
+ char *name __maybe_unused,
+ char const *msg __maybe_unused)
{
}
diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 0dda25d..49a256e 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -41,7 +41,7 @@
#include "symbol.h"
#include "thread.h"
#include "debugfs.h"
-#include "trace-event.h" /* For __unused */
+#include "trace-event.h" /* For __maybe_unused */
#include "probe-event.h"
#include "probe-finder.h"
#include "session.h"
@@ -647,8 +647,8 @@
}
static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
- struct probe_trace_event **tevs __unused,
- int max_tevs __unused, const char *target)
+ struct probe_trace_event **tevs __maybe_unused,
+ int max_tevs __maybe_unused, const char *target)
{
if (perf_probe_event_need_dwarf(pev)) {
pr_warning("Debuginfo-analysis is not supported.\n");
@@ -661,17 +661,18 @@
return 0;
}
-int show_line_range(struct line_range *lr __unused, const char *module __unused)
+int show_line_range(struct line_range *lr __maybe_unused,
+ const char *module __maybe_unused)
{
pr_warning("Debuginfo-analysis is not supported.\n");
return -ENOSYS;
}
-int show_available_vars(struct perf_probe_event *pevs __unused,
- int npevs __unused, int max_vls __unused,
- const char *module __unused,
- struct strfilter *filter __unused,
- bool externs __unused)
+int show_available_vars(struct perf_probe_event *pevs __maybe_unused,
+ int npevs __maybe_unused, int max_vls __maybe_unused,
+ const char *module __maybe_unused,
+ struct strfilter *filter __maybe_unused,
+ bool externs __maybe_unused)
{
pr_warning("Debuginfo-analysis is not supported.\n");
return -ENOSYS;
@@ -1099,6 +1100,7 @@
struct probe_trace_point *tp = &tev->point;
char pr;
char *p;
+ char *argv0_str = NULL, *fmt, *fmt1_str, *fmt2_str, *fmt3_str;
int ret, i, argc;
char **argv;
@@ -1115,14 +1117,27 @@
}
/* Scan event and group name. */
- ret = sscanf(argv[0], "%c:%a[^/ \t]/%a[^ \t]",
- &pr, (float *)(void *)&tev->group,
- (float *)(void *)&tev->event);
- if (ret != 3) {
+ argv0_str = strdup(argv[0]);
+ if (argv0_str == NULL) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ fmt1_str = strtok_r(argv0_str, ":", &fmt);
+ fmt2_str = strtok_r(NULL, "/", &fmt);
+ fmt3_str = strtok_r(NULL, " \t", &fmt);
+ if (fmt1_str == NULL || strlen(fmt1_str) != 1 || fmt2_str == NULL
+ || fmt3_str == NULL) {
semantic_error("Failed to parse event name: %s\n", argv[0]);
ret = -EINVAL;
goto out;
}
+ pr = fmt1_str[0];
+ tev->group = strdup(fmt2_str);
+ tev->event = strdup(fmt3_str);
+ if (tev->group == NULL || tev->event == NULL) {
+ ret = -ENOMEM;
+ goto out;
+ }
pr_debug("Group:%s Event:%s probe:%c\n", tev->group, tev->event, pr);
tp->retprobe = (pr == 'r');
@@ -1134,10 +1149,17 @@
p++;
} else
p = argv[1];
- ret = sscanf(p, "%a[^+]+%lu", (float *)(void *)&tp->symbol,
- &tp->offset);
- if (ret == 1)
+ fmt1_str = strtok_r(p, "+", &fmt);
+ tp->symbol = strdup(fmt1_str);
+ if (tp->symbol == NULL) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ fmt2_str = strtok_r(NULL, "", &fmt);
+ if (fmt2_str == NULL)
tp->offset = 0;
+ else
+ tp->offset = strtoul(fmt2_str, NULL, 10);
tev->nargs = argc - 2;
tev->args = zalloc(sizeof(struct probe_trace_arg) * tev->nargs);
@@ -1161,6 +1183,7 @@
}
ret = 0;
out:
+ free(argv0_str);
argv_free(argv);
return ret;
}
@@ -2183,7 +2206,7 @@
* If a symbol corresponds to a function with global binding and
* matches filter return 0. For all others return 1.
*/
-static int filter_available_functions(struct map *map __unused,
+static int filter_available_functions(struct map *map __maybe_unused,
struct symbol *sym)
{
if (sym->binding == STB_GLOBAL &&
@@ -2307,10 +2330,17 @@
function = NULL;
}
if (!pev->group) {
- char *ptr1, *ptr2;
+ char *ptr1, *ptr2, *exec_copy;
pev->group = zalloc(sizeof(char *) * 64);
- ptr1 = strdup(basename(exec));
+ exec_copy = strdup(exec);
+ if (!exec_copy) {
+ ret = -ENOMEM;
+ pr_warning("Failed to copy exec string.\n");
+ goto out;
+ }
+
+ ptr1 = strdup(basename(exec_copy));
if (ptr1) {
ptr2 = strpbrk(ptr1, "-._");
if (ptr2)
@@ -2319,6 +2349,7 @@
ptr1);
free(ptr1);
}
+ free(exec_copy);
}
free(pp->function);
pp->function = zalloc(sizeof(char *) * MAX_PROBE_ARGS);
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index d448984..1daf5c1 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -207,7 +207,7 @@
#else
/* With older elfutils, this just support kernel module... */
static int debuginfo__init_online_kernel_dwarf(struct debuginfo *self,
- Dwarf_Addr addr __used)
+ Dwarf_Addr addr __maybe_unused)
{
const char *path = kernel_get_module_path("kernel");
@@ -525,8 +525,10 @@
return -ENOENT;
}
/* Verify it is a data structure */
- if (dwarf_tag(&type) != DW_TAG_structure_type) {
- pr_warning("%s is not a data structure.\n", varname);
+ tag = dwarf_tag(&type);
+ if (tag != DW_TAG_structure_type && tag != DW_TAG_union_type) {
+ pr_warning("%s is not a data structure nor an union.\n",
+ varname);
return -EINVAL;
}
@@ -539,8 +541,9 @@
*ref_ptr = ref;
} else {
/* Verify it is a data structure */
- if (tag != DW_TAG_structure_type) {
- pr_warning("%s is not a data structure.\n", varname);
+ if (tag != DW_TAG_structure_type && tag != DW_TAG_union_type) {
+ pr_warning("%s is not a data structure nor an union.\n",
+ varname);
return -EINVAL;
}
if (field->name[0] == '[') {
@@ -567,10 +570,15 @@
}
/* Get the offset of the field */
- ret = die_get_data_member_location(die_mem, &offs);
- if (ret < 0) {
- pr_warning("Failed to get the offset of %s.\n", field->name);
- return ret;
+ if (tag == DW_TAG_union_type) {
+ offs = 0;
+ } else {
+ ret = die_get_data_member_location(die_mem, &offs);
+ if (ret < 0) {
+ pr_warning("Failed to get the offset of %s.\n",
+ field->name);
+ return ret;
+ }
}
ref->offset += (long)offs;
@@ -1419,7 +1427,7 @@
}
static int line_range_walk_cb(const char *fname, int lineno,
- Dwarf_Addr addr __used,
+ Dwarf_Addr addr __maybe_unused,
void *data)
{
struct line_finder *lf = data;
diff --git a/tools/perf/util/python-ext-sources b/tools/perf/util/python-ext-sources
index 2884e67..c40c2d3 100644
--- a/tools/perf/util/python-ext-sources
+++ b/tools/perf/util/python-ext-sources
@@ -1,5 +1,5 @@
#
-# List of files needed by perf python extention
+# List of files needed by perf python extension
#
# Each source file must be placed on its own line so that it can be
# processed by Makefile and util/setup.py accordingly.
@@ -10,10 +10,12 @@
util/evlist.c
util/evsel.c
util/cpumap.c
+util/hweight.c
util/thread_map.c
util/util.c
util/xyarray.c
util/cgroup.c
util/debugfs.c
+util/rblist.c
util/strlist.c
../../lib/rbtree.c
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 0688bfb..9181bf2 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -627,7 +627,7 @@
* This will group just the fds for this single evsel, to group
* multiple events, use evlist.open().
*/
- if (perf_evsel__open(evsel, cpus, threads, group, NULL) < 0) {
+ if (perf_evsel__open(evsel, cpus, threads) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
@@ -672,7 +672,7 @@
};
static int pyrf_evlist__init(struct pyrf_evlist *pevlist,
- PyObject *args, PyObject *kwargs __used)
+ PyObject *args, PyObject *kwargs __maybe_unused)
{
PyObject *pcpus = NULL, *pthreads = NULL;
struct cpu_map *cpus;
@@ -733,7 +733,8 @@
}
static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,
- PyObject *args __used, PyObject *kwargs __used)
+ PyObject *args __maybe_unused,
+ PyObject *kwargs __maybe_unused)
{
struct perf_evlist *evlist = &pevlist->evlist;
PyObject *list = PyList_New(0);
@@ -765,7 +766,8 @@
static PyObject *pyrf_evlist__add(struct pyrf_evlist *pevlist,
- PyObject *args, PyObject *kwargs __used)
+ PyObject *args,
+ PyObject *kwargs __maybe_unused)
{
struct perf_evlist *evlist = &pevlist->evlist;
PyObject *pevsel;
@@ -803,7 +805,7 @@
if (pyevent == NULL)
return PyErr_NoMemory();
- err = perf_evlist__parse_sample(evlist, event, &pevent->sample, false);
+ err = perf_evlist__parse_sample(evlist, event, &pevent->sample);
if (err)
return PyErr_Format(PyExc_OSError,
"perf: can't parse sample, err=%d", err);
@@ -824,7 +826,10 @@
if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|OOii", kwlist, &group))
return NULL;
- if (perf_evlist__open(evlist, group) < 0) {
+ if (group)
+ perf_evlist__set_leader(evlist);
+
+ if (perf_evlist__open(evlist) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
diff --git a/tools/perf/util/scripting-engines/trace-event-perl.c b/tools/perf/util/scripting-engines/trace-event-perl.c
index 02dfa19..f80605e 100644
--- a/tools/perf/util/scripting-engines/trace-event-perl.c
+++ b/tools/perf/util/scripting-engines/trace-event-perl.c
@@ -25,16 +25,16 @@
#include <ctype.h>
#include <errno.h>
-#include "../../perf.h"
#include "../util.h"
+#include <EXTERN.h>
+#include <perl.h>
+
+#include "../../perf.h"
#include "../thread.h"
#include "../event.h"
#include "../trace-event.h"
#include "../evsel.h"
-#include <EXTERN.h>
-#include <perl.h>
-
void boot_Perf__Trace__Context(pTHX_ CV *cv);
void boot_DynaLoader(pTHX_ CV *cv);
typedef PerlInterpreter * INTERP;
@@ -237,16 +237,16 @@
define_event_symbols(event, ev_name, args->next);
}
-static inline
-struct event_format *find_cache_event(struct pevent *pevent, int type)
+static inline struct event_format *find_cache_event(struct perf_evsel *evsel)
{
static char ev_name[256];
struct event_format *event;
+ int type = evsel->attr.config;
if (events[type])
return events[type];
- events[type] = event = pevent_find_event(pevent, type);
+ events[type] = event = evsel->tp_format;
if (!event)
return NULL;
@@ -257,23 +257,22 @@
return event;
}
-static void perl_process_tracepoint(union perf_event *perf_event __unused,
- struct pevent *pevent,
+static void perl_process_tracepoint(union perf_event *perf_event __maybe_unused,
struct perf_sample *sample,
struct perf_evsel *evsel,
- struct machine *machine __unused,
- struct thread *thread)
+ struct machine *machine __maybe_unused,
+ struct addr_location *al)
{
struct format_field *field;
static char handler[256];
unsigned long long val;
unsigned long s, ns;
struct event_format *event;
- int type;
int pid;
int cpu = sample->cpu;
void *data = sample->raw_data;
unsigned long long nsecs = sample->time;
+ struct thread *thread = al->thread;
char *comm = thread->comm;
dSP;
@@ -281,13 +280,11 @@
if (evsel->attr.type != PERF_TYPE_TRACEPOINT)
return;
- type = trace_parse_common_type(pevent, data);
-
- event = find_cache_event(pevent, type);
+ event = find_cache_event(evsel);
if (!event)
- die("ug! no event found for type %d", type);
+ die("ug! no event found for type %" PRIu64, evsel->attr.config);
- pid = trace_parse_common_pid(pevent, data);
+ pid = raw_field_value(event, "common_pid", data);
sprintf(handler, "%s::%s", event->system, event->name);
@@ -320,7 +317,7 @@
offset = field->offset;
XPUSHs(sv_2mortal(newSVpv((char *)data + offset, 0)));
} else { /* FIELD_IS_NUMERIC */
- val = read_size(pevent, data + field->offset,
+ val = read_size(event, data + field->offset,
field->size);
if (field->flags & FIELD_IS_SIGNED) {
XPUSHs(sv_2mortal(newSViv(val)));
@@ -349,11 +346,11 @@
LEAVE;
}
-static void perl_process_event_generic(union perf_event *pevent __unused,
+static void perl_process_event_generic(union perf_event *event,
struct perf_sample *sample,
- struct perf_evsel *evsel __unused,
- struct machine *machine __unused,
- struct thread *thread __unused)
+ struct perf_evsel *evsel,
+ struct machine *machine __maybe_unused,
+ struct addr_location *al __maybe_unused)
{
dSP;
@@ -363,7 +360,7 @@
ENTER;
SAVETMPS;
PUSHMARK(SP);
- XPUSHs(sv_2mortal(newSVpvn((const char *)pevent, pevent->header.size)));
+ XPUSHs(sv_2mortal(newSVpvn((const char *)event, event->header.size)));
XPUSHs(sv_2mortal(newSVpvn((const char *)&evsel->attr, sizeof(evsel->attr))));
XPUSHs(sv_2mortal(newSVpvn((const char *)sample, sizeof(*sample))));
XPUSHs(sv_2mortal(newSVpvn((const char *)sample->raw_data, sample->raw_size)));
@@ -376,14 +373,13 @@
}
static void perl_process_event(union perf_event *event,
- struct pevent *pevent,
struct perf_sample *sample,
struct perf_evsel *evsel,
struct machine *machine,
- struct thread *thread)
+ struct addr_location *al)
{
- perl_process_tracepoint(event, pevent, sample, evsel, machine, thread);
- perl_process_event_generic(event, sample, evsel, machine, thread);
+ perl_process_tracepoint(event, sample, evsel, machine, al);
+ perl_process_event_generic(event, sample, evsel, machine, al);
}
static void run_start_sub(void)
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index ce4d1b0..730c663 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -27,10 +27,12 @@
#include <errno.h>
#include "../../perf.h"
+#include "../evsel.h"
#include "../util.h"
#include "../event.h"
#include "../thread.h"
#include "../trace-event.h"
+#include "../evsel.h"
PyMODINIT_FUNC initperf_trace_context(void);
@@ -194,16 +196,21 @@
define_event_symbols(event, ev_name, args->next);
}
-static inline
-struct event_format *find_cache_event(struct pevent *pevent, int type)
+static inline struct event_format *find_cache_event(struct perf_evsel *evsel)
{
static char ev_name[256];
struct event_format *event;
+ int type = evsel->attr.config;
+ /*
+ * XXX: Do we really need to cache this since now we have evsel->tp_format
+ * cached already? Need to re-read this "cache" routine that as well calls
+ * define_event_symbols() :-\
+ */
if (events[type])
return events[type];
- events[type] = event = pevent_find_event(pevent, type);
+ events[type] = event = evsel->tp_format;
if (!event)
return NULL;
@@ -214,12 +221,12 @@
return event;
}
-static void python_process_event(union perf_event *perf_event __unused,
- struct pevent *pevent,
+static void python_process_tracepoint(union perf_event *perf_event
+ __maybe_unused,
struct perf_sample *sample,
- struct perf_evsel *evsel __unused,
- struct machine *machine __unused,
- struct thread *thread)
+ struct perf_evsel *evsel,
+ struct machine *machine __maybe_unused,
+ struct addr_location *al)
{
PyObject *handler, *retval, *context, *t, *obj, *dict = NULL;
static char handler_name[256];
@@ -228,24 +235,22 @@
unsigned long s, ns;
struct event_format *event;
unsigned n = 0;
- int type;
int pid;
int cpu = sample->cpu;
void *data = sample->raw_data;
unsigned long long nsecs = sample->time;
+ struct thread *thread = al->thread;
char *comm = thread->comm;
t = PyTuple_New(MAX_FIELDS);
if (!t)
Py_FatalError("couldn't create Python tuple");
- type = trace_parse_common_type(pevent, data);
-
- event = find_cache_event(pevent, type);
+ event = find_cache_event(evsel);
if (!event)
- die("ug! no event found for type %d", type);
+ die("ug! no event found for type %d", (int)evsel->attr.config);
- pid = trace_parse_common_pid(pevent, data);
+ pid = raw_field_value(event, "common_pid", data);
sprintf(handler_name, "%s__%s", event->system, event->name);
@@ -290,7 +295,7 @@
offset = field->offset;
obj = PyString_FromString((char *)data + offset);
} else { /* FIELD_IS_NUMERIC */
- val = read_size(pevent, data + field->offset,
+ val = read_size(event, data + field->offset,
field->size);
if (field->flags & FIELD_IS_SIGNED) {
if ((long long)val >= LONG_MIN &&
@@ -335,6 +340,84 @@
Py_DECREF(t);
}
+static void python_process_general_event(union perf_event *perf_event
+ __maybe_unused,
+ struct perf_sample *sample,
+ struct perf_evsel *evsel,
+ struct machine *machine __maybe_unused,
+ struct addr_location *al)
+{
+ PyObject *handler, *retval, *t, *dict;
+ static char handler_name[64];
+ unsigned n = 0;
+ struct thread *thread = al->thread;
+
+ /*
+ * Use the MAX_FIELDS to make the function expandable, though
+ * currently there is only one item for the tuple.
+ */
+ t = PyTuple_New(MAX_FIELDS);
+ if (!t)
+ Py_FatalError("couldn't create Python tuple");
+
+ dict = PyDict_New();
+ if (!dict)
+ Py_FatalError("couldn't create Python dictionary");
+
+ snprintf(handler_name, sizeof(handler_name), "%s", "process_event");
+
+ handler = PyDict_GetItemString(main_dict, handler_name);
+ if (!handler || !PyCallable_Check(handler))
+ goto exit;
+
+ PyDict_SetItemString(dict, "ev_name", PyString_FromString(perf_evsel__name(evsel)));
+ PyDict_SetItemString(dict, "attr", PyString_FromStringAndSize(
+ (const char *)&evsel->attr, sizeof(evsel->attr)));
+ PyDict_SetItemString(dict, "sample", PyString_FromStringAndSize(
+ (const char *)sample, sizeof(*sample)));
+ PyDict_SetItemString(dict, "raw_buf", PyString_FromStringAndSize(
+ (const char *)sample->raw_data, sample->raw_size));
+ PyDict_SetItemString(dict, "comm",
+ PyString_FromString(thread->comm));
+ if (al->map) {
+ PyDict_SetItemString(dict, "dso",
+ PyString_FromString(al->map->dso->name));
+ }
+ if (al->sym) {
+ PyDict_SetItemString(dict, "symbol",
+ PyString_FromString(al->sym->name));
+ }
+
+ PyTuple_SetItem(t, n++, dict);
+ if (_PyTuple_Resize(&t, n) == -1)
+ Py_FatalError("error resizing Python tuple");
+
+ retval = PyObject_CallObject(handler, t);
+ if (retval == NULL)
+ handler_call_die(handler_name);
+exit:
+ Py_DECREF(dict);
+ Py_DECREF(t);
+}
+
+static void python_process_event(union perf_event *perf_event,
+ struct perf_sample *sample,
+ struct perf_evsel *evsel,
+ struct machine *machine,
+ struct addr_location *al)
+{
+ switch (evsel->attr.type) {
+ case PERF_TYPE_TRACEPOINT:
+ python_process_tracepoint(perf_event, sample, evsel,
+ machine, al);
+ break;
+ /* Reserve for future process_hw/sw/raw APIs */
+ default:
+ python_process_general_event(perf_event, sample, evsel,
+ machine, al);
+ }
+}
+
static int run_start_sub(void)
{
PyObject *handler, *retval;
diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index 2437fb0..8cdd232 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -15,6 +15,9 @@
#include "util.h"
#include "cpumap.h"
#include "event-parse.h"
+#include "perf_regs.h"
+#include "unwind.h"
+#include "vdso.h"
static int perf_session__open(struct perf_session *self, bool force)
{
@@ -209,6 +212,7 @@
machine__exit(&self->host_machine);
close(self->fd);
free(self);
+ vdso__exit();
}
void machine__remove_thread(struct machine *self, struct thread *th)
@@ -288,10 +292,11 @@
return bi;
}
-int machine__resolve_callchain(struct machine *self,
- struct thread *thread,
- struct ip_callchain *chain,
- struct symbol **parent)
+static int machine__resolve_callchain_sample(struct machine *machine,
+ struct thread *thread,
+ struct ip_callchain *chain,
+ struct symbol **parent)
+
{
u8 cpumode = PERF_RECORD_MISC_USER;
unsigned int i;
@@ -316,11 +321,14 @@
if (ip >= PERF_CONTEXT_MAX) {
switch (ip) {
case PERF_CONTEXT_HV:
- cpumode = PERF_RECORD_MISC_HYPERVISOR; break;
+ cpumode = PERF_RECORD_MISC_HYPERVISOR;
+ break;
case PERF_CONTEXT_KERNEL:
- cpumode = PERF_RECORD_MISC_KERNEL; break;
+ cpumode = PERF_RECORD_MISC_KERNEL;
+ break;
case PERF_CONTEXT_USER:
- cpumode = PERF_RECORD_MISC_USER; break;
+ cpumode = PERF_RECORD_MISC_USER;
+ break;
default:
pr_debug("invalid callchain context: "
"%"PRId64"\n", (s64) ip);
@@ -335,7 +343,7 @@
}
al.filtered = false;
- thread__find_addr_location(thread, self, cpumode,
+ thread__find_addr_location(thread, machine, cpumode,
MAP__FUNCTION, ip, &al, NULL);
if (al.sym != NULL) {
if (sort__has_parent && !*parent &&
@@ -354,49 +362,92 @@
return 0;
}
-static int process_event_synth_tracing_data_stub(union perf_event *event __used,
- struct perf_session *session __used)
+static int unwind_entry(struct unwind_entry *entry, void *arg)
+{
+ struct callchain_cursor *cursor = arg;
+ return callchain_cursor_append(cursor, entry->ip,
+ entry->map, entry->sym);
+}
+
+int machine__resolve_callchain(struct machine *machine,
+ struct perf_evsel *evsel,
+ struct thread *thread,
+ struct perf_sample *sample,
+ struct symbol **parent)
+
+{
+ int ret;
+
+ callchain_cursor_reset(&callchain_cursor);
+
+ ret = machine__resolve_callchain_sample(machine, thread,
+ sample->callchain, parent);
+ if (ret)
+ return ret;
+
+ /* Can we do dwarf post unwind? */
+ if (!((evsel->attr.sample_type & PERF_SAMPLE_REGS_USER) &&
+ (evsel->attr.sample_type & PERF_SAMPLE_STACK_USER)))
+ return 0;
+
+ /* Bail out if nothing was captured. */
+ if ((!sample->user_regs.regs) ||
+ (!sample->user_stack.size))
+ return 0;
+
+ return unwind__get_entries(unwind_entry, &callchain_cursor, machine,
+ thread, evsel->attr.sample_regs_user,
+ sample);
+
+}
+
+static int process_event_synth_tracing_data_stub(union perf_event *event
+ __maybe_unused,
+ struct perf_session *session
+ __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
}
-static int process_event_synth_attr_stub(union perf_event *event __used,
- struct perf_evlist **pevlist __used)
+static int process_event_synth_attr_stub(union perf_event *event __maybe_unused,
+ struct perf_evlist **pevlist
+ __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
}
-static int process_event_sample_stub(struct perf_tool *tool __used,
- union perf_event *event __used,
- struct perf_sample *sample __used,
- struct perf_evsel *evsel __used,
- struct machine *machine __used)
+static int process_event_sample_stub(struct perf_tool *tool __maybe_unused,
+ union perf_event *event __maybe_unused,
+ struct perf_sample *sample __maybe_unused,
+ struct perf_evsel *evsel __maybe_unused,
+ struct machine *machine __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
}
-static int process_event_stub(struct perf_tool *tool __used,
- union perf_event *event __used,
- struct perf_sample *sample __used,
- struct machine *machine __used)
+static int process_event_stub(struct perf_tool *tool __maybe_unused,
+ union perf_event *event __maybe_unused,
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
}
-static int process_finished_round_stub(struct perf_tool *tool __used,
- union perf_event *event __used,
- struct perf_session *perf_session __used)
+static int process_finished_round_stub(struct perf_tool *tool __maybe_unused,
+ union perf_event *event __maybe_unused,
+ struct perf_session *perf_session
+ __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
}
-static int process_event_type_stub(struct perf_tool *tool __used,
- union perf_event *event __used)
+static int process_event_type_stub(struct perf_tool *tool __maybe_unused,
+ union perf_event *event __maybe_unused)
{
dump_printf(": unhandled!\n");
return 0;
@@ -473,7 +524,7 @@
}
static void perf_event__all64_swap(union perf_event *event,
- bool sample_id_all __used)
+ bool sample_id_all __maybe_unused)
{
struct perf_event_header *hdr = &event->header;
mem_bswap_64(hdr + 1, event->header.size - sizeof(*hdr));
@@ -487,7 +538,7 @@
if (sample_id_all) {
void *data = &event->comm.comm;
- data += ALIGN(strlen(data) + 1, sizeof(u64));
+ data += PERF_ALIGN(strlen(data) + 1, sizeof(u64));
swap_sample_id_all(event, data);
}
}
@@ -504,7 +555,7 @@
if (sample_id_all) {
void *data = &event->mmap.filename;
- data += ALIGN(strlen(data) + 1, sizeof(u64));
+ data += PERF_ALIGN(strlen(data) + 1, sizeof(u64));
swap_sample_id_all(event, data);
}
}
@@ -584,7 +635,7 @@
}
static void perf_event__hdr_attr_swap(union perf_event *event,
- bool sample_id_all __used)
+ bool sample_id_all __maybe_unused)
{
size_t size;
@@ -596,14 +647,14 @@
}
static void perf_event__event_type_swap(union perf_event *event,
- bool sample_id_all __used)
+ bool sample_id_all __maybe_unused)
{
event->event_type.event_type.event_id =
bswap_64(event->event_type.event_type.event_id);
}
static void perf_event__tracing_data_swap(union perf_event *event,
- bool sample_id_all __used)
+ bool sample_id_all __maybe_unused)
{
event->tracing_data.size = bswap_32(event->tracing_data.size);
}
@@ -652,7 +703,7 @@
struct perf_tool *tool,
u64 file_offset);
-static void flush_sample_queue(struct perf_session *s,
+static int flush_sample_queue(struct perf_session *s,
struct perf_tool *tool)
{
struct ordered_samples *os = &s->ordered_samples;
@@ -665,19 +716,21 @@
int ret;
if (!tool->ordered_samples || !limit)
- return;
+ return 0;
list_for_each_entry_safe(iter, tmp, head, list) {
if (iter->timestamp > limit)
break;
- ret = perf_evlist__parse_sample(s->evlist, iter->event, &sample,
- s->header.needs_swap);
+ ret = perf_evlist__parse_sample(s->evlist, iter->event, &sample);
if (ret)
pr_err("Can't parse sample, err = %d\n", ret);
- else
- perf_session_deliver_event(s, iter->event, &sample, tool,
- iter->file_offset);
+ else {
+ ret = perf_session_deliver_event(s, iter->event, &sample, tool,
+ iter->file_offset);
+ if (ret)
+ return ret;
+ }
os->last_flush = iter->timestamp;
list_del(&iter->list);
@@ -697,6 +750,8 @@
}
os->nr_samples = 0;
+
+ return 0;
}
/*
@@ -739,13 +794,14 @@
* etc...
*/
static int process_finished_round(struct perf_tool *tool,
- union perf_event *event __used,
+ union perf_event *event __maybe_unused,
struct perf_session *session)
{
- flush_sample_queue(session, tool);
- session->ordered_samples.next_flush = session->ordered_samples.max_timestamp;
+ int ret = flush_sample_queue(session, tool);
+ if (!ret)
+ session->ordered_samples.next_flush = session->ordered_samples.max_timestamp;
- return 0;
+ return ret;
}
/* The queue is ordered by time */
@@ -860,6 +916,34 @@
sample->branch_stack->entries[i].to);
}
+static void regs_dump__printf(u64 mask, u64 *regs)
+{
+ unsigned rid, i = 0;
+
+ for_each_set_bit(rid, (unsigned long *) &mask, sizeof(mask) * 8) {
+ u64 val = regs[i++];
+
+ printf(".... %-5s 0x%" PRIx64 "\n",
+ perf_reg_name(rid), val);
+ }
+}
+
+static void regs_user__printf(struct perf_sample *sample, u64 mask)
+{
+ struct regs_dump *user_regs = &sample->user_regs;
+
+ if (user_regs->regs) {
+ printf("... user regs: mask 0x%" PRIx64 "\n", mask);
+ regs_dump__printf(mask, user_regs->regs);
+ }
+}
+
+static void stack_user__printf(struct stack_dump *dump)
+{
+ printf("... ustack: size %" PRIu64 ", offset 0x%x\n",
+ dump->size, dump->offset);
+}
+
static void perf_session__print_tstamp(struct perf_session *session,
union perf_event *event,
struct perf_sample *sample)
@@ -897,7 +981,7 @@
event->header.size, perf_event__name(event->header.type));
}
-static void dump_sample(struct perf_session *session, union perf_event *event,
+static void dump_sample(struct perf_evsel *evsel, union perf_event *event,
struct perf_sample *sample)
{
u64 sample_type;
@@ -909,13 +993,19 @@
event->header.misc, sample->pid, sample->tid, sample->ip,
sample->period, sample->addr);
- sample_type = perf_evlist__sample_type(session->evlist);
+ sample_type = evsel->attr.sample_type;
if (sample_type & PERF_SAMPLE_CALLCHAIN)
callchain__printf(sample);
if (sample_type & PERF_SAMPLE_BRANCH_STACK)
branch_stack__printf(sample);
+
+ if (sample_type & PERF_SAMPLE_REGS_USER)
+ regs_user__printf(sample, evsel->attr.sample_regs_user);
+
+ if (sample_type & PERF_SAMPLE_STACK_USER)
+ stack_user__printf(&sample->user_stack);
}
static struct machine *
@@ -973,7 +1063,7 @@
switch (event->header.type) {
case PERF_RECORD_SAMPLE:
- dump_sample(session, event, sample);
+ dump_sample(evsel, event, sample);
if (evsel == NULL) {
++session->hists.stats.nr_unknown_id;
return 0;
@@ -1083,8 +1173,7 @@
/*
* For all kernel events we get the sample data
*/
- ret = perf_evlist__parse_sample(session->evlist, event, &sample,
- session->header.needs_swap);
+ ret = perf_evlist__parse_sample(session->evlist, event, &sample);
if (ret)
return ret;
@@ -1369,7 +1458,7 @@
err = 0;
/* do the final flush for ordered samples */
session->ordered_samples.next_flush = ULLONG_MAX;
- flush_sample_queue(session, tool);
+ err = flush_sample_queue(session, tool);
out_err:
perf_session__warn_about_errors(session, tool);
perf_session_free_sample_buffers(session);
@@ -1498,9 +1587,9 @@
return NULL;
}
-void perf_event__print_ip(union perf_event *event, struct perf_sample *sample,
- struct machine *machine, int print_sym,
- int print_dso, int print_symoffset)
+void perf_evsel__print_ip(struct perf_evsel *evsel, union perf_event *event,
+ struct perf_sample *sample, struct machine *machine,
+ int print_sym, int print_dso, int print_symoffset)
{
struct addr_location al;
struct callchain_cursor_node *node;
@@ -1514,8 +1603,9 @@
if (symbol_conf.use_callchain && sample->callchain) {
- if (machine__resolve_callchain(machine, al.thread,
- sample->callchain, NULL) != 0) {
+
+ if (machine__resolve_callchain(machine, evsel, al.thread,
+ sample, NULL) != 0) {
if (verbose)
error("Failed to resolve callchain. Skipping\n");
return;
diff --git a/tools/perf/util/session.h b/tools/perf/util/session.h
index 1f7ec87..aab414f 100644
--- a/tools/perf/util/session.h
+++ b/tools/perf/util/session.h
@@ -36,9 +36,7 @@
struct pevent *pevent;
/*
* FIXME: Need to split this up further, we need global
- * stats + per event stats. 'perf diff' also needs
- * to properly support multiple events in a single
- * perf.data file.
+ * stats + per event stats.
*/
struct hists hists;
int fd;
@@ -129,9 +127,9 @@
struct perf_evsel *perf_session__find_first_evtype(struct perf_session *session,
unsigned int type);
-void perf_event__print_ip(union perf_event *event, struct perf_sample *sample,
- struct machine *machine, int print_sym,
- int print_dso, int print_symoffset);
+void perf_evsel__print_ip(struct perf_evsel *evsel, union perf_event *event,
+ struct perf_sample *sample, struct machine *machine,
+ int print_sym, int print_dso, int print_symoffset);
int perf_session__cpu_bitmap(struct perf_session *session,
const char *cpu_list, unsigned long *cpu_bitmap);
diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c
index 0f5a0a4..b5b1b92 100644
--- a/tools/perf/util/sort.c
+++ b/tools/perf/util/sort.c
@@ -8,12 +8,11 @@
const char *sort_order = default_sort_order;
int sort__need_collapse = 0;
int sort__has_parent = 0;
+int sort__has_sym = 0;
int sort__branch_mode = -1; /* -1 = means not set */
enum sort_type sort__first_dimension;
-char * field_sep;
-
LIST_HEAD(hist_entry__sort_list);
static int repsep_snprintf(char *bf, size_t size, const char *fmt, ...)
@@ -23,11 +22,11 @@
va_start(ap, fmt);
n = vsnprintf(bf, size, fmt, ap);
- if (field_sep && n > 0) {
+ if (symbol_conf.field_sep && n > 0) {
char *sep = bf;
while (1) {
- sep = strchr(sep, *field_sep);
+ sep = strchr(sep, *symbol_conf.field_sep);
if (sep == NULL)
break;
*sep = '.';
@@ -172,7 +171,7 @@
static int _hist_entry__sym_snprintf(struct map *map, struct symbol *sym,
u64 ip, char level, char *bf, size_t size,
- unsigned int width __used)
+ unsigned int width __maybe_unused)
{
size_t ret = 0;
@@ -207,7 +206,8 @@
};
static int hist_entry__sym_snprintf(struct hist_entry *self, char *bf,
- size_t size, unsigned int width __used)
+ size_t size,
+ unsigned int width __maybe_unused)
{
return _hist_entry__sym_snprintf(self->ms.map, self->ms.sym, self->ip,
self->level, bf, size, width);
@@ -250,7 +250,8 @@
}
static int hist_entry__srcline_snprintf(struct hist_entry *self, char *bf,
- size_t size, unsigned int width __used)
+ size_t size,
+ unsigned int width __maybe_unused)
{
FILE *fp;
char cmd[PATH_MAX + 2], *path = self->srcline, *nl;
@@ -399,7 +400,8 @@
}
static int hist_entry__sym_from_snprintf(struct hist_entry *self, char *bf,
- size_t size, unsigned int width __used)
+ size_t size,
+ unsigned int width __maybe_unused)
{
struct addr_map_symbol *from = &self->branch_info->from;
return _hist_entry__sym_snprintf(from->map, from->sym, from->addr,
@@ -408,7 +410,8 @@
}
static int hist_entry__sym_to_snprintf(struct hist_entry *self, char *bf,
- size_t size, unsigned int width __used)
+ size_t size,
+ unsigned int width __maybe_unused)
{
struct addr_map_symbol *to = &self->branch_info->to;
return _hist_entry__sym_snprintf(to->map, to->sym, to->addr,
@@ -509,6 +512,10 @@
return -EINVAL;
}
sort__has_parent = 1;
+ } else if (sd->entry == &sort_sym ||
+ sd->entry == &sort_sym_from ||
+ sd->entry == &sort_sym_to) {
+ sort__has_sym = 1;
}
if (sd->taken)
diff --git a/tools/perf/util/sort.h b/tools/perf/util/sort.h
index e724b26..12d6347 100644
--- a/tools/perf/util/sort.h
+++ b/tools/perf/util/sort.h
@@ -31,8 +31,8 @@
extern const char default_sort_order[];
extern int sort__need_collapse;
extern int sort__has_parent;
+extern int sort__has_sym;
extern int sort__branch_mode;
-extern char *field_sep;
extern struct sort_entry sort_comm;
extern struct sort_entry sort_dso;
extern struct sort_entry sort_sym;
diff --git a/tools/perf/util/stat.c b/tools/perf/util/stat.c
new file mode 100644
index 0000000..2374212
--- /dev/null
+++ b/tools/perf/util/stat.c
@@ -0,0 +1,57 @@
+#include <math.h>
+
+#include "stat.h"
+
+void update_stats(struct stats *stats, u64 val)
+{
+ double delta;
+
+ stats->n++;
+ delta = val - stats->mean;
+ stats->mean += delta / stats->n;
+ stats->M2 += delta*(val - stats->mean);
+}
+
+double avg_stats(struct stats *stats)
+{
+ return stats->mean;
+}
+
+/*
+ * http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
+ *
+ * (\Sum n_i^2) - ((\Sum n_i)^2)/n
+ * s^2 = -------------------------------
+ * n - 1
+ *
+ * http://en.wikipedia.org/wiki/Stddev
+ *
+ * The std dev of the mean is related to the std dev by:
+ *
+ * s
+ * s_mean = -------
+ * sqrt(n)
+ *
+ */
+double stddev_stats(struct stats *stats)
+{
+ double variance, variance_mean;
+
+ if (!stats->n)
+ return 0.0;
+
+ variance = stats->M2 / (stats->n - 1);
+ variance_mean = variance / stats->n;
+
+ return sqrt(variance_mean);
+}
+
+double rel_stddev_stats(double stddev, double avg)
+{
+ double pct = 0.0;
+
+ if (avg)
+ pct = 100.0 * stddev/avg;
+
+ return pct;
+}
diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h
new file mode 100644
index 0000000..588367c
--- /dev/null
+++ b/tools/perf/util/stat.h
@@ -0,0 +1,16 @@
+#ifndef __PERF_STATS_H
+#define __PERF_STATS_H
+
+#include "types.h"
+
+struct stats
+{
+ double n, mean, M2;
+};
+
+void update_stats(struct stats *stats, u64 val);
+double avg_stats(struct stats *stats);
+double stddev_stats(struct stats *stats);
+double rel_stddev_stats(double stddev, double avg);
+
+#endif
diff --git a/tools/perf/util/string.c b/tools/perf/util/string.c
index 199bc4d..3217059 100644
--- a/tools/perf/util/string.c
+++ b/tools/perf/util/string.c
@@ -1,5 +1,5 @@
#include "util.h"
-#include "string.h"
+#include "linux/string.h"
#define K 1024LL
/*
@@ -335,3 +335,19 @@
return s;
}
+
+/**
+ * memdup - duplicate region of memory
+ * @src: memory region to duplicate
+ * @len: memory region length
+ */
+void *memdup(const void *src, size_t len)
+{
+ void *p;
+
+ p = malloc(len);
+ if (p)
+ memcpy(p, src, len);
+
+ return p;
+}
diff --git a/tools/perf/util/strlist.c b/tools/perf/util/strlist.c
index 95856ff..155d8b7 100644
--- a/tools/perf/util/strlist.c
+++ b/tools/perf/util/strlist.c
@@ -93,7 +93,7 @@
void strlist__remove(struct strlist *slist, struct str_node *snode)
{
- str_node__delete(snode, slist->dupstr);
+ rblist__remove_node(&slist->rblist, &snode->rb_node);
}
struct str_node *strlist__find(struct strlist *slist, const char *entry)
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
new file mode 100644
index 0000000..db0cc92
--- /dev/null
+++ b/tools/perf/util/symbol-elf.c
@@ -0,0 +1,841 @@
+#include <libelf.h>
+#include <gelf.h>
+#include <elf.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <unistd.h>
+#include <inttypes.h>
+
+#include "symbol.h"
+#include "debug.h"
+
+#ifndef NT_GNU_BUILD_ID
+#define NT_GNU_BUILD_ID 3
+#endif
+
+/**
+ * elf_symtab__for_each_symbol - iterate thru all the symbols
+ *
+ * @syms: struct elf_symtab instance to iterate
+ * @idx: uint32_t idx
+ * @sym: GElf_Sym iterator
+ */
+#define elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) \
+ for (idx = 0, gelf_getsym(syms, idx, &sym);\
+ idx < nr_syms; \
+ idx++, gelf_getsym(syms, idx, &sym))
+
+static inline uint8_t elf_sym__type(const GElf_Sym *sym)
+{
+ return GELF_ST_TYPE(sym->st_info);
+}
+
+static inline int elf_sym__is_function(const GElf_Sym *sym)
+{
+ return elf_sym__type(sym) == STT_FUNC &&
+ sym->st_name != 0 &&
+ sym->st_shndx != SHN_UNDEF;
+}
+
+static inline bool elf_sym__is_object(const GElf_Sym *sym)
+{
+ return elf_sym__type(sym) == STT_OBJECT &&
+ sym->st_name != 0 &&
+ sym->st_shndx != SHN_UNDEF;
+}
+
+static inline int elf_sym__is_label(const GElf_Sym *sym)
+{
+ return elf_sym__type(sym) == STT_NOTYPE &&
+ sym->st_name != 0 &&
+ sym->st_shndx != SHN_UNDEF &&
+ sym->st_shndx != SHN_ABS;
+}
+
+static bool elf_sym__is_a(GElf_Sym *sym, enum map_type type)
+{
+ switch (type) {
+ case MAP__FUNCTION:
+ return elf_sym__is_function(sym);
+ case MAP__VARIABLE:
+ return elf_sym__is_object(sym);
+ default:
+ return false;
+ }
+}
+
+static inline const char *elf_sym__name(const GElf_Sym *sym,
+ const Elf_Data *symstrs)
+{
+ return symstrs->d_buf + sym->st_name;
+}
+
+static inline const char *elf_sec__name(const GElf_Shdr *shdr,
+ const Elf_Data *secstrs)
+{
+ return secstrs->d_buf + shdr->sh_name;
+}
+
+static inline int elf_sec__is_text(const GElf_Shdr *shdr,
+ const Elf_Data *secstrs)
+{
+ return strstr(elf_sec__name(shdr, secstrs), "text") != NULL;
+}
+
+static inline bool elf_sec__is_data(const GElf_Shdr *shdr,
+ const Elf_Data *secstrs)
+{
+ return strstr(elf_sec__name(shdr, secstrs), "data") != NULL;
+}
+
+static bool elf_sec__is_a(GElf_Shdr *shdr, Elf_Data *secstrs,
+ enum map_type type)
+{
+ switch (type) {
+ case MAP__FUNCTION:
+ return elf_sec__is_text(shdr, secstrs);
+ case MAP__VARIABLE:
+ return elf_sec__is_data(shdr, secstrs);
+ default:
+ return false;
+ }
+}
+
+static size_t elf_addr_to_index(Elf *elf, GElf_Addr addr)
+{
+ Elf_Scn *sec = NULL;
+ GElf_Shdr shdr;
+ size_t cnt = 1;
+
+ while ((sec = elf_nextscn(elf, sec)) != NULL) {
+ gelf_getshdr(sec, &shdr);
+
+ if ((addr >= shdr.sh_addr) &&
+ (addr < (shdr.sh_addr + shdr.sh_size)))
+ return cnt;
+
+ ++cnt;
+ }
+
+ return -1;
+}
+
+static Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
+ GElf_Shdr *shp, const char *name,
+ size_t *idx)
+{
+ Elf_Scn *sec = NULL;
+ size_t cnt = 1;
+
+ /* Elf is corrupted/truncated, avoid calling elf_strptr. */
+ if (!elf_rawdata(elf_getscn(elf, ep->e_shstrndx), NULL))
+ return NULL;
+
+ while ((sec = elf_nextscn(elf, sec)) != NULL) {
+ char *str;
+
+ gelf_getshdr(sec, shp);
+ str = elf_strptr(elf, ep->e_shstrndx, shp->sh_name);
+ if (!strcmp(name, str)) {
+ if (idx)
+ *idx = cnt;
+ break;
+ }
+ ++cnt;
+ }
+
+ return sec;
+}
+
+#define elf_section__for_each_rel(reldata, pos, pos_mem, idx, nr_entries) \
+ for (idx = 0, pos = gelf_getrel(reldata, 0, &pos_mem); \
+ idx < nr_entries; \
+ ++idx, pos = gelf_getrel(reldata, idx, &pos_mem))
+
+#define elf_section__for_each_rela(reldata, pos, pos_mem, idx, nr_entries) \
+ for (idx = 0, pos = gelf_getrela(reldata, 0, &pos_mem); \
+ idx < nr_entries; \
+ ++idx, pos = gelf_getrela(reldata, idx, &pos_mem))
+
+/*
+ * We need to check if we have a .dynsym, so that we can handle the
+ * .plt, synthesizing its symbols, that aren't on the symtabs (be it
+ * .dynsym or .symtab).
+ * And always look at the original dso, not at debuginfo packages, that
+ * have the PLT data stripped out (shdr_rel_plt.sh_type == SHT_NOBITS).
+ */
+int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss, struct map *map,
+ symbol_filter_t filter)
+{
+ uint32_t nr_rel_entries, idx;
+ GElf_Sym sym;
+ u64 plt_offset;
+ GElf_Shdr shdr_plt;
+ struct symbol *f;
+ GElf_Shdr shdr_rel_plt, shdr_dynsym;
+ Elf_Data *reldata, *syms, *symstrs;
+ Elf_Scn *scn_plt_rel, *scn_symstrs, *scn_dynsym;
+ size_t dynsym_idx;
+ GElf_Ehdr ehdr;
+ char sympltname[1024];
+ Elf *elf;
+ int nr = 0, symidx, err = 0;
+
+ if (!ss->dynsym)
+ return 0;
+
+ elf = ss->elf;
+ ehdr = ss->ehdr;
+
+ scn_dynsym = ss->dynsym;
+ shdr_dynsym = ss->dynshdr;
+ dynsym_idx = ss->dynsym_idx;
+
+ if (scn_dynsym == NULL)
+ goto out_elf_end;
+
+ scn_plt_rel = elf_section_by_name(elf, &ehdr, &shdr_rel_plt,
+ ".rela.plt", NULL);
+ if (scn_plt_rel == NULL) {
+ scn_plt_rel = elf_section_by_name(elf, &ehdr, &shdr_rel_plt,
+ ".rel.plt", NULL);
+ if (scn_plt_rel == NULL)
+ goto out_elf_end;
+ }
+
+ err = -1;
+
+ if (shdr_rel_plt.sh_link != dynsym_idx)
+ goto out_elf_end;
+
+ if (elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL) == NULL)
+ goto out_elf_end;
+
+ /*
+ * Fetch the relocation section to find the idxes to the GOT
+ * and the symbols in the .dynsym they refer to.
+ */
+ reldata = elf_getdata(scn_plt_rel, NULL);
+ if (reldata == NULL)
+ goto out_elf_end;
+
+ syms = elf_getdata(scn_dynsym, NULL);
+ if (syms == NULL)
+ goto out_elf_end;
+
+ scn_symstrs = elf_getscn(elf, shdr_dynsym.sh_link);
+ if (scn_symstrs == NULL)
+ goto out_elf_end;
+
+ symstrs = elf_getdata(scn_symstrs, NULL);
+ if (symstrs == NULL)
+ goto out_elf_end;
+
+ if (symstrs->d_size == 0)
+ goto out_elf_end;
+
+ nr_rel_entries = shdr_rel_plt.sh_size / shdr_rel_plt.sh_entsize;
+ plt_offset = shdr_plt.sh_offset;
+
+ if (shdr_rel_plt.sh_type == SHT_RELA) {
+ GElf_Rela pos_mem, *pos;
+
+ elf_section__for_each_rela(reldata, pos, pos_mem, idx,
+ nr_rel_entries) {
+ symidx = GELF_R_SYM(pos->r_info);
+ plt_offset += shdr_plt.sh_entsize;
+ gelf_getsym(syms, symidx, &sym);
+ snprintf(sympltname, sizeof(sympltname),
+ "%s@plt", elf_sym__name(&sym, symstrs));
+
+ f = symbol__new(plt_offset, shdr_plt.sh_entsize,
+ STB_GLOBAL, sympltname);
+ if (!f)
+ goto out_elf_end;
+
+ if (filter && filter(map, f))
+ symbol__delete(f);
+ else {
+ symbols__insert(&dso->symbols[map->type], f);
+ ++nr;
+ }
+ }
+ } else if (shdr_rel_plt.sh_type == SHT_REL) {
+ GElf_Rel pos_mem, *pos;
+ elf_section__for_each_rel(reldata, pos, pos_mem, idx,
+ nr_rel_entries) {
+ symidx = GELF_R_SYM(pos->r_info);
+ plt_offset += shdr_plt.sh_entsize;
+ gelf_getsym(syms, symidx, &sym);
+ snprintf(sympltname, sizeof(sympltname),
+ "%s@plt", elf_sym__name(&sym, symstrs));
+
+ f = symbol__new(plt_offset, shdr_plt.sh_entsize,
+ STB_GLOBAL, sympltname);
+ if (!f)
+ goto out_elf_end;
+
+ if (filter && filter(map, f))
+ symbol__delete(f);
+ else {
+ symbols__insert(&dso->symbols[map->type], f);
+ ++nr;
+ }
+ }
+ }
+
+ err = 0;
+out_elf_end:
+ if (err == 0)
+ return nr;
+ pr_debug("%s: problems reading %s PLT info.\n",
+ __func__, dso->long_name);
+ return 0;
+}
+
+/*
+ * Align offset to 4 bytes as needed for note name and descriptor data.
+ */
+#define NOTE_ALIGN(n) (((n) + 3) & -4U)
+
+static int elf_read_build_id(Elf *elf, void *bf, size_t size)
+{
+ int err = -1;
+ GElf_Ehdr ehdr;
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ Elf_Scn *sec;
+ Elf_Kind ek;
+ void *ptr;
+
+ if (size < BUILD_ID_SIZE)
+ goto out;
+
+ ek = elf_kind(elf);
+ if (ek != ELF_K_ELF)
+ goto out;
+
+ if (gelf_getehdr(elf, &ehdr) == NULL) {
+ pr_err("%s: cannot get elf header.\n", __func__);
+ goto out;
+ }
+
+ /*
+ * Check following sections for notes:
+ * '.note.gnu.build-id'
+ * '.notes'
+ * '.note' (VDSO specific)
+ */
+ do {
+ sec = elf_section_by_name(elf, &ehdr, &shdr,
+ ".note.gnu.build-id", NULL);
+ if (sec)
+ break;
+
+ sec = elf_section_by_name(elf, &ehdr, &shdr,
+ ".notes", NULL);
+ if (sec)
+ break;
+
+ sec = elf_section_by_name(elf, &ehdr, &shdr,
+ ".note", NULL);
+ if (sec)
+ break;
+
+ return err;
+
+ } while (0);
+
+ data = elf_getdata(sec, NULL);
+ if (data == NULL)
+ goto out;
+
+ ptr = data->d_buf;
+ while (ptr < (data->d_buf + data->d_size)) {
+ GElf_Nhdr *nhdr = ptr;
+ size_t namesz = NOTE_ALIGN(nhdr->n_namesz),
+ descsz = NOTE_ALIGN(nhdr->n_descsz);
+ const char *name;
+
+ ptr += sizeof(*nhdr);
+ name = ptr;
+ ptr += namesz;
+ if (nhdr->n_type == NT_GNU_BUILD_ID &&
+ nhdr->n_namesz == sizeof("GNU")) {
+ if (memcmp(name, "GNU", sizeof("GNU")) == 0) {
+ size_t sz = min(size, descsz);
+ memcpy(bf, ptr, sz);
+ memset(bf + sz, 0, size - sz);
+ err = descsz;
+ break;
+ }
+ }
+ ptr += descsz;
+ }
+
+out:
+ return err;
+}
+
+int filename__read_build_id(const char *filename, void *bf, size_t size)
+{
+ int fd, err = -1;
+ Elf *elf;
+
+ if (size < BUILD_ID_SIZE)
+ goto out;
+
+ fd = open(filename, O_RDONLY);
+ if (fd < 0)
+ goto out;
+
+ elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
+ if (elf == NULL) {
+ pr_debug2("%s: cannot read %s ELF file.\n", __func__, filename);
+ goto out_close;
+ }
+
+ err = elf_read_build_id(elf, bf, size);
+
+ elf_end(elf);
+out_close:
+ close(fd);
+out:
+ return err;
+}
+
+int sysfs__read_build_id(const char *filename, void *build_id, size_t size)
+{
+ int fd, err = -1;
+
+ if (size < BUILD_ID_SIZE)
+ goto out;
+
+ fd = open(filename, O_RDONLY);
+ if (fd < 0)
+ goto out;
+
+ while (1) {
+ char bf[BUFSIZ];
+ GElf_Nhdr nhdr;
+ size_t namesz, descsz;
+
+ if (read(fd, &nhdr, sizeof(nhdr)) != sizeof(nhdr))
+ break;
+
+ namesz = NOTE_ALIGN(nhdr.n_namesz);
+ descsz = NOTE_ALIGN(nhdr.n_descsz);
+ if (nhdr.n_type == NT_GNU_BUILD_ID &&
+ nhdr.n_namesz == sizeof("GNU")) {
+ if (read(fd, bf, namesz) != (ssize_t)namesz)
+ break;
+ if (memcmp(bf, "GNU", sizeof("GNU")) == 0) {
+ size_t sz = min(descsz, size);
+ if (read(fd, build_id, sz) == (ssize_t)sz) {
+ memset(build_id + sz, 0, size - sz);
+ err = 0;
+ break;
+ }
+ } else if (read(fd, bf, descsz) != (ssize_t)descsz)
+ break;
+ } else {
+ int n = namesz + descsz;
+ if (read(fd, bf, n) != n)
+ break;
+ }
+ }
+ close(fd);
+out:
+ return err;
+}
+
+int filename__read_debuglink(const char *filename, char *debuglink,
+ size_t size)
+{
+ int fd, err = -1;
+ Elf *elf;
+ GElf_Ehdr ehdr;
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ Elf_Scn *sec;
+ Elf_Kind ek;
+
+ fd = open(filename, O_RDONLY);
+ if (fd < 0)
+ goto out;
+
+ elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
+ if (elf == NULL) {
+ pr_debug2("%s: cannot read %s ELF file.\n", __func__, filename);
+ goto out_close;
+ }
+
+ ek = elf_kind(elf);
+ if (ek != ELF_K_ELF)
+ goto out_close;
+
+ if (gelf_getehdr(elf, &ehdr) == NULL) {
+ pr_err("%s: cannot get elf header.\n", __func__);
+ goto out_close;
+ }
+
+ sec = elf_section_by_name(elf, &ehdr, &shdr,
+ ".gnu_debuglink", NULL);
+ if (sec == NULL)
+ goto out_close;
+
+ data = elf_getdata(sec, NULL);
+ if (data == NULL)
+ goto out_close;
+
+ /* the start of this section is a zero-terminated string */
+ strncpy(debuglink, data->d_buf, size);
+
+ elf_end(elf);
+
+out_close:
+ close(fd);
+out:
+ return err;
+}
+
+static int dso__swap_init(struct dso *dso, unsigned char eidata)
+{
+ static unsigned int const endian = 1;
+
+ dso->needs_swap = DSO_SWAP__NO;
+
+ switch (eidata) {
+ case ELFDATA2LSB:
+ /* We are big endian, DSO is little endian. */
+ if (*(unsigned char const *)&endian != 1)
+ dso->needs_swap = DSO_SWAP__YES;
+ break;
+
+ case ELFDATA2MSB:
+ /* We are little endian, DSO is big endian. */
+ if (*(unsigned char const *)&endian != 0)
+ dso->needs_swap = DSO_SWAP__YES;
+ break;
+
+ default:
+ pr_err("unrecognized DSO data encoding %d\n", eidata);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+bool symsrc__possibly_runtime(struct symsrc *ss)
+{
+ return ss->dynsym || ss->opdsec;
+}
+
+bool symsrc__has_symtab(struct symsrc *ss)
+{
+ return ss->symtab != NULL;
+}
+
+void symsrc__destroy(struct symsrc *ss)
+{
+ free(ss->name);
+ elf_end(ss->elf);
+ close(ss->fd);
+}
+
+int symsrc__init(struct symsrc *ss, struct dso *dso, const char *name,
+ enum dso_binary_type type)
+{
+ int err = -1;
+ GElf_Ehdr ehdr;
+ Elf *elf;
+ int fd;
+
+ fd = open(name, O_RDONLY);
+ if (fd < 0)
+ return -1;
+
+ elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
+ if (elf == NULL) {
+ pr_debug("%s: cannot read %s ELF file.\n", __func__, name);
+ goto out_close;
+ }
+
+ if (gelf_getehdr(elf, &ehdr) == NULL) {
+ pr_debug("%s: cannot get elf header.\n", __func__);
+ goto out_elf_end;
+ }
+
+ if (dso__swap_init(dso, ehdr.e_ident[EI_DATA]))
+ goto out_elf_end;
+
+ /* Always reject images with a mismatched build-id: */
+ if (dso->has_build_id) {
+ u8 build_id[BUILD_ID_SIZE];
+
+ if (elf_read_build_id(elf, build_id, BUILD_ID_SIZE) < 0)
+ goto out_elf_end;
+
+ if (!dso__build_id_equal(dso, build_id))
+ goto out_elf_end;
+ }
+
+ ss->symtab = elf_section_by_name(elf, &ehdr, &ss->symshdr, ".symtab",
+ NULL);
+ if (ss->symshdr.sh_type != SHT_SYMTAB)
+ ss->symtab = NULL;
+
+ ss->dynsym_idx = 0;
+ ss->dynsym = elf_section_by_name(elf, &ehdr, &ss->dynshdr, ".dynsym",
+ &ss->dynsym_idx);
+ if (ss->dynshdr.sh_type != SHT_DYNSYM)
+ ss->dynsym = NULL;
+
+ ss->opdidx = 0;
+ ss->opdsec = elf_section_by_name(elf, &ehdr, &ss->opdshdr, ".opd",
+ &ss->opdidx);
+ if (ss->opdshdr.sh_type != SHT_PROGBITS)
+ ss->opdsec = NULL;
+
+ if (dso->kernel == DSO_TYPE_USER) {
+ GElf_Shdr shdr;
+ ss->adjust_symbols = (ehdr.e_type == ET_EXEC ||
+ elf_section_by_name(elf, &ehdr, &shdr,
+ ".gnu.prelink_undo",
+ NULL) != NULL);
+ } else {
+ ss->adjust_symbols = 0;
+ }
+
+ ss->name = strdup(name);
+ if (!ss->name)
+ goto out_elf_end;
+
+ ss->elf = elf;
+ ss->fd = fd;
+ ss->ehdr = ehdr;
+ ss->type = type;
+
+ return 0;
+
+out_elf_end:
+ elf_end(elf);
+out_close:
+ close(fd);
+ return err;
+}
+
+int dso__load_sym(struct dso *dso, struct map *map,
+ struct symsrc *syms_ss, struct symsrc *runtime_ss,
+ symbol_filter_t filter, int kmodule)
+{
+ struct kmap *kmap = dso->kernel ? map__kmap(map) : NULL;
+ struct map *curr_map = map;
+ struct dso *curr_dso = dso;
+ Elf_Data *symstrs, *secstrs;
+ uint32_t nr_syms;
+ int err = -1;
+ uint32_t idx;
+ GElf_Ehdr ehdr;
+ GElf_Shdr shdr;
+ Elf_Data *syms, *opddata = NULL;
+ GElf_Sym sym;
+ Elf_Scn *sec, *sec_strndx;
+ Elf *elf;
+ int nr = 0;
+
+ dso->symtab_type = syms_ss->type;
+
+ if (!syms_ss->symtab) {
+ syms_ss->symtab = syms_ss->dynsym;
+ syms_ss->symshdr = syms_ss->dynshdr;
+ }
+
+ elf = syms_ss->elf;
+ ehdr = syms_ss->ehdr;
+ sec = syms_ss->symtab;
+ shdr = syms_ss->symshdr;
+
+ if (runtime_ss->opdsec)
+ opddata = elf_rawdata(runtime_ss->opdsec, NULL);
+
+ syms = elf_getdata(sec, NULL);
+ if (syms == NULL)
+ goto out_elf_end;
+
+ sec = elf_getscn(elf, shdr.sh_link);
+ if (sec == NULL)
+ goto out_elf_end;
+
+ symstrs = elf_getdata(sec, NULL);
+ if (symstrs == NULL)
+ goto out_elf_end;
+
+ sec_strndx = elf_getscn(elf, ehdr.e_shstrndx);
+ if (sec_strndx == NULL)
+ goto out_elf_end;
+
+ secstrs = elf_getdata(sec_strndx, NULL);
+ if (secstrs == NULL)
+ goto out_elf_end;
+
+ nr_syms = shdr.sh_size / shdr.sh_entsize;
+
+ memset(&sym, 0, sizeof(sym));
+ dso->adjust_symbols = runtime_ss->adjust_symbols;
+ elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
+ struct symbol *f;
+ const char *elf_name = elf_sym__name(&sym, symstrs);
+ char *demangled = NULL;
+ int is_label = elf_sym__is_label(&sym);
+ const char *section_name;
+ bool used_opd = false;
+
+ if (kmap && kmap->ref_reloc_sym && kmap->ref_reloc_sym->name &&
+ strcmp(elf_name, kmap->ref_reloc_sym->name) == 0)
+ kmap->ref_reloc_sym->unrelocated_addr = sym.st_value;
+
+ if (!is_label && !elf_sym__is_a(&sym, map->type))
+ continue;
+
+ /* Reject ARM ELF "mapping symbols": these aren't unique and
+ * don't identify functions, so will confuse the profile
+ * output: */
+ if (ehdr.e_machine == EM_ARM) {
+ if (!strcmp(elf_name, "$a") ||
+ !strcmp(elf_name, "$d") ||
+ !strcmp(elf_name, "$t"))
+ continue;
+ }
+
+ if (runtime_ss->opdsec && sym.st_shndx == runtime_ss->opdidx) {
+ u32 offset = sym.st_value - syms_ss->opdshdr.sh_addr;
+ u64 *opd = opddata->d_buf + offset;
+ sym.st_value = DSO__SWAP(dso, u64, *opd);
+ sym.st_shndx = elf_addr_to_index(runtime_ss->elf,
+ sym.st_value);
+ used_opd = true;
+ }
+
+ sec = elf_getscn(runtime_ss->elf, sym.st_shndx);
+ if (!sec)
+ goto out_elf_end;
+
+ gelf_getshdr(sec, &shdr);
+
+ if (is_label && !elf_sec__is_a(&shdr, secstrs, map->type))
+ continue;
+
+ section_name = elf_sec__name(&shdr, secstrs);
+
+ /* On ARM, symbols for thumb functions have 1 added to
+ * the symbol address as a flag - remove it */
+ if ((ehdr.e_machine == EM_ARM) &&
+ (map->type == MAP__FUNCTION) &&
+ (sym.st_value & 1))
+ --sym.st_value;
+
+ if (dso->kernel != DSO_TYPE_USER || kmodule) {
+ char dso_name[PATH_MAX];
+
+ if (strcmp(section_name,
+ (curr_dso->short_name +
+ dso->short_name_len)) == 0)
+ goto new_symbol;
+
+ if (strcmp(section_name, ".text") == 0) {
+ curr_map = map;
+ curr_dso = dso;
+ goto new_symbol;
+ }
+
+ snprintf(dso_name, sizeof(dso_name),
+ "%s%s", dso->short_name, section_name);
+
+ curr_map = map_groups__find_by_name(kmap->kmaps, map->type, dso_name);
+ if (curr_map == NULL) {
+ u64 start = sym.st_value;
+
+ if (kmodule)
+ start += map->start + shdr.sh_offset;
+
+ curr_dso = dso__new(dso_name);
+ if (curr_dso == NULL)
+ goto out_elf_end;
+ curr_dso->kernel = dso->kernel;
+ curr_dso->long_name = dso->long_name;
+ curr_dso->long_name_len = dso->long_name_len;
+ curr_map = map__new2(start, curr_dso,
+ map->type);
+ if (curr_map == NULL) {
+ dso__delete(curr_dso);
+ goto out_elf_end;
+ }
+ curr_map->map_ip = identity__map_ip;
+ curr_map->unmap_ip = identity__map_ip;
+ curr_dso->symtab_type = dso->symtab_type;
+ map_groups__insert(kmap->kmaps, curr_map);
+ dsos__add(&dso->node, curr_dso);
+ dso__set_loaded(curr_dso, map->type);
+ } else
+ curr_dso = curr_map->dso;
+
+ goto new_symbol;
+ }
+
+ if ((used_opd && runtime_ss->adjust_symbols)
+ || (!used_opd && syms_ss->adjust_symbols)) {
+ pr_debug4("%s: adjusting symbol: st_value: %#" PRIx64 " "
+ "sh_addr: %#" PRIx64 " sh_offset: %#" PRIx64 "\n", __func__,
+ (u64)sym.st_value, (u64)shdr.sh_addr,
+ (u64)shdr.sh_offset);
+ sym.st_value -= shdr.sh_addr - shdr.sh_offset;
+ }
+ /*
+ * We need to figure out if the object was created from C++ sources
+ * DWARF DW_compile_unit has this, but we don't always have access
+ * to it...
+ */
+ demangled = bfd_demangle(NULL, elf_name, DMGL_PARAMS | DMGL_ANSI);
+ if (demangled != NULL)
+ elf_name = demangled;
+new_symbol:
+ f = symbol__new(sym.st_value, sym.st_size,
+ GELF_ST_BIND(sym.st_info), elf_name);
+ free(demangled);
+ if (!f)
+ goto out_elf_end;
+
+ if (filter && filter(curr_map, f))
+ symbol__delete(f);
+ else {
+ symbols__insert(&curr_dso->symbols[curr_map->type], f);
+ nr++;
+ }
+ }
+
+ /*
+ * For misannotated, zeroed, ASM function sizes.
+ */
+ if (nr > 0) {
+ symbols__fixup_duplicate(&dso->symbols[map->type]);
+ symbols__fixup_end(&dso->symbols[map->type]);
+ if (kmap) {
+ /*
+ * We need to fixup this here too because we create new
+ * maps here, for things like vsyscall sections.
+ */
+ __map_groups__fixup_end(kmap->kmaps, map->type);
+ }
+ }
+ err = nr;
+out_elf_end:
+ return err;
+}
+
+void symbol__elf_init(void)
+{
+ elf_version(EV_CURRENT);
+}
diff --git a/tools/perf/util/symbol-minimal.c b/tools/perf/util/symbol-minimal.c
new file mode 100644
index 0000000..259f8f2
--- /dev/null
+++ b/tools/perf/util/symbol-minimal.c
@@ -0,0 +1,307 @@
+#include "symbol.h"
+
+#include <elf.h>
+#include <stdio.h>
+#include <fcntl.h>
+#include <string.h>
+#include <byteswap.h>
+#include <sys/stat.h>
+
+
+static bool check_need_swap(int file_endian)
+{
+ const int data = 1;
+ u8 *check = (u8 *)&data;
+ int host_endian;
+
+ if (check[0] == 1)
+ host_endian = ELFDATA2LSB;
+ else
+ host_endian = ELFDATA2MSB;
+
+ return host_endian != file_endian;
+}
+
+#define NOTE_ALIGN(sz) (((sz) + 3) & ~3)
+
+#define NT_GNU_BUILD_ID 3
+
+static int read_build_id(void *note_data, size_t note_len, void *bf,
+ size_t size, bool need_swap)
+{
+ struct {
+ u32 n_namesz;
+ u32 n_descsz;
+ u32 n_type;
+ } *nhdr;
+ void *ptr;
+
+ ptr = note_data;
+ while (ptr < (note_data + note_len)) {
+ const char *name;
+ size_t namesz, descsz;
+
+ nhdr = ptr;
+ if (need_swap) {
+ nhdr->n_namesz = bswap_32(nhdr->n_namesz);
+ nhdr->n_descsz = bswap_32(nhdr->n_descsz);
+ nhdr->n_type = bswap_32(nhdr->n_type);
+ }
+
+ namesz = NOTE_ALIGN(nhdr->n_namesz);
+ descsz = NOTE_ALIGN(nhdr->n_descsz);
+
+ ptr += sizeof(*nhdr);
+ name = ptr;
+ ptr += namesz;
+ if (nhdr->n_type == NT_GNU_BUILD_ID &&
+ nhdr->n_namesz == sizeof("GNU")) {
+ if (memcmp(name, "GNU", sizeof("GNU")) == 0) {
+ size_t sz = min(size, descsz);
+ memcpy(bf, ptr, sz);
+ memset(bf + sz, 0, size - sz);
+ return 0;
+ }
+ }
+ ptr += descsz;
+ }
+
+ return -1;
+}
+
+int filename__read_debuglink(const char *filename __maybe_unused,
+ char *debuglink __maybe_unused,
+ size_t size __maybe_unused)
+{
+ return -1;
+}
+
+/*
+ * Just try PT_NOTE header otherwise fails
+ */
+int filename__read_build_id(const char *filename, void *bf, size_t size)
+{
+ FILE *fp;
+ int ret = -1;
+ bool need_swap = false;
+ u8 e_ident[EI_NIDENT];
+ size_t buf_size;
+ void *buf;
+ int i;
+
+ fp = fopen(filename, "r");
+ if (fp == NULL)
+ return -1;
+
+ if (fread(e_ident, sizeof(e_ident), 1, fp) != 1)
+ goto out;
+
+ if (memcmp(e_ident, ELFMAG, SELFMAG) ||
+ e_ident[EI_VERSION] != EV_CURRENT)
+ goto out;
+
+ need_swap = check_need_swap(e_ident[EI_DATA]);
+
+ /* for simplicity */
+ fseek(fp, 0, SEEK_SET);
+
+ if (e_ident[EI_CLASS] == ELFCLASS32) {
+ Elf32_Ehdr ehdr;
+ Elf32_Phdr *phdr;
+
+ if (fread(&ehdr, sizeof(ehdr), 1, fp) != 1)
+ goto out;
+
+ if (need_swap) {
+ ehdr.e_phoff = bswap_32(ehdr.e_phoff);
+ ehdr.e_phentsize = bswap_16(ehdr.e_phentsize);
+ ehdr.e_phnum = bswap_16(ehdr.e_phnum);
+ }
+
+ buf_size = ehdr.e_phentsize * ehdr.e_phnum;
+ buf = malloc(buf_size);
+ if (buf == NULL)
+ goto out;
+
+ fseek(fp, ehdr.e_phoff, SEEK_SET);
+ if (fread(buf, buf_size, 1, fp) != 1)
+ goto out_free;
+
+ for (i = 0, phdr = buf; i < ehdr.e_phnum; i++, phdr++) {
+ void *tmp;
+
+ if (need_swap) {
+ phdr->p_type = bswap_32(phdr->p_type);
+ phdr->p_offset = bswap_32(phdr->p_offset);
+ phdr->p_filesz = bswap_32(phdr->p_filesz);
+ }
+
+ if (phdr->p_type != PT_NOTE)
+ continue;
+
+ buf_size = phdr->p_filesz;
+ tmp = realloc(buf, buf_size);
+ if (tmp == NULL)
+ goto out_free;
+
+ buf = tmp;
+ fseek(fp, phdr->p_offset, SEEK_SET);
+ if (fread(buf, buf_size, 1, fp) != 1)
+ goto out_free;
+
+ ret = read_build_id(buf, buf_size, bf, size, need_swap);
+ if (ret == 0)
+ ret = size;
+ break;
+ }
+ } else {
+ Elf64_Ehdr ehdr;
+ Elf64_Phdr *phdr;
+
+ if (fread(&ehdr, sizeof(ehdr), 1, fp) != 1)
+ goto out;
+
+ if (need_swap) {
+ ehdr.e_phoff = bswap_64(ehdr.e_phoff);
+ ehdr.e_phentsize = bswap_16(ehdr.e_phentsize);
+ ehdr.e_phnum = bswap_16(ehdr.e_phnum);
+ }
+
+ buf_size = ehdr.e_phentsize * ehdr.e_phnum;
+ buf = malloc(buf_size);
+ if (buf == NULL)
+ goto out;
+
+ fseek(fp, ehdr.e_phoff, SEEK_SET);
+ if (fread(buf, buf_size, 1, fp) != 1)
+ goto out_free;
+
+ for (i = 0, phdr = buf; i < ehdr.e_phnum; i++, phdr++) {
+ void *tmp;
+
+ if (need_swap) {
+ phdr->p_type = bswap_32(phdr->p_type);
+ phdr->p_offset = bswap_64(phdr->p_offset);
+ phdr->p_filesz = bswap_64(phdr->p_filesz);
+ }
+
+ if (phdr->p_type != PT_NOTE)
+ continue;
+
+ buf_size = phdr->p_filesz;
+ tmp = realloc(buf, buf_size);
+ if (tmp == NULL)
+ goto out_free;
+
+ buf = tmp;
+ fseek(fp, phdr->p_offset, SEEK_SET);
+ if (fread(buf, buf_size, 1, fp) != 1)
+ goto out_free;
+
+ ret = read_build_id(buf, buf_size, bf, size, need_swap);
+ if (ret == 0)
+ ret = size;
+ break;
+ }
+ }
+out_free:
+ free(buf);
+out:
+ fclose(fp);
+ return ret;
+}
+
+int sysfs__read_build_id(const char *filename, void *build_id, size_t size)
+{
+ int fd;
+ int ret = -1;
+ struct stat stbuf;
+ size_t buf_size;
+ void *buf;
+
+ fd = open(filename, O_RDONLY);
+ if (fd < 0)
+ return -1;
+
+ if (fstat(fd, &stbuf) < 0)
+ goto out;
+
+ buf_size = stbuf.st_size;
+ buf = malloc(buf_size);
+ if (buf == NULL)
+ goto out;
+
+ if (read(fd, buf, buf_size) != (ssize_t) buf_size)
+ goto out_free;
+
+ ret = read_build_id(buf, buf_size, build_id, size, false);
+out_free:
+ free(buf);
+out:
+ close(fd);
+ return ret;
+}
+
+int symsrc__init(struct symsrc *ss, struct dso *dso __maybe_unused,
+ const char *name,
+ enum dso_binary_type type)
+{
+ int fd = open(name, O_RDONLY);
+ if (fd < 0)
+ return -1;
+
+ ss->name = strdup(name);
+ if (!ss->name)
+ goto out_close;
+
+ ss->type = type;
+
+ return 0;
+out_close:
+ close(fd);
+ return -1;
+}
+
+bool symsrc__possibly_runtime(struct symsrc *ss __maybe_unused)
+{
+ /* Assume all sym sources could be a runtime image. */
+ return true;
+}
+
+bool symsrc__has_symtab(struct symsrc *ss __maybe_unused)
+{
+ return false;
+}
+
+void symsrc__destroy(struct symsrc *ss)
+{
+ free(ss->name);
+ close(ss->fd);
+}
+
+int dso__synthesize_plt_symbols(struct dso *dso __maybe_unused,
+ struct symsrc *ss __maybe_unused,
+ struct map *map __maybe_unused,
+ symbol_filter_t filter __maybe_unused)
+{
+ return 0;
+}
+
+int dso__load_sym(struct dso *dso, struct map *map __maybe_unused,
+ struct symsrc *ss,
+ struct symsrc *runtime_ss __maybe_unused,
+ symbol_filter_t filter __maybe_unused,
+ int kmodule __maybe_unused)
+{
+ unsigned char *build_id[BUILD_ID_SIZE];
+
+ if (filename__read_build_id(ss->name, build_id, BUILD_ID_SIZE) > 0) {
+ dso__set_build_id(dso, build_id);
+ return 1;
+ }
+ return 0;
+}
+
+void symbol__elf_init(void)
+{
+}
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 8b63b67..e2e8c69 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -15,8 +15,6 @@
#include "symbol.h"
#include "strlist.h"
-#include <libelf.h>
-#include <gelf.h>
#include <elf.h>
#include <limits.h>
#include <sys/utsname.h>
@@ -25,15 +23,7 @@
#define KSYM_NAME_LEN 256
#endif
-#ifndef NT_GNU_BUILD_ID
-#define NT_GNU_BUILD_ID 3
-#endif
-
static void dso_cache__free(struct rb_root *root);
-static bool dso__build_id_equal(const struct dso *dso, u8 *build_id);
-static int elf_read_build_id(Elf *elf, void *bf, size_t size);
-static void dsos__add(struct list_head *head, struct dso *dso);
-static struct map *map__new2(u64 start, struct dso *dso, enum map_type type);
static int dso__load_kernel_sym(struct dso *dso, struct map *map,
symbol_filter_t filter);
static int dso__load_guest_kernel_sym(struct dso *dso, struct map *map,
@@ -170,7 +160,7 @@
return SYMBOL_B;
}
-static void symbols__fixup_duplicate(struct rb_root *symbols)
+void symbols__fixup_duplicate(struct rb_root *symbols)
{
struct rb_node *nd;
struct symbol *curr, *next;
@@ -199,7 +189,7 @@
}
}
-static void symbols__fixup_end(struct rb_root *symbols)
+void symbols__fixup_end(struct rb_root *symbols)
{
struct rb_node *nd, *prevnd = rb_first(symbols);
struct symbol *curr, *prev;
@@ -222,7 +212,7 @@
curr->end = roundup(curr->start, 4096);
}
-static void __map_groups__fixup_end(struct map_groups *mg, enum map_type type)
+void __map_groups__fixup_end(struct map_groups *mg, enum map_type type)
{
struct map *prev, *curr;
struct rb_node *nd, *prevnd = rb_first(&mg->maps[type]);
@@ -252,8 +242,7 @@
__map_groups__fixup_end(mg, i);
}
-static struct symbol *symbol__new(u64 start, u64 len, u8 binding,
- const char *name)
+struct symbol *symbol__new(u64 start, u64 len, u8 binding, const char *name)
{
size_t namelen = strlen(name) + 1;
struct symbol *sym = calloc(1, (symbol_conf.priv_size +
@@ -390,7 +379,7 @@
dso->has_build_id = 1;
}
-static void symbols__insert(struct rb_root *symbols, struct symbol *sym)
+void symbols__insert(struct rb_root *symbols, struct symbol *sym)
{
struct rb_node **p = &symbols->rb_node;
struct rb_node *parent = NULL;
@@ -574,7 +563,7 @@
int kallsyms__parse(const char *filename, void *arg,
int (*process_symbol)(void *arg, const char *name,
- char type, u64 start, u64 end))
+ char type, u64 start))
{
char *line = NULL;
size_t n;
@@ -614,13 +603,8 @@
break;
}
- /*
- * module symbols are not sorted so we add all
- * symbols with zero length and rely on
- * symbols__fixup_end() to fix it up.
- */
err = process_symbol(arg, symbol_name,
- symbol_type, start, start);
+ symbol_type, start);
if (err)
break;
}
@@ -647,7 +631,7 @@
}
static int map__process_kallsym_symbol(void *arg, const char *name,
- char type, u64 start, u64 end)
+ char type, u64 start)
{
struct symbol *sym;
struct process_kallsyms_args *a = arg;
@@ -656,8 +640,12 @@
if (!symbol_type__is_a(type, a->map->type))
return 0;
- sym = symbol__new(start, end - start + 1,
- kallsyms2elf_type(type), name);
+ /*
+ * module symbols are not sorted so we add all
+ * symbols, setting length to 0, and rely on
+ * symbols__fixup_end() to fix it up.
+ */
+ sym = symbol__new(start, 0, kallsyms2elf_type(type), name);
if (sym == NULL)
return -ENOMEM;
/*
@@ -904,556 +892,7 @@
return -1;
}
-/**
- * elf_symtab__for_each_symbol - iterate thru all the symbols
- *
- * @syms: struct elf_symtab instance to iterate
- * @idx: uint32_t idx
- * @sym: GElf_Sym iterator
- */
-#define elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) \
- for (idx = 0, gelf_getsym(syms, idx, &sym);\
- idx < nr_syms; \
- idx++, gelf_getsym(syms, idx, &sym))
-
-static inline uint8_t elf_sym__type(const GElf_Sym *sym)
-{
- return GELF_ST_TYPE(sym->st_info);
-}
-
-static inline int elf_sym__is_function(const GElf_Sym *sym)
-{
- return elf_sym__type(sym) == STT_FUNC &&
- sym->st_name != 0 &&
- sym->st_shndx != SHN_UNDEF;
-}
-
-static inline bool elf_sym__is_object(const GElf_Sym *sym)
-{
- return elf_sym__type(sym) == STT_OBJECT &&
- sym->st_name != 0 &&
- sym->st_shndx != SHN_UNDEF;
-}
-
-static inline int elf_sym__is_label(const GElf_Sym *sym)
-{
- return elf_sym__type(sym) == STT_NOTYPE &&
- sym->st_name != 0 &&
- sym->st_shndx != SHN_UNDEF &&
- sym->st_shndx != SHN_ABS;
-}
-
-static inline const char *elf_sec__name(const GElf_Shdr *shdr,
- const Elf_Data *secstrs)
-{
- return secstrs->d_buf + shdr->sh_name;
-}
-
-static inline int elf_sec__is_text(const GElf_Shdr *shdr,
- const Elf_Data *secstrs)
-{
- return strstr(elf_sec__name(shdr, secstrs), "text") != NULL;
-}
-
-static inline bool elf_sec__is_data(const GElf_Shdr *shdr,
- const Elf_Data *secstrs)
-{
- return strstr(elf_sec__name(shdr, secstrs), "data") != NULL;
-}
-
-static inline const char *elf_sym__name(const GElf_Sym *sym,
- const Elf_Data *symstrs)
-{
- return symstrs->d_buf + sym->st_name;
-}
-
-static Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
- GElf_Shdr *shp, const char *name,
- size_t *idx)
-{
- Elf_Scn *sec = NULL;
- size_t cnt = 1;
-
- while ((sec = elf_nextscn(elf, sec)) != NULL) {
- char *str;
-
- gelf_getshdr(sec, shp);
- str = elf_strptr(elf, ep->e_shstrndx, shp->sh_name);
- if (!strcmp(name, str)) {
- if (idx)
- *idx = cnt;
- break;
- }
- ++cnt;
- }
-
- return sec;
-}
-
-#define elf_section__for_each_rel(reldata, pos, pos_mem, idx, nr_entries) \
- for (idx = 0, pos = gelf_getrel(reldata, 0, &pos_mem); \
- idx < nr_entries; \
- ++idx, pos = gelf_getrel(reldata, idx, &pos_mem))
-
-#define elf_section__for_each_rela(reldata, pos, pos_mem, idx, nr_entries) \
- for (idx = 0, pos = gelf_getrela(reldata, 0, &pos_mem); \
- idx < nr_entries; \
- ++idx, pos = gelf_getrela(reldata, idx, &pos_mem))
-
-/*
- * We need to check if we have a .dynsym, so that we can handle the
- * .plt, synthesizing its symbols, that aren't on the symtabs (be it
- * .dynsym or .symtab).
- * And always look at the original dso, not at debuginfo packages, that
- * have the PLT data stripped out (shdr_rel_plt.sh_type == SHT_NOBITS).
- */
-static int
-dso__synthesize_plt_symbols(struct dso *dso, char *name, struct map *map,
- symbol_filter_t filter)
-{
- uint32_t nr_rel_entries, idx;
- GElf_Sym sym;
- u64 plt_offset;
- GElf_Shdr shdr_plt;
- struct symbol *f;
- GElf_Shdr shdr_rel_plt, shdr_dynsym;
- Elf_Data *reldata, *syms, *symstrs;
- Elf_Scn *scn_plt_rel, *scn_symstrs, *scn_dynsym;
- size_t dynsym_idx;
- GElf_Ehdr ehdr;
- char sympltname[1024];
- Elf *elf;
- int nr = 0, symidx, fd, err = 0;
-
- fd = open(name, O_RDONLY);
- if (fd < 0)
- goto out;
-
- elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
- if (elf == NULL)
- goto out_close;
-
- if (gelf_getehdr(elf, &ehdr) == NULL)
- goto out_elf_end;
-
- scn_dynsym = elf_section_by_name(elf, &ehdr, &shdr_dynsym,
- ".dynsym", &dynsym_idx);
- if (scn_dynsym == NULL)
- goto out_elf_end;
-
- scn_plt_rel = elf_section_by_name(elf, &ehdr, &shdr_rel_plt,
- ".rela.plt", NULL);
- if (scn_plt_rel == NULL) {
- scn_plt_rel = elf_section_by_name(elf, &ehdr, &shdr_rel_plt,
- ".rel.plt", NULL);
- if (scn_plt_rel == NULL)
- goto out_elf_end;
- }
-
- err = -1;
-
- if (shdr_rel_plt.sh_link != dynsym_idx)
- goto out_elf_end;
-
- if (elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL) == NULL)
- goto out_elf_end;
-
- /*
- * Fetch the relocation section to find the idxes to the GOT
- * and the symbols in the .dynsym they refer to.
- */
- reldata = elf_getdata(scn_plt_rel, NULL);
- if (reldata == NULL)
- goto out_elf_end;
-
- syms = elf_getdata(scn_dynsym, NULL);
- if (syms == NULL)
- goto out_elf_end;
-
- scn_symstrs = elf_getscn(elf, shdr_dynsym.sh_link);
- if (scn_symstrs == NULL)
- goto out_elf_end;
-
- symstrs = elf_getdata(scn_symstrs, NULL);
- if (symstrs == NULL)
- goto out_elf_end;
-
- nr_rel_entries = shdr_rel_plt.sh_size / shdr_rel_plt.sh_entsize;
- plt_offset = shdr_plt.sh_offset;
-
- if (shdr_rel_plt.sh_type == SHT_RELA) {
- GElf_Rela pos_mem, *pos;
-
- elf_section__for_each_rela(reldata, pos, pos_mem, idx,
- nr_rel_entries) {
- symidx = GELF_R_SYM(pos->r_info);
- plt_offset += shdr_plt.sh_entsize;
- gelf_getsym(syms, symidx, &sym);
- snprintf(sympltname, sizeof(sympltname),
- "%s@plt", elf_sym__name(&sym, symstrs));
-
- f = symbol__new(plt_offset, shdr_plt.sh_entsize,
- STB_GLOBAL, sympltname);
- if (!f)
- goto out_elf_end;
-
- if (filter && filter(map, f))
- symbol__delete(f);
- else {
- symbols__insert(&dso->symbols[map->type], f);
- ++nr;
- }
- }
- } else if (shdr_rel_plt.sh_type == SHT_REL) {
- GElf_Rel pos_mem, *pos;
- elf_section__for_each_rel(reldata, pos, pos_mem, idx,
- nr_rel_entries) {
- symidx = GELF_R_SYM(pos->r_info);
- plt_offset += shdr_plt.sh_entsize;
- gelf_getsym(syms, symidx, &sym);
- snprintf(sympltname, sizeof(sympltname),
- "%s@plt", elf_sym__name(&sym, symstrs));
-
- f = symbol__new(plt_offset, shdr_plt.sh_entsize,
- STB_GLOBAL, sympltname);
- if (!f)
- goto out_elf_end;
-
- if (filter && filter(map, f))
- symbol__delete(f);
- else {
- symbols__insert(&dso->symbols[map->type], f);
- ++nr;
- }
- }
- }
-
- err = 0;
-out_elf_end:
- elf_end(elf);
-out_close:
- close(fd);
-
- if (err == 0)
- return nr;
-out:
- pr_debug("%s: problems reading %s PLT info.\n",
- __func__, dso->long_name);
- return 0;
-}
-
-static bool elf_sym__is_a(GElf_Sym *sym, enum map_type type)
-{
- switch (type) {
- case MAP__FUNCTION:
- return elf_sym__is_function(sym);
- case MAP__VARIABLE:
- return elf_sym__is_object(sym);
- default:
- return false;
- }
-}
-
-static bool elf_sec__is_a(GElf_Shdr *shdr, Elf_Data *secstrs,
- enum map_type type)
-{
- switch (type) {
- case MAP__FUNCTION:
- return elf_sec__is_text(shdr, secstrs);
- case MAP__VARIABLE:
- return elf_sec__is_data(shdr, secstrs);
- default:
- return false;
- }
-}
-
-static size_t elf_addr_to_index(Elf *elf, GElf_Addr addr)
-{
- Elf_Scn *sec = NULL;
- GElf_Shdr shdr;
- size_t cnt = 1;
-
- while ((sec = elf_nextscn(elf, sec)) != NULL) {
- gelf_getshdr(sec, &shdr);
-
- if ((addr >= shdr.sh_addr) &&
- (addr < (shdr.sh_addr + shdr.sh_size)))
- return cnt;
-
- ++cnt;
- }
-
- return -1;
-}
-
-static int dso__swap_init(struct dso *dso, unsigned char eidata)
-{
- static unsigned int const endian = 1;
-
- dso->needs_swap = DSO_SWAP__NO;
-
- switch (eidata) {
- case ELFDATA2LSB:
- /* We are big endian, DSO is little endian. */
- if (*(unsigned char const *)&endian != 1)
- dso->needs_swap = DSO_SWAP__YES;
- break;
-
- case ELFDATA2MSB:
- /* We are little endian, DSO is big endian. */
- if (*(unsigned char const *)&endian != 0)
- dso->needs_swap = DSO_SWAP__YES;
- break;
-
- default:
- pr_err("unrecognized DSO data encoding %d\n", eidata);
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int dso__load_sym(struct dso *dso, struct map *map, const char *name,
- int fd, symbol_filter_t filter, int kmodule,
- int want_symtab)
-{
- struct kmap *kmap = dso->kernel ? map__kmap(map) : NULL;
- struct map *curr_map = map;
- struct dso *curr_dso = dso;
- Elf_Data *symstrs, *secstrs;
- uint32_t nr_syms;
- int err = -1;
- uint32_t idx;
- GElf_Ehdr ehdr;
- GElf_Shdr shdr, opdshdr;
- Elf_Data *syms, *opddata = NULL;
- GElf_Sym sym;
- Elf_Scn *sec, *sec_strndx, *opdsec;
- Elf *elf;
- int nr = 0;
- size_t opdidx = 0;
-
- elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
- if (elf == NULL) {
- pr_debug("%s: cannot read %s ELF file.\n", __func__, name);
- goto out_close;
- }
-
- if (gelf_getehdr(elf, &ehdr) == NULL) {
- pr_debug("%s: cannot get elf header.\n", __func__);
- goto out_elf_end;
- }
-
- if (dso__swap_init(dso, ehdr.e_ident[EI_DATA]))
- goto out_elf_end;
-
- /* Always reject images with a mismatched build-id: */
- if (dso->has_build_id) {
- u8 build_id[BUILD_ID_SIZE];
-
- if (elf_read_build_id(elf, build_id, BUILD_ID_SIZE) < 0)
- goto out_elf_end;
-
- if (!dso__build_id_equal(dso, build_id))
- goto out_elf_end;
- }
-
- sec = elf_section_by_name(elf, &ehdr, &shdr, ".symtab", NULL);
- if (sec == NULL) {
- if (want_symtab)
- goto out_elf_end;
-
- sec = elf_section_by_name(elf, &ehdr, &shdr, ".dynsym", NULL);
- if (sec == NULL)
- goto out_elf_end;
- }
-
- opdsec = elf_section_by_name(elf, &ehdr, &opdshdr, ".opd", &opdidx);
- if (opdshdr.sh_type != SHT_PROGBITS)
- opdsec = NULL;
- if (opdsec)
- opddata = elf_rawdata(opdsec, NULL);
-
- syms = elf_getdata(sec, NULL);
- if (syms == NULL)
- goto out_elf_end;
-
- sec = elf_getscn(elf, shdr.sh_link);
- if (sec == NULL)
- goto out_elf_end;
-
- symstrs = elf_getdata(sec, NULL);
- if (symstrs == NULL)
- goto out_elf_end;
-
- sec_strndx = elf_getscn(elf, ehdr.e_shstrndx);
- if (sec_strndx == NULL)
- goto out_elf_end;
-
- secstrs = elf_getdata(sec_strndx, NULL);
- if (secstrs == NULL)
- goto out_elf_end;
-
- nr_syms = shdr.sh_size / shdr.sh_entsize;
-
- memset(&sym, 0, sizeof(sym));
- if (dso->kernel == DSO_TYPE_USER) {
- dso->adjust_symbols = (ehdr.e_type == ET_EXEC ||
- elf_section_by_name(elf, &ehdr, &shdr,
- ".gnu.prelink_undo",
- NULL) != NULL);
- } else {
- dso->adjust_symbols = 0;
- }
- elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
- struct symbol *f;
- const char *elf_name = elf_sym__name(&sym, symstrs);
- char *demangled = NULL;
- int is_label = elf_sym__is_label(&sym);
- const char *section_name;
-
- if (kmap && kmap->ref_reloc_sym && kmap->ref_reloc_sym->name &&
- strcmp(elf_name, kmap->ref_reloc_sym->name) == 0)
- kmap->ref_reloc_sym->unrelocated_addr = sym.st_value;
-
- if (!is_label && !elf_sym__is_a(&sym, map->type))
- continue;
-
- /* Reject ARM ELF "mapping symbols": these aren't unique and
- * don't identify functions, so will confuse the profile
- * output: */
- if (ehdr.e_machine == EM_ARM) {
- if (!strcmp(elf_name, "$a") ||
- !strcmp(elf_name, "$d") ||
- !strcmp(elf_name, "$t"))
- continue;
- }
-
- if (opdsec && sym.st_shndx == opdidx) {
- u32 offset = sym.st_value - opdshdr.sh_addr;
- u64 *opd = opddata->d_buf + offset;
- sym.st_value = DSO__SWAP(dso, u64, *opd);
- sym.st_shndx = elf_addr_to_index(elf, sym.st_value);
- }
-
- sec = elf_getscn(elf, sym.st_shndx);
- if (!sec)
- goto out_elf_end;
-
- gelf_getshdr(sec, &shdr);
-
- if (is_label && !elf_sec__is_a(&shdr, secstrs, map->type))
- continue;
-
- section_name = elf_sec__name(&shdr, secstrs);
-
- /* On ARM, symbols for thumb functions have 1 added to
- * the symbol address as a flag - remove it */
- if ((ehdr.e_machine == EM_ARM) &&
- (map->type == MAP__FUNCTION) &&
- (sym.st_value & 1))
- --sym.st_value;
-
- if (dso->kernel != DSO_TYPE_USER || kmodule) {
- char dso_name[PATH_MAX];
-
- if (strcmp(section_name,
- (curr_dso->short_name +
- dso->short_name_len)) == 0)
- goto new_symbol;
-
- if (strcmp(section_name, ".text") == 0) {
- curr_map = map;
- curr_dso = dso;
- goto new_symbol;
- }
-
- snprintf(dso_name, sizeof(dso_name),
- "%s%s", dso->short_name, section_name);
-
- curr_map = map_groups__find_by_name(kmap->kmaps, map->type, dso_name);
- if (curr_map == NULL) {
- u64 start = sym.st_value;
-
- if (kmodule)
- start += map->start + shdr.sh_offset;
-
- curr_dso = dso__new(dso_name);
- if (curr_dso == NULL)
- goto out_elf_end;
- curr_dso->kernel = dso->kernel;
- curr_dso->long_name = dso->long_name;
- curr_dso->long_name_len = dso->long_name_len;
- curr_map = map__new2(start, curr_dso,
- map->type);
- if (curr_map == NULL) {
- dso__delete(curr_dso);
- goto out_elf_end;
- }
- curr_map->map_ip = identity__map_ip;
- curr_map->unmap_ip = identity__map_ip;
- curr_dso->symtab_type = dso->symtab_type;
- map_groups__insert(kmap->kmaps, curr_map);
- dsos__add(&dso->node, curr_dso);
- dso__set_loaded(curr_dso, map->type);
- } else
- curr_dso = curr_map->dso;
-
- goto new_symbol;
- }
-
- if (curr_dso->adjust_symbols) {
- pr_debug4("%s: adjusting symbol: st_value: %#" PRIx64 " "
- "sh_addr: %#" PRIx64 " sh_offset: %#" PRIx64 "\n", __func__,
- (u64)sym.st_value, (u64)shdr.sh_addr,
- (u64)shdr.sh_offset);
- sym.st_value -= shdr.sh_addr - shdr.sh_offset;
- }
- /*
- * We need to figure out if the object was created from C++ sources
- * DWARF DW_compile_unit has this, but we don't always have access
- * to it...
- */
- demangled = bfd_demangle(NULL, elf_name, DMGL_PARAMS | DMGL_ANSI);
- if (demangled != NULL)
- elf_name = demangled;
-new_symbol:
- f = symbol__new(sym.st_value, sym.st_size,
- GELF_ST_BIND(sym.st_info), elf_name);
- free(demangled);
- if (!f)
- goto out_elf_end;
-
- if (filter && filter(curr_map, f))
- symbol__delete(f);
- else {
- symbols__insert(&curr_dso->symbols[curr_map->type], f);
- nr++;
- }
- }
-
- /*
- * For misannotated, zeroed, ASM function sizes.
- */
- if (nr > 0) {
- symbols__fixup_duplicate(&dso->symbols[map->type]);
- symbols__fixup_end(&dso->symbols[map->type]);
- if (kmap) {
- /*
- * We need to fixup this here too because we create new
- * maps here, for things like vsyscall sections.
- */
- __map_groups__fixup_end(kmap->kmaps, map->type);
- }
- }
- err = nr;
-out_elf_end:
- elf_end(elf);
-out_close:
- return err;
-}
-
-static bool dso__build_id_equal(const struct dso *dso, u8 *build_id)
+bool dso__build_id_equal(const struct dso *dso, u8 *build_id)
{
return memcmp(dso->build_id, build_id, sizeof(dso->build_id)) == 0;
}
@@ -1480,216 +919,11 @@
return have_build_id;
}
-/*
- * Align offset to 4 bytes as needed for note name and descriptor data.
- */
-#define NOTE_ALIGN(n) (((n) + 3) & -4U)
-
-static int elf_read_build_id(Elf *elf, void *bf, size_t size)
-{
- int err = -1;
- GElf_Ehdr ehdr;
- GElf_Shdr shdr;
- Elf_Data *data;
- Elf_Scn *sec;
- Elf_Kind ek;
- void *ptr;
-
- if (size < BUILD_ID_SIZE)
- goto out;
-
- ek = elf_kind(elf);
- if (ek != ELF_K_ELF)
- goto out;
-
- if (gelf_getehdr(elf, &ehdr) == NULL) {
- pr_err("%s: cannot get elf header.\n", __func__);
- goto out;
- }
-
- /*
- * Check following sections for notes:
- * '.note.gnu.build-id'
- * '.notes'
- * '.note' (VDSO specific)
- */
- do {
- sec = elf_section_by_name(elf, &ehdr, &shdr,
- ".note.gnu.build-id", NULL);
- if (sec)
- break;
-
- sec = elf_section_by_name(elf, &ehdr, &shdr,
- ".notes", NULL);
- if (sec)
- break;
-
- sec = elf_section_by_name(elf, &ehdr, &shdr,
- ".note", NULL);
- if (sec)
- break;
-
- return err;
-
- } while (0);
-
- data = elf_getdata(sec, NULL);
- if (data == NULL)
- goto out;
-
- ptr = data->d_buf;
- while (ptr < (data->d_buf + data->d_size)) {
- GElf_Nhdr *nhdr = ptr;
- size_t namesz = NOTE_ALIGN(nhdr->n_namesz),
- descsz = NOTE_ALIGN(nhdr->n_descsz);
- const char *name;
-
- ptr += sizeof(*nhdr);
- name = ptr;
- ptr += namesz;
- if (nhdr->n_type == NT_GNU_BUILD_ID &&
- nhdr->n_namesz == sizeof("GNU")) {
- if (memcmp(name, "GNU", sizeof("GNU")) == 0) {
- size_t sz = min(size, descsz);
- memcpy(bf, ptr, sz);
- memset(bf + sz, 0, size - sz);
- err = descsz;
- break;
- }
- }
- ptr += descsz;
- }
-
-out:
- return err;
-}
-
-int filename__read_build_id(const char *filename, void *bf, size_t size)
-{
- int fd, err = -1;
- Elf *elf;
-
- if (size < BUILD_ID_SIZE)
- goto out;
-
- fd = open(filename, O_RDONLY);
- if (fd < 0)
- goto out;
-
- elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
- if (elf == NULL) {
- pr_debug2("%s: cannot read %s ELF file.\n", __func__, filename);
- goto out_close;
- }
-
- err = elf_read_build_id(elf, bf, size);
-
- elf_end(elf);
-out_close:
- close(fd);
-out:
- return err;
-}
-
-int sysfs__read_build_id(const char *filename, void *build_id, size_t size)
-{
- int fd, err = -1;
-
- if (size < BUILD_ID_SIZE)
- goto out;
-
- fd = open(filename, O_RDONLY);
- if (fd < 0)
- goto out;
-
- while (1) {
- char bf[BUFSIZ];
- GElf_Nhdr nhdr;
- size_t namesz, descsz;
-
- if (read(fd, &nhdr, sizeof(nhdr)) != sizeof(nhdr))
- break;
-
- namesz = NOTE_ALIGN(nhdr.n_namesz);
- descsz = NOTE_ALIGN(nhdr.n_descsz);
- if (nhdr.n_type == NT_GNU_BUILD_ID &&
- nhdr.n_namesz == sizeof("GNU")) {
- if (read(fd, bf, namesz) != (ssize_t)namesz)
- break;
- if (memcmp(bf, "GNU", sizeof("GNU")) == 0) {
- size_t sz = min(descsz, size);
- if (read(fd, build_id, sz) == (ssize_t)sz) {
- memset(build_id + sz, 0, size - sz);
- err = 0;
- break;
- }
- } else if (read(fd, bf, descsz) != (ssize_t)descsz)
- break;
- } else {
- int n = namesz + descsz;
- if (read(fd, bf, n) != n)
- break;
- }
- }
- close(fd);
-out:
- return err;
-}
-
-static int filename__read_debuglink(const char *filename,
- char *debuglink, size_t size)
-{
- int fd, err = -1;
- Elf *elf;
- GElf_Ehdr ehdr;
- GElf_Shdr shdr;
- Elf_Data *data;
- Elf_Scn *sec;
- Elf_Kind ek;
-
- fd = open(filename, O_RDONLY);
- if (fd < 0)
- goto out;
-
- elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
- if (elf == NULL) {
- pr_debug2("%s: cannot read %s ELF file.\n", __func__, filename);
- goto out_close;
- }
-
- ek = elf_kind(elf);
- if (ek != ELF_K_ELF)
- goto out_close;
-
- if (gelf_getehdr(elf, &ehdr) == NULL) {
- pr_err("%s: cannot get elf header.\n", __func__);
- goto out_close;
- }
-
- sec = elf_section_by_name(elf, &ehdr, &shdr,
- ".gnu_debuglink", NULL);
- if (sec == NULL)
- goto out_close;
-
- data = elf_getdata(sec, NULL);
- if (data == NULL)
- goto out_close;
-
- /* the start of this section is a zero-terminated string */
- strncpy(debuglink, data->d_buf, size);
-
- elf_end(elf);
-
-out_close:
- close(fd);
-out:
- return err;
-}
-
char dso__symtab_origin(const struct dso *dso)
{
static const char origin[] = {
[DSO_BINARY_TYPE__KALLSYMS] = 'k',
+ [DSO_BINARY_TYPE__VMLINUX] = 'v',
[DSO_BINARY_TYPE__JAVA_JIT] = 'j',
[DSO_BINARY_TYPE__DEBUGLINK] = 'l',
[DSO_BINARY_TYPE__BUILD_ID_CACHE] = 'B',
@@ -1700,6 +934,7 @@
[DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE] = 'K',
[DSO_BINARY_TYPE__GUEST_KALLSYMS] = 'g',
[DSO_BINARY_TYPE__GUEST_KMODULE] = 'G',
+ [DSO_BINARY_TYPE__GUEST_VMLINUX] = 'V',
};
if (dso == NULL || dso->symtab_type == DSO_BINARY_TYPE__NOT_FOUND)
@@ -1775,7 +1010,9 @@
default:
case DSO_BINARY_TYPE__KALLSYMS:
+ case DSO_BINARY_TYPE__VMLINUX:
case DSO_BINARY_TYPE__GUEST_KALLSYMS:
+ case DSO_BINARY_TYPE__GUEST_VMLINUX:
case DSO_BINARY_TYPE__JAVA_JIT:
case DSO_BINARY_TYPE__NOT_FOUND:
ret = -1;
@@ -1789,11 +1026,12 @@
{
char *name;
int ret = -1;
- int fd;
u_int i;
struct machine *machine;
char *root_dir = (char *) "";
- int want_symtab;
+ int ss_pos = 0;
+ struct symsrc ss_[2];
+ struct symsrc *syms_ss = NULL, *runtime_ss = NULL;
dso__set_loaded(dso, map->type);
@@ -1835,54 +1073,69 @@
root_dir = machine->root_dir;
/* Iterate over candidate debug images.
- * On the first pass, only load images if they have a full symtab.
- * Failing that, do a second pass where we accept .dynsym also
+ * Keep track of "interesting" ones (those which have a symtab, dynsym,
+ * and/or opd section) for processing.
*/
- want_symtab = 1;
-restart:
for (i = 0; i < DSO_BINARY_TYPE__SYMTAB_CNT; i++) {
+ struct symsrc *ss = &ss_[ss_pos];
+ bool next_slot = false;
- dso->symtab_type = binary_type_symtab[i];
+ enum dso_binary_type symtab_type = binary_type_symtab[i];
- if (dso__binary_type_file(dso, dso->symtab_type,
+ if (dso__binary_type_file(dso, symtab_type,
root_dir, name, PATH_MAX))
continue;
/* Name is now the name of the next image to try */
- fd = open(name, O_RDONLY);
- if (fd < 0)
+ if (symsrc__init(ss, dso, name, symtab_type) < 0)
continue;
- ret = dso__load_sym(dso, map, name, fd, filter, 0,
- want_symtab);
- close(fd);
-
- /*
- * Some people seem to have debuginfo files _WITHOUT_ debug
- * info!?!?
- */
- if (!ret)
- continue;
-
- if (ret > 0) {
- int nr_plt;
-
- nr_plt = dso__synthesize_plt_symbols(dso, name, map, filter);
- if (nr_plt > 0)
- ret += nr_plt;
- break;
+ if (!syms_ss && symsrc__has_symtab(ss)) {
+ syms_ss = ss;
+ next_slot = true;
}
+
+ if (!runtime_ss && symsrc__possibly_runtime(ss)) {
+ runtime_ss = ss;
+ next_slot = true;
+ }
+
+ if (next_slot) {
+ ss_pos++;
+
+ if (syms_ss && runtime_ss)
+ break;
+ }
+
}
- /*
- * If we wanted a full symtab but no image had one,
- * relax our requirements and repeat the search.
- */
- if (ret <= 0 && want_symtab) {
- want_symtab = 0;
- goto restart;
+ if (!runtime_ss && !syms_ss)
+ goto out_free;
+
+ if (runtime_ss && !syms_ss) {
+ syms_ss = runtime_ss;
}
+ /* We'll have to hope for the best */
+ if (!runtime_ss && syms_ss)
+ runtime_ss = syms_ss;
+
+ if (syms_ss)
+ ret = dso__load_sym(dso, map, syms_ss, runtime_ss, filter, 0);
+ else
+ ret = -1;
+
+ if (ret > 0) {
+ int nr_plt;
+
+ nr_plt = dso__synthesize_plt_symbols(dso, runtime_ss, map, filter);
+ if (nr_plt > 0)
+ ret += nr_plt;
+ }
+
+ for (; ss_pos > 0; ss_pos--)
+ symsrc__destroy(&ss_[ss_pos - 1]);
+out_free:
free(name);
if (ret < 0 && strstr(dso->name, " (deleted)") != NULL)
return 0;
@@ -2030,25 +1283,6 @@
return map_groups__set_modules_path_dir(&machine->kmaps, modules_path);
}
-/*
- * Constructor variant for modules (where we know from /proc/modules where
- * they are loaded) and for vmlinux, where only after we load all the
- * symbols we'll know where it starts and ends.
- */
-static struct map *map__new2(u64 start, struct dso *dso, enum map_type type)
-{
- struct map *map = calloc(1, (sizeof(*map) +
- (dso->kernel ? sizeof(struct kmap) : 0)));
- if (map != NULL) {
- /*
- * ->end will be filled after we load all the symbols
- */
- map__init(map, type, start, 0, 0, dso);
- }
-
- return map;
-}
-
struct map *machine__new_module(struct machine *machine, u64 start,
const char *filename)
{
@@ -2141,22 +1375,30 @@
int dso__load_vmlinux(struct dso *dso, struct map *map,
const char *vmlinux, symbol_filter_t filter)
{
- int err = -1, fd;
+ int err = -1;
+ struct symsrc ss;
char symfs_vmlinux[PATH_MAX];
+ enum dso_binary_type symtab_type;
snprintf(symfs_vmlinux, sizeof(symfs_vmlinux), "%s%s",
symbol_conf.symfs, vmlinux);
- fd = open(symfs_vmlinux, O_RDONLY);
- if (fd < 0)
+
+ if (dso->kernel == DSO_TYPE_GUEST_KERNEL)
+ symtab_type = DSO_BINARY_TYPE__GUEST_VMLINUX;
+ else
+ symtab_type = DSO_BINARY_TYPE__VMLINUX;
+
+ if (symsrc__init(&ss, dso, symfs_vmlinux, symtab_type))
return -1;
- dso__set_long_name(dso, (char *)vmlinux);
- dso__set_loaded(dso, map->type);
- err = dso__load_sym(dso, map, symfs_vmlinux, fd, filter, 0, 0);
- close(fd);
+ err = dso__load_sym(dso, map, &ss, &ss, filter, 0);
+ symsrc__destroy(&ss);
- if (err > 0)
+ if (err > 0) {
+ dso__set_long_name(dso, (char *)vmlinux);
+ dso__set_loaded(dso, map->type);
pr_debug("Using %s for symbols\n", symfs_vmlinux);
+ }
return err;
}
@@ -2173,10 +1415,8 @@
filename = dso__build_id_filename(dso, NULL, 0);
if (filename != NULL) {
err = dso__load_vmlinux(dso, map, filename, filter);
- if (err > 0) {
- dso__set_long_name(dso, filename);
+ if (err > 0)
goto out;
- }
free(filename);
}
@@ -2291,9 +1531,8 @@
free(kallsyms_allocated_filename);
if (err > 0) {
+ dso__set_long_name(dso, strdup("[kernel.kallsyms]"));
out_fixup:
- if (kallsyms_filename != NULL)
- dso__set_long_name(dso, strdup("[kernel.kallsyms]"));
map__fixup_start(map);
map__fixup_end(map);
}
@@ -2352,12 +1591,12 @@
return err;
}
-static void dsos__add(struct list_head *head, struct dso *dso)
+void dsos__add(struct list_head *head, struct dso *dso)
{
list_add_tail(&dso->node, head);
}
-static struct dso *dsos__find(struct list_head *head, const char *name)
+struct dso *dsos__find(struct list_head *head, const char *name)
{
struct dso *pos;
@@ -2516,7 +1755,7 @@
};
static int symbol__in_kernel(void *arg, const char *name,
- char type __used, u64 start, u64 end __used)
+ char type __maybe_unused, u64 start)
{
struct process_args *args = arg;
@@ -2752,9 +1991,10 @@
if (symbol_conf.initialized)
return 0;
- symbol_conf.priv_size = ALIGN(symbol_conf.priv_size, sizeof(u64));
+ symbol_conf.priv_size = PERF_ALIGN(symbol_conf.priv_size, sizeof(u64));
- elf_version(EV_CURRENT);
+ symbol__elf_init();
+
if (symbol_conf.sort_by_name)
symbol_conf.priv_size += (sizeof(struct symbol_name_rb_node) -
sizeof(struct symbol));
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 1fe733a..b441b07 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -10,22 +10,31 @@
#include <linux/rbtree.h>
#include <stdio.h>
#include <byteswap.h>
+#include <libgen.h>
+
+#ifndef NO_LIBELF_SUPPORT
+#include <libelf.h>
+#include <gelf.h>
+#include <elf.h>
+#endif
#ifdef HAVE_CPLUS_DEMANGLE
extern char *cplus_demangle(const char *, int);
-static inline char *bfd_demangle(void __used *v, const char *c, int i)
+static inline char *bfd_demangle(void __maybe_unused *v, const char *c, int i)
{
return cplus_demangle(c, i);
}
#else
#ifdef NO_DEMANGLE
-static inline char *bfd_demangle(void __used *v, const char __used *c,
- int __used i)
+static inline char *bfd_demangle(void __maybe_unused *v,
+ const char __maybe_unused *c,
+ int __maybe_unused i)
{
return NULL;
}
#else
+#define PACKAGE 'perf'
#include <bfd.h>
#endif
#endif
@@ -158,6 +167,8 @@
enum dso_binary_type {
DSO_BINARY_TYPE__KALLSYMS = 0,
DSO_BINARY_TYPE__GUEST_KALLSYMS,
+ DSO_BINARY_TYPE__VMLINUX,
+ DSO_BINARY_TYPE__GUEST_VMLINUX,
DSO_BINARY_TYPE__JAVA_JIT,
DSO_BINARY_TYPE__DEBUGLINK,
DSO_BINARY_TYPE__BUILD_ID_CACHE,
@@ -217,6 +228,36 @@
char name[0];
};
+struct symsrc {
+ char *name;
+ int fd;
+ enum dso_binary_type type;
+
+#ifndef NO_LIBELF_SUPPORT
+ Elf *elf;
+ GElf_Ehdr ehdr;
+
+ Elf_Scn *opdsec;
+ size_t opdidx;
+ GElf_Shdr opdshdr;
+
+ Elf_Scn *symtab;
+ GElf_Shdr symshdr;
+
+ Elf_Scn *dynsym;
+ size_t dynsym_idx;
+ GElf_Shdr dynshdr;
+
+ bool adjust_symbols;
+#endif
+};
+
+void symsrc__destroy(struct symsrc *ss);
+int symsrc__init(struct symsrc *ss, struct dso *dso, const char *name,
+ enum dso_binary_type type);
+bool symsrc__has_symtab(struct symsrc *ss);
+bool symsrc__possibly_runtime(struct symsrc *ss);
+
#define DSO__SWAP(dso, type, val) \
({ \
type ____r = val; \
@@ -254,6 +295,8 @@
void dso__sort_by_name(struct dso *dso, enum map_type type);
+void dsos__add(struct list_head *head, struct dso *dso);
+struct dso *dsos__find(struct list_head *head, const char *name);
struct dso *__dsos__findnew(struct list_head *head, const char *name);
int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter);
@@ -283,6 +326,7 @@
char dso__symtab_origin(const struct dso *dso);
void dso__set_long_name(struct dso *dso, char *name);
void dso__set_build_id(struct dso *dso, void *build_id);
+bool dso__build_id_equal(const struct dso *dso, u8 *build_id);
void dso__read_running_kernel_build_id(struct dso *dso,
struct machine *machine);
struct map *dso__new_map(const char *name);
@@ -297,7 +341,9 @@
int build_id__sprintf(const u8 *build_id, int len, char *bf);
int kallsyms__parse(const char *filename, void *arg,
int (*process_symbol)(void *arg, const char *name,
- char type, u64 start, u64 end));
+ char type, u64 start));
+int filename__read_debuglink(const char *filename, char *debuglink,
+ size_t size);
void machine__destroy_kernel_maps(struct machine *machine);
int __machine__create_kernel_maps(struct machine *machine, struct dso *kernel);
@@ -309,6 +355,8 @@
int symbol__init(void);
void symbol__exit(void);
+void symbol__elf_init(void);
+struct symbol *symbol__new(u64 start, u64 len, u8 binding, const char *name);
size_t symbol__fprintf_symname_offs(const struct symbol *sym,
const struct addr_location *al, FILE *fp);
size_t symbol__fprintf_symname(const struct symbol *sym, FILE *fp);
@@ -326,4 +374,15 @@
struct machine *machine, u64 addr,
u8 *data, ssize_t size);
int dso__test_data(void);
+int dso__load_sym(struct dso *dso, struct map *map, struct symsrc *syms_ss,
+ struct symsrc *runtime_ss, symbol_filter_t filter,
+ int kmodule);
+int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss,
+ struct map *map, symbol_filter_t filter);
+
+void symbols__insert(struct rb_root *symbols, struct symbol *sym);
+void symbols__fixup_duplicate(struct rb_root *symbols);
+void symbols__fixup_end(struct rb_root *symbols);
+void __map_groups__fixup_end(struct map_groups *mg, enum map_type type);
+
#endif /* __PERF_SYMBOL */
diff --git a/tools/perf/util/target.c b/tools/perf/util/target.c
index 051eaa6..065528b 100644
--- a/tools/perf/util/target.c
+++ b/tools/perf/util/target.c
@@ -117,8 +117,8 @@
if (err != buf) {
size_t len = strlen(err);
- char *c = mempcpy(buf, err, min(buflen - 1, len));
- *c = '\0';
+ memcpy(buf, err, min(buflen - 1, len));
+ *(buf + min(buflen - 1, len)) = '\0';
}
return 0;
diff --git a/tools/perf/util/thread.h b/tools/perf/util/thread.h
index 70c2c13..f66610b 100644
--- a/tools/perf/util/thread.h
+++ b/tools/perf/util/thread.h
@@ -16,6 +16,8 @@
bool comm_set;
char *comm;
int comm_len;
+
+ void *priv;
};
struct machine;
diff --git a/tools/perf/util/top.c b/tools/perf/util/top.c
index 7eeebce..884dde9 100644
--- a/tools/perf/util/top.c
+++ b/tools/perf/util/top.c
@@ -58,8 +58,7 @@
}
if (top->evlist->nr_entries == 1) {
- struct perf_evsel *first;
- first = list_entry(top->evlist->entries.next, struct perf_evsel, node);
+ struct perf_evsel *first = perf_evlist__first(top->evlist);
ret += SNPRINTF(bf + ret, size - ret, "%" PRIu64 "%s ",
(uint64_t)first->attr.sample_period,
top->freq ? "Hz" : "");
diff --git a/tools/perf/util/top.h b/tools/perf/util/top.h
index 33347ca..86ff1b1 100644
--- a/tools/perf/util/top.h
+++ b/tools/perf/util/top.h
@@ -5,6 +5,7 @@
#include "types.h"
#include <stddef.h>
#include <stdbool.h>
+#include <termios.h>
struct perf_evlist;
struct perf_evsel;
diff --git a/tools/perf/util/trace-event-parse.c b/tools/perf/util/trace-event-parse.c
index 0715c84..3aabcd6 100644
--- a/tools/perf/util/trace-event-parse.c
+++ b/tools/perf/util/trace-event-parse.c
@@ -162,25 +162,16 @@
return pevent_data_pid(pevent, &record);
}
-unsigned long long read_size(struct pevent *pevent, void *ptr, int size)
+unsigned long long read_size(struct event_format *event, void *ptr, int size)
{
- return pevent_read_number(pevent, ptr, size);
+ return pevent_read_number(event->pevent, ptr, size);
}
-void print_trace_event(struct pevent *pevent, int cpu, void *data, int size)
+void event_format__print(struct event_format *event,
+ int cpu, void *data, int size)
{
- struct event_format *event;
struct pevent_record record;
struct trace_seq s;
- int type;
-
- type = trace_parse_common_type(pevent, data);
-
- event = pevent_find_event(pevent, type);
- if (!event) {
- warning("ug! no event found for type %d", type);
- return;
- }
memset(&record, 0, sizeof(record));
record.cpu = cpu;
@@ -192,6 +183,19 @@
trace_seq_do_printf(&s);
}
+void print_trace_event(struct pevent *pevent, int cpu, void *data, int size)
+{
+ int type = trace_parse_common_type(pevent, data);
+ struct event_format *event = pevent_find_event(pevent, type);
+
+ if (!event) {
+ warning("ug! no event found for type %d", type);
+ return;
+ }
+
+ event_format__print(event, cpu, data, size);
+}
+
void print_event(struct pevent *pevent, int cpu, void *data, int size,
unsigned long long nsecs, char *comm)
{
@@ -217,7 +221,7 @@
}
void parse_proc_kallsyms(struct pevent *pevent,
- char *file, unsigned int size __unused)
+ char *file, unsigned int size __maybe_unused)
{
unsigned long long addr;
char *func;
@@ -225,31 +229,29 @@
char *next = NULL;
char *addr_str;
char *mod;
- char ch;
+ char *fmt;
line = strtok_r(file, "\n", &next);
while (line) {
mod = NULL;
- sscanf(line, "%as %c %as\t[%as",
- (float *)(void *)&addr_str, /* workaround gcc warning */
- &ch, (float *)(void *)&func, (float *)(void *)&mod);
+ addr_str = strtok_r(line, " ", &fmt);
addr = strtoull(addr_str, NULL, 16);
- free(addr_str);
-
- /* truncate the extra ']' */
+ /* skip character */
+ strtok_r(NULL, " ", &fmt);
+ func = strtok_r(NULL, "\t", &fmt);
+ mod = strtok_r(NULL, "]", &fmt);
+ /* truncate the extra '[' */
if (mod)
- mod[strlen(mod) - 1] = 0;
+ mod = mod + 1;
pevent_register_function(pevent, func, addr, mod);
- free(func);
- free(mod);
line = strtok_r(NULL, "\n", &next);
}
}
void parse_ftrace_printk(struct pevent *pevent,
- char *file, unsigned int size __unused)
+ char *file, unsigned int size __maybe_unused)
{
unsigned long long addr;
char *printk;
@@ -289,7 +291,7 @@
{
static int idx;
- if (!pevent->events)
+ if (!pevent || !pevent->events)
return NULL;
if (!event) {
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index 474aa7a..8715a10 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -35,12 +35,11 @@
return 0;
}
-static void process_event_unsupported(union perf_event *event __unused,
- struct pevent *pevent __unused,
- struct perf_sample *sample __unused,
- struct perf_evsel *evsel __unused,
- struct machine *machine __unused,
- struct thread *thread __unused)
+static void process_event_unsupported(union perf_event *event __maybe_unused,
+ struct perf_sample *sample __maybe_unused,
+ struct perf_evsel *evsel __maybe_unused,
+ struct machine *machine __maybe_unused,
+ struct addr_location *al __maybe_unused)
{
}
@@ -53,17 +52,19 @@
"\n etc.\n");
}
-static int python_start_script_unsupported(const char *script __unused,
- int argc __unused,
- const char **argv __unused)
+static int python_start_script_unsupported(const char *script __maybe_unused,
+ int argc __maybe_unused,
+ const char **argv __maybe_unused)
{
print_python_unsupported_msg();
return -1;
}
-static int python_generate_script_unsupported(struct pevent *pevent __unused,
- const char *outfile __unused)
+static int python_generate_script_unsupported(struct pevent *pevent
+ __maybe_unused,
+ const char *outfile
+ __maybe_unused)
{
print_python_unsupported_msg();
@@ -115,17 +116,18 @@
"\n etc.\n");
}
-static int perl_start_script_unsupported(const char *script __unused,
- int argc __unused,
- const char **argv __unused)
+static int perl_start_script_unsupported(const char *script __maybe_unused,
+ int argc __maybe_unused,
+ const char **argv __maybe_unused)
{
print_perl_unsupported_msg();
return -1;
}
-static int perl_generate_script_unsupported(struct pevent *pevent __unused,
- const char *outfile __unused)
+static int perl_generate_script_unsupported(struct pevent *pevent
+ __maybe_unused,
+ const char *outfile __maybe_unused)
{
print_perl_unsupported_msg();
diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
index 8fef1d6..a55fd37 100644
--- a/tools/perf/util/trace-event.h
+++ b/tools/perf/util/trace-event.h
@@ -9,7 +9,6 @@
struct perf_sample;
union perf_event;
struct perf_tool;
-struct thread;
extern int header_page_size_size;
extern int header_page_ts_size;
@@ -32,6 +31,8 @@
struct pevent *read_trace_init(int file_bigendian, int host_bigendian);
void print_trace_event(struct pevent *pevent, int cpu, void *data, int size);
+void event_format__print(struct event_format *event,
+ int cpu, void *data, int size);
void print_event(struct pevent *pevent, int cpu, void *data, int size,
unsigned long long nsecs, char *comm);
@@ -56,7 +57,7 @@
struct event_format *trace_find_next_event(struct pevent *pevent,
struct event_format *event);
-unsigned long long read_size(struct pevent *pevent, void *ptr, int size);
+unsigned long long read_size(struct event_format *event, void *ptr, int size);
unsigned long long eval_flag(const char *flag);
struct pevent_record *trace_read_data(struct pevent *pevent, int cpu);
@@ -74,16 +75,19 @@
void tracing_data_put(struct tracing_data *tdata);
+struct addr_location;
+
+struct perf_session;
+
struct scripting_ops {
const char *name;
int (*start_script) (const char *script, int argc, const char **argv);
int (*stop_script) (void);
void (*process_event) (union perf_event *event,
- struct pevent *pevent,
struct perf_sample *sample,
struct perf_evsel *evsel,
struct machine *machine,
- struct thread *thread);
+ struct addr_location *al);
int (*generate_script) (struct pevent *pevent, const char *outfile);
};
diff --git a/tools/perf/util/unwind.c b/tools/perf/util/unwind.c
new file mode 100644
index 0000000..958723b
--- /dev/null
+++ b/tools/perf/util/unwind.c
@@ -0,0 +1,571 @@
+/*
+ * Post mortem Dwarf CFI based unwinding on top of regs and stack dumps.
+ *
+ * Lots of this code have been borrowed or heavily inspired from parts of
+ * the libunwind 0.99 code which are (amongst other contributors I may have
+ * forgotten):
+ *
+ * Copyright (C) 2002-2007 Hewlett-Packard Co
+ * Contributed by David Mosberger-Tang <davidm@hpl.hp.com>
+ *
+ * And the bugs have been added by:
+ *
+ * Copyright (C) 2010, Frederic Weisbecker <fweisbec@gmail.com>
+ * Copyright (C) 2012, Jiri Olsa <jolsa@redhat.com>
+ *
+ */
+
+#include <elf.h>
+#include <gelf.h>
+#include <fcntl.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/mman.h>
+#include <linux/list.h>
+#include <libunwind.h>
+#include <libunwind-ptrace.h>
+#include "thread.h"
+#include "session.h"
+#include "perf_regs.h"
+#include "unwind.h"
+#include "util.h"
+
+extern int
+UNW_OBJ(dwarf_search_unwind_table) (unw_addr_space_t as,
+ unw_word_t ip,
+ unw_dyn_info_t *di,
+ unw_proc_info_t *pi,
+ int need_unwind_info, void *arg);
+
+#define dwarf_search_unwind_table UNW_OBJ(dwarf_search_unwind_table)
+
+#define DW_EH_PE_FORMAT_MASK 0x0f /* format of the encoded value */
+#define DW_EH_PE_APPL_MASK 0x70 /* how the value is to be applied */
+
+/* Pointer-encoding formats: */
+#define DW_EH_PE_omit 0xff
+#define DW_EH_PE_ptr 0x00 /* pointer-sized unsigned value */
+#define DW_EH_PE_udata4 0x03 /* unsigned 32-bit value */
+#define DW_EH_PE_udata8 0x04 /* unsigned 64-bit value */
+#define DW_EH_PE_sdata4 0x0b /* signed 32-bit value */
+#define DW_EH_PE_sdata8 0x0c /* signed 64-bit value */
+
+/* Pointer-encoding application: */
+#define DW_EH_PE_absptr 0x00 /* absolute value */
+#define DW_EH_PE_pcrel 0x10 /* rel. to addr. of encoded value */
+
+/*
+ * The following are not documented by LSB v1.3, yet they are used by
+ * GCC, presumably they aren't documented by LSB since they aren't
+ * used on Linux:
+ */
+#define DW_EH_PE_funcrel 0x40 /* start-of-procedure-relative */
+#define DW_EH_PE_aligned 0x50 /* aligned pointer */
+
+/* Flags intentionaly not handled, since they're not needed:
+ * #define DW_EH_PE_indirect 0x80
+ * #define DW_EH_PE_uleb128 0x01
+ * #define DW_EH_PE_udata2 0x02
+ * #define DW_EH_PE_sleb128 0x09
+ * #define DW_EH_PE_sdata2 0x0a
+ * #define DW_EH_PE_textrel 0x20
+ * #define DW_EH_PE_datarel 0x30
+ */
+
+struct unwind_info {
+ struct perf_sample *sample;
+ struct machine *machine;
+ struct thread *thread;
+ u64 sample_uregs;
+};
+
+#define dw_read(ptr, type, end) ({ \
+ type *__p = (type *) ptr; \
+ type __v; \
+ if ((__p + 1) > (type *) end) \
+ return -EINVAL; \
+ __v = *__p++; \
+ ptr = (typeof(ptr)) __p; \
+ __v; \
+ })
+
+static int __dw_read_encoded_value(u8 **p, u8 *end, u64 *val,
+ u8 encoding)
+{
+ u8 *cur = *p;
+ *val = 0;
+
+ switch (encoding) {
+ case DW_EH_PE_omit:
+ *val = 0;
+ goto out;
+ case DW_EH_PE_ptr:
+ *val = dw_read(cur, unsigned long, end);
+ goto out;
+ default:
+ break;
+ }
+
+ switch (encoding & DW_EH_PE_APPL_MASK) {
+ case DW_EH_PE_absptr:
+ break;
+ case DW_EH_PE_pcrel:
+ *val = (unsigned long) cur;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if ((encoding & 0x07) == 0x00)
+ encoding |= DW_EH_PE_udata4;
+
+ switch (encoding & DW_EH_PE_FORMAT_MASK) {
+ case DW_EH_PE_sdata4:
+ *val += dw_read(cur, s32, end);
+ break;
+ case DW_EH_PE_udata4:
+ *val += dw_read(cur, u32, end);
+ break;
+ case DW_EH_PE_sdata8:
+ *val += dw_read(cur, s64, end);
+ break;
+ case DW_EH_PE_udata8:
+ *val += dw_read(cur, u64, end);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ out:
+ *p = cur;
+ return 0;
+}
+
+#define dw_read_encoded_value(ptr, end, enc) ({ \
+ u64 __v; \
+ if (__dw_read_encoded_value(&ptr, end, &__v, enc)) { \
+ return -EINVAL; \
+ } \
+ __v; \
+ })
+
+static Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
+ GElf_Shdr *shp, const char *name)
+{
+ Elf_Scn *sec = NULL;
+
+ while ((sec = elf_nextscn(elf, sec)) != NULL) {
+ char *str;
+
+ gelf_getshdr(sec, shp);
+ str = elf_strptr(elf, ep->e_shstrndx, shp->sh_name);
+ if (!strcmp(name, str))
+ break;
+ }
+
+ return sec;
+}
+
+static u64 elf_section_offset(int fd, const char *name)
+{
+ Elf *elf;
+ GElf_Ehdr ehdr;
+ GElf_Shdr shdr;
+ u64 offset = 0;
+
+ elf = elf_begin(fd, PERF_ELF_C_READ_MMAP, NULL);
+ if (elf == NULL)
+ return 0;
+
+ do {
+ if (gelf_getehdr(elf, &ehdr) == NULL)
+ break;
+
+ if (!elf_section_by_name(elf, &ehdr, &shdr, name))
+ break;
+
+ offset = shdr.sh_offset;
+ } while (0);
+
+ elf_end(elf);
+ return offset;
+}
+
+struct table_entry {
+ u32 start_ip_offset;
+ u32 fde_offset;
+};
+
+struct eh_frame_hdr {
+ unsigned char version;
+ unsigned char eh_frame_ptr_enc;
+ unsigned char fde_count_enc;
+ unsigned char table_enc;
+
+ /*
+ * The rest of the header is variable-length and consists of the
+ * following members:
+ *
+ * encoded_t eh_frame_ptr;
+ * encoded_t fde_count;
+ */
+
+ /* A single encoded pointer should not be more than 8 bytes. */
+ u64 enc[2];
+
+ /*
+ * struct {
+ * encoded_t start_ip;
+ * encoded_t fde_addr;
+ * } binary_search_table[fde_count];
+ */
+ char data[0];
+} __packed;
+
+static int unwind_spec_ehframe(struct dso *dso, struct machine *machine,
+ u64 offset, u64 *table_data, u64 *segbase,
+ u64 *fde_count)
+{
+ struct eh_frame_hdr hdr;
+ u8 *enc = (u8 *) &hdr.enc;
+ u8 *end = (u8 *) &hdr.data;
+ ssize_t r;
+
+ r = dso__data_read_offset(dso, machine, offset,
+ (u8 *) &hdr, sizeof(hdr));
+ if (r != sizeof(hdr))
+ return -EINVAL;
+
+ /* We dont need eh_frame_ptr, just skip it. */
+ dw_read_encoded_value(enc, end, hdr.eh_frame_ptr_enc);
+
+ *fde_count = dw_read_encoded_value(enc, end, hdr.fde_count_enc);
+ *segbase = offset;
+ *table_data = (enc - (u8 *) &hdr) + offset;
+ return 0;
+}
+
+static int read_unwind_spec(struct dso *dso, struct machine *machine,
+ u64 *table_data, u64 *segbase, u64 *fde_count)
+{
+ int ret = -EINVAL, fd;
+ u64 offset;
+
+ fd = dso__data_fd(dso, machine);
+ if (fd < 0)
+ return -EINVAL;
+
+ offset = elf_section_offset(fd, ".eh_frame_hdr");
+ close(fd);
+
+ if (offset)
+ ret = unwind_spec_ehframe(dso, machine, offset,
+ table_data, segbase,
+ fde_count);
+
+ /* TODO .debug_frame check if eh_frame_hdr fails */
+ return ret;
+}
+
+static struct map *find_map(unw_word_t ip, struct unwind_info *ui)
+{
+ struct addr_location al;
+
+ thread__find_addr_map(ui->thread, ui->machine, PERF_RECORD_MISC_USER,
+ MAP__FUNCTION, ip, &al);
+ return al.map;
+}
+
+static int
+find_proc_info(unw_addr_space_t as, unw_word_t ip, unw_proc_info_t *pi,
+ int need_unwind_info, void *arg)
+{
+ struct unwind_info *ui = arg;
+ struct map *map;
+ unw_dyn_info_t di;
+ u64 table_data, segbase, fde_count;
+
+ map = find_map(ip, ui);
+ if (!map || !map->dso)
+ return -EINVAL;
+
+ pr_debug("unwind: find_proc_info dso %s\n", map->dso->name);
+
+ if (read_unwind_spec(map->dso, ui->machine,
+ &table_data, &segbase, &fde_count))
+ return -EINVAL;
+
+ memset(&di, 0, sizeof(di));
+ di.format = UNW_INFO_FORMAT_REMOTE_TABLE;
+ di.start_ip = map->start;
+ di.end_ip = map->end;
+ di.u.rti.segbase = map->start + segbase;
+ di.u.rti.table_data = map->start + table_data;
+ di.u.rti.table_len = fde_count * sizeof(struct table_entry)
+ / sizeof(unw_word_t);
+ return dwarf_search_unwind_table(as, ip, &di, pi,
+ need_unwind_info, arg);
+}
+
+static int access_fpreg(unw_addr_space_t __maybe_unused as,
+ unw_regnum_t __maybe_unused num,
+ unw_fpreg_t __maybe_unused *val,
+ int __maybe_unused __write,
+ void __maybe_unused *arg)
+{
+ pr_err("unwind: access_fpreg unsupported\n");
+ return -UNW_EINVAL;
+}
+
+static int get_dyn_info_list_addr(unw_addr_space_t __maybe_unused as,
+ unw_word_t __maybe_unused *dil_addr,
+ void __maybe_unused *arg)
+{
+ return -UNW_ENOINFO;
+}
+
+static int resume(unw_addr_space_t __maybe_unused as,
+ unw_cursor_t __maybe_unused *cu,
+ void __maybe_unused *arg)
+{
+ pr_err("unwind: resume unsupported\n");
+ return -UNW_EINVAL;
+}
+
+static int
+get_proc_name(unw_addr_space_t __maybe_unused as,
+ unw_word_t __maybe_unused addr,
+ char __maybe_unused *bufp, size_t __maybe_unused buf_len,
+ unw_word_t __maybe_unused *offp, void __maybe_unused *arg)
+{
+ pr_err("unwind: get_proc_name unsupported\n");
+ return -UNW_EINVAL;
+}
+
+static int access_dso_mem(struct unwind_info *ui, unw_word_t addr,
+ unw_word_t *data)
+{
+ struct addr_location al;
+ ssize_t size;
+
+ thread__find_addr_map(ui->thread, ui->machine, PERF_RECORD_MISC_USER,
+ MAP__FUNCTION, addr, &al);
+ if (!al.map) {
+ pr_debug("unwind: no map for %lx\n", (unsigned long)addr);
+ return -1;
+ }
+
+ if (!al.map->dso)
+ return -1;
+
+ size = dso__data_read_addr(al.map->dso, al.map, ui->machine,
+ addr, (u8 *) data, sizeof(*data));
+
+ return !(size == sizeof(*data));
+}
+
+static int reg_value(unw_word_t *valp, struct regs_dump *regs, int id,
+ u64 sample_regs)
+{
+ int i, idx = 0;
+
+ if (!(sample_regs & (1 << id)))
+ return -EINVAL;
+
+ for (i = 0; i < id; i++) {
+ if (sample_regs & (1 << i))
+ idx++;
+ }
+
+ *valp = regs->regs[idx];
+ return 0;
+}
+
+static int access_mem(unw_addr_space_t __maybe_unused as,
+ unw_word_t addr, unw_word_t *valp,
+ int __write, void *arg)
+{
+ struct unwind_info *ui = arg;
+ struct stack_dump *stack = &ui->sample->user_stack;
+ unw_word_t start, end;
+ int offset;
+ int ret;
+
+ /* Don't support write, probably not needed. */
+ if (__write || !stack || !ui->sample->user_regs.regs) {
+ *valp = 0;
+ return 0;
+ }
+
+ ret = reg_value(&start, &ui->sample->user_regs, PERF_REG_SP,
+ ui->sample_uregs);
+ if (ret)
+ return ret;
+
+ end = start + stack->size;
+
+ /* Check overflow. */
+ if (addr + sizeof(unw_word_t) < addr)
+ return -EINVAL;
+
+ if (addr < start || addr + sizeof(unw_word_t) >= end) {
+ ret = access_dso_mem(ui, addr, valp);
+ if (ret) {
+ pr_debug("unwind: access_mem %p not inside range %p-%p\n",
+ (void *)addr, (void *)start, (void *)end);
+ *valp = 0;
+ return ret;
+ }
+ return 0;
+ }
+
+ offset = addr - start;
+ *valp = *(unw_word_t *)&stack->data[offset];
+ pr_debug("unwind: access_mem addr %p, val %lx, offset %d\n",
+ (void *)addr, (unsigned long)*valp, offset);
+ return 0;
+}
+
+static int access_reg(unw_addr_space_t __maybe_unused as,
+ unw_regnum_t regnum, unw_word_t *valp,
+ int __write, void *arg)
+{
+ struct unwind_info *ui = arg;
+ int id, ret;
+
+ /* Don't support write, I suspect we don't need it. */
+ if (__write) {
+ pr_err("unwind: access_reg w %d\n", regnum);
+ return 0;
+ }
+
+ if (!ui->sample->user_regs.regs) {
+ *valp = 0;
+ return 0;
+ }
+
+ id = unwind__arch_reg_id(regnum);
+ if (id < 0)
+ return -EINVAL;
+
+ ret = reg_value(valp, &ui->sample->user_regs, id, ui->sample_uregs);
+ if (ret) {
+ pr_err("unwind: can't read reg %d\n", regnum);
+ return ret;
+ }
+
+ pr_debug("unwind: reg %d, val %lx\n", regnum, (unsigned long)*valp);
+ return 0;
+}
+
+static void put_unwind_info(unw_addr_space_t __maybe_unused as,
+ unw_proc_info_t *pi __maybe_unused,
+ void *arg __maybe_unused)
+{
+ pr_debug("unwind: put_unwind_info called\n");
+}
+
+static int entry(u64 ip, struct thread *thread, struct machine *machine,
+ unwind_entry_cb_t cb, void *arg)
+{
+ struct unwind_entry e;
+ struct addr_location al;
+
+ thread__find_addr_location(thread, machine,
+ PERF_RECORD_MISC_USER,
+ MAP__FUNCTION, ip, &al, NULL);
+
+ e.ip = ip;
+ e.map = al.map;
+ e.sym = al.sym;
+
+ pr_debug("unwind: %s:ip = 0x%" PRIx64 " (0x%" PRIx64 ")\n",
+ al.sym ? al.sym->name : "''",
+ ip,
+ al.map ? al.map->map_ip(al.map, ip) : (u64) 0);
+
+ return cb(&e, arg);
+}
+
+static void display_error(int err)
+{
+ switch (err) {
+ case UNW_EINVAL:
+ pr_err("unwind: Only supports local.\n");
+ break;
+ case UNW_EUNSPEC:
+ pr_err("unwind: Unspecified error.\n");
+ break;
+ case UNW_EBADREG:
+ pr_err("unwind: Register unavailable.\n");
+ break;
+ default:
+ break;
+ }
+}
+
+static unw_accessors_t accessors = {
+ .find_proc_info = find_proc_info,
+ .put_unwind_info = put_unwind_info,
+ .get_dyn_info_list_addr = get_dyn_info_list_addr,
+ .access_mem = access_mem,
+ .access_reg = access_reg,
+ .access_fpreg = access_fpreg,
+ .resume = resume,
+ .get_proc_name = get_proc_name,
+};
+
+static int get_entries(struct unwind_info *ui, unwind_entry_cb_t cb,
+ void *arg)
+{
+ unw_addr_space_t addr_space;
+ unw_cursor_t c;
+ int ret;
+
+ addr_space = unw_create_addr_space(&accessors, 0);
+ if (!addr_space) {
+ pr_err("unwind: Can't create unwind address space.\n");
+ return -ENOMEM;
+ }
+
+ ret = unw_init_remote(&c, addr_space, ui);
+ if (ret)
+ display_error(ret);
+
+ while (!ret && (unw_step(&c) > 0)) {
+ unw_word_t ip;
+
+ unw_get_reg(&c, UNW_REG_IP, &ip);
+ ret = entry(ip, ui->thread, ui->machine, cb, arg);
+ }
+
+ unw_destroy_addr_space(addr_space);
+ return ret;
+}
+
+int unwind__get_entries(unwind_entry_cb_t cb, void *arg,
+ struct machine *machine, struct thread *thread,
+ u64 sample_uregs, struct perf_sample *data)
+{
+ unw_word_t ip;
+ struct unwind_info ui = {
+ .sample = data,
+ .sample_uregs = sample_uregs,
+ .thread = thread,
+ .machine = machine,
+ };
+ int ret;
+
+ if (!data->user_regs.regs)
+ return -EINVAL;
+
+ ret = reg_value(&ip, &data->user_regs, PERF_REG_IP, sample_uregs);
+ if (ret)
+ return ret;
+
+ ret = entry(ip, thread, machine, cb, arg);
+ if (ret)
+ return -ENOMEM;
+
+ return get_entries(&ui, cb, arg);
+}
diff --git a/tools/perf/util/unwind.h b/tools/perf/util/unwind.h
new file mode 100644
index 0000000..a78c8b3
--- /dev/null
+++ b/tools/perf/util/unwind.h
@@ -0,0 +1,35 @@
+#ifndef __UNWIND_H
+#define __UNWIND_H
+
+#include "types.h"
+#include "event.h"
+#include "symbol.h"
+
+struct unwind_entry {
+ struct map *map;
+ struct symbol *sym;
+ u64 ip;
+};
+
+typedef int (*unwind_entry_cb_t)(struct unwind_entry *entry, void *arg);
+
+#ifndef NO_LIBUNWIND_SUPPORT
+int unwind__get_entries(unwind_entry_cb_t cb, void *arg,
+ struct machine *machine,
+ struct thread *thread,
+ u64 sample_uregs,
+ struct perf_sample *data);
+int unwind__arch_reg_id(int regnum);
+#else
+static inline int
+unwind__get_entries(unwind_entry_cb_t cb __maybe_unused,
+ void *arg __maybe_unused,
+ struct machine *machine __maybe_unused,
+ struct thread *thread __maybe_unused,
+ u64 sample_uregs __maybe_unused,
+ struct perf_sample *data __maybe_unused)
+{
+ return 0;
+}
+#endif /* NO_LIBUNWIND_SUPPORT */
+#endif /* __UNWIND_H */
diff --git a/tools/perf/util/util.c b/tools/perf/util/util.c
index d03599f..2055cf3 100644
--- a/tools/perf/util/util.c
+++ b/tools/perf/util/util.c
@@ -1,6 +1,11 @@
#include "../perf.h"
#include "util.h"
#include <sys/mman.h>
+#ifndef NO_BACKTRACE
+#include <execinfo.h>
+#endif
+#include <stdio.h>
+#include <stdlib.h>
/*
* XXX We need to find a better place for these things...
@@ -158,3 +163,23 @@
return n;
}
+
+/* Obtain a backtrace and print it to stdout. */
+#ifndef NO_BACKTRACE
+void dump_stack(void)
+{
+ void *array[16];
+ size_t size = backtrace(array, ARRAY_SIZE(array));
+ char **strings = backtrace_symbols(array, size);
+ size_t i;
+
+ printf("Obtained %zd stack frames.\n", size);
+
+ for (i = 0; i < size; i++)
+ printf("%s\n", strings[i]);
+
+ free(strings);
+}
+#else
+void dump_stack(void) {}
+#endif
diff --git a/tools/perf/util/util.h b/tools/perf/util/util.h
index b13c733..70fa70b 100644
--- a/tools/perf/util/util.h
+++ b/tools/perf/util/util.h
@@ -69,13 +69,8 @@
#include <sys/poll.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
-#include <sys/select.h>
-#include <netinet/in.h>
-#include <netinet/tcp.h>
-#include <arpa/inet.h>
-#include <netdb.h>
#include <inttypes.h>
-#include "../../../include/linux/magic.h"
+#include <linux/magic.h>
#include "types.h"
#include <sys/ttydefaults.h>
@@ -266,4 +261,6 @@
char *rtrim(char *s);
+void dump_stack(void);
+
#endif
diff --git a/tools/perf/util/vdso.c b/tools/perf/util/vdso.c
new file mode 100644
index 0000000..e60951f
--- /dev/null
+++ b/tools/perf/util/vdso.c
@@ -0,0 +1,111 @@
+
+#include <unistd.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdlib.h>
+#include <linux/kernel.h>
+
+#include "vdso.h"
+#include "util.h"
+#include "symbol.h"
+#include "linux/string.h"
+
+static bool vdso_found;
+static char vdso_file[] = "/tmp/perf-vdso.so-XXXXXX";
+
+static int find_vdso_map(void **start, void **end)
+{
+ FILE *maps;
+ char line[128];
+ int found = 0;
+
+ maps = fopen("/proc/self/maps", "r");
+ if (!maps) {
+ pr_err("vdso: cannot open maps\n");
+ return -1;
+ }
+
+ while (!found && fgets(line, sizeof(line), maps)) {
+ int m = -1;
+
+ /* We care only about private r-x mappings. */
+ if (2 != sscanf(line, "%p-%p r-xp %*x %*x:%*x %*u %n",
+ start, end, &m))
+ continue;
+ if (m < 0)
+ continue;
+
+ if (!strncmp(&line[m], VDSO__MAP_NAME,
+ sizeof(VDSO__MAP_NAME) - 1))
+ found = 1;
+ }
+
+ fclose(maps);
+ return !found;
+}
+
+static char *get_file(void)
+{
+ char *vdso = NULL;
+ char *buf = NULL;
+ void *start, *end;
+ size_t size;
+ int fd;
+
+ if (vdso_found)
+ return vdso_file;
+
+ if (find_vdso_map(&start, &end))
+ return NULL;
+
+ size = end - start;
+
+ buf = memdup(start, size);
+ if (!buf)
+ return NULL;
+
+ fd = mkstemp(vdso_file);
+ if (fd < 0)
+ goto out;
+
+ if (size == (size_t) write(fd, buf, size))
+ vdso = vdso_file;
+
+ close(fd);
+
+ out:
+ free(buf);
+
+ vdso_found = (vdso != NULL);
+ return vdso;
+}
+
+void vdso__exit(void)
+{
+ if (vdso_found)
+ unlink(vdso_file);
+}
+
+struct dso *vdso__dso_findnew(struct list_head *head)
+{
+ struct dso *dso = dsos__find(head, VDSO__MAP_NAME);
+
+ if (!dso) {
+ char *file;
+
+ file = get_file();
+ if (!file)
+ return NULL;
+
+ dso = dso__new(VDSO__MAP_NAME);
+ if (dso != NULL) {
+ dsos__add(head, dso);
+ dso__set_long_name(dso, file);
+ }
+ }
+
+ return dso;
+}
diff --git a/tools/perf/util/vdso.h b/tools/perf/util/vdso.h
new file mode 100644
index 0000000..0f76e7c
--- /dev/null
+++ b/tools/perf/util/vdso.h
@@ -0,0 +1,18 @@
+#ifndef __PERF_VDSO__
+#define __PERF_VDSO__
+
+#include <linux/types.h>
+#include <string.h>
+#include <stdbool.h>
+
+#define VDSO__MAP_NAME "[vdso]"
+
+static inline bool is_vdso_map(const char *filename)
+{
+ return !strcmp(filename, VDSO__MAP_NAME);
+}
+
+struct dso *vdso__dso_findnew(struct list_head *head);
+void vdso__exit(void);
+
+#endif /* __PERF_VDSO__ */
diff --git a/tools/perf/util/wrapper.c b/tools/perf/util/wrapper.c
index 73e900e..19f15b6 100644
--- a/tools/perf/util/wrapper.c
+++ b/tools/perf/util/wrapper.c
@@ -7,7 +7,8 @@
* There's no pack memory to release - but stay close to the Git
* version so wrap this away:
*/
-static inline void release_pack_memory(size_t size __used, int flag __used)
+static inline void release_pack_memory(size_t size __maybe_unused,
+ int flag __maybe_unused)
{
}
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include
index bde8521..96ce80a 100644
--- a/tools/scripts/Makefile.include
+++ b/tools/scripts/Makefile.include
@@ -1,6 +1,8 @@
ifeq ("$(origin O)", "command line")
- OUTPUT := $(O)/
- COMMAND_O := O=$(O)
+ dummy := $(if $(shell test -d $(O) || echo $(O)),$(error O=$(O) does not exist),)
+ ABSOLUTE_O := $(shell cd $(O) ; pwd)
+ OUTPUT := $(ABSOLUTE_O)/
+ COMMAND_O := O=$(ABSOLUTE_O)
endif
ifneq ($(OUTPUT),)
diff --git a/tools/testing/ktest/examples/include/defaults.conf b/tools/testing/ktest/examples/include/defaults.conf
index 323a552c..63a1a83 100644
--- a/tools/testing/ktest/examples/include/defaults.conf
+++ b/tools/testing/ktest/examples/include/defaults.conf
@@ -33,7 +33,7 @@
THIS_DIR := ${PWD}
-# to orginize your configs, having each machine save their configs
+# to organize your configs, having each machine save their configs
# into a separate directly is useful.
CONFIG_DIR := ${THIS_DIR}/configs/${MACHINE}
diff --git a/tools/testing/ktest/examples/include/tests.conf b/tools/testing/ktest/examples/include/tests.conf
index 4fdb811..60cedb1 100644
--- a/tools/testing/ktest/examples/include/tests.conf
+++ b/tools/testing/ktest/examples/include/tests.conf
@@ -47,7 +47,7 @@
# Build, install, boot and test with a randconfg 10 times.
# It is important that you have set MIN_CONFIG in the config
# that includes this file otherwise it is likely that the
-# randconfig will not have the neccessary configs needed to
+# randconfig will not have the necessary configs needed to
# boot your box. This version of the test requires a min
# config that has enough to make sure the target has network
# working.
diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 52b7959..c05bcd2 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -840,7 +840,9 @@
if ($rest =~ /\sIF\s+(.*)/) {
# May be a ELSE IF section.
- if (!process_if($name, $1)) {
+ if (process_if($name, $1)) {
+ $if_set = 1;
+ } else {
$skip = 1;
}
$rest = "";
diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests
index 8b40bd5..4c53cae 100644
--- a/tools/testing/selftests/vm/run_vmtests
+++ b/tools/testing/selftests/vm/run_vmtests
@@ -36,7 +36,7 @@
mount -t hugetlbfs none $mnt
echo "--------------------"
-echo "runing hugepage-mmap"
+echo "running hugepage-mmap"
echo "--------------------"
./hugepage-mmap
if [ $? -ne 0 ]; then
@@ -50,7 +50,7 @@
echo 268435456 > /proc/sys/kernel/shmmax
echo 4194304 > /proc/sys/kernel/shmall
echo "--------------------"
-echo "runing hugepage-shm"
+echo "running hugepage-shm"
echo "--------------------"
./hugepage-shm
if [ $? -ne 0 ]; then
@@ -62,7 +62,7 @@
echo $shmall > /proc/sys/kernel/shmall
echo "--------------------"
-echo "runing map_hugetlb"
+echo "running map_hugetlb"
echo "--------------------"
./map_hugetlb
if [ $? -ne 0 ]; then
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 2468523..d617f69 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1976,9 +1976,10 @@
if (copy_from_user(&csigset, sigmask_arg->sigset,
sizeof csigset))
goto out;
- }
- sigset_from_compat(&sigset, &csigset);
- r = kvm_vcpu_ioctl_set_sigmask(vcpu, &sigset);
+ sigset_from_compat(&sigset, &csigset);
+ r = kvm_vcpu_ioctl_set_sigmask(vcpu, &sigset);
+ } else
+ r = kvm_vcpu_ioctl_set_sigmask(vcpu, NULL);
break;
}
default: